When lawmakers needed $1.5 billion to pay for an expanded student visa program last week, they turned to an unexpected funding source: mortgage giants Fannie Mae and Freddie Mac.

The proposed law, which passed the House last Friday but has stalled in the Senate, is the latest in a series of bills that tap the government-backed mortgage financiers to cover the cost of new government programs.

Congress set this dangerous precedent last year by raising guarantee fees — the upfront and annual fees charged on all mortgages owned or guaranteed by Fannie and Freddie — to defray the cost of a payroll tax cut. In other words, to pay for one middle class tax cut, lawmakers levied a tax on middle class homeowners.

Congress needs to stop using Fannie and Freddie as piggy banks. Guarantee fees are the primary source of revenue for the companies, allowing them to offset operational costs and reserve against credit risk on the home loans they back. Without that income, Fannie and Freddie cannot provide liquidity to the U.S. mortgage market. Since the federal government placed the companies in conservatorship four years ago, their regulator, the Federal Housing Finance Agency, has been charged with setting actuarially sound fees that cover associated risks and protects taxpayers from future losses.

If that income is diverted to other programs, it translates to a smaller capital buffer or smaller profits for Fannie and Freddie, meaning it will take longer to pay back their $140 billion tab with taxpayers. Using the fee to pay for other government programs is essentially robbing Peter to pay Paul.

Even the companies' harshest critics are uncomfortable with this arrangement.

"Directing these fees to unrelated spending is bad policy," said Rep. Jeb Hensarling (R-Texas), who recently sponsored a bill to abolish Fannie and Freddie. "I am committed to ensuring that any future guarantee fee increases are not misdirected."

Any increase in the guarantee fee will be passed directly on to consumers. Since Fannie and Freddie currently back about 70% of mortgage originations in the U.S., a significant fee increase (or a series of small increases) could meaningfully increase the cost of homeownership and price some creditworthy borrowers out of the market entirely.

Amidst tense fiscal negotiations in Washington, lawmakers on both sides of the aisle will be tempted to raise fees further to generate much-needed government revenues, especially since both companies have been posting healthy profits in recent months.

But U.S. homeowners should not be asked to pick up the tab for programs that do nothing to help the housing market or improve the long-term health of Fannie and Freddie. To the extent that Fannie and Freddie need to raise fees to cover risks and improve their business, those funds needs to stay with the companies.

John Griffith is a policy analyst with the housing team at the Center for American Progress. Julia Gordon is the Center's director for Housing Finance and Policy.