Banks made a great deal of money in mortgage banking last year, even those who just dabbled in it, but this year is vastly different. Higher rates have reduced volume, refinancings have all but vanished, margins have shrunk, and cut-throat pricing is proving again that in mortgage banking, you're only as smart as your most desperate competitors.

For those wondering if they should keep doing mortgage banking, here are six reasons why it might make sense to get out.

Interest Rate Risk: Mortgage rates have been dropping for 30 years, and it's a good bet that they've already hit their bottom with only one direction to go. Mortgage banking operators always claim to be fully hedged, but sudden spikes in interest rates seem to always catch some of them off-guard. Managing pipeline risk is much different than basic asset-liability matching, and one bad month can and often does wipe out the profit from six to eight good months.

Earnings Volatility and Refis: Mortgage earnings are generally joined at the hip with loan volume, and loan volume is greatly driven by refinance activity. My firm did a study of past refinance waves, and from the peak of a refi boom to the next quarter bust, the drop-off in volume ranged from 84% to 93%. Overall mortgage volume has already dropped 50% to 60% at many banks, with more decreases likely to come.

Effect on Bank Multiples: Mortgage banking is like the little girl in the nursery rhyme: "When she was good, she was very, very good, and when she was bad, she was horrid." When rates are low and volume high, mortgage banking profits are unbelievably great, and generating a 1.5% return on assets is not uncommon. Unfortunately, these profits can disappear overnight and even turn into losses, and the markets often acknowledge this risk by according a lower multiple to mortgage banking-oriented depositories. Not much is accomplished if you double your earnings but see your price-to-earnings multiple potentially cut in half.

Compliance Risk: Does this really need an explanation? The Consumer Financial Protection Bureau's Supervision and Examination Manual is well over 900 pages, and most of it is devoted to mortgage lending. If you retain the servicing rights when you sell your loans, it gets even more complicated. The broader issue is that mortgage compliance is extraordinarily complex, and as someone once put it, if you're dabbling in the mortgage business, you can't be a dabbler anymore.

Repurchases: It used to be that once you sold a mortgage loan, you were typically done with it forever. That's no longer the case. The last five years have devastated lenders that have had to re-purchase loans they had previously sold. Buybacks have slackened, but the point is that this is a whole new area of risk.

Do You Really Need to Offer Mortgages? One of the great myths of our industry is that a mortgage is the foundational product for consumer relationships. With many people having their mortgage payment automatically taken from their checking account, a significant number of borrowers don't even know who their mortgage lender is. And mortgage borrowers are much more interested in getting the lowest rate than in getting a mortgage from their primary bank.

There are many commercial banks that do just fine not offering mortgages. Some offer it through a private-label mortgage company. Some refer borrowers to local mortgage bankers, and most simply don't offer it. Mortgage banking does not generate deposits from customers, and to the extent that customer deposits are a major part of what makes a franchise valuable, mortgage banking does not help.

I cannot think of a single banker who was ever criticized for getting out of mortgage banking, but there are plenty who stayed in too long and lost their job and even lost their bank. Yes, mortgage banking can be very lucrative when times are good, but bank executives must know when to cut back, and they must also have the courage to simply exit this business when it no longer adds value to the bank and its franchise.

Joe Garrett is a principal in Garrett, McAuley, a bank advisory firm in Berkeley, CA. He had been the CEO of two community banks.