The Risks of Oversimplifying Stress Test Models

One of the objections we've heard from banks' internal and external validators and the Federal Reserve during the last Comprehensive Capital Analysis and Review is that stress-testing models need to be very simple.

The notion is that since stress-testing results are to be woven into risk management processes, humble folks, such as C-level executives at major banks, need to understand exactly how the models work. We do not disagree with the thrust of this sentiment, though we have somewhat more faith in the ability of bank managers to understand well-thought-out econometric models than it appears some others have.

By "simple" we mean, specifically, an over-reliance on too few macroeconomic drivers. We do not question the imperative to avoid complex functional forms, unnecessary nonlinearities, and counterproductive levels of portfolio segmentation in developing stress-testing models. In terms of these three features, we come down squarely in favor of the standard "Keep It Simple, Stupid" maxim.

The way the Supervisory Stress Scenarios are defined suggests that very simple models involving only two or three economic drivers will fail to adequately represent the effect of each scenario on the bank's credit losses and pre-provision net revenue calculations. The easiest way to see this is to examine the "adverse" scenario juxtaposed with its "severely" adverse counterpart. Given that the severely adverse installment ostensibly represents a repeat of the Great Recession–though, admittedly, coming without an intervening boom period–a model that tracks a small handful of key performance measures should do a reasonable job of projecting performance under renewed stress.

The merely "adverse" scenario is another matter altogether. This event, while clearly milder than the severely adverse scenario, has a distinct nature that is fully at odds with anything that has occurred in the U.S. economy since at least the 1980s. Put simply, the scenario is stagflationary–elevated inflation and interest rates coupled with a very weak real economy, rising unemployment, and falling house prices.

In terms of only the last two concepts mentioned, the adverse scenario is roughly half as bad as the severely adverse event (see charts above). 

Now suppose that our credit-loss model was built using only these two variables, i.e. housing prices and unemployment.  Many banks that are currently stress-testing are using models with exactly this feature. Credit-loss projections will, necessarily, follow contours that closely match either a linear or nonlinear combination of these two drivers alone. The question is, given this situation: How can stagflation possibly be reflected in the adverse scenario projections developed using these models? To answer this question in a way that even C-level executives will understand: It cannot be.

But inflation has an important impact on many aspects of credit performance. For one thing, the real repayment burden for borrowers with fixed-interest loans will fall as inflation causes nominal income to rise quickly but does not affect repayment amounts. Though the ability to prepay may be harmed as interest rates rise, the desirability of refinancing loans whose real repayments are falling in an environment of rising interest rates is likewise reduced and fewer prepayments equates to a larger denominator for the loan portfolio, further diluting the loss rate. Applying these fairly uncontroversial statements to the problem of credit-loss assessment, we should expect losses, in real terms, to be lower than suggested by the dynamics witnessed in unemployment and house prices alone. This implies that real capital charges, under the adverse scenario, should also be lower than otherwise indicated by the simple model. At the very least, stagflation presents for banks an interesting research question that, clearly, cannot be addressed with a model involving only two concepts.