methodology for forecasting and stress-testing abs and rmbs deals · 2019-06-01 · moodys...

19
Methodology for Forecasting and Stress-Testing ABS and RMBS Deals August 5, 2010 Prepared By ECONOMIC & CONSUMER CREDIT ANALYTICS Juan Carlos Calcagno Senior Economist [email protected] Anthony Hughes Senior Director [email protected] Ioannis Stamatopoulos Assistant Director [email protected]

Upload: others

Post on 20-Jun-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

Methodology for Forecasting and Stress-Testing ABS and RMBS Deals

August 5, 2010

Prepared By

ECONOMIC & CONSUMER CREDIT ANALYTICS

Juan Carlos CalcagnoSenior [email protected]

Anthony HughesSenior [email protected]

Ioannis StamatopoulosAssistant [email protected]

Page 2: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

Methodology for Forecasting and Stress-Testing ABS and RMBS Deals

In the recent debate about the future of structured finance, an analogy with another sort of ABS is instructive. Several incidents and auto recalls over the past year were attributed to software problems in the antilock braking systems of certain vehicles. Though the recent events were tragic,

no one would dispute the fact that ABS technology has saved many lives since becoming standard equipment on vehicles. Most motorists understand nothing of the complex software that stops cars a critical fraction of a second sooner. However, neither their lack of technical understanding nor their awareness of the software flaws—since corrected—is leading people to demand a return to old-style brakes, because they rightly perceive the benefits of ABS as far outweighing the risks.

Some people do argue, however, that asset-backed securities—equally complex and misunderstood—pose an unacceptable threat to economic prosperity. For the financial version of ABS, the benefits are arguably more opaque than for the vehicular version,1 though they are just as real. The harm that ensued from extreme exuberance in the ABS market during the housing boom was also more wide-spread than the vehicular ABS problem. However, just as the answer to faulty brakes lay in applying expert know-how to improve safety, the answer to the dormant market for structured securities lies in research and development. We already have products such as Moody’s Analytics’ Structured Finance Workstation that enables users to value and price even the most complex bonds so long as forecasts of the cash flow from the loans backing the deal are accurate. Using this tool, the multifaceted hierar-chy that governs how borrowers’ repayments are distributed to investors of differing seniority can be precisely calculated once one has predicted the extent to which the loans in the pools backing the deal either amortize, are paid off early, or default and funds from the sale of collateral are recovered.

In this article, we present new research on how best to forecast collateral losses that bolsters the case for a vibrant market for structured asset-backed securities. With the waterfall modeling problem solved, we will concentrate on the more controversial issue of how best, in an uncertain world, to pre-dict the underlying cash flow from the assets backing each deal. A natural extension of this pursuit is a system that enables investors and originators to conduct meaningful stress tests so they can more confidently assess the probability of elevated capital losses. Moody’s Analytics has engaged in a num-ber of consulting projects in recent years using this modeling approach.

1 For a broad discussion on the benefits of securitization, see Fabbozzi and Kothari (2008), Green and Wachter (2005), and Kendall and Fishman (2000).

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 1

Page 3: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2

In general, structured finance deals are put together by pooling groups of consumer loans and placing the principal and interest paid by borrowers into a kitty to be distribut-ed to investors who own the rights to differ-ent tranches representing different degrees of risk. A deal is often composed of numer-ous pools of loans, each with distinct charac-teristics. Complex rules then govern how the money in the kitty is distributed to investors. Some investors may be more exposed to a particular pool within a deal than others who hold bonds carrying the same rating. Conversely, the pools may all pay into a common kitty, in which case the division into pools serves a purely administrative function. The value of the bonds is thus determined by the extent to which the mortgage pools are able to contribute cash to the kitty. Therefore, the accuracy of forecasts of these cash streams should be the objective of any model used to assess bond value. Whether or not individual borrowers default is of no consequence to investors in these securities; they care only about accurately predicting the underlying aggregate cash flow.

In this paper, we explore the efficacy of sev-eral credit risk modeling strategies. We begin by demonstrating, through Monte Carlo simu-lations in a rarefied but realistic setting, that simple aggregate specifications outperform loan-level models even when the loan-level model is correctly specified. We then turn our attention to modeling pool-level data derived from actual ABS deals and describe what we view as the best approach to forecasting cash flow, one that considers economic condi-tions at loan origination, loan characteristics, past pool performance, and dynamics in the macroeconomic environment over time. This approach will be applied to U.S. residential mortgage-backed securities pools, and the models will be validated against simple alter-native forecasting approaches. The same basic methodology has been successfully applied to other types of asset-backed securities such as international RMBS, auto ABS, student loans, credit cards, and business leases. In all cases, the methodology is immediately and globally available, allowing any type of security, provid-ed data are available, to be accurately valued and stress-tested.

Aggregate Models Prove SuperiorA. The Experiment

When loss is determined by aggregate forecast accuracy, simple, parsimonious speci-fications are generally found to outperform more complex models (Armstrong (1985), Allen and Fildes (2001) inter alia). In many cases, even carefully built econometric mod-els forecast less accurately than simple pre-dictors like an AR(1) or even a naïve forecast. Standard methods used to assess individual-level models such as the Gini coefficient or KS statistic weigh on questions unrelated to the central issue of aggregate forecasting abil-ity. A credit score that more accurately ranks borrowers may indeed enjoy better aggregate forecasting performance than one that poorly ranks them, but this amounts to nothing if the optimal scoring system fails to beat the naïve forecast in terms of aggregate prediction ac-curacy. In this section, we will demonstrate, using Monte Carlo simulations, that loan-level models predict future credit performance poorly when compared with simple aggre-gate specifications, even when the loan-level model is correctly specified.

The key to a successful simulation ex-ercise is to design a scenario that is both understandable in terms of its intricate detail and broadly realistic. If a technique is shown to perform optimally under readily under-standable circumstances, it should perform well in real life. Here, we will assume that a large number of individuals are granted loans on the first day of a given month and are ob-ligated to pay on the final day of the month. Failure to pay constitutes a default, which is the pathology of primary interest. This situa-tion most closely resembles common payday loans. In the experiment, 5,000 such loans are originated each month, and the outcome of each individual contract is observed. The dependent variable is thus binary, mean-ing a logistic regression is an appropriate individual-level model. Each individual has a credit score drawn from a uniform distribu-tion. Any credit score is really just an ordinal measure whose scale is irrelevant. This distributional assumption can therefore be made without any loss of generality.

Each individual has a positive probability of becoming unemployed during the period of the contract. We consider both the case in which credit scores and individual unem-ployment incidence are uncorrelated and the more realistic case in which they are moder-ately negatively correlated. This means that those with weak credit histories are assumed to be more likely to suffer unemployment in any given period. The actual unemployment rate among our 5,000 individuals follows the observed U.S. unemployment rate. In each case, we use 50 consecutive months of data to define the in-sample period and then construct forecasts and stressed scenarios for each of the subsequent 24 months. This means that there are 250,000 observations in the individual-level estimation sample and 50 observed default rates with which to esti-mate the aggregate model.

We consider two points in time. The first is in early 2006, when Moody’s Analytics predicted unemployment would inch higher during the 24-month forecast period yet the observed rate fell through 2007 before turn-ing upward toward the end of the forecast horizon. The second is in early 2008, the forecast period that included the rapid up-swing in unemployment in 2008-2009 as the recent recession took hold. Though the actual unemployment rate eventually nudged 10%, Moody’s Analytics, similar to the consensus of economic opinion at the time, had forecast an increase to only about 6%. For both periods, we also include results of an archived stress test unemployment rate made concurrently with the forecasts. For the 2008-2010 fore-casting exercise, this stress scenario lines up quite neatly with the actual unemployment rate’s ultimate path. These situations are de-picted in Charts 1 and 2.

The individual default outcomes were then generated as a function of the credit score and the individual’s unemployment status during the life of the loan. We consid-er a variety of situations here: neither unem-ployment nor the credit score drive default; the credit score is important but unemploy-ment is not; unemployment is important and the credit score is not; and both unemploy-ment and the credit score are important driv-ers of default. Defaults are generated in such

ECONOMIC & CONSUMER CREDIT ANALYTICS

Page 4: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 3

ECONOMIC & CONSUMER CREDIT ANALYTICS

a way that the average default rate equals 10% through the 74-month period.

It is assumed that the only individual-level characteristic the modelers observe, be-sides the in-sample default outcome, is the credit score. Individual unemployment out-comes are not observed—only the aggregate unemployment rate. For individual-level models, the aggregate unemployment rate is used as a proxy for the unobserved individual incidence of unemployment during the con-tract period. This situation is very realistic; credit bureaus or other data collectors rarely, if ever, observe individual employment out-comes. Even banks generally do not know whether a client has become unemployed.

The competing models we consider are:1. An individual logit model of default

using the aggregate unemployment rate and the observed individual credit score as independent variables.

2. The same as (1) but with an interac-tion term between the two regressors. Though this is an overfitted model, given the data-generating process, the inclusion of an interaction may alleviate the proxy variable issues resulting from our nonob-servance of unemployment outcomes.

3. An aggregate linear regression model of default rate on average credit score (which is actually excluded from the model because it is constant over time and thus perfectly correlated with the intercept term) and the aggregate un-employment rate.

The two individual-level models are cali-brated so that they each predict the aggre-gate default rate correctly during part of the in-sample period—we consider using both the last three months of observed history as well as the last year to this end. This calibra-tion is necessary because uncalibrated logit models tend to underpredict the occurrence of rare events such as defaults quite severely. A calibration such as this is the standard ap-proach when using a logistic regression ap-plied to individual level data.

We replicated this simulation 100 times.

B. Results When the Unemployment Rate Forecast Proves Accurate

The results in Table 1 represent the mean squared error of the aggregate-level model relative to each individual-level model. Numbers greater than unity rep-resent cases in which the aggregate model outperforms the loan-level variant. The tables indicate that, conditional on the unemployment rate forecast proving accu-rate, the aggregate model always outper-forms the individual-level models based on squared error loss. Conditional on know-ing the future path of the unemployment rate, we can categorically conclude that aggregate models are preferable for risk forecasting in this context.

In one instance, when the forecast pe-riod is 2008-2010, the correlation between unemployment and the credit score is nega-tive, and both credit scores and unemploy-

ment are important drivers of default, the loan-level model’s performance, without an interaction term, approaches that of the ag-gregate model but does not surpass it. Bear in mind that when parameters are switched on in the data-generating process, they are set high to test the various methodologies under extreme circumstances. The loan-level models seem to work adequately only when all parameters are set extremely high.

In most cases, the aggregate model’s per-formance far surpasses that of either loan-level specification. For instance, when the 2008-2010 period is used, the correlation is zero, and unemployment is the only factor driving default, the individual-level models, regardless of the details, suffer about 10,000 times as much mean squared forecast error as the aggregate model. In general, when credit scores are not directly relevant to the determination of default, perhaps not surprisingly, individual-level models do es-pecially poorly relative to aggregate models in predicting the default probability under the conditions established in the experiment. The other circumstance in which individual-level models do especially poorly is when the correlation between unemployment and the credit score is zero and when both unem-ployment and the credit score are factors in determining default outcomes. Interestingly, this circumstance is the primary one in which the inclusion of an interaction term can lift the individual-level model’s performance. However, in about an equal number of cases,

FROM MOODY’S ECONOMY.COM 1 FROM MOODY’S ECONOMY.COM 1

4.0

4.5

5.0

5.5

6.0

6.5

02 03 04 05 06 07 08

Baseline Stressed Actual

Chart 1: Unemployment Rate Forecasts and Actuals Used in Monte Carlo Simulation 2006-08 Forecast Period, %

Source: Moody’s Analytics

FROM MOODY’S ECONOMY.COM 2 FROM MOODY’S ECONOMY.COM 2

4

5

6

7

8

9

10

11

04 05 06 07 08 09 10

Baseline Stressed Actual

Chart 2: Unemployment Rate Forecasts and Actuals Used in Monte Carlo Simulation 2008-10 Forecast Period, %

Source: Moody’s Analytics

Page 5: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 4

ECONOMIC & CONSUMER CREDIT ANALYTICS

interaction terms significantly detract from forecasting performance and overall add or detract little from forecasting performance.

For forecast accuracy, we find that it is generally better to use one year as opposed to one quarter to compute the calibration for the individual-level models. In one or two cases, however, using a longer calibration period hurts performance. The calibration process is very interesting from the perspec-tive of a forecaster because it introduces an additional source of potential error and variance. Using annual calibration periods in the Monte Carlo simulations reduced these errors, improving the performance of the loan-level models relative to aggregate specifications. Put simply, any system that requires in-sample calibration will be less parsimonious than a system that does not involve such a process. Aggregate models do not require calibration to provide effective forecasts, which constitutes another check mark in their favor.

C. Results When the Unemployment Rate Forecast Is Wrong

We now turn to the case in which the unemployment forecasts turn out to be wrong. This part of the Monte Carlo study comes with important caveats. The pri-mary interest here is to identify any biases or inadequacies in the credit models, not to assess the efficacy of forecasts of the macroeconomy more broadly. In assessing cases in which, for example, unemployment is overpredicted, we must bear in mind that the forecast could have just as easily turned out to be too optimistic. Just because a technique “wins” when one of the input variables is forecast with a particular error, it does not mean it will win on average for the multitude of possibilities. The previous section gives a better sense of the average performance of the techniques assuming unbiased economic forecasts. The other point to note is that for stress–testing, we are asking questions that are conditional on

the event in the underlying macroeconomic scenario. If the stress does not occur, this in no way invalidates the results of the stress test. In the 2008-2010 forecast period, a highly stressful situation occurred and, con-ditional on knowing this a priori, the aggre-gate model performed significantly better than any of the individual-level specifica-tions in predicting default outcomes.

When the unemployment rate forecasts turn out to be wrong, most of the results de-scribed above still hold. For the 2006-2008 forecast period, the one-year calibrated indi-vidual-level models are able to eke out vic-tories of only a handful of percentage points in a couple rarefied circumstances. More in-teresting outcomes occur in the 2008-2010 forecast period, when unemployment spikes to 10%, 4 percentage points higher than forecast. Where the correlation between un-employment and the credit score is high and where only unemployment incidence drives defaults, the individual-level model suffers

TABLE 1

Monte Carlo Simulation ResultsUnemployment Rate Forecast Coincides With Observed Actuals

Correlation UE Coefficient CS CoefficientOne Qtr - No Interaction

One Qtr - Interaction

One Year - No Interaction

One Year - Interaction

2006-2008 Forecast Period

Zero Zero Zero 0.0003 0.0005 0.0003 0.0004

Zero Zero Negative 0.8594 0.8594 0.9551 0.9521

Zero Positive Zero 0.0023 0.0024 0.0022 0.0022

Zero Positive Negative 0.2063 0.7042 0.1337 0.6429

Negative Zero Zero 0.0003 0.0007 0.0002 0.0004

Negative Zero Negative 0.8352 0.8352 0.9385 0.9366

Negative Positive Zero 0.2652 0.0622 0.1574 0.0294

Negative Positive Negative 0.8476 0.8476 0.9298 0.9264

2008-2010 Forecast Period

Zero Zero Zero 0.0002 0.0003 0.0002 0.0002

Zero Zero Negative 0.7634 0.7422 0.7957 0.7170

Zero Positive Zero 0.0001 0.0001 0.0001 0.0001

Zero Positive Negative 0.0041 0.0054 0.0040 0.0083

Negative Zero Zero 0.0001 0.0002 0.0002 0.0003

Negative Zero Negative 0.7306 0.7279 0.7447 0.7292

Negative Positive Zero 0.0045 0.0033 0.0035 0.0025

Negative Positive Negative 0.8864 0.8830 1.0000 0.9732

Page 6: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 5

TABLE 2

Monte Carlo Simulation ResultsUnemployment Rate as Contemporaneously Forecasted By Moody’s Analytics

Correlation UE Coefficient CS Coefficient ForecastOne Qtr - No Interaction

One Qtr - Interaction

One Year - No Interaction

One Year - Interaction

2006-2008 Forecast Period

Zero Zero Zero Baseline 0.0043 0.0057 0.0015 0.0016

Zero Zero Negative Baseline 0.8933 0.8942 0.9447 0.9478

Zero Positive Zero Baseline 0.0025 0.0032 0.0029 0.0030

Zero Positive Negative Baseline 0.7968 0.8811 1.0396 0.9052

Negative Zero Zero Baseline 0.0047 0.0061 0.0010 0.0016

Negative Zero Negative Baseline 0.8569 0.8577 0.9802 0.9790

Negative Positive Zero Baseline 0.8321 0.5924 1.0531 0.4458

Negative Positive Negative Baseline 0.9078 0.9078 0.9966 1.0023

Zero Zero Zero Stressed 0.0004 0.0009 0.0007 0.0010

Zero Zero Negative Stressed 0.8670 0.8670 0.9788 0.9811

Zero Positive Zero Stressed 0.0001 0.0002 0.0002 0.0003

Zero Positive Negative Stressed 0.2697 0.8956 0.4313 0.9896

Negative Zero Zero Stressed 0.0006 0.0009 0.0006 0.0014

Negative Zero Negative Stressed 0.8535 0.8535 0.9661 0.9651

Negative Positive Zero Stressed 0.3400 0.1322 0.4472 0.1943

Negative Positive Negative Stressed 0.8667 0.8614 0.9384 0.9320

2008-2010 Forecast Period

Zero Zero Zero Baseline 0.0001 0.0001 0.0001 0.0001

Zero Zero Negative Baseline 0.7973 0.8045 0.8689 0.8689

Zero Positive Zero Baseline 0.0011 0.0011 0.0010 0.0010

Zero Positive Negative Baseline 4.3411 1.0378 3.9093 0.9879

Negative Zero Zero Baseline 0.0001 0.0001 0.0001 0.0001

Negative Zero Negative Baseline 0.8279 0.8279 0.8948 0.8939

Negative Positive Zero Baseline 3.7652 3.1453 4.4201 1.2926

Negative Positive Negative Baseline 1.0330 1.0366 1.0512 1.0579

Zero Zero Zero Stressed 0.0001 0.0001 0.0001 0.0001

Zero Zero Negative Stressed 0.7948 0.7745 0.8818 0.8661

Zero Positive Zero Stressed 0.0001 0.0001 0.0001 0.0001

Zero Positive Negative Stressed 0.0113 0.0386 0.0099 0.0438

Negative Zero Zero Stressed 0.0001 0.0002 0.0001 0.0002

Negative Zero Negative Stressed 0.7831 0.7529 0.8571 0.8072

Negative Positive Zero Stressed 0.0141 0.0084 0.0139 0.0080

Negative Positive Negative Stressed 1.2161 1.1076 1.1477 1.0491

ECONOMIC & CONSUMER CREDIT ANALYTICS

Page 7: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 6

ECONOMIC & CONSUMER CREDIT ANALYTICS

only about a quarter of the mean squared forecast error of the aggregate model. In this case, the credit score, which all modelers are assumed to know at all points, includ-ing during the forecast period, is a better predictor of future unemployment incidence than the (as it turned out) faulty aggregate unemployment rate forecasts being used. The implication here is that the only case in which individual-level models unambigu-ously defeat aggregate-based specifications is when additional loan-level information can dilute the distortions caused by incorrect input forecasts. When we apply the stressed unemployment rate, which turned out to be a more prescient forecast, this pattern largely disappears, leaving aggregate models dominant in virtually every situation.

In all but the most extreme situations, even when the unemployment forecast is in-accurate, aggregate-based models outperform loan-level specifications. We believe there are two primary reasons for this: First, loan-level models often lack important loan-level information such as individual house prices or the individual incidence of unemployment, which then have to be proxied, imperfectly, by macroeconomic data. Second, loan-level modeling systems, involving predictions of thousands of individuals and an often-unreliable calibration process, are inherently less parsimonious than simple aggregate ap-proaches. A third reason, not considered in the Monte Carlo simulation, is that loan-level models cannot easily cope with correlations among the individuals they purport to model. Aggregate-based models, drawn from the macroeconomics literature, are all about correlations among individuals and the subse-quent propagation of business cycles in eco-nomic data. Getting these large-scale dynam-ic features right is far more important for bond valuation than knowing whether John Smith is more likely to default than Mary Chang.

The academic economic forecasting literature has shown, in detail, that using disaggregate information of any sort and at any level does not help when one’s aim is to forecast aggregates2. Taking the concept

2 See for example, Hendry & Hubrich (2010) & Lutkepohl (2006).

of disaggregation to the extreme – using individual-level information – does not al-ter the validity of this law. Although some might suggest the results of this Monte Carlo experiment are specific and not generally applicable, we are aware of no alternative specific yet realistic scenarios in which loan-level specifications consistently outperform aggregate models built with forecasting in mind. The law of parsimony alone precludes the existence of such examples.

The Moody’s Analytics Approach

We now consider the approach Moody’s Analytics uses to forecast pool performance. Interestingly, in the case of U.S. mortgages, loan-level information is often available. For other forms of ABS and for RMBS from other countries, no such data sets – built in a reliable and consistent manner – seem to be publicly available. Given that pool-level models are more likely to provide accurate forecasts, the existence of loan-level data is irrelevant to the work presented here. The fact that our approach is equally applicable to mortgages anywhere or other types of ABS, however, gives the pool-level approach far wider applicability than loan-level meth-odologies at present.

The data used for this study are drawn primarily from two sources: Moody’s Analyt-ics’ U.S. macroeconomic forecast databases and the Moody’s Analytics’ Performance Data Services product.

Moody’s Analytics maintains one of the largest repositories of macroeconomic, demographic and financial data from a mul-titude of government and private sources. The data set covers the national accounts, banking and finance, demographics, personal income, prices, retail sales, labor markets, energy, financial markets and many other indicators. Moody’s Analytics, each month, produces projections, both baseline and stressed, for the world economies and fi-nancial markets using large-scale structural macroeconometric models. In the broadest terms, the models are specified to reflect the interaction between aggregate demand and

aggregate supply. In the short run, fluctua-tions in economic activity are primarily de-termined by shifts in aggregate demand, in-cluding personal consumption, gross private investment, net exports, and government expenditures. The level of resources and technology available for production is, in the short term, taken as given. Prices and wages then adjust slowly to equate aggregate de-mand and supply and thus move the econo-my toward equilibrium. In the longer term, changes in aggregate supply determine the growth potential of the economy. The rate of expansion of the resource and technology base is the principal determinant of overall economic growth, which feeds, interactively, to other factors in the model. The model and subsequent forecast are overseen by a team of about 60 economists who cover the performance of the economy in real time.

The forecasts undergo continual revision and adjustment to reflect new trends and changing data. Recent advances in macroeco-nomic theory, new econometric techniques, and increased computing power also govern development of the models. The system has been changing to accommodate increased de-mand by clients who want to use the models to generate alternative scenarios and to un-derstand the sensitivity of the macroeconomy to changing economic and financial condi-tions. Moody’s Analytics produces one upside and several downside alternative scenarios each month for each economy it covers. A de-tailed description of some recent alternative scenarios is included in Appendix 2.

In terms of the ABS credit data, mean-while, PDS is a comprehensive and standard-ized dataset that includes raw information on all active and inactive ABS and RMBS deals rated by Moody’s Investors Service in the U.S. and globally. For a more compre-hensive picture of the U.S. RMBS market during the model development stage, and to avoid any selection bias, we have supple-mented the PDS dataset with information from deals that are not rated by MIS; this supplemental database is available only in-ternally. Given that MIS-rated deals cover about 90% of the U.S. RMBS universe, the PDS product alone is close to comprehen-sive, given the aims of this paper.

Page 8: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 7

ECONOMIC & CONSUMER CREDIT ANALYTICS

The dataset is composed of deal informa-tion at origination, augmented by continu-ously updated performance information from servicer and trustee monitoring reports. The data files are cleaned and validated by the MIS monitoring team as part of the ongoing ratings surveillance process. The data are of excellent quality and of a perfect structure for the forecasting models con-sidered in this article. Included in the files are over 100 performance statistics at the deal, pool, and tranche level for over 10,000 ABS and RMBS deals in the U.S., Europe, the Middle East and Africa as well as in the Asia- Pacific region. PDS coverage in each region matches the market share of MIS in rating the underlying deals. The data are not only consistent within each region or country, they are also consistent across and between regions, allowing for international analysis if so desired. Our analysis typically con-centrates on key pool performance metrics such as delinquency rates as a proportion of original balance, longer-term indicators of default, prepayment rates, and severity of losses or loss given default.

For the purposes of this study, because of the lack of data on a few key variables, we restrict our attention to U.S. RMBS deals

originated after December 2001. Many performance measures, however, are avail-able back to 1995. Table 3 provides basic descriptive statistics about the pools in the sample used for modeling purposes. The pools employed are concentrated in sub-prime, alt-A and jumbo categories. Given that originations grew so quickly from 2004 to 2007, unsurprisingly, these cohorts are also very well represented. Data for other vintages and product types are sufficiently available for their determining factors to be accurately estimated.

Once all data files are merged, the final dataset used for the analysis is a multi-dimensional, longitudinal panel dataset. Our estimation sample is composed of 12,148 unique asset pools observed over, in most cases, several years. In total, our es-timation sample for most dependent vari-ables amounts to about 660,000 unique observations with a few categories, most notably loss severities, having access to only about 140,000 separate observations. The data cover a full business cycle, includ-ing two recessions, allowing us to correctly measure the impact of cyclical factors and to weigh these factors against internal pool-specific information.

A. The econometric modelThe panel nature of the data on struc-

tured securities provides a rich tapestry with which to construct high-quality forecasting models. In this paper, we consider a model of the form

or, decomposing into its constituent

components:

where indicates a series of

pools contained in the data set of interest, is a time series defining the co-

hort, or vintage, in which the pool was origi-nated, is a standard time series indicator of the period in which the pool’s performance is observed and is the dependent variable of interest, appropriately transformed, be it a delinquency rate, a de-fault rate, a prepayment rate, a volume or a rate of loss severity. The vector is an unknown random error term and defines the unobserved, time invariant, pool specific het-erogeneity in the data.

The vectors , , , and contain independent variables thought to ex-

TABLE 3

Number of pools by collateral type and closing year

Pools Collateral Type

Deal Closing Year

Total2002 2003 2004 2005 2006 2007 2008 2009 2010

HELOC 14 23 61 50 46 20 2 — — 216

High LTV 14 10 6 7 6 1 — — — 44

Home Equity/Closed End 2nds 8 8 6 6 19 14 — — — 61

Subprime 287 526 749 795 777 386 4 — — 3,524

Alt-A 155 463 1,080 1,335 904 676 12 — 1 4,626

FHA-VA 9 17 17 25 20 2 — — — 90

Jumbo 207 583 534 452 315 252 30 2 1 2,376

Prime Conforming — — 5 — — — — 1 — 6

Scratch & Dent 10 25 63 43 49 41 — — — 231

Subprime 2nds 2 13 27 51 83 41 — — — 217

Option Arms — 5 71 221 284 173 3 — — 757

Total Pools 706 1,673 2,619 2,985 2,503 1,606 51 3 2 12,148

Average Time-Series Available 92 82 70 58 46 36 23 10 2

(1)

Page 9: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 8

plain the behavior of . For instance, would include factors that specifically

relate to the given pool in question but that do not change over time. This set of vari-ables would thus include origination charac-teristics such as the average FICO score or origination LTV. Conversely, one could sim-ply include a full set of fixed-effects dum-mies in this matrix, though doing so would preclude the identification of pool-specific factors and thus rule applying the model to newly originated pools. This action would also breach the principle of parsimony, which is paramount given the application of the models to forecasting. The vector, meanwhile, contains factors that pertain to a particular vintage but are common to all pools that are members of the vintage and that do not change over time; the main fac-tors that would be represented in de-scribe economic conditions that existed when the pools in question were being origi-nated. The inclusion of these dynamic origi-nation conditions variables is crucial for the problem of forecasting the way newly mint-ed loans – or loans that will soon be origi-nated – are likely to perform. Note that the inclusion of a full set of pool-level fixed ef-fects would also preclude estimation of this component, reducing the utility derived from the model.

The vector contains factors that change over time but which are common to all pools. This is therefore the component that explains the way the external macroeco-nomic environment affects the performance of all legacy pools. This vector may include lags of macro variables and variables such as structural break dummies that indicate past policy changes. Macroeconomic variables, therefore, enter the model twice, describing both conditions at origination as well as ongo-ing conditions as the pool matures. Because origination standards tend to decline during booms and rise during recessions, the two components will generally run counter to each other, with opposite signs on variables that are common to the two components.

The nonlinear, baseline lifecycle is con-tained in since the age of the pool is a simple function of time and vintage. This lifecycle component arises because delin-

quencies, defaults and prepayments tend to rise from low levels in the early months of the pool’s existence before peaking and then gradually declining as the pool matures. This inherently nonlinear behavior can be parsimoniously modeled using cubic spline functions with a small number of knots to capture shape changes across the pool’s life-cycle. We employ the Stata routine rc_spline to this end, which fits cubic splines between each knot and linear functions before the first knot and after the last.3

The last component, , includes any factors that vary across pool, vintage and time. This component could include updates of pool-specific information, such as refreshed FICOs, but, unfortunately, these factors are unavailable for the cur-rent exercise. The vector may also include interaction terms between any or all the individual components described above. For instance, it is reasonable to believe that poorer quality pools may be more sensitive to changes in the macroeconomic environment than pools of better stock. An interaction between FICO, say, or LTV and home prices should therefore be considered for inclusion. Conversely, the shape of the lifecycle may be influenced by pool qual-ity or even by the timing of events in the broader macro environment. For example, a recession occurring close to the peak in the lifecycle may have a more pronounced effect on pool performance than a recession that hits a more mature pool. A parsimoni-ous forecasting model may leave many, or all, these interaction terms on the cutting-room floor, but a serious modeler should at least consider their inclusion in the pre-ferred forecasting specification.

The final members of the compo-nent are forecasts and actuals of other pool-specific performance metrics that enter into the determination of the dependent variable of interest. For example, if we are modeling a 60-day delinquency rate, it makes sense for the 30-day rate, observed with a one-month lag, to be a key leading indicator. This is akin to the inclusion of a “roll rate” or migration

3 We also used STATA optimization routines to define the number and location of knots (Royston & Sauerbrei, 2007).

term in the estimated model. Models of pathologies that occur later in the default process, like REO or foreclosure rates, may conceivably include lagged 30-day, 60-day and 90-day delinquencies or a myriad of oth-er indicators from earlier in the process. Pre-payment rates may weigh on default rates, as advocates of competing risk models often suggest they should. In considering the in-clusion of these terms, one must weigh the threat of promulgating forecast errors across different models. Roll rate terms should be used sparingly, though they should certainly be considered in the model specification process. We refer to these roll rate terms as “pipeline connections”.

Taken together, it is clear that the multi-dimensional, vintage panel data framework provides us with a huge array of factors that may influence cash flow.

B. EstimationIn this section, we turn to estimation is-

sues, followed by some practical matters of model specification for the problem at hand. This section will focus on functional form selection as well as independent variable and lag length determination. In modeling a large panel dataset such as this one, the standard approach would be to set out using a fixed-effects estimator and merely produce the de-sired forecasts.4 One could conceivably test the restrictions implied by a random-effects estimator, but with more than half a million observations, this model will almost certain-ly be rejected. As mentioned earlier, we also reject the fixed-effects approach because its use likely affects forecasting performance5 while reducing the utility of the model in a number of important ways.

Several alternatives to the fixed-effects ap-proach are available via the econometrics lit-erature. The most simple option would be to assume that all pool-level heterogeneity can be explained by available economic data and pool-specific features and leave out of the model altogether; this is known in the litera-ture as the pooled OLS estimator. The ability

4 For general descriptions of panel data models, see Baltagi (2001), Hsiao (2002), or Wooldridge (2002)

5 Though assertion is consistent with the principles of parsi-mony and has been verified by, for example, Baltagi (2008).

ECONOMIC & CONSUMER CREDIT ANALYTICS

Page 10: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 9

to identify unobserved individual heterogene-ity is, however, one of the primary advantages of panel data, and this simple approach would be unable to reap these rewards. The second alternative involves the standard random-effects estimator. Using this approach, we could identify the effects of observed time invariant factors, but this estimator is built on a few strong simplifying assumptions that are unnecessary given the plethora of avail-able data at our disposal. Most notably, the random-effects estimator assumes that un-observed individual effects are uncorrelated with other included factors.6 The third option – the Hausman-Taylor (1981) estimator – was built to overcome the problems inherent in both the random-effects and fixed-effects estimators. For these reasons, we prefer the Hausman-Taylor estimator.

The HT estimator allows for unobserved individual heterogeneity and the estimation of time invariant factors while avoiding the assumption of no correlation between the various terms included in the model. If we rewrite (1) as

where and represent variables that are uncorrelated with the error term and that are time varying and time invariant, re-spectively. The variables and rep-resent equivalent sets of factors, albeit those that are correlated with the unobserved innovations. In this model, the unobserved individual heterogeneity, , is assumed to be random, independent and identically dis-tributed with a zero mean and finite variance. The usual assumptions are made regarding .

Note that the impact of the time invariant factors, , cannot be directly separated from the impact of the unobserved, time in-variant random effects, . HT estimation then proceeds using an instrumental vari-ables approach. The variables in , which are uncorrelated with and by as-

6 Using formal statistical validation like the Hausman test (Hausman, 1978), we rejected the null hypothesis of no cor-relation. As a result, we do not present results using random-effect model here.

sumption, represent the instruments for the endogenous time invariant factors contained in . In this analysis, because we seek to retain the ability of the models to project the performance of newly originated mortgage pools, the most interesting factors for con-sideration in are those describing pool-level characteristics such as average LTV or average FICO. As instruments for these fac-tors, we employ various macroeconomic se-ries, observed at origination, that we assume to be uncorrelated with the errors but that can help explain shifts in underwriting stan-dards indicated by variables such as average FICO scores. As mentioned earlier, lending standards tend to fall during a boom, mean-ing that average FICOs should, for example, rise as the economy sinks into recession. We concede that there may be some residual correlation between macro conditions at origination and but weigh this against the utility derived from estimates of the pool-specific factors.

C. Practical considerationsOur ultimate aim in the model building

process is to accurately forecast the constant default rate (CDR), constant prepayment rate (CPR), and rate of loss severity – the three inputs to the waterfall engine. To support the forecasts for these factors, we also construct models for 30-day, 60-day and 90+-day de-linquencies, bankruptcies, foreclosures, the rate at which defaulted properties are owned by the servicer (known as “real estate owned” or REO), and principal repayment rates. There may be considerable interest in these other factors in their own right, but we use these factors chiefly to serve the broader issue of accurate cash flow projection.

In terms of functional form, since all de-pendent variables of interest are bound by zero and unity, we apply a logistic transfor-mation of the form

In many cases, we could have effectively applied a simple log transform to the data, which asymptotes only at zero but not one. In some cases, however, particularly in

short-term delinquency rates on subprime mortgages, the data start to reach into the higher part of the available number space, where the upper bound starts to come into play. Further, for loss severities in home eq-uity lines and some mortgages, we find most of the action close to the uppermost bound. In all cases, the logistic transform is found to perform well in terms of forecast accuracy, allowing a simple, theoretically tenable and consistent transformation to be applied.

A key aspect of model development is variable selection – identifying which credit and economic variables best explain the dynamic behavior of the dependent variable in question. For panel data structures such as this, one must also decide on the number and nature of the lags for each variable in the equation. Aligned with principles of modern econometrics, we prefer to choose the vari-ables based on a combination of economic theory or intuition, together with a consid-eration of the statistical properties of the estimated model. We feel that models built using pure data-mining techniques or prin-ciples such as machine learning, though they may fit the existing data well, are more likely to fail in a changing external environment because they lack theoretical underpinnings. A strong signal can be derived from a consid-eration not only of how mortgage markets are seen to work but of how they should work given our understanding of markets and human folly. The best prediction models em-ploy a combination of statistical rigor with a healthy dose of economic principle. Models built this way enjoy the additional benefit of ease of interpretation.

The selection of the explanatory macro-economic variables was also based on our ability to forecast them easily and sufficiently accurately. In econometric forecasting, the future realizations of the dependent variable rely on accurate forecasts for the independent variables. Forecast errors can easily be multi-plied if input variable forecasts are not closely monitored or if the data backing the forecasts are noisy and thus inherently difficult to project. Identifying macro variables that are useful in this context is straightforward: They are the factors that often have the power to change the direction of financial markets and

ECONOMIC & CONSUMER CREDIT ANALYTICS

(2)

Page 11: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 10

that are thus routinely debated in the public square. Variables such as unemployment rates, home price indices, key interest rates, and retail sales numbers are generally bet-ter candidates than industry-level mortgage foreclosure rates, for instance, which are of interest only during certain brief periods. We find that a combination of such core variables generally act as a reasonable proxy for these more peripheral factors. Since core variables are generally easier to predict, we find them more useful in building optimal forecasts of ABS pool performance.

Model ResultsA summary of the variables included in

each regression for the vectors we modeled is presented in Table 4. The first group of variables, the pool origination factors ( in (2)), include average LTV and FICO, a set of dummies for vintage origination year, dum-mies defining loan or collateral type (i.e. whether the pool represents subprime or jumbo mortgages, for instance) and a set of dummies indicating whether the originator of the pool in question was in the top 10 in terms of total number of pools originated during the in-sample period. The vintage year, collateral type and originator dummies are included in most regressions in order to control for group heterogeneity in the data. Weighted average LTV and FICO at origina-tion are key pieces of information not only because they determine pool quality and hence performance, but also because we seek to retain the ability of the models to project the performance of newly originated mortgage pools using information that is readily available at deal origination.

The second group of variables includes the economic condition factors at deal origi-nation, which is our set of instrumental vari-ables, . In this category, we found that gen-eral measures of economic activity, like GDP and the unemployment rate, are correlated with changes in pool performance; we also find good support for factors specific to the mortgage market such as home prices and interest rates. Broadly speaking, house price changes and an average benchmark interest rate at origination tend to have a counter-

cyclical impact on measures of performance observed early in the default process. Labor market conditions at deal origination, how-ever, become more important when model-ing later stages in the default process, includ-ing foreclosure, REO, and losses.

The third set of variables is the macro-economic series or those factors related to the pools’ exposure to the business cycle. We found that the types of macroeconomic factors that affect delinquency are standard fare – labor and housing market variables, income, and overall economic activity – with close attention paid to lag structures and transformations applied to the economic data being used. House price changes and refinancings become much more important drivers at later stages in the foreclosure pro-cess. We use both current home price chang-es and price changes since origination as key housing market factors. This second variable allows for a differential “negative equity” ef-fect to be explicitly modeled. The models of foreclosure, REO, charge-offs, and severity are tied largely to home prices and various other factors affecting housing and legal cost structures. We also found prepayment rates to be more sensitive to interest rates, refi-nancing activity, and existing-home sales.

As the structure of the model is recursive in nature, the next set of covariates consti-tute the so-called “pipeline connections”. We model delinquencies largely independently of one another but model subsequent events – prepayments, foreclosures, charge-offs, and severity – conditionally on the delinquency performance the pools attain. We also take advantage of our panel data estimation techniques to measure the unobserved pool heterogeneity embedded within several early performance measures and then use these estimates as an exogenous quality metric in the subsequent net charge-off, prepayment, and principal payment models.

Finally, the models include a cubic spline baseline lifecycle with four knots7 and inter-action terms between the spline variables, current economic conditions, and collateral type dummies. By introducing these inter-actions, we measure the differential impact

7 Three in the case of the LGD model.

of economic drivers on different types of assets. For example, we allow the sensitiv-ity to unemployment rate changes to vary between pools classified as subprime and those defined as jumbo deals. Similarly, by interacting the spline with collateral type, we allow the shape of the lifecycle to depend upon the underlying quality of the pools under consideration.

Table 4 summarizes the specifications used to model each of the 11 vectors studied in this article. All the variables (or groups of variables) are significant at the 5% level, and all variables have appropriate signs con-sistent with economic theory. The models have been tested rigorously to ensure that they are optimal, among the set considered, for forecasting purposes. In Table 5 we pres-ent the results for one regression, that for the foreclosure rate, as an example of the estimation results achieved for all models. Other specifications are, naturally, available from the authors on request.

Given that our general model specifica-tion included factors from all the subcom-ponents described above, we can reasonably conclude that factors specific and internal to the pool, together with external mac-roeconomic factors, are given appropriate loadings in the calculation of performance forecasts. Once we feed in forecasts of macroeconomic variables derived from our structural macroeconomic model, generating pool-level forecasts is straightforward. Ex-tending this principle to stress tests – which really constitute little more than pessimistic economic forecasts – is equally simple. This process delivers a set of vectors under each alternative macro scenario, including a base-line projection. The models can be applied to custom scenarios provided all necessary macroeconomic variables are covered by the scenario. The particular shape of these fore-casts and scenarios depends on the estimat-ed elasticity of the risk vector to the included macroeconomic series. As mentioned earlier, the elasticities have been found, in many cases, to be heterogeneous across different collateral groups. Moreover, we find that the slopes vary depending on the underlying quality of the pool in question and the time at which the pool was originated.

ECONOMIC & CONSUMER CREDIT ANALYTICS

Page 12: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 11

TABLE 4

Summary of Model’s inputs

Group Variable

Vector

30 day delinquency

60 day delinquency

90+ day delinquency CDR Bankruptcy Foreclosure REO LGD

Net Chargeoff CPR Principal

Orig

inat

ion

C

ondi

tion

s

LTV X X X X X X X

FICO X X X X X X X X X

Top Originators X X X X X X X X X X X

Vintage Year X X X X X X X X X

Loan Type X X X X X X X X X X X

Econ

omic

C

ondi

tion

s at

O

rigin

atio

n

Fed Funds X X X X (relative to WAC)

Home Prices X X X X

GDP X

Unemployment Rate X X X X

Cur

rent

Eco

nom

ic C

ondi

tion

s

Disposable Income t, t-3 t, t-3 t-3

Home Prices (HP) t, t-6 t, t-6 t-6 t-3 t, t-6

Change in HP since 0 t t t

Negative Equity Dummy t t t

Unemployment Rate t-3 t-3 t-3 t-1 t-6 t

Avg. Hourly Earnings t-6 t-6 t-6 t-6

GDP t t

Personal Bankruptcies t-6

REFI Volume t-12 t t

Fed Funds t

Debt Service Burden t-6 t

Existing Home Sales t-12

Mort. REFI Originations t-12 t

Pipe

line

Con

nect

ions

30 day delinquency t

90+ day delinquency t-3 t-3 t-3

Bankruptcy t-3

Foreclosure t-1

REO t

Unobserved effect (CDR) X X X

Oth

er V

aria

bles Lifecycle * Loan Type X X X X X X

Lifecycle * Economic X X X

Loan Type*Economic X X X X

Dec 2005 Bankruptcy Law X

Life

cycl

e

Number of knots 4 4 4 4 4 4 4 3 4 4 4

Notes: All models include seasonality factors (month) and dummies for zeros

ECONOMIC & CONSUMER CREDIT ANALYTICS

Page 13: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 12

ECONOMIC & CONSUMER CREDIT ANALYTICS

To illustrate these features, we present forecasts and scenarios for a small number of pools drawn from the subprime, alt-A, jumbo, and option ARM collateral types under two alternative macroeconomic scenarios – the current baseline forecast (a recovery) and an alternative, depression-like event. These projections are depicted in Charts 3 and 4. Under baseline assumptions, we find that foreclosure rates will stay flat for another quarter as the real estate market stabilizes and then steadily decrease as bad accounts move out of the foreclosure state and are flushed from the system; the vector then converges to the steady-state baseline lifecycle. A depression scenario, however,

generates a sharp double dip, with figures across collateral types reaching historically high levels in 2011, similar to the peaks ob-served in mid-2009.

A. Out of sample validationOne of the most important findings in

the forecasting literature is that the model that best fits the data is not necessarily the one that will provide the most accurate out-of-sample forecast (Fildes and Makridakis (1995)). Typically, to assess accuracy, the data will be split into two data sets: The first set, a development sample, is used to specify the model and estimate its coefficients; the second set is used to evaluate forecast accu-

racy and is known as the hold-out sample. To validate our preferred models, we hold out the last six available data points at the end of the time series for each pool. The models were fitted to the remaining historical data. The forecasts generated using the models were then compared with actual values observed during the hold-out sample, and aggregate root mean squared forecast errors were computed.

Underlying our forecast accuracy evalua-tion was the need to test our preferred mod-el against reasonable alternatives (Baltagi (2008)). For this purpose, the out-of-sample performance of the HT estimator was com-pared against that of three benchmark mod-

FROM MOODY’S ECONOMY.COM 3 FROM MOODY’S ECONOMY.COM 3

0

5

10

15

20

25

30

35

07 08 09 10 11 12 13 14 15

Baseline

Alt. Scenario

Chart 3: Projections for Alt-A and Subprime Pools Foreclosure rates, %

Source: Moody’s Analytics

Subprime

Alt-A

FROM MOODY’S ECONOMY.COM 4 FROM MOODY’S ECONOMY.COM 4

0

5

10

15

20

25

07 08 09 10 11 12 13 14 15

Baseline

Alt. Scenario

Chart 4: Projections for Jumbo and Option Arm pools Foreclosure rates, %

Source: Moody’s Analytics

Option ARM

Jumbo

TABLE 5

Partial Regression Results for Foreclosure RateCoefficient Standard Error t-statistic

LTV 0.053 0.004 11.809

FICO (non-linear spline 1) -0.006 0.001 -8.402

FICO (non-linear spline 2) -0.001 0.001 -1.889

Unemployment Rate, % at origination -0.443 0.014 -30.842

House Prices, % change year ago lagged 12 months -0.028 0.000 -119.093

GDP, % change year ago -0.015 0.001 -27.163

Dummy indicator for Negative Equity 0.108 0.004 29.156

90+ day delinquency lagged 3 months 0.110 0.001 173.511

Bankruptcy lagged 3 months 0.081 0.001 151.339

Observations 576651

Number of unique pools 11727

Note: model includes lifecycle, seasonality, and dummies for zeros, originators, negative equity and loan type and a constant term

Page 14: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 13

ECONOMIC & CONSUMER CREDIT ANALYTICS

els. Our first benchmark was a naïve forecast that assumed no change from the most recently observed value in the development sample. The other two benchmarks were specified similarly to our preferred specifica-tion but were estimated using pooled OLS and the fixed-effects estimator, respectively. These approaches are described in previous sections; both should provide far stiffer com-petition to the preferred specification than the corresponding naïve forecast.

Errors for each approach were aggregated and results presented in Table 6. As in the Monte Carlo results, we present the perfor-mance of the alternative models relative to the preferred HT specification. Numbers greater than unity therefore indicate cases in which the alternative approach is able to beat the pre-ferred method in a head-to-head competition.

The results of the validation exercise are clear-cut. The preferred HT estimated panel data model suffers less squared forecast error than each of the three considered alternative specifications for all 11 estimated vectors. The one exception is for charge-offs, in which the fixed-effects estimator is able to eke out a small victory by just a couple of percentage points. We feel that any test applied enough times will yield contrary results and place this finding in that category. Folding in the fact that the fixed-effects estimator reduces model utility, we can easily conclude that a 2% improvement is not enough to warrant a change in technique.

In terms of the three primary inputs to the Structured Finance Workstation – CDR, CPR and severity – the HT estimated model easily dominates its competitors and can thus be recommended. Vectors derived from this approach are likely to be optimal in terms of deriving accurate bond valuations.

Final RemarksWe have presented results to demon-

strate that pool-level performance aggre-gates are well forecast using aggregate panel data specifications that accurately capture the impact of macroeconomic dynamics on the behavior of loans. The models described are simple, parsimonious structures in which every coefficient estimate can be readily understood using simple economic intuition. The subsequent models forecast the aggre-gate performance characteristics accurately and easily lend themselves to the construc-tion of relevant stress-testing scenarios. Traditional approaches using individual-level scoring models are not designed for forecast-ing and are thus unlikely to succeed when that is the intended application.

Structured finance is a complex business, make no bones about it. When a failure of the models used in the industry was exposed

during the recent recession, many naturally sought even more complex solutions and models in a bid to fill the cracks that were so clearly exposed. Though it will seem coun-terintuitive to many, the correct response to the failure of complex models in 2007-2009 was a flight to simplicity and functionality. The models that form the backbone of the Moody’s Analytics system are simple yet comprehensive and, importantly, and are designed specifically to tackle the problem of forecasting and stress-testing the collateral underlying ABS deals.

When one considers that accurate forecasting completely defines the objec-tive function of the modeling process, the use of models such as those presented should be more widespread. Indeed, the availability of accurate forecasts may well prove a necessary condition for a return to a normal securitization market in the post-subprime world.

TABLE 6

Out of Sample Forecasting ResultsRatio between Hausman-Taylor and benchmark models RMSE

Vector Naïve Forecast Pooled OLS Fixed Effects

30 day delinquency 0.935 0.742 0.941

60 day delinquency 0.871 0.853 0.904

90+ day delinquency 0.688 0.479 0.531

CDR 0.830 0.802 0.875

Bankruptcy 0.948 0.720 0.711

Foreclosure 0.572 0.636 0.777

REO 0.973 0.645 0.758

CPR 0.884 0.957 0.993

Principal 0.668 0.586 0.667

Severity 0.911 0.928 0.850

Chargeoff 0.762 0.905 1.020

Page 15: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 14

ECONOMIC & CONSUMER CREDIT ANALYTICS

References 1. Armstrong, J. S. (1985). Long-Range Forecasting: From Crystal Ball to Computer. New York, John Wiley & Sons2. Allen, P. G. and Fildes, R. (2001). Econometric Forecasting. Principles of Forecasting. J. S. Armstrong. Norwell, MA, Kluwer Academic

Publishers.3. Baltagi Badi H. (2001). Econometric Analysis of Panel Data. Wiley and Sons, Chichester, UK.4. Baltagi, Badi H. (2008). Forecasting with panel data, Journal of Forecasting, 27(2), pp 153-173.5. Hendry, David F. and Hubrich, Kirstin, (2009). Combining Disaggregate Forecasts or Combining Disaggregate Information to Forecast an

Aggregate. ECB Working Paper No. 1155.6. Fabbozzi, Frank and Kothari, Vinod, (2008). Introduction to Securitization, Wiley.7. Fildes, R and Makridakis, S (1995), The impact of empirical accuracy studies on time series analysis and forecasting, International Statis-

tical Review, 63, pp. 289-308.8. Green, Richard and Wachter, Susan M. (2005), The American Mortgage in Historical and International Context. Journal of Economic Per-

spectives, 19(4), pp. 93-114. 9. Hausman, J.A. (1978). Specification Tests in Econometrics, Econometrica, 46 (6), pp. 1251–1271.10. Hausman Jerry A. and Taylor William E. (1981) Panel Data and Unobservable Individual Effects. Econometrica 49, pp.1377–98.11. Hsiao (2002). Analysis of Panel Data. Cambridge University Press12. Kendall, Leon, and Fishman, Michael (2000). A Primer on Securitization. The MIT Press13. Lutkepohl, H. (2006). Forecasting with VARMA processes, in G. Elliott, C.W.J. Granger & A. Timmermann (eds), Handbook of Economic

Forecasting, vol. 1, pp 287-325, Elsevier.14. Royston, Patrick, and Sauerbrei, Willi (2007). Multivariable Modeling with Cubic Regression Splines: A Principled Approach. The Stata

Journal 7(1), pp. 45–7015. Wooldridge Jeffrey M. (2002) Econometric Analysis of Cross Section and Panel Data. MIT Press, Cambridge, MA.

Page 16: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 15

ECONOMIC & CONSUMER CREDIT ANALYTICS

Appendix A: Definition of Variables Field Description

Pool Performance

30 day delinquency Amount of receivables that are 30-59 days past due divided by the original collateral balance (%).

60 day delinquency Amount of receivables that are 60-89 days past due divided by the original collateral balance (%).

90+ day delinquency Amount of receivables that are 90 or more days past due divided by the original collateral balance (%).

CDR Constant default rate based on amount of receivables in default in the current month, annualized (%)

Bankruptcy Amount of receivables in which the obligor has declared bankruptcy divided by the current collateral balance (%)

Foreclosure Amount of receivables in foreclosure divided by the current collateral balance (%).

REO Amount of receivables that are real estate owned divided by the current collateral balance (%).

Loss-Given Default (LGD) Monthly net losses during the related monthly period divided by gross losses during the related monthly period (%).

Net Chargeoff Net losses during the related monthly period divided by the prior month’s ending collateral balance (%).

CPR Constant prepayment rate based on unscheduled principal paid by obligors from the current month, annualized (%).

Principal Total principal collected during reporting period divided by the current collateral balance (%).

Pool Origination

LTV Weighted Average Loan To Value at deal origination (%).

FICO Weighted FICO score at deal origination (%).

Top Originators Indicator variables for Top Mortgage originators during the sample time period including Countrywide Bank, Wells Fargo, IndyMac, BofA, Citibank, Greenpoint Bank (CapOne), Option One Mortgage, and Residential Funding (GMAC)

Vintage Year Vintage origination year (2001 to 2010)

Loan Type Loan collateral type: Heloc, High LTV, Home Equity/Closed End 2nds, Subprime, Alt-A, FHA-VA, Jumbo, Prime Con-forming, Scratch and Dent, Subprime 2nds, Option Arm

Macroeconomic series

Disposable Income Income: Per capital disposable income, (Constant$, SAAR, annual growth rate)

Home Prices Median Sales Price Existing Single-Family Homes, (Ths. $, SA, annual growth rate)

Change in HP since 0 Difference in Median Sales Price since Deal Origination (%)

Negative Equity Dummy 1 if change in HP since 0 is negative; 0 otherwise

Unemployment Rate Household survey: Unemployment rate, (%, SA)

Avg. Hourly Earnings Avg. Hrly. Earnings: Total Private, ($ Per Hrs., SA, annual growth rate)

GDP NIPA: Gross domestic product, (Bil. 2000 $, SAAR, annual growth rate)

Personal Bankruptcies Bankruptcies: Personal, Total, (# 3-Month Ending, SAAR, annual growth rate)

REFI Volume MBA: Percent REFI Volume, (%, annual growth rate)

Fed Funds Interest Rates: Federal Funds Rate, (%,P.A.)

Debt Service Burden Debt Service Burden: Total, (% of Disposable Personal Income)

Existing Home Sales Existing Home Sales: Single-Family, (Mil., SAAR, annual growth rate)

Mort. REFI Originations Mortgage Originations: Refinances, (Mil. $, SAAR, annual growth rate)

Page 17: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 16

ECONOMIC & CONSUMER CREDIT ANALYTICS

Appendix B: Summary of U.S. Economics Scenarios (July 2010) Scenario Real GDP Median Home Price Fed Funds Target Unemployment

S1 Stronger Recovery in 2010

Real growth of 3.9% in 2010, 4.6% in 2011

Expected to rise 2.9% in 2010 and 2.0% in 2011

The funds rate is expected to end 2010 at 1.6% and 2011 at 2.5%

Peaks at 10.1% in Q4 2009 and ends 2010 at 8.3%

BL Baseline, Current Real growth of 2.9% in 2010, 3.6% in 2011

Peak-to-trough decline of 26%, turnaround in mid 2011

The funds rate is expected to end 2010 at 0.2% and 2011 at 1.8%

Peaks at 10.0% in Q1 2011

S2 Mild Second Recession

Real growth of 2.0% in 2010, 1.3% in 2011

Peak-to-trough decline of 36%, turnaround after mid 2011

The funds rate is expected to end 2010 at 0.1% and 2011 at 0.5%

Peaks at 11.8% in Q2 2011

S3 Deeper Second Recession

Real growth of 1.6% in 2010, -0.7% in 2011

Peak-to-trough decline of 40%, turnaround at beginning of 2012

The funds rate is expected to remain below 1% until Q3 2012

Peaks at 14.0% in Q4 2011

S4 Complete Collapse, Depression

Real growth of 1.3% in 2010, -1.9% in 2011

Peak-to-trough decline of 45%, turnaround in late 2012

The funds rate is expected to end 2010 at 0.1% and 2011 at 0.2%

Peaks at 15.1% in mid 2012

S5Aborted Recovery,

Below-Trend Long-Term Growth

Real growth of 2.1% in 2010, 1.8% in 2011

Peak-to-trough decline of 29%, turnaround in mid 2012

The funds rate is expected to end 2010 at 0.1% and 2011 at 0.4%

Peaks at 11.4% in mid 2011

S6 Fiscal Crisis, Dollar Crashes, Inflation

Real growth of 2.2% in 2010, 2.6% in 2011

Peak-to-trough decline of 37%, turnaround in mid 2013

The funds rate is expected to end 2010 at 0.1% and 2011 at 1.9%

Peaks at 13.0% in early 2013, ends 2010 at 10.2%

CREDITFORECAST.COM Exclusive forecasts and analysis of household finances based on credit bureau data from Equifax.www.creditforecast.com

ADVISORY SERVICESCutting-edge economic research, modeling capabilities & data analytics at your disposal.www.economy.com/consulting

MOODY’S CREDITCYCLETM

Integrating regional economics with industry-leading consumer credit forecasting & stress testing.www.economy.com/mcc

U.S./CANADA +1.866.275.3266

EMEA+44.(0) 20.7772.1646

ASIA/PACIFIC +61.2.9270 8111

OTHER LOCATIONS+1.610.235.5299

Email: [email protected]

More on Consumer Credit Analytics from Moody’s Analytics

Page 18: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 17

© 2010, Moody’s Analytics, Inc. and/or its licensors and affiliates (together, “Moody’s”). All rights reserved. ALL INFORMATION CONTAINED HEREIN IS PROTECTED BY COPYRIGHT LAW AND NONE OF SUCH INFORMATION MAY BE COPIED OR OTHERWISE REPRODUCED, REPACKAGED, FURTHER TRANSMITTED, TRANSFERRED, DISSEMINATED, REDISTRIBUTED OR RESOLD, OR STORED FOR SUBSEQUENT USE FOR ANY PURPOSE, IN WHOLE OR IN PART, IN ANY FORM OR MANNER OR BY ANY MEANS WHATSOEVER, BY ANY PERSON WITHOUT MOODY’S PRIOR WRITTEN CONSENT. All information contained herein is obtained by Moody’s from sources believed by it to be accurate and reliable. Because of the possibility of human and mechanical error as well as other factors, however, all information contained herein is provided “AS IS” without warranty of any kind. Under no circumstances shall Moody’s have any liability to any person or entity for (a) any loss or damage in whole or in part caused by, resulting from, or relating to, any error (negligent or otherwise) or other circumstance or contingency within or outside the control of Moody’s or any of its directors, officers, employees or agents in connection with the procurement, collection, compilation, analysis, interpretation, communication, publication or delivery of any such information, or (b) any direct, indirect, special, consequential, compensatory or incidental damages whatsoever (including without limitation, lost profits), even if Moody’s is advised in advance of the possibility of such damages, resulting from the use of or inability to use, any such information. The financial reporting, analysis, projections, observations, and other information contained herein are, and must be construed solely as, statements of opinion and not statements of fact or recommendations to purchase, sell, or hold any securities. NO WARRANTY, EXPRESS OR IMPLIED, AS TO THE ACCURACY, TIMELINESS, COMPLETENESS, MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OF ANY SUCH OPINION OR INFORMATION IS GIVEN OR MADE BY MOODY’S IN ANY FORM OR MANNER WHATSOEVER. Each opinion must be weighed solely as one factor in any investment decision made by or on behalf of any user of the information contained herein, and each such user must accordingly make its own study and evaluation prior to investing.

About Moody’s Economy.comMoody’s Economy.com, a division of Moody's Analytics Inc., is a leading independent provider of economic research, analysis and data. As a well-recognized source of pro-prietary information on national and regional economies, industries, financial markets, and credit risk, we support strategic planning, product and sales forecasting, risk and sensitivity management, and investment research. Our clients include multinational corporations, governments at all levels, central banks, retailers, mutual funds, financial institutions, utilities, residential and commercial real estate firms, insurance companies, and professional investors.

With one of the largest assembled financial, economic and demographic databases, Moody's Economy.com helps companies assess what trends in consumer credit and behavior, mortgage markets, population, income, and property prices will mean for their business. Our web and print periodicals and special publications cover every U.S. state and metropolitan area; countries throughout Europe, Asia and the Americas; and the world's major cities, plus the U.S. housing market and 58 other industries. We also provide up-to-the-minute reporting and analysis on the world's major economies on our real-time Dismal Scientist web site from our offices in the U.S., the United Kingdom, and Australia. Our staff of more than 50 economists, a third of whom hold PhD's, offers wide expertise in regional economics, public finance, credit risk and sensitivity analysis, pricing, and macro and financial forecasting.

Moody's Economy.com became part of the Moody’s Corporation in 2005. Moody's Economy.com is headquartered in West Chester PA, a suburb of Philadelphia, with offices in London and Sydney. More information is available at www.economy.com.

Page 19: Methodology for Forecasting and Stress-Testing ABS and RMBS Deals · 2019-06-01 · MOODYS ANALYTICS / METHODOLOGY FOR FORECASTING AND STRESS-TESTING ABS AND RMBS DEALS 2 In general,

Copyright © 2010, Moody’s Analytics, Inc. All Rights Reserved.

Email us: [email protected] Or visit us: www.economy.com

U.S./CANADA +1 866.275.3266

EMEA+44 (0) 20.7772.1646

ASIA/PACIFIC +61 2 9270 8111

OTHER LOCATIONS+1 610.235.5299

CONTACT US