1

PredictingEconomicRecessionsUsingMachineLearningAlgorithms

RickardNyman1andPaulOrmerod2

December2016

Acknowledgement:weacknowledgetheassistanceofOliverRiceincheckingourresultsinPython

1PerianderLtdandUniversityCollegeLondon(UCL);r.nyman@periander.co.uk2 University College London (UCL), Volterra Partners LLP, London and Periander Ltd.;pormerod@ucl.ac.uk.Correspondingauthor

2

AbstractEven at the beginning of 2008, the economic recession of 2008/09was not beingpredictedbytheeconomicforecastingcommunity.Thefailuretopredictrecessionsisapersistenttheme ineconomicforecasting. TheSurveyofProfessionalForecasters(SPF)providesdataonpredictionsmadeforthegrowthoftotaloutput,GDP,intheUnitedStatesforone,two,threeandfourquartersahead,goingbacktotheendofthe1960s.Overathreequartersaheadhorizon,themeanpredictionmadeforGDPgrowthhasneverbeennegativeoverthisperiod.ThecorrelationbetweenthemeanSPFthreequartersaheadforecastandthedataisverylow,andoverthemostrecent25yearsisnotsignificantlydifferentfromzero.Here, we show that the machine learning technique of random forests has thepotential to give early warning of recessions. We use a small set of explanatoryvariablesfromfinancialmarketswhichwouldhavebeenavailabletoaforecasteratthe timeofmaking the forecast. Wetrain thealgorithmover the1970Q2-1990Q1period, andmakepredictionsone, threeand six quartersahead. We then re-trainover1970Q2-1990Q2andmakeafurthersetofpredictions,andsoon.Wedidnotattemptanyoptimisationofpredictions,usingonlythedefault inputparameterstothealgorithmwedownloadedinthepackageR.We compare thepredictionsmade from1990 to thepresentwith theactual data.One quarter ahead, the algorithm is not able to improve on the SPF predictions.Threeandsixquartersahead,thecorrelationsbetweenactualandpredictedarelow,but they are very significantly different from zero. Although the timing is slightlywrong, a serious downturn in the first half of 2009 could have been predicted sixquartersaheadinlate2007.Thealgorithmneverpredictsarecessionwhenonedidnotoccur.WeobtainevenstrongerresultswithrandomforestmachinelearningtechniquesinthecaseoftheUnitedKingdom.JELclassification:C53;E37;E44Keywords: random forests; macroeconomic forecasting; business cycle; monetaryfactors

3

1. Introduction

Overthepastfiftyyearsorso,atrackrecordofmacroeconomicforecastsandtheiraccuracyhas been built up. Economists disagree about how the economy operates, and thesedisagreementsarereflectedin,amongstotherthings,thespecificationoftherelationshipsinmacro-economicmodels. But, over time, no single approach has a better forecastingrecordthananyother.Thereare somegeneralpointswhichemerge fromthe literatureon forecastingaccuracy,especiallywithrespecttotheoneyearaheadpredictionsforgrowthrateofGDP. Twentyyearsago, forexample,amajorsurveyarticleconcludedthat there isnorealevidencetosuggest that accuracy was improving over time (Fildes and Stekler, 2000). This haspersisted.Indeed,thereissomesuggestionthataccuracyhasdeteroriatedinrecentyears(see for example the review of the UK Monetary Policy Committee’s forecasts Stockton(2012)).Aparticularproblemistheverypoorrecordofpredictingrecessions.Thefailuretoforecastthefinancialcrisisrecessionofthelate2000siswellknown.Butthesamepointwasmadein1993byZarnowitzandBraun.IntheUnitedStates,forexample,theypointoutthattherecessions following the1974and1981peaks in the levelofoutputwerenot recognisedevenastheytookplace.Thepurposeofthispaperistoexaminewhethermachinelearningalgorithmscanimproveforecasting accuracy. Varian (2014) gives a wide range of examples of the potentialapplicationofsuchalgorithms.Ourspecificfocusisonshort-termforecastsofrealGDPgrowthintheUnitedStates,andinparticular inwhethertherecessionofthe late2000scouldhavebeenpredicted. WealsoincludetheUnitedKingdominouranalysis.RealGDPbegantofallintheUSonaquarter-byquarterbasisinasustainedway(therewasa temporary fall in 2008Q1) in 2008Q3, and growthwasnot resumeduntil 2009Q3. ThepreviouspeaklevelofrealGDPin2008Q2wasnotreacheduntil2011Q2.Thecumulativefall in output in the 2008/09 period was 4.1 per cent at an annualised rate, and thecumulativefallfromthe2008Q2peakuntil2011Q2was22.9percent.Therecessionwasbysomemarginthebiggestsincequarterlynationalaccountsdatabeganin1947Q1.Tosetitincontext,duringthepreviousglobalfinancialcrisisinthe1930s,realGDPfellineachyearbetween1930and1933,withacumulativefallof29percent.The1929peaklevelwasnotreacheduntil1936,makingacumulativelossofoutputbelowthepreviouspeakof93percent. So although the recession of the late 2000swasmild in comparison, it was still aseriousone.

4

Section2describesthebenchmarksforforecastingaccuracygivenbythemeanpredictionsoftheSurveyofProfessionalForecasters,maintainedandpublishedbytheFederalReserveBank of Philadelphia (https://www.philadelphiafed.org/research-and-data/real-time-center/survey-of-professional-forecasters/).Section3setsoutthemethodologywefollow,section4describes the results for theUS, sectionsummarises the results for theUK,andsection6offersashortconclusion.

2. TheSurveyofProfessionalForecastersbenchmarks

TheSurveyofProfessionalForecasterspublishesawiderangeofinformationoneconomicforecasts.ItistheoldestquarterlysurveyofmacroeconomicforecastsintheUnitedStates.The surveybegan in1968andwasconductedby theAmericanStatisticalAssociationandtheNationalBureauofEconomicResearch.TheFederalReserveBankofPhiladelphiatookoverthesurveyin1990.Wefocushereontheshort-termpredictionsforrealGDPgrowth.Acompleteseriesgoingbackto1968Q4isavailableforone,twoandthreequarteraheadforecastsofGDPgrowth(atanannualisedrate).Inotherwords,theSPFdatain1968Q4fortheonequarteraheadpredictionisthepredictionfortheoutturnin1969Q1.In termsof theonequarter aheadpredictions, although theoverall explanatorypower islow, a regressionof the actual growth rateon themeanpredictionpublishedby the SPFshowsthatthepredictionsareunbiased.Table1showstheresultsofOLSregressionsofthemean SPF prediction on each of themost recent estimate of GDP growth and the thirdpublishedestimate3.Thethirdestimateisreleasedneartotheendofthethirdmonthaftertheendofthequartertowhichthedatarefers.WealsosetoutinTable1theregressionsfrom1990Q2,forreasonsexplainedinsection3below.WereporttheresultswiththemostrecentestimatesofGDPgrowthoutofinterest,ourcomparatorfortheresultsreportedinsection4isthethirdestimatedata.Inn-steppredictionslaterinthepaper,laggedvaluesofvariablesmustbeused,andsotoenableexactcomparisonswiththeSPFpredictionrecord,westartthesamplein1970Q2.

3ThevariousvintagesofGDPgrowthestimatesareavailableathttp://www.bea.gov/National/index.htm

5

Table1 RegressionsofGDPquarteronquartergrowth,annualisedrate,percent,onSPFforecasts

Dependentvariable(GDPquarteronquartergrowth,annualisedrate,percent)

-----------------------------------------------------------------------------------------------SamplePeriod 1970Q2–2016Q2 1990Q2–2016Q2MostRecentEstimateThirdEstimateMostRecentEstimateThirdEstimate(1) (2) (3) (4)SPF0.940 1.054 1.102 1.175(0.123) (0.121)(0.225) (0.207)Constant0.248-0.171 -0.445 -0.482(0.386)(0.379) (0.611) (0.563)Observations185185 105 105AdjustedR20.2370.289 0.181 0.230ResidualStd.Error2.851 2.799 2.229 2.051Thesituationisquitedifferentwiththethreequarteraheadpredictions.Forexample,threequartersahead,themeanSPFpredictionhasneverbeenfornegativegrowthovertheentire1970Q2– 2016Q2period. This reinforces thepointmade in section 1 above thatmacroforecastsareverypoorintermsoftheirtrackrecordonrecessions.AregressionofthethreequartersaheadpredictionforrealGDPgrowthonthemostrecentestimategivesanRbarsquaredof0.040,andonthethirdestimateofGDPof0.050.Overthe1990Q2-2016Q2period,theyare-0.005and0.009respectively.The historical accuracy of the SPF predictions therefore provides two benchmarks for amachinelearningapproach:

• Givendatawhichwasavailableatthetimeapredictionwouldhavebeenmade,isitpossibletoobtainmoreexplanatorypowerononeperiodaheadpredictions

• Givendatawhichwasavailableatthetimeapredictionwouldhavebeenmade,isitpossibletoobtainanyexplanatorypoweronthreeperiodaheadpredictions

Inaddition,wealsoexaminesixquarteraheadpredictionsoutof interest,thoughtheSPFdoesnotpublishthese.

6

3. Methodology

3.1 DataWetrytoreplicateascloselyaspossiblethesituationinwhichaforecastwouldactuallybemade. Thedependentvariableinouranalysis isthethirdestimateofrealGDPgrowthintherelevantquarter,ratherthanthemostrecentestimatewhichisnowavailable.Thethirdestimateisingeneraltheoneonwhichpolicymakerswouldrelywhentryingtojudgethecurrentstateoftheeconomyatthetime.Weshouldstressthattheexplanatorydatasetwasselectedonatheoreticalbasis,withoutpriorinvestigationofhowanyofthevariablescorrelatedwithGDPgrowth.Onceselected,we did not amend the data base in anyway to try to improve statistical fits, a commonpracticeinmuchmacroeconomictimeseriesinvestigations.Themotivationwasthatthereis a distinct tradition within economics of regarding recessions, especially deep ones, asbeingofmonetaryorigin. FriedmanandSchwartz(1963) isakeytexthere, inwhichtheyarguedthattheFederalReserve’smonetarypolicieswerelargelytoblamefortheseverityoftheGreatDepression.We choose explanatory variables principally from financial markets, where in theoryinformationshouldexistaboutthefuturestateoftheeconomy.Perhapsmoreimportantlyinthecontextoftryingtoreplicateagenuineforecastingsituation,thesevariablesarebothavailableatthetime,andtheirvaluesarenotsubsequentlyrevised. Weusethe3monthTreasuryBillrate,theyieldon10yearUSgovernmentbonds,andthequarterlypercentagechangeintheStandardandPoors500index.WeaddtothesetofexplanatoryvariablestheratioofprivatesectordebttocurrentpriceGDP, thedebtdatabeingtakenfromtheBankof InternationalSettlementswebsite. Thisdatausuallyappearswithalagofthreeorfourmonths,sothemostrecentquarterinanyforecasting situation would have to have been estimated. Further, current price GDP issubject to revisions, but given that it is used to divide another large number of similarmagnitude,anysuchrevisionsintheratiowillbeverysmall.

3.2 Modelestimationtechniques

Wecomparetheresultsoftwodifferentestimationtechniques:

• Ordinaryleastsquaresregression• Randomforestmachinelearning

7

Randomforests(Breiman,2001,2002)aremachine-learningmodelsknownfortheirabilitytocopewithnoisy,non-linear,high-dimensionalpredictionproblems.ManyproofsoftheirpropertieswhichextendtheoriginalworkofBreimanareavailablein,forexample,Biauetal.(2008)andBiau(2012).Theyconstructa largenumberofdecision trees in trainingbysamplingwith replacementfromtheobservations.Eachtreeinthecollectionisformedbyfirstselectingatrandom,ateachnode,asmallgroupof inputcoordinatestosplitonand,secondly,bycalculatingthebestsplitbasedonthesefeaturesinthetrainingset.Eachtreegivesaprediction,andthepredictions are averaged. From the point of view of the bias-variance trade-off, theensembleofalargenumberoftreestrainedonindependentbootstrapsamples,eachwithrelatively large variance but low bias, achieve much reduced variance without theintroductionofadditionalbias.Fernandez-Delgadoetal.(2014),inapaperwhosecitationsarerisingrapidly,compare179classificationalgorithmsfrom17“families”suchasBayesian,neuralnetworks, logisticandmultinomialregression,Theyexaminetheirperformanceon121datasetsintheUniversityof California at Irvine machine learning repository. This repository is in standard use inmachine learning research. Theauthors find that the random forest familyof algorithmsachievesthebestresults.Theclosestrivalissupportvectormachines.Thereareafewofotherswhichhave good results. But the authorsnote that the remainder,which includeBayesian and logistic regression algorithms, “are not competitive at all”. (p.3175). In aneconomiccontext,AlessiandDetken(2011)andAlessietal.(2015)reportgoodresultswithrandomforestalgorithmsinthecontextofearlywarningofbankingcrises.ToillustrateourapproachusingtheexampleofOLSandtheonequarteraheadpredictions,weregresstheGDPgrowthvariableonlaggedvaluesoftheexplanatoryvariables.Weusedlaggedvaluesbecausethesevalueswouldhavebeenavailabletoaforecasteratthetime.Weuselagsfromonethroughfour.Initially,wecarryouttheregressionovertheperiod1970Q2through1990Q1,andusetheresulting equation to predict 1990Q2. We then repeat the regression over the period1970Q2through1990Q2,andpredict1990Q3,andsoonuntilweregressovertheperiod1970Q2through2016Q1andpredict2016Q2.We save each of the predictions, and then regress the data series composed of thesepredictionson,inturn,thethirdestimatesofGDPgrowthoverthe1990Q2through2016Q2period.TheresultsarethencomparedtothebenchmarkSPFaccuracyregressionsreportedinTable1above.

8

Forthreequartersahead,weusethesamesampleperiods,butusethethirdthroughsixthlagsoftheexplanatoryvariables.Inotherwords,weregressGDPgrowthattimetontheexplanatoryvariablesattimest-3,t-4,t-5andt-6.Thesearethevalueswhichwouldhavebeenavailableatthetimeatwhichathreequarteraheadpredictionwasmade.Wealso carryout the sameprocedurewitha sixquarteraheadprediction, justusing thesixth and seventh lags of the explanatory variables. No direct comparison with the SPFaccuracy is available,but it isof interest to seewhat levelof accuracy couldbeobtainedoverthislongertimehorizon.The same procedure is followed with the random forest algorithm. We describe theprocedureaboveusingtheexampleofOLSsimplybecauseitismuchmorefamiliarandsolesscumbersometodescribe.WeusethestatisticalprogramR,anddownloadedthepackagerandomForesttocarryouttherandomforestanalysis4.Weemphasisethatweusedthedefaultvaluesforthevariousoptionsavailableforinputsintherandomforestalgorithm.Inotherwords,wedidnotattemptinanywaytooptimisetheaccuracyofthepredictionsbytryingdifferentcombinationsofinputparameters.Itisverylikelythatbetterresultscouldbeobtainedinthisway,butthiswouldviolatetheprincipleofreplicatingasfaraspossibleanactualforecastingsituation.

4. ResultsfortheUnitedStatesIntermsofonestepaheadforecasts,neitherofthetwoapproachesgaveresultsintermsofexplanatorypowerwhichareasgoodasthesimpleregressionofthethirdestimateofGDPgrowthonthemeanSPFpredictions,reportedinTable1above.Ingeneral,thepredictionswerealsobiased.The interestoftheresults is inthethreeandsixstepaheadpredictions. Asnotedabove,theSPFtrackrecordisverypooroverthethreequarterhorizon,witharegressionofGDPgrowthonthemeanSPFpredictionhavingzeroexplanatorypoweroverthe1990Q2-2016Q2period.

4 Oliver Rice kindly used the RandomForestRegressor tool in the scikit-learn package tocheckthevalidityofourresults.ThesameparametersasthedefaultsintheRpackagewereusedasfaraspossible

9

The regression of GDP growth on the predictions from the linear model gave an R bar-squaredofonly0.009,not significantlydifferent fromzero. The random forest approachgivesaconsiderablyhigherRbarsquaredof0.149.Over the six period ahead horizon, the linear model has an R bar-squared of 0.025,significantly different from zero at a p-value of 0.059. Although in scientific terms theexplanatory power of the random forest approach remains lowwith anR bar-squaredof0.091,itissignificantlydifferentfromzeroatap-valueof0.001.WereporttheregressionoftheactualthirdestimateofGDPgrowthontherandomforestpredictionsoverthethreeandsixquarterhorizonsinTable2belowTable2 RegressionsofGDPquarteronquartergrowth,annualisedrate,percent,thirdestimateonrandomforestpredictionsmadethreeandsixquarterspreviously

Dependentvariable:GDPquarteronquartergrowth,annualisedrate,percent,third

estimate---------------------------Threequartersahead SixquartersaheadPrediction0.674 0.548 (0.154) (0.162)Constant0.570 1.105 (0.488) (0.466)Observations105 105AdjustedR20.149 0.091ResidualStd.Error2.157 2.229The predictions are in each case biased, the coefficients on the predictions beingsignificantly different from one, and in the case of the six quarter ahead predictions theconstant term is significantly different from zero. However, they each have explanatorypowerwhichissignificantlydifferentfromzero.Inthecaseofthethreequarterahead,thiscompares with the zero explanatory power of the mean SPF forecasts, the SPF notpublishingatrackrecordonsixquarteraheadprediction.

10

Figure 1 plots the actual third estimate GDP growth and the predictions made by therandomforestapproachsixquarterspreviously,overthe1990Q2-2016Q2period.

Figure1 ActualannualisedquarteronquarterthirdestimateUSGDPgrowth,percent,andrandomforestpredictionsmadesixquarterspreviously,1990Q2–2016Q2

Note: solidblacklineisactual,dottedredlinepredictedIntriguingly,althoughtherandomforestpredictionswouldnothavegottheexacttimingofthe recession in the winter of 2008/09 correct, a serious recession would have beenpredictedforearly2009eighteenmonthspreviously.Period Actualannualisedthirdestimate Randomforestpredictionsmade quarteronquarterrealGDPgrowth sixquarterspreviously2008Q1 0.96 2.402008Q2 2.83 2.692008Q3 -0.51 2.112008Q4 -6.34 2.072009Q1 -5.49 -2.632009Q2 -0.74 -3.602009Q3 2.24 -1.932009Q4 5.55 0.33

11

The results are really quite striking. For example, in 2007Q3, a prediction of a negativegrowthrateof-2.63percent in2009Q1couldhavebeenmade. Theoveralldepthoftherecession predicted for 2009Q1-2009Q3was obviously not as big as the actual recessionitself, but it is very similar to the recession of the early 1980s, which until 2008/09wasdistinctlythelargestrecessionexperiencedsincethe1930s.There are twoother periods of recession in theperiodonwhichwe focus: thewinter of1990/91and2001 (technically, therewasonlyoneperiodofnegativegrowth in the thirdestimatedatafor2001,butgrowthwasclosetozeroinotherquarters).Inbothcases,therandom forest approach predicts six quarters ahead a marked slowdown in growth, butagainslightly later thanoccurred. Theapproachdoesnotpredictarecession,butseveralperiodsofgrowthunder1percent(atanannualisedrate).Ingeneral,periodswithgrowthatthislowlevelareassociated,forexample,withrisingunemployment.The random forest doesnot predict a recession in periodswhenonedidnot in fact takeplace.

5. ResultsfortheUnitedKingdomWedescribetheseresultsconsiderablymorebriefly,adoptingthesameapproachasabove.ThereisnohistoricalrecordofactualforecastsmadewhichissimilartothatoftheSPFinthe US, so we are unable to make this direct comparison. But, in general, themacroeconomicforecastingrecordissimilarinthetwocountries.As explanatory variables,we use the 3month Treasury Bill rate, the yield on 10 yearUKgovernment bonds, the quarterly percentage change in the FTSE All Share Index and theratioofprivatesectordebt tocurrentpriceGDP. TheBankofEnglandstatisticdatabase,ourmain source,doesnotgobackquiteas faras is the casewith theUS sources, sowepredict over the period 1995Q1 through 2016Q1, using the Office for National StatisticssecondestimateofrealGDP.Looking three quarters ahead, the linear regression using four relevant lags of theexplanatory variables gives an R bar squared for the regression of the actual secondestimateofGDPgrowthonitspredictedvalueof0.004.IncontrasttheRbarsquaredusingthepredictionsfromtherandomforestmodelis0.246.Lookingsixquartersahead,thelinearregressionusingfourrelevantlagsoftheexplanatoryvariables gives anRbar squared for the regressionof the actual secondestimateofGDPgrowthonitspredictedvalueof0.042.IncontrasttheRbarsquaredusingthepredictionsfromtherandomforestmodelis0.290.

12

Figure2plotstheactualandthesixquarteraheadprediction.

Figure2 ActualannualisedquarteronquarterthirdestimateUKGDPgrowth,percent,andrandomforestpredictionsmadesixquarterspreviously,1995Q1–2016Q1

Note: solidblacklineisactual,dottedredlinepredicted

6. ConcludingremarksWehavetried,asfarasitispossible,toreplicateanactualforecastingsituationstartingfortheUnitedStatesin1990Q2andmovingforwardaquarteratatimethroughto2016.Weuse a small number of lags on a small number of financial variables in order to makepredictions.In terms of one step ahead predictions of real GDP growth, we have not been able toimproveuponthemeanforecastsmadebytheSocietyofProfessionalForecasters.

13

However,evenjustthreequartersahead,theSPFtrackrecordisverypoor.Aregressionofactual GDP growth on the mean prediction made three quarters previously has zeroexplanatory power, and the SPF predictions never indicated a single quarter of negativegrowth.Therandomforestapproachimprovesveryconsiderablyonthis.Evenmore strikingly,overa sixperiodaheadhorizon, the randomforestapproachwouldhavepredicted,duringthewinterof2007/08,asevererecessionintheUnitedStatesduring2009,endingin2009Q4.Againtoemphasise,wehavenotattempted inanywaytooptimisetheseresults inanexpostmanner. We use only the default values of the input parameters into themachinelearningalgorithm,anduseonlyasmallnumberofexplanatoryvariables.We obtain qualitatively similar results for the UK, though the predictive power of therandomforestalgorithmisevenbetterthanitisfortheUnitedStates.AsOrmerod andMounfield (2000) show, usingmodern signal processing techniques, thetimeseriesGDPgrowthdataisdominatedbynoiseratherthanbysignal.Sothereisalmostcertainlyaquiterestrictiveupperboundonthedegreeofaccuracyofpredictionwhichcanbeachieved.However,machinelearningtechniquesdoseemtohaveconsiderablepromiseinextendingusefulforecastinghorizonsandprovidingbetter informationtopolicymakersoversuchhorizons. ReferencesAlessi,L.andDetken,C.,2011.Quasirealtimeearlywarningindicatorsforcostlyassetpriceboom/bustcycles:Arole forglobal liquidity.EuropeanJournalofPoliticalEconomy,27(3),pp.520-533Alessi,L.,Antunes,A.,Babecký,J.,Baltussen,S.,Behn,M.,Bonfim,D.,Bush,O.,Detken,C.,Frost,J.,Guimaraes,R.andHavranek,T.,2015.Comparingdifferentearlywarningsystems:Results fromahorse race competitionamongmembersof themacro-prudential researchnetwork.Biau,G.,2012.Analysisofarandomforestsmodel.JournalofMachineLearningResearch,13(Apr),pp.1063-1095.Biau, G., Devroye, L. and Lugosi, G., 2008. Consistency of random forests and otheraveragingclassifiers.JournalofMachineLearningResearch,9(Sep),pp.2015-2033.Breiman,L.,2001.Randomforests.Machinelearning,45(1),pp.5-32Breiman, L., 2002.Manual on setting up, using, and understanding random forests v3. 1.StatisticsDepartmentUniversityofCaliforniaBerkeley,CA,USA

14

Fildes, R. and Stekler, H., 2002. The state of macroeconomic forecasting. Journal ofMacroeconomics,24(4),pp.435-468Ormerod, P. and Mounfield, C., 2000. Random matrix theory and the failure of macro-economic forecasts.Physica A: StatisticalMechanics and its Applications, 280(3), pp.497-504.Stockton, D., 2012. Review of the Monetary Policy Committee’s Forecasting Capability,presented to the Court of the Bank of Englandhttp://www.bankofengland.co.uk/publications/Documents/news/2012/cr3stockton.pdfVarian, H.R., 2014. Big data: New tricks for econometrics. The Journal of EconomicPerspectives,28(2),pp.3-27Zarnowitz,V.andBraun,P.,1993.Twenty-twoyearsof theNBER-ASAquarterlyeconomicoutlook surveys: aspects and comparisons of forecasting performance. InBusiness cycles,IndicatorsandForecasting(pp.11-94).UniversityofChicagoPress.