STATE TAX REVENUE

FORECASTING

ACCURACY

Technical Report

September 2014

Donald J Boyd and Lucy Dadayan

Rockefeller Institute of Government

State University of New York

Support for this project was provided by

The Pew Charitable Trusts

The Public PolicyResearch Arm of theState Universityof New York

411 State StreetAlbany NY 12203-1003(518) 443-5522

wwwrockinstorg

R E V E N U E F O R E C A S T I N G

Acknowledgements

We thank revenue forecasters and analysts in the states for responding to the survey on revenueforecasting conducted in the course of this project

We graciously thank The Pew Charitable Trusts for providing funding for the research and releaseof this report

The views expressed herein are those of the authors and do not necessarily reflect the views ofThe Pew Charitable Trusts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page ii wwwrockinstorg

A Note on the Term ldquoForecast Errorrdquo

Throughout this report we refer to the difference between a forecast and actual results as aldquoforecast errorrdquo This term is common in analyses of forecasts whether they be forecasts of theeconomy or of the weather or of state tax revenue It does not imply that the forecaster made anavoidable mistake or that the forecaster was somehow unprofessional All forecasts of economicactivity will be wrong to some extent regardless of the expertise of the forecaster or the qualityof the tools used Forecasting errors are inevitable because the task is so difficult Revenue fore-casts are based in part upon economic forecasts and economic forecasts prepared by profes-sional forecasting firms often are subject to substantial error tax revenue is volatile anddependent upon idiosyncratic behavior of individual taxpayers which is notoriously difficult topredict and tax revenue is subject to legislative and administrative changes that also are difficultto predict

STATE TAX

REVENUE

FORECASTING

ACCURACY

Technical Report

September 2014

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page iii wwwrockinstorg

ContentsExecutive Summary v

Description and Summary of Data Used in This Study v

Revenue Forecasting Accuracy and Revenue Volatility vi

Revenue Forecasting Accuracy and the Timing andFrequency of Forecasts vii

Revenue Forecasting Accuracy and InstitutionalArrangements vii

Introduction 1

Description And Summary of Data Used in This Study 2

Description of Data 2

Measures of Forecasting Error 5

Forecasting Errors as Percentage of Actual Revenue 7

Absolute Value of Percentage Forecasting Errors 11

Constructing a Forecast Difficulty Measure 12

Conclusions and Policy Implications FromDescriptive Analysis 15

Revenue Forecasting Accuracy and Revenue Volatility 18

Prior Research on Revenue Volatility and RevenueForecasting 18

Trends in Revenue Volatility and How It Relates toForecasting Accuracy 20

Reducing Volatility by Changing Tax Structure 23

Managing Volatility and Errors 24

Conclusions and Policy Implications RegardingForecasting Accuracy and Revenue Volatility 25

Revenue Forecasting Accuracy and the Timing andFrequency of Forecasts 27

Prior Research on Forecasting Accuracy andForecast Timing and Frequency 27

Measuring Timing of Forecasts 27

Accuracy and the Timing of Forecast 28

Accuracy and the Frequency of Forecast Updates 31

Conclusions and Policy Implications RegardingForecasting Accuracy and the Timing and Frequencyof Forecasts 33

Revenue Forecasting Accuracy and InstitutionalArrangements 34

Prior Research on Forecasting Accuracy andInstitutional Arrangements 34

Observations From the Survey 37

Descriptive Analysis 39

Conclusions and Policy Recommendations RegardingForecasting Accuracy and Institutional Arrangements 39

Appendix A Survey of State Officials 41

Endnotes 43

Bibliography 44

Executive Summary

This report updates data on state revenue forecasting errorsthat was initially presented in ldquoCracks in the Crystal Ballrdquo a2011 report on revenue forecasting in the states produced in

collaboration with The Pew Charitable Trusts It supplementsthese data with additional data including the results of a surveyof state forecasting officials conducted by the Rockefeller InstituteIn addition to examining how revenue forecasting errors havechanged since 2009 which was the last data point in the prior pro-ject we examine the relationship between revenue forecasting ac-curacy and

Tax revenue volatility

Timing and frequency of forecasts and

Forecasting institutions and processes

Our main conclusions and recommendations follow

Description and Summary of Data Used in This Study

Our analysis for this report is based on four main sourcesFirst as with the 2011 report we computerized data on revenueestimates and revenue collections for the personal income salesand corporate income taxes from the Fall Fiscal Survey of the Statesfrom the National Association of State Budget Officers (NASBO)for each year from 1987 through 2013 covering a total oftwenty-seven years We define the forecasting error as actual mi-nus the forecast and thus include both overforecasting andunderforecasting In other words a positive number is an under-estimate of revenue (actual revenue is greater than the forecast)and a negative number is an overestimate

Second we developed a new measure of forecast difficultythat we used both in descriptive analyses of the data and in statis-tical analyses to allow more accurate estimates of the influence ofother factors on forecast error after controlling for the difficulty ofa forecast The measure of forecast difficulty is the error that re-sults from a uniformly applied simple or ldquonaiumlverdquo forecastingmodel which we then compare to forecasting errors that resultfrom statesrsquo idiosyncratic and more elaborate revenue forecastingprocesses

Third as before we included other secondary data in ouranalyses including measures of year-over-year change in state taxrevenue from the Census Bureau and measures of the nationaland state economies from the Bureau of Economic Analysis In ad-dition several other researchers provided us with data that theyhad developed on the institutional and political environment inwhich revenue forecasts are made and we incorporated some ofthese data into our analyses

Fourth we surveyed state government officials involved instate revenue forecasting We did this to gain a more detailed un-derstanding of the NASBO data on forecasts and results and alsoto learn more about the institutional forecasting arrangements in

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page vi wwwrockinstorg

STATE TAX

REVENUE

FORECASTING

ACCURACY

Technical Report

September 2014

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page iii wwwrockinstorg

ContentsExecutive Summary v

Description and Summary of Data Used in This Study v

Revenue Forecasting Accuracy and Revenue Volatility vi

Revenue Forecasting Accuracy and the Timing andFrequency of Forecasts vii

Revenue Forecasting Accuracy and InstitutionalArrangements vii

Introduction 1

Description And Summary of Data Used in This Study 2

Description of Data 2

Measures of Forecasting Error 5

Forecasting Errors as Percentage of Actual Revenue 7

Absolute Value of Percentage Forecasting Errors 11

Constructing a Forecast Difficulty Measure 12

Conclusions and Policy Implications FromDescriptive Analysis 15

Revenue Forecasting Accuracy and Revenue Volatility 18

Prior Research on Revenue Volatility and RevenueForecasting 18

Trends in Revenue Volatility and How It Relates toForecasting Accuracy 20

Reducing Volatility by Changing Tax Structure 23

Managing Volatility and Errors 24

Conclusions and Policy Implications RegardingForecasting Accuracy and Revenue Volatility 25

Revenue Forecasting Accuracy and the Timing andFrequency of Forecasts 27

Prior Research on Forecasting Accuracy andForecast Timing and Frequency 27

Measuring Timing of Forecasts 27

Accuracy and the Timing of Forecast 28

Accuracy and the Frequency of Forecast Updates 31

Conclusions and Policy Implications RegardingForecasting Accuracy and the Timing and Frequencyof Forecasts 33

Revenue Forecasting Accuracy and InstitutionalArrangements 34

Prior Research on Forecasting Accuracy andInstitutional Arrangements 34

Observations From the Survey 37

Descriptive Analysis 39

Conclusions and Policy Recommendations RegardingForecasting Accuracy and Institutional Arrangements 39

Appendix A Survey of State Officials 41

Endnotes 43

Bibliography 44

Executive Summary

This report updates data on state revenue forecasting errorsthat was initially presented in ldquoCracks in the Crystal Ballrdquo a2011 report on revenue forecasting in the states produced in

collaboration with The Pew Charitable Trusts It supplementsthese data with additional data including the results of a surveyof state forecasting officials conducted by the Rockefeller InstituteIn addition to examining how revenue forecasting errors havechanged since 2009 which was the last data point in the prior pro-ject we examine the relationship between revenue forecasting ac-curacy and

Tax revenue volatility

Timing and frequency of forecasts and

Forecasting institutions and processes

Our main conclusions and recommendations follow

Description and Summary of Data Used in This Study

Our analysis for this report is based on four main sourcesFirst as with the 2011 report we computerized data on revenueestimates and revenue collections for the personal income salesand corporate income taxes from the Fall Fiscal Survey of the Statesfrom the National Association of State Budget Officers (NASBO)for each year from 1987 through 2013 covering a total oftwenty-seven years We define the forecasting error as actual mi-nus the forecast and thus include both overforecasting andunderforecasting In other words a positive number is an under-estimate of revenue (actual revenue is greater than the forecast)and a negative number is an overestimate

Second we developed a new measure of forecast difficultythat we used both in descriptive analyses of the data and in statis-tical analyses to allow more accurate estimates of the influence ofother factors on forecast error after controlling for the difficulty ofa forecast The measure of forecast difficulty is the error that re-sults from a uniformly applied simple or ldquonaiumlverdquo forecastingmodel which we then compare to forecasting errors that resultfrom statesrsquo idiosyncratic and more elaborate revenue forecastingprocesses

Third as before we included other secondary data in ouranalyses including measures of year-over-year change in state taxrevenue from the Census Bureau and measures of the nationaland state economies from the Bureau of Economic Analysis In ad-dition several other researchers provided us with data that theyhad developed on the institutional and political environment inwhich revenue forecasts are made and we incorporated some ofthese data into our analyses

Fourth we surveyed state government officials involved instate revenue forecasting We did this to gain a more detailed un-derstanding of the NASBO data on forecasts and results and alsoto learn more about the institutional forecasting arrangements in

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page vi wwwrockinstorg

Executive Summary

This report updates data on state revenue forecasting errorsthat was initially presented in ldquoCracks in the Crystal Ballrdquo a2011 report on revenue forecasting in the states produced in

collaboration with The Pew Charitable Trusts It supplementsthese data with additional data including the results of a surveyof state forecasting officials conducted by the Rockefeller InstituteIn addition to examining how revenue forecasting errors havechanged since 2009 which was the last data point in the prior pro-ject we examine the relationship between revenue forecasting ac-curacy and

Tax revenue volatility

Timing and frequency of forecasts and

Forecasting institutions and processes

Our main conclusions and recommendations follow

Description and Summary of Data Used in This Study

Our analysis for this report is based on four main sourcesFirst as with the 2011 report we computerized data on revenueestimates and revenue collections for the personal income salesand corporate income taxes from the Fall Fiscal Survey of the Statesfrom the National Association of State Budget Officers (NASBO)for each year from 1987 through 2013 covering a total oftwenty-seven years We define the forecasting error as actual mi-nus the forecast and thus include both overforecasting andunderforecasting In other words a positive number is an under-estimate of revenue (actual revenue is greater than the forecast)and a negative number is an overestimate

Second we developed a new measure of forecast difficultythat we used both in descriptive analyses of the data and in statis-tical analyses to allow more accurate estimates of the influence ofother factors on forecast error after controlling for the difficulty ofa forecast The measure of forecast difficulty is the error that re-sults from a uniformly applied simple or ldquonaiumlverdquo forecastingmodel which we then compare to forecasting errors that resultfrom statesrsquo idiosyncratic and more elaborate revenue forecastingprocesses

Third as before we included other secondary data in ouranalyses including measures of year-over-year change in state taxrevenue from the Census Bureau and measures of the nationaland state economies from the Bureau of Economic Analysis In ad-dition several other researchers provided us with data that theyhad developed on the institutional and political environment inwhich revenue forecasts are made and we incorporated some ofthese data into our analyses

Fourth we surveyed state government officials involved instate revenue forecasting We did this to gain a more detailed un-derstanding of the NASBO data on forecasts and results and alsoto learn more about the institutional forecasting arrangements in

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page vi wwwrockinstorg

each state and how forecasts are used in the budget processAmong other things this survey allowed us to develop a measureof the typical lag in each state between the time a revenue forecastis prepared and the start of the fiscal year which we used in ouranalysis of the impact of the timing and frequency of forecasts onforecast accuracy

Our main conclusions from our initial descriptive analyses ofthese data sources are

Corporate income tax forecasting errors are much largerthan errors for other taxes followed by the personal in-come tax and then the sales tax The median absolute per-centage error was 118 percent for the corporate incometax 44 percent for the personal income tax and 23 per-cent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) mdash such as Alaska Montana Nevada NewHampshire and North Dakota mdash tend to have larger er-rors Those statesrsquo errors also tend to be more variable(Among these states the taxes reported to NASBO byAlaska Delaware Montana New Hampshire Vermontand Wyoming are a relatively small share of total taxes asmeasured by the Census Bureau Their errors for totaltaxes in their budget which also include other less volatiletaxes appear to be smaller than those examined hereHowever these states also have large errors from naiumlveforecasting models using full Census data)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

States have returned to ldquomore normalrdquo forecasting errorswith the 2007 recession now behind us

Revenue Forecasting Accuracy and Revenue Volatility

Increases in forecasting errors have been driven by increasesin revenue volatility which in turn have been driven in large part

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page vii wwwrockinstorg

by volatile capital gains which have grown as a share of adjustedgross income over the last several decades

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtuallyeliminating the corporate income tax and significantly increasingreliance on the sales tax relative to the personal income tax couldthe typical state reduce revenue forecasting errors and even thenmost tax combinations would not reduce forecast errors verymuch Changing the structure of individual taxes may be morepromising but it can raise difficult tax policy issues For examplemaking income taxes less progressive might also make them lessvolatile and easier to forecast but that may run counter to distri-butional objectives that some policymakers hope for

Because it is so hard to reduce forecasting errors by changingtax structure it is especially important for states to be able to man-age the effects of volatility Rainy day funds are one importanttool states can use However the data suggest that states withlarger forecast errors and more difficult forecasting tasks do nothave larger rainy day funds perhaps they should

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed based on the information collected from oursurvey In both cases the effect is meaningful the data suggestthat accuracy worsens by well over a percentage point in the fore-cast for the second year of a biennium and worsens by about ahalf a percentage point for every ten weeks of lag between thetime a forecast is prepared and the start of the forecast period

There is no evidence from our data that frequent forecast up-dates lead to greater accuracy This is consistent with pastresearch

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

Revenue Forecasting Accuracyand Institutional Arrangements

Our reading of the literature is that there is very little relation-ship between consensus forecasting and forecast accuracy That is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page viii wwwrockinstorg

consistent with our examination of these data also However aswe have noted before the evidence in favor of examining andcombining forecasts is overwhelming Combining forecasts tendsto lead to lower errors Processes that encourage this to happenmay also lead to lower errors

Beyond that it is good practice to try to insulate forecastingfrom the political process and consensus forecasting can help toachieve that

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page ix wwwrockinstorg

Introduction

This report is based upon work conducted in 2013 and 2014and updates and extends analysis by the Rockefeller Insti-tute of Government for ldquoCracks in the Crystal Ballrdquo a 2011 re-

port on revenue forecasting in the states produced incollaboration with The Pew Charitable Trusts1 Among otherthings the 2011 report concluded

hellip errors in the annual revenue estimates have worsenedRevenues have become more difficult to predict accu-ratelyhellip [R]evenue overestimates during the nationrsquos pastthree recessions grew progressively larger as did the un-derestimates in the past two periods of economic growth

This analysis updates data on state revenue forecasting errorsdeveloped during that project and supplements it with data froma survey of state forecasting officials conducted by the RockefellerInstitute In addition this report examines the relationship be-tween revenue forecasting accuracy and

Tax revenue volatility

Timing and frequency of forecasts

Forecasting institutions and processes

Finally we offer policy recommendations for how to improverevenue forecasting and to manage outcomes of the process

STATE TAX REVENUE

FORECASTING

ACCURACY

Technical ReportState University of NewYork411 State StreetAlbany New York 12203(518) 443-5522wwwrockinstorg

Carl HaydenChair Board of Overseers

Thomas GaisDirector

Robert BullockDeputy Director forOperations

Patricia StrachDeputy Director for Research

Michael CooperDirector of Publications

Michele CharbonneauStaff Assistant forPublications

Nancy L ZimpherChancellor

Rockefeller Institute Page 1 wwwrockinstorg

R E V E N U E F O R E C A S T I N G

by volatile capital gains which have grown as a share of adjustedgross income over the last several decades

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtuallyeliminating the corporate income tax and significantly increasingreliance on the sales tax relative to the personal income tax couldthe typical state reduce revenue forecasting errors and even thenmost tax combinations would not reduce forecast errors verymuch Changing the structure of individual taxes may be morepromising but it can raise difficult tax policy issues For examplemaking income taxes less progressive might also make them lessvolatile and easier to forecast but that may run counter to distri-butional objectives that some policymakers hope for

Because it is so hard to reduce forecasting errors by changingtax structure it is especially important for states to be able to man-age the effects of volatility Rainy day funds are one importanttool states can use However the data suggest that states withlarger forecast errors and more difficult forecasting tasks do nothave larger rainy day funds perhaps they should

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed based on the information collected from oursurvey In both cases the effect is meaningful the data suggestthat accuracy worsens by well over a percentage point in the fore-cast for the second year of a biennium and worsens by about ahalf a percentage point for every ten weeks of lag between thetime a forecast is prepared and the start of the forecast period

There is no evidence from our data that frequent forecast up-dates lead to greater accuracy This is consistent with pastresearch

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

Revenue Forecasting Accuracyand Institutional Arrangements

Our reading of the literature is that there is very little relation-ship between consensus forecasting and forecast accuracy That is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page viii wwwrockinstorg

consistent with our examination of these data also However aswe have noted before the evidence in favor of examining andcombining forecasts is overwhelming Combining forecasts tendsto lead to lower errors Processes that encourage this to happenmay also lead to lower errors

Beyond that it is good practice to try to insulate forecastingfrom the political process and consensus forecasting can help toachieve that

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page ix wwwrockinstorg

Introduction

This report is based upon work conducted in 2013 and 2014and updates and extends analysis by the Rockefeller Insti-tute of Government for ldquoCracks in the Crystal Ballrdquo a 2011 re-

port on revenue forecasting in the states produced incollaboration with The Pew Charitable Trusts1 Among otherthings the 2011 report concluded

hellip errors in the annual revenue estimates have worsenedRevenues have become more difficult to predict accu-ratelyhellip [R]evenue overestimates during the nationrsquos pastthree recessions grew progressively larger as did the un-derestimates in the past two periods of economic growth

This analysis updates data on state revenue forecasting errorsdeveloped during that project and supplements it with data froma survey of state forecasting officials conducted by the RockefellerInstitute In addition this report examines the relationship be-tween revenue forecasting accuracy and

Tax revenue volatility

Timing and frequency of forecasts

Forecasting institutions and processes

Finally we offer policy recommendations for how to improverevenue forecasting and to manage outcomes of the process

STATE TAX REVENUE

FORECASTING

ACCURACY

Technical ReportState University of NewYork411 State StreetAlbany New York 12203(518) 443-5522wwwrockinstorg

Carl HaydenChair Board of Overseers

Thomas GaisDirector

Robert BullockDeputy Director forOperations

Patricia StrachDeputy Director for Research

Michael CooperDirector of Publications

Michele CharbonneauStaff Assistant forPublications

Nancy L ZimpherChancellor

Rockefeller Institute Page 1 wwwrockinstorg

R E V E N U E F O R E C A S T I N G

consistent with our examination of these data also However aswe have noted before the evidence in favor of examining andcombining forecasts is overwhelming Combining forecasts tendsto lead to lower errors Processes that encourage this to happenmay also lead to lower errors

Beyond that it is good practice to try to insulate forecastingfrom the political process and consensus forecasting can help toachieve that

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page ix wwwrockinstorg

Introduction

This report is based upon work conducted in 2013 and 2014and updates and extends analysis by the Rockefeller Insti-tute of Government for ldquoCracks in the Crystal Ballrdquo a 2011 re-

port on revenue forecasting in the states produced incollaboration with The Pew Charitable Trusts1 Among otherthings the 2011 report concluded

hellip errors in the annual revenue estimates have worsenedRevenues have become more difficult to predict accu-ratelyhellip [R]evenue overestimates during the nationrsquos pastthree recessions grew progressively larger as did the un-derestimates in the past two periods of economic growth

This analysis updates data on state revenue forecasting errorsdeveloped during that project and supplements it with data froma survey of state forecasting officials conducted by the RockefellerInstitute In addition this report examines the relationship be-tween revenue forecasting accuracy and

Tax revenue volatility

Timing and frequency of forecasts

Forecasting institutions and processes

Finally we offer policy recommendations for how to improverevenue forecasting and to manage outcomes of the process

STATE TAX REVENUE

FORECASTING

ACCURACY

Technical ReportState University of NewYork411 State StreetAlbany New York 12203(518) 443-5522wwwrockinstorg

Carl HaydenChair Board of Overseers

Thomas GaisDirector

Robert BullockDeputy Director forOperations

Patricia StrachDeputy Director for Research

Michael CooperDirector of Publications

Michele CharbonneauStaff Assistant forPublications

Nancy L ZimpherChancellor

Rockefeller Institute Page 1 wwwrockinstorg

R E V E N U E F O R E C A S T I N G

Introduction

This report is based upon work conducted in 2013 and 2014and updates and extends analysis by the Rockefeller Insti-tute of Government for ldquoCracks in the Crystal Ballrdquo a 2011 re-

port on revenue forecasting in the states produced incollaboration with The Pew Charitable Trusts1 Among otherthings the 2011 report concluded

hellip errors in the annual revenue estimates have worsenedRevenues have become more difficult to predict accu-ratelyhellip [R]evenue overestimates during the nationrsquos pastthree recessions grew progressively larger as did the un-derestimates in the past two periods of economic growth

This analysis updates data on state revenue forecasting errorsdeveloped during that project and supplements it with data froma survey of state forecasting officials conducted by the RockefellerInstitute In addition this report examines the relationship be-tween revenue forecasting accuracy and

Tax revenue volatility

Timing and frequency of forecasts

Forecasting institutions and processes

Finally we offer policy recommendations for how to improverevenue forecasting and to manage outcomes of the process

STATE TAX REVENUE

FORECASTING

ACCURACY

Technical ReportState University of NewYork411 State StreetAlbany New York 12203(518) 443-5522wwwrockinstorg

Carl HaydenChair Board of Overseers

Thomas GaisDirector

Robert BullockDeputy Director forOperations

Patricia StrachDeputy Director for Research

Michael CooperDirector of Publications

Michele CharbonneauStaff Assistant forPublications

Nancy L ZimpherChancellor

Rockefeller Institute Page 1 wwwrockinstorg

R E V E N U E F O R E C A S T I N G

Description and Summary of Data Used in This Study

Description of Data

Data on Forecasting Errors From NASBO

Each fall the National Association of State Budget Officers(NASBO) ask state budget officers for revenue estimates used atthe time of budget enactment and for preliminary actual revenuecollections2 They publish these results in the fall Fiscal Survey ofStates The fall Survey typically is conducted in the months of Julythrough September and published in December

Below is an example of the question as it was asked in August2013 for fiscal year 2013 preliminary actuals and fiscal year 2014estimates

Please complete the following table with respect to actualrevenue collections for FY 2012 estimates used at FY2013 budget enactment preliminary actual revenue col-lections for FY 2013 amp estimates used at budget enact-ment for FY 2014 ($ in millions)

TaxesActualCollectionsFY 2012

EstimatesUsed WhenBudget wasAdopted FY2013

PreliminaryActuals FY2013

EstimatesUsed WhenBudgetAdopted FY2014

Sales Tax

Collections

PersonalIncome TaxCollections

CorporateIncome TaxCollections

We computerized data on revenue estimates and revenue col-lections for the personal income sales and corporate income taxesfrom the NASBO Fall Fiscal Survey of States for each year from1987 through 2013 covering a total of twenty-seven years In thesurvey the ldquooriginal estimatesrdquo are intended to be the forecasts onwhich the adopted budget was based and the ldquocurrent estimatesrdquoare the preliminary actual estimates for the fall after the year wasclosed (eg the estimate in fall 2013 for the fiscal year that endedin June 2013) In 1987 and 1988 the survey covered personal in-come and sales taxes only from 1989 forward it included personalincome sales and corporate income taxes Because NASBO datado not include the District of Columbia we did not incorporatethe District in this analysis

The NASBO data held several great advantages for the pur-poses of our analysis They are self-reported by states and thus re-flect statesrsquo own assessment of appropriate data they are collected

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 2 wwwrockinstorg

by a single source they are intended to be collected under a com-mon set of definitions they are collected for all fifty states in mostyears and they go back more than twenty-five years covering allor part of three recessions (in 1990 2001 and 2007) It would beimpractical to assemble such a data set from scratch collectinghistorical forecast and actual revenue data from individual statesover nearly three decades records disappear memories fade andstaff move on making it difficult to reconstruct these data

As with any self-reported numbers there were some anomaliesin the NASBO data which we were diligent in cleaning We elimi-nated data in the following situations cases in which only the origi-nal estimate was reported but not the current and vice versa casesin which the original and current estimates were identical for twoor more taxes suggesting a possible reporting error and cases inwhich we believed the estimating errors were too large to be plausi-ble (the top 1 percent of cases with the highest absolute value offorecast error) California was missing from the NASBO data in2001 and 2009 Because it was such a large state we contacted stateofficials in California directly and supplemented the NASBO datawith estimates of what would have been submitted to NASBO inthose years Data were also missing for Texas in 1996 1997 and1999 However as noted below it was not practical to obtain com-parable data for missing values for Texas

After these adjustments we have 3347 observations of fore-cast errors for individual taxes In addition where data allowedwe computed forecast errors for the sum of the three taxes foreach state and year for an additional 1311 values3

Survey of State Officials

We supplemented forecast-error data from NASBO with a sur-vey we conducted of state government officials4 Our intent wasto strengthen our ability to use the NASBO data in regressionanalyses and to gain a richer understanding of state forecastingprocesses more generally The survey had two main purposes

1 To gain a clearer understanding of what exactly theforecast data provided by states to NASBO represent togather information on when those forecasts are pre-pared and to understand how they are used in the bud-get process in the states

2 To gain a richer understanding of the variety of fore-casting arrangements in the states and how these pro-cesses relate to the budgeting process

We conducted the survey in two rounds During the firstround we asked questions of state forecasting officials related tothe data reported to NASBO The survey questions were designedto help us get a better understanding of when estimates that un-derlie state budgets are developed as that influences the difficultyof the forecasting job and to be sure we understand how the fore-casts are used in the budget process

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 3 wwwrockinstorg

The main purpose of the second round was to get a better un-derstanding of general procedures and practices of revenue esti-mating processes in the states The survey questions weretargeted at getting more information about the timing of forecastsfrequency of forecast updates the long-term forecasting practicesparties involved in the forecasting processes and whether there isa formal group that plays a major role in the revenue estimatingprocess

Both surveys were conducted online using Google Docs Thefirst round was conducted during July-August of 2013 while thesecond round was conducted during August-September of 2013

For the first round of the survey we sent an online surveyquestionnaire to state budget officers in all fifty states who werealso the respondents for the NASBOrsquos survey The list of respon-dents was retrieved from the Fiscal Survey of the States report Theresponse rate for the first round of the survey was 68 percent (34usable responses were received)

For the second round of the survey we sent an online surveyquestionnaire to both legislative and executive branch officials inall fifty states who were likely to be involved in the forecastingprocess The response rate for this round was much higher 90percent (forty-five responses) for the executive branch and 92 per-cent (forty-six responses) for the legislative branch In aggregate(executive and legislative branches combined) we have receivedresponses from all fifty states

The responses from the first round indicate that in most statesthe estimates provided to NASBO are the final estimates used forthe adopted budget States vary widely in how far in advance ofthe budget they prepare their estimates Some states prepare reve-nue estimates as early as thirty-five weeks ahead of the start of thefiscal year (eg Alabama) while other states use revenue esti-mates updated about two weeks before the start of the state fiscalyear (ie Delaware Pennsylvania and Tennessee) The survey re-sults also revealed that in general states with a biennial budgetcycle provide updated estimates to NASBO for the second year ofthe biennium in order to account for legislative changes and up-dated economic forecast conditions Nonetheless on average esti-mates for the second year of a biennium are prepared further inadvance of the second year than are estimates for the first year

The responses from the second round indicate wide variationamong the states in the timing of the forecasts prepared for theadopted budget as well as in frequency of revenue estimate up-dates In terms of long-term forecasting states indicated that theytypically generate revenue estimates for one or more years be-yond the upcoming budget period most states release revenue es-timates for two to five years beyond the budget period

The state officials were also asked to indicate who is involvedin the revenue estimating process The survey results indicate thatin most states one or more agencies of the executive branch are in-volved in the revenue estimating process and in a handful of

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 4 wwwrockinstorg

states the legislative branch also plays a major role in revenue esti-mating However only very few states also involve academiciansandor experts from the private sector in the revenue estimatingprocess Finally in a few states the revenue estimating process isunder the jurisdiction of the nonpartisan independent groupagency the members of which are normally appointed by thegovernor andor legislative branch According to the survey re-sponses in approximately two-thirds of the states the participantsin the revenue estimating process come to an agreement on asingle number

In general the survey results indicate that there is a widerange of practices and processes involved among the fifty states interms of revenue estimates There are no universally agreed uponstandards and practices among the fifty states The survey re-sponses as well as various discussions with the state governmentofficials indicate that it is highly desirable to have more frequentforecasts as well as to have a minimum time gap between the fore-cast and the adoption of the budget so that the states can take intoaccount the latest economic conditions as well as the impact ofany legislative andor administrative changes Overall it is alsodesirable to have a consensus revenue estimating practice in thestate so that various parties involved in the forecasting processcan come into an agreement

Other Data

In addition to the NASBO data we used US Census state taxcollection figures in estimating the size of errors across years oracross the type of tax We also used data on national price changes(the gross domestic product price index) gross domestic productby state and state personal income from the US Bureau of Eco-nomic Analysis We also obtained data from several other re-searchers and describe as needed below5 Finally we constructeda measure of forecast difficulty from US Census state tax collec-tion data which we describe in the section Constructing a ForecastDifficulty Measure

How do the revenue estimating data relate to Census tax reve-nue data Not all states rely heavily on the three taxes thatNASBO collects forecast data for Six states relied on these threetaxes for less than half of their tax revenue in the typical yearAlaska Delaware Montana New Hampshire Vermont and Wy-oming (see Table 1)

Measures of Forecasting Error

There are several common ways of measuring forecast errorSome of these measures do not distinguish positive from negativeerrors and are useful for gauging magnitude of error and othersmaintain the sign and are useful for gauging bias and asymme-tries One common measure is the root-mean-squared error(RMSE) or more appropriate when comparing across states theroot-mean-squared percent error (RMSPE)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 5 wwwrockinstorg

One very common measure in the literature examin-ing state revenue forecasting is the absolute value of theerror as a percentage of the actual result (known as abso-lute percentage error) or variants on this (Voorhees 2004)Summary measures of estimating error across states oryears often use the mean or median of the absolute valueof the percentage error (mean absolute percentage error isquite common and is known as the MAPE) These mea-sures which treat positive and negative errors the samemay not be the most appropriate in all situations In par-ticular other measures are needed to examine questionsof whether forecasts are biased (eg more likely to under-estimate revenue than to overestimate) or whether fore-casters believe the costs of overestimates are greater thanthe costs of underestimates

As many observers have noted the revenue estimat-ing error is quite large relative to what policymakersmight find acceptable In 2002 when the previous fiscalcrisis hit sharply twelve states had revenue estimatingerrors of 10 percent or more (Willoughby and Guo 2008)To provide some perspective a 4 percentage point errorin the state of California would be more than $4 billionmdash an amount that policymakers would find quitedisconcerting

We define the forecasting error as actual minus theforecast Thus a positive number is an underestimate ofrevenue (actual revenue is greater than the forecast) anda negative number is an overestimate Forecasters oftenview underestimates as better than overestimates Ourprimary measure of error is the error as a percentage ofthe actual revenue Thus if a forecast of revenue was $90million and the actual revenue was $100 million then

the error was $10 million (an underestimate) and the percentageerror was 10 percent

We use two main versions of this percentage mdash as is whereit can be either positive or negative and the absolute valuewhich is always positive In general the absolute value is usefulwhen we are interested in the concept of accuracy without re-gard to whether revenue is above target or below although theas-is value can be useful too and we use that in much of theanalysis below When we want to examine bias (not a significantfocus of this report) the absolute value measure is not usefulbecause we care about the direction of error and we focus solelyon the as-is value

When we want to summarize error across states years taxesor other groupings we almost always use the median as a mea-sure of central tendency The reason is that our data are fairlynoisy and we donrsquot generally place great faith in any single datapoint although we do believe the data tell truthful stories in ag-gregate Because any single data point might be wrong or an

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 6 wwwrockinstorg

Table 1 Personal Income Sales and Corporate

Income Taxes as Percent of Total Tax Revenue

outlier it could have an undue impact on the mean pulling itaway from what we think is the true central tendency of the data

When we want to measure how much the errors vary acrossstates years taxes or other groupings we use either the standarddeviation or the interquartile range

In the next section we provide simple descriptive statistics oferrors first using the percentage error ldquoas isrdquo and then the abso-lute value There are some useful insights which we summarizein the concluding subsection of this section

Forecasting Errors as Percentage of Actual Revenue

Below we provide summary tables of errors by tax by fiscalyear and by state and explore patterns with several plots

Table 2 displays selected summary statistics for forecastingerrors by tax for 1987-2013 The corporate income tax median fore-casting error at 28 percent was much larger than the median forother taxes The personal income tax median forecasting error was18 percent while the sales tax forecasting error was 03 percentThe standard deviation of the error mdash a measure of variability inforecast error and a potential indication of forecasting difficulty mdashwas largest by far for the corporate income tax nearly three timesas large as for the personal income tax and five times as large asfor the sales tax

Figure 1 on the following page displays the median forecasterror by tax

There is some evidence that forecasting errors are ldquoseriallycorrelatedrdquo mdash that is that a revenue shortfall in one year is morelikely to be followed by a shortfall in the next than by an overageand vice versa Rudolph Penner a former director of the Congres-sional Budget Office concluded this about federal governmentforecasts ldquohellip if CBO makes an error in an overly optimistic orpessimistic direction one year it is highly probable that it willmake an error in the same direction the following yearrdquo (Penner2008) Inspection of NASBO data in time periods surrounding re-cessions shows that at least for the states in aggregate revenueoverestimates tended to be followed by additional overestimates

Figure 2 provides median forecasting errors by fiscal year fortwenty-seven years spanning from 1987 to 2013 As depicted onFigure 2 states normally underestimate during expansions andoverestimate during the downturns Moreover overestimating

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 7 wwwrockinstorg

ofobservations

Meanerror

Standarddeviation

25Median(50)

75

Personal income tax 1100 11 77 (23) 18 55Sales tax 1189 02 46 (20) 03 26Corporate income tax 1058 13 226 (93) 28 140Sum of PIT sales amp CIT 1311 10 76 (22) 15 44

Forecast errors as percentage of actual revenue

Source Rockefeller Institute analysis of NASBO forecast error data

Tax

Table 2 State Revenue Forecasting Percentage Errors by Tax 1987-2013

errors were particu-larly large near the2001 and 2007 reces-sions States now re-turned to theirpattern of underesti-mation after theworst of the fiscalcrisis was behindthem

Table 3 on thefollowing page pro-vides summary sta-tistics of forecastingerrors by state from1987 to 2013 Over-all smaller statesand states depend-ent on a few sectorsof the economy (par-ticularly states reli-ant on oil or naturalgas or gambling)such as AlaskaMontana NevadaNew Hampshireand North Dakotatend to have largererrors Those statesrsquoerrors also tend tobe more variable asmeasured by thestandard deviationor the interquartilerange As alreadymentioned abovepersonal incomesales and corporateincome taxes inAlaska DelawareMontana NewHampshire Ver-mont and Wyoming

represent less than half of total taxes as measured by the CensusBureau Their errors for total taxes in their budget which also in-clude other less volatile taxes appear to be smaller than those ex-amined here We strongly caution not to read too much into theerrors for any single state because the data are noisy and influ-enced by many forces and larger errors need not indicate lesseffective revenue forecasting

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 8 wwwrockinstorg

03

18

28

15

00

05

10

15

20

25

30

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 1 State Revenue Forecast Errors as Percent of Actual Tax Median for 1987-2013

12

10

8

6

4

2

0

2

4

6

8

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal SurveysNote Forecasting error is for the sum of personal income sales and corporate income taxes

1990 91 recession 2001 recession Great recession

Figure 2 Median State Revenue Forecast Errors as Percent of Actual Tax by Fiscal Year

Figure 3 on the following page maps the distributionover time of state forecasting errors for the sum of thethree taxes (personal income tax sales tax and corporateincome tax) and also shows the extent of missing dataWe are missing data for Texas in 1996 1997 and 1999but other than that no large state is missing in any yearIt is difficult to obtain data from state documents that iscomparable to data in the NASBO survey and as notedearlier we were unable to obtain such data with reason-able effort for Texas

The figure shows how forecasting errors have rolledout over time across the country The revenue shortfallsin the middle of the country in 1987 likely are related tothe oil price bust that hit Texas and other oil-extractionstates in the mid-1980s The effects of the 1990 recessionwhich lingered for four years appear to have led to sig-nificant revenue shortfalls on the coasts and in the northbut not as much in the center of the country The 2001recession and the effects of the September 11th attacksare evidenced in revenue shortfalls that began on theeast coast but spread to the rest of the country revenueshortfalls of more than 8 percent were far more commonthan in the 1990 recession The 2005-07 years were agolden period of revenue overages in most of the coun-try The 2007 recession appears to have hit early in thehousing bust states of Arizona Florida and Nevada aswell as Michigan It spread in 2008 and hit the rest of thecountry with force in 2009 and 2010 before state fore-casters were once again on more-solid ground in 2011through 2013

Forecasts may be intentionally biased mdash more likelyto be off in one direction than another mdash because fore-casters believe there is an ldquoasymmetric riskrdquo and that themost accurate estimate is not always the best estimateMany executive-branch forecasters are assumed to be-lieve that the costs of overestimating revenue are greaterthan the costs of underestimating revenue because thepolitical and managerial complications of managing def-icits are believed to be worse than the consequences ofmanaging surpluses These forecasters sometimes ex-plicitly and other times unknowingly may choose con-

servative forecasts of the revenue that is available to financespending By contrast legislative forecasters or their principalsnot generally responsible for financial management might not bebothered in the least by revenue shortfalls that the governor has tomanage but may be troubled to learn of ldquoextrardquo revenue that theycould have spent on their priorities mdash that is they may be morewilling to risk revenue overestimates and revenue shortfalls thanis the executive

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 9 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 01 43 (11) 19 25AK 76 222 (90) 103 152AL 05 61 (13) 15 38AR 21 41 (02) 21 35AZ (04) 97 (16) 14 44CA (03) 74 (46) 25 43CO 09 85 (02) 16 55CT 05 71 (24) 28 53DE 10 68 (28) (01) 56FL (10) 52 (23) (04) 21GA (03) 50 (41) 15 33HI 12 65 (27) 15 49IA 19 37 (03) 18 48ID 19 68 01 15 51IL 07 53 (23) 19 28IN (09) 49 (25) 09 20KS 18 45 (00) 11 20KY (11) 51 (36) 05 26LA 06 66 (38) 17 41MA 12 75 (22) 23 57MD 04 56 (29) 16 40ME 17 46 (01) 14 39MI (17) 52 (42) (10) 23MN 17 51 (06) 18 55MO (02) 54 (13) 11 36MS 10 52 (24) 18 43MT 42 112 (10) 66 93NC 05 59 (12) 25 42ND 93 189 07 30 138NE 10 52 (15) 17 44NH (04) 161 (34) 21 70NJ (07) 56 (36) 10 25NM 07 60 (25) 06 45NV 22 70 (39) 39 71NY (25) 38 (52) (21) 03OH (02) 53 (19) 13 26OK 03 70 (15) 11 43OR 03 100 (17) 21 49PA (02) 37 (19) 02 32RI 03 57 (28) (04) 43SC (06) 86 (51) 14 46SD 11 25 (02) 08 24TN 03 45 (14) 06 34TX 33 64 00 26 67UT 15 61 02 29 53VA (01) 67 (23) 15 26VT 29 63 (08) 42 75WA 03 70 (10) 13 33WI 02 33 (00) 09 18WV 16 34 (11) 16 32WY 34 71 03 20 71Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 3 Forecast Errors as

Percent of Actual Tax by State

Figure 4 on the following page displays boxplots of eachstatersquos percentage forecasting error for the sum of personal in-come tax sales tax and corporate income tax The top horizontalline of each box marks the 75th percentile of forecast errors andthe lower horizontal line marks the 25th percentile Each box alsohas vertical lines (sometimes called ldquowhiskersrdquo) extending fromits top and bottom indicating variability in the top and bottom 25percent of the data (longer lines indicate more variability in thispart of the data) and dots marking extreme outliers6 The tallerthe box (the greater the difference between the 75th and 25th per-centiles) and the longer the lines the greater the variability inforecast errors for a given state

Only five states have a negative median forecasting error sug-gesting again that states may be ldquotryingrdquo to have small positiveforecasting errors mdash underforecasts of revenue However in acommon formal but rather strict test of unbiasedness known asthe Mincer-Zarnowitz test we reject unbiasedness at the 5 percentlevel for only eight states Arkansas Iowa Montana New YorkNorth Dakota Texas Vermont and Wyoming7 For each of thesestates virtually all of the box in Figure 4 is above the line (aboutthree quarters of the errors are positive) or else below the line (inthe case of New York)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 10 wwwrockinstorg

Figure 3 Percentage Forecast Errors for Sum of PIT CIT and Sales Taxes 1987-2013

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

by a single source they are intended to be collected under a com-mon set of definitions they are collected for all fifty states in mostyears and they go back more than twenty-five years covering allor part of three recessions (in 1990 2001 and 2007) It would beimpractical to assemble such a data set from scratch collectinghistorical forecast and actual revenue data from individual statesover nearly three decades records disappear memories fade andstaff move on making it difficult to reconstruct these data

As with any self-reported numbers there were some anomaliesin the NASBO data which we were diligent in cleaning We elimi-nated data in the following situations cases in which only the origi-nal estimate was reported but not the current and vice versa casesin which the original and current estimates were identical for twoor more taxes suggesting a possible reporting error and cases inwhich we believed the estimating errors were too large to be plausi-ble (the top 1 percent of cases with the highest absolute value offorecast error) California was missing from the NASBO data in2001 and 2009 Because it was such a large state we contacted stateofficials in California directly and supplemented the NASBO datawith estimates of what would have been submitted to NASBO inthose years Data were also missing for Texas in 1996 1997 and1999 However as noted below it was not practical to obtain com-parable data for missing values for Texas

After these adjustments we have 3347 observations of fore-cast errors for individual taxes In addition where data allowedwe computed forecast errors for the sum of the three taxes foreach state and year for an additional 1311 values3

Survey of State Officials

We supplemented forecast-error data from NASBO with a sur-vey we conducted of state government officials4 Our intent wasto strengthen our ability to use the NASBO data in regressionanalyses and to gain a richer understanding of state forecastingprocesses more generally The survey had two main purposes

1 To gain a clearer understanding of what exactly theforecast data provided by states to NASBO represent togather information on when those forecasts are pre-pared and to understand how they are used in the bud-get process in the states

2 To gain a richer understanding of the variety of fore-casting arrangements in the states and how these pro-cesses relate to the budgeting process

We conducted the survey in two rounds During the firstround we asked questions of state forecasting officials related tothe data reported to NASBO The survey questions were designedto help us get a better understanding of when estimates that un-derlie state budgets are developed as that influences the difficultyof the forecasting job and to be sure we understand how the fore-casts are used in the budget process

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 3 wwwrockinstorg

The main purpose of the second round was to get a better un-derstanding of general procedures and practices of revenue esti-mating processes in the states The survey questions weretargeted at getting more information about the timing of forecastsfrequency of forecast updates the long-term forecasting practicesparties involved in the forecasting processes and whether there isa formal group that plays a major role in the revenue estimatingprocess

Both surveys were conducted online using Google Docs Thefirst round was conducted during July-August of 2013 while thesecond round was conducted during August-September of 2013

For the first round of the survey we sent an online surveyquestionnaire to state budget officers in all fifty states who werealso the respondents for the NASBOrsquos survey The list of respon-dents was retrieved from the Fiscal Survey of the States report Theresponse rate for the first round of the survey was 68 percent (34usable responses were received)

For the second round of the survey we sent an online surveyquestionnaire to both legislative and executive branch officials inall fifty states who were likely to be involved in the forecastingprocess The response rate for this round was much higher 90percent (forty-five responses) for the executive branch and 92 per-cent (forty-six responses) for the legislative branch In aggregate(executive and legislative branches combined) we have receivedresponses from all fifty states

The responses from the first round indicate that in most statesthe estimates provided to NASBO are the final estimates used forthe adopted budget States vary widely in how far in advance ofthe budget they prepare their estimates Some states prepare reve-nue estimates as early as thirty-five weeks ahead of the start of thefiscal year (eg Alabama) while other states use revenue esti-mates updated about two weeks before the start of the state fiscalyear (ie Delaware Pennsylvania and Tennessee) The survey re-sults also revealed that in general states with a biennial budgetcycle provide updated estimates to NASBO for the second year ofthe biennium in order to account for legislative changes and up-dated economic forecast conditions Nonetheless on average esti-mates for the second year of a biennium are prepared further inadvance of the second year than are estimates for the first year

The responses from the second round indicate wide variationamong the states in the timing of the forecasts prepared for theadopted budget as well as in frequency of revenue estimate up-dates In terms of long-term forecasting states indicated that theytypically generate revenue estimates for one or more years be-yond the upcoming budget period most states release revenue es-timates for two to five years beyond the budget period

The state officials were also asked to indicate who is involvedin the revenue estimating process The survey results indicate thatin most states one or more agencies of the executive branch are in-volved in the revenue estimating process and in a handful of

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 4 wwwrockinstorg

states the legislative branch also plays a major role in revenue esti-mating However only very few states also involve academiciansandor experts from the private sector in the revenue estimatingprocess Finally in a few states the revenue estimating process isunder the jurisdiction of the nonpartisan independent groupagency the members of which are normally appointed by thegovernor andor legislative branch According to the survey re-sponses in approximately two-thirds of the states the participantsin the revenue estimating process come to an agreement on asingle number

In general the survey results indicate that there is a widerange of practices and processes involved among the fifty states interms of revenue estimates There are no universally agreed uponstandards and practices among the fifty states The survey re-sponses as well as various discussions with the state governmentofficials indicate that it is highly desirable to have more frequentforecasts as well as to have a minimum time gap between the fore-cast and the adoption of the budget so that the states can take intoaccount the latest economic conditions as well as the impact ofany legislative andor administrative changes Overall it is alsodesirable to have a consensus revenue estimating practice in thestate so that various parties involved in the forecasting processcan come into an agreement

Other Data

In addition to the NASBO data we used US Census state taxcollection figures in estimating the size of errors across years oracross the type of tax We also used data on national price changes(the gross domestic product price index) gross domestic productby state and state personal income from the US Bureau of Eco-nomic Analysis We also obtained data from several other re-searchers and describe as needed below5 Finally we constructeda measure of forecast difficulty from US Census state tax collec-tion data which we describe in the section Constructing a ForecastDifficulty Measure

How do the revenue estimating data relate to Census tax reve-nue data Not all states rely heavily on the three taxes thatNASBO collects forecast data for Six states relied on these threetaxes for less than half of their tax revenue in the typical yearAlaska Delaware Montana New Hampshire Vermont and Wy-oming (see Table 1)

Measures of Forecasting Error

There are several common ways of measuring forecast errorSome of these measures do not distinguish positive from negativeerrors and are useful for gauging magnitude of error and othersmaintain the sign and are useful for gauging bias and asymme-tries One common measure is the root-mean-squared error(RMSE) or more appropriate when comparing across states theroot-mean-squared percent error (RMSPE)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 5 wwwrockinstorg

One very common measure in the literature examin-ing state revenue forecasting is the absolute value of theerror as a percentage of the actual result (known as abso-lute percentage error) or variants on this (Voorhees 2004)Summary measures of estimating error across states oryears often use the mean or median of the absolute valueof the percentage error (mean absolute percentage error isquite common and is known as the MAPE) These mea-sures which treat positive and negative errors the samemay not be the most appropriate in all situations In par-ticular other measures are needed to examine questionsof whether forecasts are biased (eg more likely to under-estimate revenue than to overestimate) or whether fore-casters believe the costs of overestimates are greater thanthe costs of underestimates

As many observers have noted the revenue estimat-ing error is quite large relative to what policymakersmight find acceptable In 2002 when the previous fiscalcrisis hit sharply twelve states had revenue estimatingerrors of 10 percent or more (Willoughby and Guo 2008)To provide some perspective a 4 percentage point errorin the state of California would be more than $4 billionmdash an amount that policymakers would find quitedisconcerting

We define the forecasting error as actual minus theforecast Thus a positive number is an underestimate ofrevenue (actual revenue is greater than the forecast) anda negative number is an overestimate Forecasters oftenview underestimates as better than overestimates Ourprimary measure of error is the error as a percentage ofthe actual revenue Thus if a forecast of revenue was $90million and the actual revenue was $100 million then

the error was $10 million (an underestimate) and the percentageerror was 10 percent

We use two main versions of this percentage mdash as is whereit can be either positive or negative and the absolute valuewhich is always positive In general the absolute value is usefulwhen we are interested in the concept of accuracy without re-gard to whether revenue is above target or below although theas-is value can be useful too and we use that in much of theanalysis below When we want to examine bias (not a significantfocus of this report) the absolute value measure is not usefulbecause we care about the direction of error and we focus solelyon the as-is value

When we want to summarize error across states years taxesor other groupings we almost always use the median as a mea-sure of central tendency The reason is that our data are fairlynoisy and we donrsquot generally place great faith in any single datapoint although we do believe the data tell truthful stories in ag-gregate Because any single data point might be wrong or an

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 6 wwwrockinstorg

Table 1 Personal Income Sales and Corporate

Income Taxes as Percent of Total Tax Revenue

outlier it could have an undue impact on the mean pulling itaway from what we think is the true central tendency of the data

When we want to measure how much the errors vary acrossstates years taxes or other groupings we use either the standarddeviation or the interquartile range

In the next section we provide simple descriptive statistics oferrors first using the percentage error ldquoas isrdquo and then the abso-lute value There are some useful insights which we summarizein the concluding subsection of this section

Forecasting Errors as Percentage of Actual Revenue

Below we provide summary tables of errors by tax by fiscalyear and by state and explore patterns with several plots

Table 2 displays selected summary statistics for forecastingerrors by tax for 1987-2013 The corporate income tax median fore-casting error at 28 percent was much larger than the median forother taxes The personal income tax median forecasting error was18 percent while the sales tax forecasting error was 03 percentThe standard deviation of the error mdash a measure of variability inforecast error and a potential indication of forecasting difficulty mdashwas largest by far for the corporate income tax nearly three timesas large as for the personal income tax and five times as large asfor the sales tax

Figure 1 on the following page displays the median forecasterror by tax

There is some evidence that forecasting errors are ldquoseriallycorrelatedrdquo mdash that is that a revenue shortfall in one year is morelikely to be followed by a shortfall in the next than by an overageand vice versa Rudolph Penner a former director of the Congres-sional Budget Office concluded this about federal governmentforecasts ldquohellip if CBO makes an error in an overly optimistic orpessimistic direction one year it is highly probable that it willmake an error in the same direction the following yearrdquo (Penner2008) Inspection of NASBO data in time periods surrounding re-cessions shows that at least for the states in aggregate revenueoverestimates tended to be followed by additional overestimates

Figure 2 provides median forecasting errors by fiscal year fortwenty-seven years spanning from 1987 to 2013 As depicted onFigure 2 states normally underestimate during expansions andoverestimate during the downturns Moreover overestimating

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 7 wwwrockinstorg

ofobservations

Meanerror

Standarddeviation

25Median(50)

75

Personal income tax 1100 11 77 (23) 18 55Sales tax 1189 02 46 (20) 03 26Corporate income tax 1058 13 226 (93) 28 140Sum of PIT sales amp CIT 1311 10 76 (22) 15 44

Forecast errors as percentage of actual revenue

Source Rockefeller Institute analysis of NASBO forecast error data

Tax

Table 2 State Revenue Forecasting Percentage Errors by Tax 1987-2013

errors were particu-larly large near the2001 and 2007 reces-sions States now re-turned to theirpattern of underesti-mation after theworst of the fiscalcrisis was behindthem

Table 3 on thefollowing page pro-vides summary sta-tistics of forecastingerrors by state from1987 to 2013 Over-all smaller statesand states depend-ent on a few sectorsof the economy (par-ticularly states reli-ant on oil or naturalgas or gambling)such as AlaskaMontana NevadaNew Hampshireand North Dakotatend to have largererrors Those statesrsquoerrors also tend tobe more variable asmeasured by thestandard deviationor the interquartilerange As alreadymentioned abovepersonal incomesales and corporateincome taxes inAlaska DelawareMontana NewHampshire Ver-mont and Wyoming

represent less than half of total taxes as measured by the CensusBureau Their errors for total taxes in their budget which also in-clude other less volatile taxes appear to be smaller than those ex-amined here We strongly caution not to read too much into theerrors for any single state because the data are noisy and influ-enced by many forces and larger errors need not indicate lesseffective revenue forecasting

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 8 wwwrockinstorg

03

18

28

15

00

05

10

15

20

25

30

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 1 State Revenue Forecast Errors as Percent of Actual Tax Median for 1987-2013

12

10

8

6

4

2

0

2

4

6

8

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal SurveysNote Forecasting error is for the sum of personal income sales and corporate income taxes

1990 91 recession 2001 recession Great recession

Figure 2 Median State Revenue Forecast Errors as Percent of Actual Tax by Fiscal Year

Figure 3 on the following page maps the distributionover time of state forecasting errors for the sum of thethree taxes (personal income tax sales tax and corporateincome tax) and also shows the extent of missing dataWe are missing data for Texas in 1996 1997 and 1999but other than that no large state is missing in any yearIt is difficult to obtain data from state documents that iscomparable to data in the NASBO survey and as notedearlier we were unable to obtain such data with reason-able effort for Texas

The figure shows how forecasting errors have rolledout over time across the country The revenue shortfallsin the middle of the country in 1987 likely are related tothe oil price bust that hit Texas and other oil-extractionstates in the mid-1980s The effects of the 1990 recessionwhich lingered for four years appear to have led to sig-nificant revenue shortfalls on the coasts and in the northbut not as much in the center of the country The 2001recession and the effects of the September 11th attacksare evidenced in revenue shortfalls that began on theeast coast but spread to the rest of the country revenueshortfalls of more than 8 percent were far more commonthan in the 1990 recession The 2005-07 years were agolden period of revenue overages in most of the coun-try The 2007 recession appears to have hit early in thehousing bust states of Arizona Florida and Nevada aswell as Michigan It spread in 2008 and hit the rest of thecountry with force in 2009 and 2010 before state fore-casters were once again on more-solid ground in 2011through 2013

Forecasts may be intentionally biased mdash more likelyto be off in one direction than another mdash because fore-casters believe there is an ldquoasymmetric riskrdquo and that themost accurate estimate is not always the best estimateMany executive-branch forecasters are assumed to be-lieve that the costs of overestimating revenue are greaterthan the costs of underestimating revenue because thepolitical and managerial complications of managing def-icits are believed to be worse than the consequences ofmanaging surpluses These forecasters sometimes ex-plicitly and other times unknowingly may choose con-

servative forecasts of the revenue that is available to financespending By contrast legislative forecasters or their principalsnot generally responsible for financial management might not bebothered in the least by revenue shortfalls that the governor has tomanage but may be troubled to learn of ldquoextrardquo revenue that theycould have spent on their priorities mdash that is they may be morewilling to risk revenue overestimates and revenue shortfalls thanis the executive

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 9 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 01 43 (11) 19 25AK 76 222 (90) 103 152AL 05 61 (13) 15 38AR 21 41 (02) 21 35AZ (04) 97 (16) 14 44CA (03) 74 (46) 25 43CO 09 85 (02) 16 55CT 05 71 (24) 28 53DE 10 68 (28) (01) 56FL (10) 52 (23) (04) 21GA (03) 50 (41) 15 33HI 12 65 (27) 15 49IA 19 37 (03) 18 48ID 19 68 01 15 51IL 07 53 (23) 19 28IN (09) 49 (25) 09 20KS 18 45 (00) 11 20KY (11) 51 (36) 05 26LA 06 66 (38) 17 41MA 12 75 (22) 23 57MD 04 56 (29) 16 40ME 17 46 (01) 14 39MI (17) 52 (42) (10) 23MN 17 51 (06) 18 55MO (02) 54 (13) 11 36MS 10 52 (24) 18 43MT 42 112 (10) 66 93NC 05 59 (12) 25 42ND 93 189 07 30 138NE 10 52 (15) 17 44NH (04) 161 (34) 21 70NJ (07) 56 (36) 10 25NM 07 60 (25) 06 45NV 22 70 (39) 39 71NY (25) 38 (52) (21) 03OH (02) 53 (19) 13 26OK 03 70 (15) 11 43OR 03 100 (17) 21 49PA (02) 37 (19) 02 32RI 03 57 (28) (04) 43SC (06) 86 (51) 14 46SD 11 25 (02) 08 24TN 03 45 (14) 06 34TX 33 64 00 26 67UT 15 61 02 29 53VA (01) 67 (23) 15 26VT 29 63 (08) 42 75WA 03 70 (10) 13 33WI 02 33 (00) 09 18WV 16 34 (11) 16 32WY 34 71 03 20 71Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 3 Forecast Errors as

Percent of Actual Tax by State

Figure 4 on the following page displays boxplots of eachstatersquos percentage forecasting error for the sum of personal in-come tax sales tax and corporate income tax The top horizontalline of each box marks the 75th percentile of forecast errors andthe lower horizontal line marks the 25th percentile Each box alsohas vertical lines (sometimes called ldquowhiskersrdquo) extending fromits top and bottom indicating variability in the top and bottom 25percent of the data (longer lines indicate more variability in thispart of the data) and dots marking extreme outliers6 The tallerthe box (the greater the difference between the 75th and 25th per-centiles) and the longer the lines the greater the variability inforecast errors for a given state

Only five states have a negative median forecasting error sug-gesting again that states may be ldquotryingrdquo to have small positiveforecasting errors mdash underforecasts of revenue However in acommon formal but rather strict test of unbiasedness known asthe Mincer-Zarnowitz test we reject unbiasedness at the 5 percentlevel for only eight states Arkansas Iowa Montana New YorkNorth Dakota Texas Vermont and Wyoming7 For each of thesestates virtually all of the box in Figure 4 is above the line (aboutthree quarters of the errors are positive) or else below the line (inthe case of New York)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 10 wwwrockinstorg

Figure 3 Percentage Forecast Errors for Sum of PIT CIT and Sales Taxes 1987-2013

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

The main purpose of the second round was to get a better un-derstanding of general procedures and practices of revenue esti-mating processes in the states The survey questions weretargeted at getting more information about the timing of forecastsfrequency of forecast updates the long-term forecasting practicesparties involved in the forecasting processes and whether there isa formal group that plays a major role in the revenue estimatingprocess

Both surveys were conducted online using Google Docs Thefirst round was conducted during July-August of 2013 while thesecond round was conducted during August-September of 2013

For the first round of the survey we sent an online surveyquestionnaire to state budget officers in all fifty states who werealso the respondents for the NASBOrsquos survey The list of respon-dents was retrieved from the Fiscal Survey of the States report Theresponse rate for the first round of the survey was 68 percent (34usable responses were received)

For the second round of the survey we sent an online surveyquestionnaire to both legislative and executive branch officials inall fifty states who were likely to be involved in the forecastingprocess The response rate for this round was much higher 90percent (forty-five responses) for the executive branch and 92 per-cent (forty-six responses) for the legislative branch In aggregate(executive and legislative branches combined) we have receivedresponses from all fifty states

The responses from the first round indicate that in most statesthe estimates provided to NASBO are the final estimates used forthe adopted budget States vary widely in how far in advance ofthe budget they prepare their estimates Some states prepare reve-nue estimates as early as thirty-five weeks ahead of the start of thefiscal year (eg Alabama) while other states use revenue esti-mates updated about two weeks before the start of the state fiscalyear (ie Delaware Pennsylvania and Tennessee) The survey re-sults also revealed that in general states with a biennial budgetcycle provide updated estimates to NASBO for the second year ofthe biennium in order to account for legislative changes and up-dated economic forecast conditions Nonetheless on average esti-mates for the second year of a biennium are prepared further inadvance of the second year than are estimates for the first year

The responses from the second round indicate wide variationamong the states in the timing of the forecasts prepared for theadopted budget as well as in frequency of revenue estimate up-dates In terms of long-term forecasting states indicated that theytypically generate revenue estimates for one or more years be-yond the upcoming budget period most states release revenue es-timates for two to five years beyond the budget period

The state officials were also asked to indicate who is involvedin the revenue estimating process The survey results indicate thatin most states one or more agencies of the executive branch are in-volved in the revenue estimating process and in a handful of

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 4 wwwrockinstorg

states the legislative branch also plays a major role in revenue esti-mating However only very few states also involve academiciansandor experts from the private sector in the revenue estimatingprocess Finally in a few states the revenue estimating process isunder the jurisdiction of the nonpartisan independent groupagency the members of which are normally appointed by thegovernor andor legislative branch According to the survey re-sponses in approximately two-thirds of the states the participantsin the revenue estimating process come to an agreement on asingle number

In general the survey results indicate that there is a widerange of practices and processes involved among the fifty states interms of revenue estimates There are no universally agreed uponstandards and practices among the fifty states The survey re-sponses as well as various discussions with the state governmentofficials indicate that it is highly desirable to have more frequentforecasts as well as to have a minimum time gap between the fore-cast and the adoption of the budget so that the states can take intoaccount the latest economic conditions as well as the impact ofany legislative andor administrative changes Overall it is alsodesirable to have a consensus revenue estimating practice in thestate so that various parties involved in the forecasting processcan come into an agreement

Other Data

In addition to the NASBO data we used US Census state taxcollection figures in estimating the size of errors across years oracross the type of tax We also used data on national price changes(the gross domestic product price index) gross domestic productby state and state personal income from the US Bureau of Eco-nomic Analysis We also obtained data from several other re-searchers and describe as needed below5 Finally we constructeda measure of forecast difficulty from US Census state tax collec-tion data which we describe in the section Constructing a ForecastDifficulty Measure

How do the revenue estimating data relate to Census tax reve-nue data Not all states rely heavily on the three taxes thatNASBO collects forecast data for Six states relied on these threetaxes for less than half of their tax revenue in the typical yearAlaska Delaware Montana New Hampshire Vermont and Wy-oming (see Table 1)

Measures of Forecasting Error

There are several common ways of measuring forecast errorSome of these measures do not distinguish positive from negativeerrors and are useful for gauging magnitude of error and othersmaintain the sign and are useful for gauging bias and asymme-tries One common measure is the root-mean-squared error(RMSE) or more appropriate when comparing across states theroot-mean-squared percent error (RMSPE)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 5 wwwrockinstorg

One very common measure in the literature examin-ing state revenue forecasting is the absolute value of theerror as a percentage of the actual result (known as abso-lute percentage error) or variants on this (Voorhees 2004)Summary measures of estimating error across states oryears often use the mean or median of the absolute valueof the percentage error (mean absolute percentage error isquite common and is known as the MAPE) These mea-sures which treat positive and negative errors the samemay not be the most appropriate in all situations In par-ticular other measures are needed to examine questionsof whether forecasts are biased (eg more likely to under-estimate revenue than to overestimate) or whether fore-casters believe the costs of overestimates are greater thanthe costs of underestimates

As many observers have noted the revenue estimat-ing error is quite large relative to what policymakersmight find acceptable In 2002 when the previous fiscalcrisis hit sharply twelve states had revenue estimatingerrors of 10 percent or more (Willoughby and Guo 2008)To provide some perspective a 4 percentage point errorin the state of California would be more than $4 billionmdash an amount that policymakers would find quitedisconcerting

We define the forecasting error as actual minus theforecast Thus a positive number is an underestimate ofrevenue (actual revenue is greater than the forecast) anda negative number is an overestimate Forecasters oftenview underestimates as better than overestimates Ourprimary measure of error is the error as a percentage ofthe actual revenue Thus if a forecast of revenue was $90million and the actual revenue was $100 million then

the error was $10 million (an underestimate) and the percentageerror was 10 percent

We use two main versions of this percentage mdash as is whereit can be either positive or negative and the absolute valuewhich is always positive In general the absolute value is usefulwhen we are interested in the concept of accuracy without re-gard to whether revenue is above target or below although theas-is value can be useful too and we use that in much of theanalysis below When we want to examine bias (not a significantfocus of this report) the absolute value measure is not usefulbecause we care about the direction of error and we focus solelyon the as-is value

When we want to summarize error across states years taxesor other groupings we almost always use the median as a mea-sure of central tendency The reason is that our data are fairlynoisy and we donrsquot generally place great faith in any single datapoint although we do believe the data tell truthful stories in ag-gregate Because any single data point might be wrong or an

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 6 wwwrockinstorg

Table 1 Personal Income Sales and Corporate

Income Taxes as Percent of Total Tax Revenue

outlier it could have an undue impact on the mean pulling itaway from what we think is the true central tendency of the data

When we want to measure how much the errors vary acrossstates years taxes or other groupings we use either the standarddeviation or the interquartile range

In the next section we provide simple descriptive statistics oferrors first using the percentage error ldquoas isrdquo and then the abso-lute value There are some useful insights which we summarizein the concluding subsection of this section

Forecasting Errors as Percentage of Actual Revenue

Below we provide summary tables of errors by tax by fiscalyear and by state and explore patterns with several plots

Table 2 displays selected summary statistics for forecastingerrors by tax for 1987-2013 The corporate income tax median fore-casting error at 28 percent was much larger than the median forother taxes The personal income tax median forecasting error was18 percent while the sales tax forecasting error was 03 percentThe standard deviation of the error mdash a measure of variability inforecast error and a potential indication of forecasting difficulty mdashwas largest by far for the corporate income tax nearly three timesas large as for the personal income tax and five times as large asfor the sales tax

Figure 1 on the following page displays the median forecasterror by tax

There is some evidence that forecasting errors are ldquoseriallycorrelatedrdquo mdash that is that a revenue shortfall in one year is morelikely to be followed by a shortfall in the next than by an overageand vice versa Rudolph Penner a former director of the Congres-sional Budget Office concluded this about federal governmentforecasts ldquohellip if CBO makes an error in an overly optimistic orpessimistic direction one year it is highly probable that it willmake an error in the same direction the following yearrdquo (Penner2008) Inspection of NASBO data in time periods surrounding re-cessions shows that at least for the states in aggregate revenueoverestimates tended to be followed by additional overestimates

Figure 2 provides median forecasting errors by fiscal year fortwenty-seven years spanning from 1987 to 2013 As depicted onFigure 2 states normally underestimate during expansions andoverestimate during the downturns Moreover overestimating

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 7 wwwrockinstorg

ofobservations

Meanerror

Standarddeviation

25Median(50)

75

Personal income tax 1100 11 77 (23) 18 55Sales tax 1189 02 46 (20) 03 26Corporate income tax 1058 13 226 (93) 28 140Sum of PIT sales amp CIT 1311 10 76 (22) 15 44

Forecast errors as percentage of actual revenue

Source Rockefeller Institute analysis of NASBO forecast error data

Tax

Table 2 State Revenue Forecasting Percentage Errors by Tax 1987-2013

errors were particu-larly large near the2001 and 2007 reces-sions States now re-turned to theirpattern of underesti-mation after theworst of the fiscalcrisis was behindthem

Table 3 on thefollowing page pro-vides summary sta-tistics of forecastingerrors by state from1987 to 2013 Over-all smaller statesand states depend-ent on a few sectorsof the economy (par-ticularly states reli-ant on oil or naturalgas or gambling)such as AlaskaMontana NevadaNew Hampshireand North Dakotatend to have largererrors Those statesrsquoerrors also tend tobe more variable asmeasured by thestandard deviationor the interquartilerange As alreadymentioned abovepersonal incomesales and corporateincome taxes inAlaska DelawareMontana NewHampshire Ver-mont and Wyoming

represent less than half of total taxes as measured by the CensusBureau Their errors for total taxes in their budget which also in-clude other less volatile taxes appear to be smaller than those ex-amined here We strongly caution not to read too much into theerrors for any single state because the data are noisy and influ-enced by many forces and larger errors need not indicate lesseffective revenue forecasting

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 8 wwwrockinstorg

03

18

28

15

00

05

10

15

20

25

30

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 1 State Revenue Forecast Errors as Percent of Actual Tax Median for 1987-2013

12

10

8

6

4

2

0

2

4

6

8

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal SurveysNote Forecasting error is for the sum of personal income sales and corporate income taxes

1990 91 recession 2001 recession Great recession

Figure 2 Median State Revenue Forecast Errors as Percent of Actual Tax by Fiscal Year

Figure 3 on the following page maps the distributionover time of state forecasting errors for the sum of thethree taxes (personal income tax sales tax and corporateincome tax) and also shows the extent of missing dataWe are missing data for Texas in 1996 1997 and 1999but other than that no large state is missing in any yearIt is difficult to obtain data from state documents that iscomparable to data in the NASBO survey and as notedearlier we were unable to obtain such data with reason-able effort for Texas

The figure shows how forecasting errors have rolledout over time across the country The revenue shortfallsin the middle of the country in 1987 likely are related tothe oil price bust that hit Texas and other oil-extractionstates in the mid-1980s The effects of the 1990 recessionwhich lingered for four years appear to have led to sig-nificant revenue shortfalls on the coasts and in the northbut not as much in the center of the country The 2001recession and the effects of the September 11th attacksare evidenced in revenue shortfalls that began on theeast coast but spread to the rest of the country revenueshortfalls of more than 8 percent were far more commonthan in the 1990 recession The 2005-07 years were agolden period of revenue overages in most of the coun-try The 2007 recession appears to have hit early in thehousing bust states of Arizona Florida and Nevada aswell as Michigan It spread in 2008 and hit the rest of thecountry with force in 2009 and 2010 before state fore-casters were once again on more-solid ground in 2011through 2013

Forecasts may be intentionally biased mdash more likelyto be off in one direction than another mdash because fore-casters believe there is an ldquoasymmetric riskrdquo and that themost accurate estimate is not always the best estimateMany executive-branch forecasters are assumed to be-lieve that the costs of overestimating revenue are greaterthan the costs of underestimating revenue because thepolitical and managerial complications of managing def-icits are believed to be worse than the consequences ofmanaging surpluses These forecasters sometimes ex-plicitly and other times unknowingly may choose con-

servative forecasts of the revenue that is available to financespending By contrast legislative forecasters or their principalsnot generally responsible for financial management might not bebothered in the least by revenue shortfalls that the governor has tomanage but may be troubled to learn of ldquoextrardquo revenue that theycould have spent on their priorities mdash that is they may be morewilling to risk revenue overestimates and revenue shortfalls thanis the executive

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 9 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 01 43 (11) 19 25AK 76 222 (90) 103 152AL 05 61 (13) 15 38AR 21 41 (02) 21 35AZ (04) 97 (16) 14 44CA (03) 74 (46) 25 43CO 09 85 (02) 16 55CT 05 71 (24) 28 53DE 10 68 (28) (01) 56FL (10) 52 (23) (04) 21GA (03) 50 (41) 15 33HI 12 65 (27) 15 49IA 19 37 (03) 18 48ID 19 68 01 15 51IL 07 53 (23) 19 28IN (09) 49 (25) 09 20KS 18 45 (00) 11 20KY (11) 51 (36) 05 26LA 06 66 (38) 17 41MA 12 75 (22) 23 57MD 04 56 (29) 16 40ME 17 46 (01) 14 39MI (17) 52 (42) (10) 23MN 17 51 (06) 18 55MO (02) 54 (13) 11 36MS 10 52 (24) 18 43MT 42 112 (10) 66 93NC 05 59 (12) 25 42ND 93 189 07 30 138NE 10 52 (15) 17 44NH (04) 161 (34) 21 70NJ (07) 56 (36) 10 25NM 07 60 (25) 06 45NV 22 70 (39) 39 71NY (25) 38 (52) (21) 03OH (02) 53 (19) 13 26OK 03 70 (15) 11 43OR 03 100 (17) 21 49PA (02) 37 (19) 02 32RI 03 57 (28) (04) 43SC (06) 86 (51) 14 46SD 11 25 (02) 08 24TN 03 45 (14) 06 34TX 33 64 00 26 67UT 15 61 02 29 53VA (01) 67 (23) 15 26VT 29 63 (08) 42 75WA 03 70 (10) 13 33WI 02 33 (00) 09 18WV 16 34 (11) 16 32WY 34 71 03 20 71Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 3 Forecast Errors as

Percent of Actual Tax by State

Figure 4 on the following page displays boxplots of eachstatersquos percentage forecasting error for the sum of personal in-come tax sales tax and corporate income tax The top horizontalline of each box marks the 75th percentile of forecast errors andthe lower horizontal line marks the 25th percentile Each box alsohas vertical lines (sometimes called ldquowhiskersrdquo) extending fromits top and bottom indicating variability in the top and bottom 25percent of the data (longer lines indicate more variability in thispart of the data) and dots marking extreme outliers6 The tallerthe box (the greater the difference between the 75th and 25th per-centiles) and the longer the lines the greater the variability inforecast errors for a given state

Only five states have a negative median forecasting error sug-gesting again that states may be ldquotryingrdquo to have small positiveforecasting errors mdash underforecasts of revenue However in acommon formal but rather strict test of unbiasedness known asthe Mincer-Zarnowitz test we reject unbiasedness at the 5 percentlevel for only eight states Arkansas Iowa Montana New YorkNorth Dakota Texas Vermont and Wyoming7 For each of thesestates virtually all of the box in Figure 4 is above the line (aboutthree quarters of the errors are positive) or else below the line (inthe case of New York)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 10 wwwrockinstorg

Figure 3 Percentage Forecast Errors for Sum of PIT CIT and Sales Taxes 1987-2013

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

states the legislative branch also plays a major role in revenue esti-mating However only very few states also involve academiciansandor experts from the private sector in the revenue estimatingprocess Finally in a few states the revenue estimating process isunder the jurisdiction of the nonpartisan independent groupagency the members of which are normally appointed by thegovernor andor legislative branch According to the survey re-sponses in approximately two-thirds of the states the participantsin the revenue estimating process come to an agreement on asingle number

In general the survey results indicate that there is a widerange of practices and processes involved among the fifty states interms of revenue estimates There are no universally agreed uponstandards and practices among the fifty states The survey re-sponses as well as various discussions with the state governmentofficials indicate that it is highly desirable to have more frequentforecasts as well as to have a minimum time gap between the fore-cast and the adoption of the budget so that the states can take intoaccount the latest economic conditions as well as the impact ofany legislative andor administrative changes Overall it is alsodesirable to have a consensus revenue estimating practice in thestate so that various parties involved in the forecasting processcan come into an agreement

Other Data

In addition to the NASBO data we used US Census state taxcollection figures in estimating the size of errors across years oracross the type of tax We also used data on national price changes(the gross domestic product price index) gross domestic productby state and state personal income from the US Bureau of Eco-nomic Analysis We also obtained data from several other re-searchers and describe as needed below5 Finally we constructeda measure of forecast difficulty from US Census state tax collec-tion data which we describe in the section Constructing a ForecastDifficulty Measure

How do the revenue estimating data relate to Census tax reve-nue data Not all states rely heavily on the three taxes thatNASBO collects forecast data for Six states relied on these threetaxes for less than half of their tax revenue in the typical yearAlaska Delaware Montana New Hampshire Vermont and Wy-oming (see Table 1)

Measures of Forecasting Error

There are several common ways of measuring forecast errorSome of these measures do not distinguish positive from negativeerrors and are useful for gauging magnitude of error and othersmaintain the sign and are useful for gauging bias and asymme-tries One common measure is the root-mean-squared error(RMSE) or more appropriate when comparing across states theroot-mean-squared percent error (RMSPE)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 5 wwwrockinstorg

One very common measure in the literature examin-ing state revenue forecasting is the absolute value of theerror as a percentage of the actual result (known as abso-lute percentage error) or variants on this (Voorhees 2004)Summary measures of estimating error across states oryears often use the mean or median of the absolute valueof the percentage error (mean absolute percentage error isquite common and is known as the MAPE) These mea-sures which treat positive and negative errors the samemay not be the most appropriate in all situations In par-ticular other measures are needed to examine questionsof whether forecasts are biased (eg more likely to under-estimate revenue than to overestimate) or whether fore-casters believe the costs of overestimates are greater thanthe costs of underestimates

As many observers have noted the revenue estimat-ing error is quite large relative to what policymakersmight find acceptable In 2002 when the previous fiscalcrisis hit sharply twelve states had revenue estimatingerrors of 10 percent or more (Willoughby and Guo 2008)To provide some perspective a 4 percentage point errorin the state of California would be more than $4 billionmdash an amount that policymakers would find quitedisconcerting

We define the forecasting error as actual minus theforecast Thus a positive number is an underestimate ofrevenue (actual revenue is greater than the forecast) anda negative number is an overestimate Forecasters oftenview underestimates as better than overestimates Ourprimary measure of error is the error as a percentage ofthe actual revenue Thus if a forecast of revenue was $90million and the actual revenue was $100 million then

the error was $10 million (an underestimate) and the percentageerror was 10 percent

We use two main versions of this percentage mdash as is whereit can be either positive or negative and the absolute valuewhich is always positive In general the absolute value is usefulwhen we are interested in the concept of accuracy without re-gard to whether revenue is above target or below although theas-is value can be useful too and we use that in much of theanalysis below When we want to examine bias (not a significantfocus of this report) the absolute value measure is not usefulbecause we care about the direction of error and we focus solelyon the as-is value

When we want to summarize error across states years taxesor other groupings we almost always use the median as a mea-sure of central tendency The reason is that our data are fairlynoisy and we donrsquot generally place great faith in any single datapoint although we do believe the data tell truthful stories in ag-gregate Because any single data point might be wrong or an

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 6 wwwrockinstorg

Table 1 Personal Income Sales and Corporate

Income Taxes as Percent of Total Tax Revenue

outlier it could have an undue impact on the mean pulling itaway from what we think is the true central tendency of the data

When we want to measure how much the errors vary acrossstates years taxes or other groupings we use either the standarddeviation or the interquartile range

In the next section we provide simple descriptive statistics oferrors first using the percentage error ldquoas isrdquo and then the abso-lute value There are some useful insights which we summarizein the concluding subsection of this section

Forecasting Errors as Percentage of Actual Revenue

Below we provide summary tables of errors by tax by fiscalyear and by state and explore patterns with several plots

Table 2 displays selected summary statistics for forecastingerrors by tax for 1987-2013 The corporate income tax median fore-casting error at 28 percent was much larger than the median forother taxes The personal income tax median forecasting error was18 percent while the sales tax forecasting error was 03 percentThe standard deviation of the error mdash a measure of variability inforecast error and a potential indication of forecasting difficulty mdashwas largest by far for the corporate income tax nearly three timesas large as for the personal income tax and five times as large asfor the sales tax

Figure 1 on the following page displays the median forecasterror by tax

There is some evidence that forecasting errors are ldquoseriallycorrelatedrdquo mdash that is that a revenue shortfall in one year is morelikely to be followed by a shortfall in the next than by an overageand vice versa Rudolph Penner a former director of the Congres-sional Budget Office concluded this about federal governmentforecasts ldquohellip if CBO makes an error in an overly optimistic orpessimistic direction one year it is highly probable that it willmake an error in the same direction the following yearrdquo (Penner2008) Inspection of NASBO data in time periods surrounding re-cessions shows that at least for the states in aggregate revenueoverestimates tended to be followed by additional overestimates

Figure 2 provides median forecasting errors by fiscal year fortwenty-seven years spanning from 1987 to 2013 As depicted onFigure 2 states normally underestimate during expansions andoverestimate during the downturns Moreover overestimating

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 7 wwwrockinstorg

ofobservations

Meanerror

Standarddeviation

25Median(50)

75

Personal income tax 1100 11 77 (23) 18 55Sales tax 1189 02 46 (20) 03 26Corporate income tax 1058 13 226 (93) 28 140Sum of PIT sales amp CIT 1311 10 76 (22) 15 44

Forecast errors as percentage of actual revenue

Source Rockefeller Institute analysis of NASBO forecast error data

Tax

Table 2 State Revenue Forecasting Percentage Errors by Tax 1987-2013

errors were particu-larly large near the2001 and 2007 reces-sions States now re-turned to theirpattern of underesti-mation after theworst of the fiscalcrisis was behindthem

Table 3 on thefollowing page pro-vides summary sta-tistics of forecastingerrors by state from1987 to 2013 Over-all smaller statesand states depend-ent on a few sectorsof the economy (par-ticularly states reli-ant on oil or naturalgas or gambling)such as AlaskaMontana NevadaNew Hampshireand North Dakotatend to have largererrors Those statesrsquoerrors also tend tobe more variable asmeasured by thestandard deviationor the interquartilerange As alreadymentioned abovepersonal incomesales and corporateincome taxes inAlaska DelawareMontana NewHampshire Ver-mont and Wyoming

represent less than half of total taxes as measured by the CensusBureau Their errors for total taxes in their budget which also in-clude other less volatile taxes appear to be smaller than those ex-amined here We strongly caution not to read too much into theerrors for any single state because the data are noisy and influ-enced by many forces and larger errors need not indicate lesseffective revenue forecasting

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 8 wwwrockinstorg

03

18

28

15

00

05

10

15

20

25

30

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 1 State Revenue Forecast Errors as Percent of Actual Tax Median for 1987-2013

12

10

8

6

4

2

0

2

4

6

8

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal SurveysNote Forecasting error is for the sum of personal income sales and corporate income taxes

1990 91 recession 2001 recession Great recession

Figure 2 Median State Revenue Forecast Errors as Percent of Actual Tax by Fiscal Year

Figure 3 on the following page maps the distributionover time of state forecasting errors for the sum of thethree taxes (personal income tax sales tax and corporateincome tax) and also shows the extent of missing dataWe are missing data for Texas in 1996 1997 and 1999but other than that no large state is missing in any yearIt is difficult to obtain data from state documents that iscomparable to data in the NASBO survey and as notedearlier we were unable to obtain such data with reason-able effort for Texas

The figure shows how forecasting errors have rolledout over time across the country The revenue shortfallsin the middle of the country in 1987 likely are related tothe oil price bust that hit Texas and other oil-extractionstates in the mid-1980s The effects of the 1990 recessionwhich lingered for four years appear to have led to sig-nificant revenue shortfalls on the coasts and in the northbut not as much in the center of the country The 2001recession and the effects of the September 11th attacksare evidenced in revenue shortfalls that began on theeast coast but spread to the rest of the country revenueshortfalls of more than 8 percent were far more commonthan in the 1990 recession The 2005-07 years were agolden period of revenue overages in most of the coun-try The 2007 recession appears to have hit early in thehousing bust states of Arizona Florida and Nevada aswell as Michigan It spread in 2008 and hit the rest of thecountry with force in 2009 and 2010 before state fore-casters were once again on more-solid ground in 2011through 2013

Forecasts may be intentionally biased mdash more likelyto be off in one direction than another mdash because fore-casters believe there is an ldquoasymmetric riskrdquo and that themost accurate estimate is not always the best estimateMany executive-branch forecasters are assumed to be-lieve that the costs of overestimating revenue are greaterthan the costs of underestimating revenue because thepolitical and managerial complications of managing def-icits are believed to be worse than the consequences ofmanaging surpluses These forecasters sometimes ex-plicitly and other times unknowingly may choose con-

servative forecasts of the revenue that is available to financespending By contrast legislative forecasters or their principalsnot generally responsible for financial management might not bebothered in the least by revenue shortfalls that the governor has tomanage but may be troubled to learn of ldquoextrardquo revenue that theycould have spent on their priorities mdash that is they may be morewilling to risk revenue overestimates and revenue shortfalls thanis the executive

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 9 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 01 43 (11) 19 25AK 76 222 (90) 103 152AL 05 61 (13) 15 38AR 21 41 (02) 21 35AZ (04) 97 (16) 14 44CA (03) 74 (46) 25 43CO 09 85 (02) 16 55CT 05 71 (24) 28 53DE 10 68 (28) (01) 56FL (10) 52 (23) (04) 21GA (03) 50 (41) 15 33HI 12 65 (27) 15 49IA 19 37 (03) 18 48ID 19 68 01 15 51IL 07 53 (23) 19 28IN (09) 49 (25) 09 20KS 18 45 (00) 11 20KY (11) 51 (36) 05 26LA 06 66 (38) 17 41MA 12 75 (22) 23 57MD 04 56 (29) 16 40ME 17 46 (01) 14 39MI (17) 52 (42) (10) 23MN 17 51 (06) 18 55MO (02) 54 (13) 11 36MS 10 52 (24) 18 43MT 42 112 (10) 66 93NC 05 59 (12) 25 42ND 93 189 07 30 138NE 10 52 (15) 17 44NH (04) 161 (34) 21 70NJ (07) 56 (36) 10 25NM 07 60 (25) 06 45NV 22 70 (39) 39 71NY (25) 38 (52) (21) 03OH (02) 53 (19) 13 26OK 03 70 (15) 11 43OR 03 100 (17) 21 49PA (02) 37 (19) 02 32RI 03 57 (28) (04) 43SC (06) 86 (51) 14 46SD 11 25 (02) 08 24TN 03 45 (14) 06 34TX 33 64 00 26 67UT 15 61 02 29 53VA (01) 67 (23) 15 26VT 29 63 (08) 42 75WA 03 70 (10) 13 33WI 02 33 (00) 09 18WV 16 34 (11) 16 32WY 34 71 03 20 71Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 3 Forecast Errors as

Percent of Actual Tax by State

Figure 4 on the following page displays boxplots of eachstatersquos percentage forecasting error for the sum of personal in-come tax sales tax and corporate income tax The top horizontalline of each box marks the 75th percentile of forecast errors andthe lower horizontal line marks the 25th percentile Each box alsohas vertical lines (sometimes called ldquowhiskersrdquo) extending fromits top and bottom indicating variability in the top and bottom 25percent of the data (longer lines indicate more variability in thispart of the data) and dots marking extreme outliers6 The tallerthe box (the greater the difference between the 75th and 25th per-centiles) and the longer the lines the greater the variability inforecast errors for a given state

Only five states have a negative median forecasting error sug-gesting again that states may be ldquotryingrdquo to have small positiveforecasting errors mdash underforecasts of revenue However in acommon formal but rather strict test of unbiasedness known asthe Mincer-Zarnowitz test we reject unbiasedness at the 5 percentlevel for only eight states Arkansas Iowa Montana New YorkNorth Dakota Texas Vermont and Wyoming7 For each of thesestates virtually all of the box in Figure 4 is above the line (aboutthree quarters of the errors are positive) or else below the line (inthe case of New York)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 10 wwwrockinstorg

Figure 3 Percentage Forecast Errors for Sum of PIT CIT and Sales Taxes 1987-2013

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

One very common measure in the literature examin-ing state revenue forecasting is the absolute value of theerror as a percentage of the actual result (known as abso-lute percentage error) or variants on this (Voorhees 2004)Summary measures of estimating error across states oryears often use the mean or median of the absolute valueof the percentage error (mean absolute percentage error isquite common and is known as the MAPE) These mea-sures which treat positive and negative errors the samemay not be the most appropriate in all situations In par-ticular other measures are needed to examine questionsof whether forecasts are biased (eg more likely to under-estimate revenue than to overestimate) or whether fore-casters believe the costs of overestimates are greater thanthe costs of underestimates

As many observers have noted the revenue estimat-ing error is quite large relative to what policymakersmight find acceptable In 2002 when the previous fiscalcrisis hit sharply twelve states had revenue estimatingerrors of 10 percent or more (Willoughby and Guo 2008)To provide some perspective a 4 percentage point errorin the state of California would be more than $4 billionmdash an amount that policymakers would find quitedisconcerting

We define the forecasting error as actual minus theforecast Thus a positive number is an underestimate ofrevenue (actual revenue is greater than the forecast) anda negative number is an overestimate Forecasters oftenview underestimates as better than overestimates Ourprimary measure of error is the error as a percentage ofthe actual revenue Thus if a forecast of revenue was $90million and the actual revenue was $100 million then

the error was $10 million (an underestimate) and the percentageerror was 10 percent

We use two main versions of this percentage mdash as is whereit can be either positive or negative and the absolute valuewhich is always positive In general the absolute value is usefulwhen we are interested in the concept of accuracy without re-gard to whether revenue is above target or below although theas-is value can be useful too and we use that in much of theanalysis below When we want to examine bias (not a significantfocus of this report) the absolute value measure is not usefulbecause we care about the direction of error and we focus solelyon the as-is value

When we want to summarize error across states years taxesor other groupings we almost always use the median as a mea-sure of central tendency The reason is that our data are fairlynoisy and we donrsquot generally place great faith in any single datapoint although we do believe the data tell truthful stories in ag-gregate Because any single data point might be wrong or an

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 6 wwwrockinstorg

Table 1 Personal Income Sales and Corporate

Income Taxes as Percent of Total Tax Revenue

outlier it could have an undue impact on the mean pulling itaway from what we think is the true central tendency of the data

When we want to measure how much the errors vary acrossstates years taxes or other groupings we use either the standarddeviation or the interquartile range

In the next section we provide simple descriptive statistics oferrors first using the percentage error ldquoas isrdquo and then the abso-lute value There are some useful insights which we summarizein the concluding subsection of this section

Forecasting Errors as Percentage of Actual Revenue

Below we provide summary tables of errors by tax by fiscalyear and by state and explore patterns with several plots

Table 2 displays selected summary statistics for forecastingerrors by tax for 1987-2013 The corporate income tax median fore-casting error at 28 percent was much larger than the median forother taxes The personal income tax median forecasting error was18 percent while the sales tax forecasting error was 03 percentThe standard deviation of the error mdash a measure of variability inforecast error and a potential indication of forecasting difficulty mdashwas largest by far for the corporate income tax nearly three timesas large as for the personal income tax and five times as large asfor the sales tax

Figure 1 on the following page displays the median forecasterror by tax

There is some evidence that forecasting errors are ldquoseriallycorrelatedrdquo mdash that is that a revenue shortfall in one year is morelikely to be followed by a shortfall in the next than by an overageand vice versa Rudolph Penner a former director of the Congres-sional Budget Office concluded this about federal governmentforecasts ldquohellip if CBO makes an error in an overly optimistic orpessimistic direction one year it is highly probable that it willmake an error in the same direction the following yearrdquo (Penner2008) Inspection of NASBO data in time periods surrounding re-cessions shows that at least for the states in aggregate revenueoverestimates tended to be followed by additional overestimates

Figure 2 provides median forecasting errors by fiscal year fortwenty-seven years spanning from 1987 to 2013 As depicted onFigure 2 states normally underestimate during expansions andoverestimate during the downturns Moreover overestimating

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 7 wwwrockinstorg

ofobservations

Meanerror

Standarddeviation

25Median(50)

75

Personal income tax 1100 11 77 (23) 18 55Sales tax 1189 02 46 (20) 03 26Corporate income tax 1058 13 226 (93) 28 140Sum of PIT sales amp CIT 1311 10 76 (22) 15 44

Forecast errors as percentage of actual revenue

Source Rockefeller Institute analysis of NASBO forecast error data

Tax

Table 2 State Revenue Forecasting Percentage Errors by Tax 1987-2013

errors were particu-larly large near the2001 and 2007 reces-sions States now re-turned to theirpattern of underesti-mation after theworst of the fiscalcrisis was behindthem

Table 3 on thefollowing page pro-vides summary sta-tistics of forecastingerrors by state from1987 to 2013 Over-all smaller statesand states depend-ent on a few sectorsof the economy (par-ticularly states reli-ant on oil or naturalgas or gambling)such as AlaskaMontana NevadaNew Hampshireand North Dakotatend to have largererrors Those statesrsquoerrors also tend tobe more variable asmeasured by thestandard deviationor the interquartilerange As alreadymentioned abovepersonal incomesales and corporateincome taxes inAlaska DelawareMontana NewHampshire Ver-mont and Wyoming

represent less than half of total taxes as measured by the CensusBureau Their errors for total taxes in their budget which also in-clude other less volatile taxes appear to be smaller than those ex-amined here We strongly caution not to read too much into theerrors for any single state because the data are noisy and influ-enced by many forces and larger errors need not indicate lesseffective revenue forecasting

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 8 wwwrockinstorg

03

18

28

15

00

05

10

15

20

25

30

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 1 State Revenue Forecast Errors as Percent of Actual Tax Median for 1987-2013

12

10

8

6

4

2

0

2

4

6

8

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal SurveysNote Forecasting error is for the sum of personal income sales and corporate income taxes

1990 91 recession 2001 recession Great recession

Figure 2 Median State Revenue Forecast Errors as Percent of Actual Tax by Fiscal Year

Figure 3 on the following page maps the distributionover time of state forecasting errors for the sum of thethree taxes (personal income tax sales tax and corporateincome tax) and also shows the extent of missing dataWe are missing data for Texas in 1996 1997 and 1999but other than that no large state is missing in any yearIt is difficult to obtain data from state documents that iscomparable to data in the NASBO survey and as notedearlier we were unable to obtain such data with reason-able effort for Texas

The figure shows how forecasting errors have rolledout over time across the country The revenue shortfallsin the middle of the country in 1987 likely are related tothe oil price bust that hit Texas and other oil-extractionstates in the mid-1980s The effects of the 1990 recessionwhich lingered for four years appear to have led to sig-nificant revenue shortfalls on the coasts and in the northbut not as much in the center of the country The 2001recession and the effects of the September 11th attacksare evidenced in revenue shortfalls that began on theeast coast but spread to the rest of the country revenueshortfalls of more than 8 percent were far more commonthan in the 1990 recession The 2005-07 years were agolden period of revenue overages in most of the coun-try The 2007 recession appears to have hit early in thehousing bust states of Arizona Florida and Nevada aswell as Michigan It spread in 2008 and hit the rest of thecountry with force in 2009 and 2010 before state fore-casters were once again on more-solid ground in 2011through 2013

Forecasts may be intentionally biased mdash more likelyto be off in one direction than another mdash because fore-casters believe there is an ldquoasymmetric riskrdquo and that themost accurate estimate is not always the best estimateMany executive-branch forecasters are assumed to be-lieve that the costs of overestimating revenue are greaterthan the costs of underestimating revenue because thepolitical and managerial complications of managing def-icits are believed to be worse than the consequences ofmanaging surpluses These forecasters sometimes ex-plicitly and other times unknowingly may choose con-

servative forecasts of the revenue that is available to financespending By contrast legislative forecasters or their principalsnot generally responsible for financial management might not bebothered in the least by revenue shortfalls that the governor has tomanage but may be troubled to learn of ldquoextrardquo revenue that theycould have spent on their priorities mdash that is they may be morewilling to risk revenue overestimates and revenue shortfalls thanis the executive

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 9 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 01 43 (11) 19 25AK 76 222 (90) 103 152AL 05 61 (13) 15 38AR 21 41 (02) 21 35AZ (04) 97 (16) 14 44CA (03) 74 (46) 25 43CO 09 85 (02) 16 55CT 05 71 (24) 28 53DE 10 68 (28) (01) 56FL (10) 52 (23) (04) 21GA (03) 50 (41) 15 33HI 12 65 (27) 15 49IA 19 37 (03) 18 48ID 19 68 01 15 51IL 07 53 (23) 19 28IN (09) 49 (25) 09 20KS 18 45 (00) 11 20KY (11) 51 (36) 05 26LA 06 66 (38) 17 41MA 12 75 (22) 23 57MD 04 56 (29) 16 40ME 17 46 (01) 14 39MI (17) 52 (42) (10) 23MN 17 51 (06) 18 55MO (02) 54 (13) 11 36MS 10 52 (24) 18 43MT 42 112 (10) 66 93NC 05 59 (12) 25 42ND 93 189 07 30 138NE 10 52 (15) 17 44NH (04) 161 (34) 21 70NJ (07) 56 (36) 10 25NM 07 60 (25) 06 45NV 22 70 (39) 39 71NY (25) 38 (52) (21) 03OH (02) 53 (19) 13 26OK 03 70 (15) 11 43OR 03 100 (17) 21 49PA (02) 37 (19) 02 32RI 03 57 (28) (04) 43SC (06) 86 (51) 14 46SD 11 25 (02) 08 24TN 03 45 (14) 06 34TX 33 64 00 26 67UT 15 61 02 29 53VA (01) 67 (23) 15 26VT 29 63 (08) 42 75WA 03 70 (10) 13 33WI 02 33 (00) 09 18WV 16 34 (11) 16 32WY 34 71 03 20 71Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 3 Forecast Errors as

Percent of Actual Tax by State

Figure 4 on the following page displays boxplots of eachstatersquos percentage forecasting error for the sum of personal in-come tax sales tax and corporate income tax The top horizontalline of each box marks the 75th percentile of forecast errors andthe lower horizontal line marks the 25th percentile Each box alsohas vertical lines (sometimes called ldquowhiskersrdquo) extending fromits top and bottom indicating variability in the top and bottom 25percent of the data (longer lines indicate more variability in thispart of the data) and dots marking extreme outliers6 The tallerthe box (the greater the difference between the 75th and 25th per-centiles) and the longer the lines the greater the variability inforecast errors for a given state

Only five states have a negative median forecasting error sug-gesting again that states may be ldquotryingrdquo to have small positiveforecasting errors mdash underforecasts of revenue However in acommon formal but rather strict test of unbiasedness known asthe Mincer-Zarnowitz test we reject unbiasedness at the 5 percentlevel for only eight states Arkansas Iowa Montana New YorkNorth Dakota Texas Vermont and Wyoming7 For each of thesestates virtually all of the box in Figure 4 is above the line (aboutthree quarters of the errors are positive) or else below the line (inthe case of New York)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 10 wwwrockinstorg

Figure 3 Percentage Forecast Errors for Sum of PIT CIT and Sales Taxes 1987-2013

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

outlier it could have an undue impact on the mean pulling itaway from what we think is the true central tendency of the data

When we want to measure how much the errors vary acrossstates years taxes or other groupings we use either the standarddeviation or the interquartile range

In the next section we provide simple descriptive statistics oferrors first using the percentage error ldquoas isrdquo and then the abso-lute value There are some useful insights which we summarizein the concluding subsection of this section

Forecasting Errors as Percentage of Actual Revenue

Below we provide summary tables of errors by tax by fiscalyear and by state and explore patterns with several plots

Table 2 displays selected summary statistics for forecastingerrors by tax for 1987-2013 The corporate income tax median fore-casting error at 28 percent was much larger than the median forother taxes The personal income tax median forecasting error was18 percent while the sales tax forecasting error was 03 percentThe standard deviation of the error mdash a measure of variability inforecast error and a potential indication of forecasting difficulty mdashwas largest by far for the corporate income tax nearly three timesas large as for the personal income tax and five times as large asfor the sales tax

Figure 1 on the following page displays the median forecasterror by tax

There is some evidence that forecasting errors are ldquoseriallycorrelatedrdquo mdash that is that a revenue shortfall in one year is morelikely to be followed by a shortfall in the next than by an overageand vice versa Rudolph Penner a former director of the Congres-sional Budget Office concluded this about federal governmentforecasts ldquohellip if CBO makes an error in an overly optimistic orpessimistic direction one year it is highly probable that it willmake an error in the same direction the following yearrdquo (Penner2008) Inspection of NASBO data in time periods surrounding re-cessions shows that at least for the states in aggregate revenueoverestimates tended to be followed by additional overestimates

Figure 2 provides median forecasting errors by fiscal year fortwenty-seven years spanning from 1987 to 2013 As depicted onFigure 2 states normally underestimate during expansions andoverestimate during the downturns Moreover overestimating

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 7 wwwrockinstorg

ofobservations

Meanerror

Standarddeviation

25Median(50)

75

Personal income tax 1100 11 77 (23) 18 55Sales tax 1189 02 46 (20) 03 26Corporate income tax 1058 13 226 (93) 28 140Sum of PIT sales amp CIT 1311 10 76 (22) 15 44

Forecast errors as percentage of actual revenue

Source Rockefeller Institute analysis of NASBO forecast error data

Tax

Table 2 State Revenue Forecasting Percentage Errors by Tax 1987-2013

errors were particu-larly large near the2001 and 2007 reces-sions States now re-turned to theirpattern of underesti-mation after theworst of the fiscalcrisis was behindthem

Table 3 on thefollowing page pro-vides summary sta-tistics of forecastingerrors by state from1987 to 2013 Over-all smaller statesand states depend-ent on a few sectorsof the economy (par-ticularly states reli-ant on oil or naturalgas or gambling)such as AlaskaMontana NevadaNew Hampshireand North Dakotatend to have largererrors Those statesrsquoerrors also tend tobe more variable asmeasured by thestandard deviationor the interquartilerange As alreadymentioned abovepersonal incomesales and corporateincome taxes inAlaska DelawareMontana NewHampshire Ver-mont and Wyoming

represent less than half of total taxes as measured by the CensusBureau Their errors for total taxes in their budget which also in-clude other less volatile taxes appear to be smaller than those ex-amined here We strongly caution not to read too much into theerrors for any single state because the data are noisy and influ-enced by many forces and larger errors need not indicate lesseffective revenue forecasting

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 8 wwwrockinstorg

03

18

28

15

00

05

10

15

20

25

30

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 1 State Revenue Forecast Errors as Percent of Actual Tax Median for 1987-2013

12

10

8

6

4

2

0

2

4

6

8

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal SurveysNote Forecasting error is for the sum of personal income sales and corporate income taxes

1990 91 recession 2001 recession Great recession

Figure 2 Median State Revenue Forecast Errors as Percent of Actual Tax by Fiscal Year

Figure 3 on the following page maps the distributionover time of state forecasting errors for the sum of thethree taxes (personal income tax sales tax and corporateincome tax) and also shows the extent of missing dataWe are missing data for Texas in 1996 1997 and 1999but other than that no large state is missing in any yearIt is difficult to obtain data from state documents that iscomparable to data in the NASBO survey and as notedearlier we were unable to obtain such data with reason-able effort for Texas

The figure shows how forecasting errors have rolledout over time across the country The revenue shortfallsin the middle of the country in 1987 likely are related tothe oil price bust that hit Texas and other oil-extractionstates in the mid-1980s The effects of the 1990 recessionwhich lingered for four years appear to have led to sig-nificant revenue shortfalls on the coasts and in the northbut not as much in the center of the country The 2001recession and the effects of the September 11th attacksare evidenced in revenue shortfalls that began on theeast coast but spread to the rest of the country revenueshortfalls of more than 8 percent were far more commonthan in the 1990 recession The 2005-07 years were agolden period of revenue overages in most of the coun-try The 2007 recession appears to have hit early in thehousing bust states of Arizona Florida and Nevada aswell as Michigan It spread in 2008 and hit the rest of thecountry with force in 2009 and 2010 before state fore-casters were once again on more-solid ground in 2011through 2013

Forecasts may be intentionally biased mdash more likelyto be off in one direction than another mdash because fore-casters believe there is an ldquoasymmetric riskrdquo and that themost accurate estimate is not always the best estimateMany executive-branch forecasters are assumed to be-lieve that the costs of overestimating revenue are greaterthan the costs of underestimating revenue because thepolitical and managerial complications of managing def-icits are believed to be worse than the consequences ofmanaging surpluses These forecasters sometimes ex-plicitly and other times unknowingly may choose con-

servative forecasts of the revenue that is available to financespending By contrast legislative forecasters or their principalsnot generally responsible for financial management might not bebothered in the least by revenue shortfalls that the governor has tomanage but may be troubled to learn of ldquoextrardquo revenue that theycould have spent on their priorities mdash that is they may be morewilling to risk revenue overestimates and revenue shortfalls thanis the executive

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 9 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 01 43 (11) 19 25AK 76 222 (90) 103 152AL 05 61 (13) 15 38AR 21 41 (02) 21 35AZ (04) 97 (16) 14 44CA (03) 74 (46) 25 43CO 09 85 (02) 16 55CT 05 71 (24) 28 53DE 10 68 (28) (01) 56FL (10) 52 (23) (04) 21GA (03) 50 (41) 15 33HI 12 65 (27) 15 49IA 19 37 (03) 18 48ID 19 68 01 15 51IL 07 53 (23) 19 28IN (09) 49 (25) 09 20KS 18 45 (00) 11 20KY (11) 51 (36) 05 26LA 06 66 (38) 17 41MA 12 75 (22) 23 57MD 04 56 (29) 16 40ME 17 46 (01) 14 39MI (17) 52 (42) (10) 23MN 17 51 (06) 18 55MO (02) 54 (13) 11 36MS 10 52 (24) 18 43MT 42 112 (10) 66 93NC 05 59 (12) 25 42ND 93 189 07 30 138NE 10 52 (15) 17 44NH (04) 161 (34) 21 70NJ (07) 56 (36) 10 25NM 07 60 (25) 06 45NV 22 70 (39) 39 71NY (25) 38 (52) (21) 03OH (02) 53 (19) 13 26OK 03 70 (15) 11 43OR 03 100 (17) 21 49PA (02) 37 (19) 02 32RI 03 57 (28) (04) 43SC (06) 86 (51) 14 46SD 11 25 (02) 08 24TN 03 45 (14) 06 34TX 33 64 00 26 67UT 15 61 02 29 53VA (01) 67 (23) 15 26VT 29 63 (08) 42 75WA 03 70 (10) 13 33WI 02 33 (00) 09 18WV 16 34 (11) 16 32WY 34 71 03 20 71Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 3 Forecast Errors as

Percent of Actual Tax by State

Figure 4 on the following page displays boxplots of eachstatersquos percentage forecasting error for the sum of personal in-come tax sales tax and corporate income tax The top horizontalline of each box marks the 75th percentile of forecast errors andthe lower horizontal line marks the 25th percentile Each box alsohas vertical lines (sometimes called ldquowhiskersrdquo) extending fromits top and bottom indicating variability in the top and bottom 25percent of the data (longer lines indicate more variability in thispart of the data) and dots marking extreme outliers6 The tallerthe box (the greater the difference between the 75th and 25th per-centiles) and the longer the lines the greater the variability inforecast errors for a given state

Only five states have a negative median forecasting error sug-gesting again that states may be ldquotryingrdquo to have small positiveforecasting errors mdash underforecasts of revenue However in acommon formal but rather strict test of unbiasedness known asthe Mincer-Zarnowitz test we reject unbiasedness at the 5 percentlevel for only eight states Arkansas Iowa Montana New YorkNorth Dakota Texas Vermont and Wyoming7 For each of thesestates virtually all of the box in Figure 4 is above the line (aboutthree quarters of the errors are positive) or else below the line (inthe case of New York)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 10 wwwrockinstorg

Figure 3 Percentage Forecast Errors for Sum of PIT CIT and Sales Taxes 1987-2013

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

errors were particu-larly large near the2001 and 2007 reces-sions States now re-turned to theirpattern of underesti-mation after theworst of the fiscalcrisis was behindthem

Table 3 on thefollowing page pro-vides summary sta-tistics of forecastingerrors by state from1987 to 2013 Over-all smaller statesand states depend-ent on a few sectorsof the economy (par-ticularly states reli-ant on oil or naturalgas or gambling)such as AlaskaMontana NevadaNew Hampshireand North Dakotatend to have largererrors Those statesrsquoerrors also tend tobe more variable asmeasured by thestandard deviationor the interquartilerange As alreadymentioned abovepersonal incomesales and corporateincome taxes inAlaska DelawareMontana NewHampshire Ver-mont and Wyoming

represent less than half of total taxes as measured by the CensusBureau Their errors for total taxes in their budget which also in-clude other less volatile taxes appear to be smaller than those ex-amined here We strongly caution not to read too much into theerrors for any single state because the data are noisy and influ-enced by many forces and larger errors need not indicate lesseffective revenue forecasting

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 8 wwwrockinstorg

03

18

28

15

00

05

10

15

20

25

30

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 1 State Revenue Forecast Errors as Percent of Actual Tax Median for 1987-2013

12

10

8

6

4

2

0

2

4

6

8

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal SurveysNote Forecasting error is for the sum of personal income sales and corporate income taxes

1990 91 recession 2001 recession Great recession

Figure 2 Median State Revenue Forecast Errors as Percent of Actual Tax by Fiscal Year

Figure 3 on the following page maps the distributionover time of state forecasting errors for the sum of thethree taxes (personal income tax sales tax and corporateincome tax) and also shows the extent of missing dataWe are missing data for Texas in 1996 1997 and 1999but other than that no large state is missing in any yearIt is difficult to obtain data from state documents that iscomparable to data in the NASBO survey and as notedearlier we were unable to obtain such data with reason-able effort for Texas

The figure shows how forecasting errors have rolledout over time across the country The revenue shortfallsin the middle of the country in 1987 likely are related tothe oil price bust that hit Texas and other oil-extractionstates in the mid-1980s The effects of the 1990 recessionwhich lingered for four years appear to have led to sig-nificant revenue shortfalls on the coasts and in the northbut not as much in the center of the country The 2001recession and the effects of the September 11th attacksare evidenced in revenue shortfalls that began on theeast coast but spread to the rest of the country revenueshortfalls of more than 8 percent were far more commonthan in the 1990 recession The 2005-07 years were agolden period of revenue overages in most of the coun-try The 2007 recession appears to have hit early in thehousing bust states of Arizona Florida and Nevada aswell as Michigan It spread in 2008 and hit the rest of thecountry with force in 2009 and 2010 before state fore-casters were once again on more-solid ground in 2011through 2013

Forecasts may be intentionally biased mdash more likelyto be off in one direction than another mdash because fore-casters believe there is an ldquoasymmetric riskrdquo and that themost accurate estimate is not always the best estimateMany executive-branch forecasters are assumed to be-lieve that the costs of overestimating revenue are greaterthan the costs of underestimating revenue because thepolitical and managerial complications of managing def-icits are believed to be worse than the consequences ofmanaging surpluses These forecasters sometimes ex-plicitly and other times unknowingly may choose con-

servative forecasts of the revenue that is available to financespending By contrast legislative forecasters or their principalsnot generally responsible for financial management might not bebothered in the least by revenue shortfalls that the governor has tomanage but may be troubled to learn of ldquoextrardquo revenue that theycould have spent on their priorities mdash that is they may be morewilling to risk revenue overestimates and revenue shortfalls thanis the executive

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 9 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 01 43 (11) 19 25AK 76 222 (90) 103 152AL 05 61 (13) 15 38AR 21 41 (02) 21 35AZ (04) 97 (16) 14 44CA (03) 74 (46) 25 43CO 09 85 (02) 16 55CT 05 71 (24) 28 53DE 10 68 (28) (01) 56FL (10) 52 (23) (04) 21GA (03) 50 (41) 15 33HI 12 65 (27) 15 49IA 19 37 (03) 18 48ID 19 68 01 15 51IL 07 53 (23) 19 28IN (09) 49 (25) 09 20KS 18 45 (00) 11 20KY (11) 51 (36) 05 26LA 06 66 (38) 17 41MA 12 75 (22) 23 57MD 04 56 (29) 16 40ME 17 46 (01) 14 39MI (17) 52 (42) (10) 23MN 17 51 (06) 18 55MO (02) 54 (13) 11 36MS 10 52 (24) 18 43MT 42 112 (10) 66 93NC 05 59 (12) 25 42ND 93 189 07 30 138NE 10 52 (15) 17 44NH (04) 161 (34) 21 70NJ (07) 56 (36) 10 25NM 07 60 (25) 06 45NV 22 70 (39) 39 71NY (25) 38 (52) (21) 03OH (02) 53 (19) 13 26OK 03 70 (15) 11 43OR 03 100 (17) 21 49PA (02) 37 (19) 02 32RI 03 57 (28) (04) 43SC (06) 86 (51) 14 46SD 11 25 (02) 08 24TN 03 45 (14) 06 34TX 33 64 00 26 67UT 15 61 02 29 53VA (01) 67 (23) 15 26VT 29 63 (08) 42 75WA 03 70 (10) 13 33WI 02 33 (00) 09 18WV 16 34 (11) 16 32WY 34 71 03 20 71Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 3 Forecast Errors as

Percent of Actual Tax by State

Figure 4 on the following page displays boxplots of eachstatersquos percentage forecasting error for the sum of personal in-come tax sales tax and corporate income tax The top horizontalline of each box marks the 75th percentile of forecast errors andthe lower horizontal line marks the 25th percentile Each box alsohas vertical lines (sometimes called ldquowhiskersrdquo) extending fromits top and bottom indicating variability in the top and bottom 25percent of the data (longer lines indicate more variability in thispart of the data) and dots marking extreme outliers6 The tallerthe box (the greater the difference between the 75th and 25th per-centiles) and the longer the lines the greater the variability inforecast errors for a given state

Only five states have a negative median forecasting error sug-gesting again that states may be ldquotryingrdquo to have small positiveforecasting errors mdash underforecasts of revenue However in acommon formal but rather strict test of unbiasedness known asthe Mincer-Zarnowitz test we reject unbiasedness at the 5 percentlevel for only eight states Arkansas Iowa Montana New YorkNorth Dakota Texas Vermont and Wyoming7 For each of thesestates virtually all of the box in Figure 4 is above the line (aboutthree quarters of the errors are positive) or else below the line (inthe case of New York)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 10 wwwrockinstorg

Figure 3 Percentage Forecast Errors for Sum of PIT CIT and Sales Taxes 1987-2013

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Figure 3 on the following page maps the distributionover time of state forecasting errors for the sum of thethree taxes (personal income tax sales tax and corporateincome tax) and also shows the extent of missing dataWe are missing data for Texas in 1996 1997 and 1999but other than that no large state is missing in any yearIt is difficult to obtain data from state documents that iscomparable to data in the NASBO survey and as notedearlier we were unable to obtain such data with reason-able effort for Texas

The figure shows how forecasting errors have rolledout over time across the country The revenue shortfallsin the middle of the country in 1987 likely are related tothe oil price bust that hit Texas and other oil-extractionstates in the mid-1980s The effects of the 1990 recessionwhich lingered for four years appear to have led to sig-nificant revenue shortfalls on the coasts and in the northbut not as much in the center of the country The 2001recession and the effects of the September 11th attacksare evidenced in revenue shortfalls that began on theeast coast but spread to the rest of the country revenueshortfalls of more than 8 percent were far more commonthan in the 1990 recession The 2005-07 years were agolden period of revenue overages in most of the coun-try The 2007 recession appears to have hit early in thehousing bust states of Arizona Florida and Nevada aswell as Michigan It spread in 2008 and hit the rest of thecountry with force in 2009 and 2010 before state fore-casters were once again on more-solid ground in 2011through 2013

Forecasts may be intentionally biased mdash more likelyto be off in one direction than another mdash because fore-casters believe there is an ldquoasymmetric riskrdquo and that themost accurate estimate is not always the best estimateMany executive-branch forecasters are assumed to be-lieve that the costs of overestimating revenue are greaterthan the costs of underestimating revenue because thepolitical and managerial complications of managing def-icits are believed to be worse than the consequences ofmanaging surpluses These forecasters sometimes ex-plicitly and other times unknowingly may choose con-

servative forecasts of the revenue that is available to financespending By contrast legislative forecasters or their principalsnot generally responsible for financial management might not bebothered in the least by revenue shortfalls that the governor has tomanage but may be troubled to learn of ldquoextrardquo revenue that theycould have spent on their priorities mdash that is they may be morewilling to risk revenue overestimates and revenue shortfalls thanis the executive

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 9 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 01 43 (11) 19 25AK 76 222 (90) 103 152AL 05 61 (13) 15 38AR 21 41 (02) 21 35AZ (04) 97 (16) 14 44CA (03) 74 (46) 25 43CO 09 85 (02) 16 55CT 05 71 (24) 28 53DE 10 68 (28) (01) 56FL (10) 52 (23) (04) 21GA (03) 50 (41) 15 33HI 12 65 (27) 15 49IA 19 37 (03) 18 48ID 19 68 01 15 51IL 07 53 (23) 19 28IN (09) 49 (25) 09 20KS 18 45 (00) 11 20KY (11) 51 (36) 05 26LA 06 66 (38) 17 41MA 12 75 (22) 23 57MD 04 56 (29) 16 40ME 17 46 (01) 14 39MI (17) 52 (42) (10) 23MN 17 51 (06) 18 55MO (02) 54 (13) 11 36MS 10 52 (24) 18 43MT 42 112 (10) 66 93NC 05 59 (12) 25 42ND 93 189 07 30 138NE 10 52 (15) 17 44NH (04) 161 (34) 21 70NJ (07) 56 (36) 10 25NM 07 60 (25) 06 45NV 22 70 (39) 39 71NY (25) 38 (52) (21) 03OH (02) 53 (19) 13 26OK 03 70 (15) 11 43OR 03 100 (17) 21 49PA (02) 37 (19) 02 32RI 03 57 (28) (04) 43SC (06) 86 (51) 14 46SD 11 25 (02) 08 24TN 03 45 (14) 06 34TX 33 64 00 26 67UT 15 61 02 29 53VA (01) 67 (23) 15 26VT 29 63 (08) 42 75WA 03 70 (10) 13 33WI 02 33 (00) 09 18WV 16 34 (11) 16 32WY 34 71 03 20 71Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 3 Forecast Errors as

Percent of Actual Tax by State

Figure 4 on the following page displays boxplots of eachstatersquos percentage forecasting error for the sum of personal in-come tax sales tax and corporate income tax The top horizontalline of each box marks the 75th percentile of forecast errors andthe lower horizontal line marks the 25th percentile Each box alsohas vertical lines (sometimes called ldquowhiskersrdquo) extending fromits top and bottom indicating variability in the top and bottom 25percent of the data (longer lines indicate more variability in thispart of the data) and dots marking extreme outliers6 The tallerthe box (the greater the difference between the 75th and 25th per-centiles) and the longer the lines the greater the variability inforecast errors for a given state

Only five states have a negative median forecasting error sug-gesting again that states may be ldquotryingrdquo to have small positiveforecasting errors mdash underforecasts of revenue However in acommon formal but rather strict test of unbiasedness known asthe Mincer-Zarnowitz test we reject unbiasedness at the 5 percentlevel for only eight states Arkansas Iowa Montana New YorkNorth Dakota Texas Vermont and Wyoming7 For each of thesestates virtually all of the box in Figure 4 is above the line (aboutthree quarters of the errors are positive) or else below the line (inthe case of New York)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 10 wwwrockinstorg

Figure 3 Percentage Forecast Errors for Sum of PIT CIT and Sales Taxes 1987-2013

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Figure 4 on the following page displays boxplots of eachstatersquos percentage forecasting error for the sum of personal in-come tax sales tax and corporate income tax The top horizontalline of each box marks the 75th percentile of forecast errors andthe lower horizontal line marks the 25th percentile Each box alsohas vertical lines (sometimes called ldquowhiskersrdquo) extending fromits top and bottom indicating variability in the top and bottom 25percent of the data (longer lines indicate more variability in thispart of the data) and dots marking extreme outliers6 The tallerthe box (the greater the difference between the 75th and 25th per-centiles) and the longer the lines the greater the variability inforecast errors for a given state

Only five states have a negative median forecasting error sug-gesting again that states may be ldquotryingrdquo to have small positiveforecasting errors mdash underforecasts of revenue However in acommon formal but rather strict test of unbiasedness known asthe Mincer-Zarnowitz test we reject unbiasedness at the 5 percentlevel for only eight states Arkansas Iowa Montana New YorkNorth Dakota Texas Vermont and Wyoming7 For each of thesestates virtually all of the box in Figure 4 is above the line (aboutthree quarters of the errors are positive) or else below the line (inthe case of New York)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 10 wwwrockinstorg

Figure 3 Percentage Forecast Errors for Sum of PIT CIT and Sales Taxes 1987-2013

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Judging by the graph there appears to be a slight tendency forstates with greater variability in the middle half of their datameasured by vertical length of the box to have slightly more-positive errors (more underforecasting) than those with lesservariability suggesting that when the possibility of large negativeerrors is greater states may be more likely to protect againstdownside risk by aiming for small positive errors mdash ie bias

Absolute Value of Percentage Forecasting Errors

As noted earlier the absolute value of the percentage error isuseful in examining accuracy Figure 5 on the following pagewhich shows median errors by tax reinforces the idea that thecorporate income tax has been the hardest tax to forecast by far

Table 4 on the following page provides another view showingthe size distribution (across states) of the median absolute per-centage error by tax It is clear that large errors are extremelycommon for the corporate income tax and much more commonfor the personal income tax than the sales tax8

Figure 6 on page 13 shows median absolute percentage errorsby fiscal year The trends shown on indicate that

Errors become particularly large in and after recessions

Variation in errors across states also increases in and afterrecessions

Errors near the 2001 and 2007 recessions were much worsethan in the recession of the early 1990s

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 11 wwwrockinstorg

Figure 4 Forty-Five States Underestimated Revenue in the Typical Year

Source Rockefeller Institute Analysis of Data From NASBO Fall Fiscal Surveys

Distribution of revenue forecasting errors 1987-2013 by state Sum of PIT CIT and sales taxesVertical axis truncated at +-20 percent error

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

States have re-turned to ldquomorenormalrdquo fore-casting errorswith the 2007 re-cession now be-hind us

Table 5 on page14 shows the me-dian absolute per-centage errors bystate As before thepattern of larger er-rors among smallerstates and statesheavily reliant onone or two large in-dustrial sectors(such as oil and gasor gambling) isclear for example

median absolute percentage errors are particularly large forAlaska Montana Nevada New Hampshire Texas and VermontIn general these states also have greater variation in their errorsover time too (Among these states the taxes reported to NASBOfor Alaska Montana New Hampshire and Vermont are a rela-tively small share of total taxes as measured by the Census Bu-reau Their errors for total taxes in their budget which alsoinclude other less volatile taxes appear to be smaller than thoseexamined here)

Constructing a Forecast Difficulty Measure

One of the problems in comparing forecast errors across statesor over time is that it is more difficult to be accurate with someforecasts than others Some taxes are more volatile than otherssome periods are more uncertain than others some states have

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 12 wwwrockinstorg

23

44

118

35

0

2

4

6

8

10

12

14

Sales PIT CIT Sum ofSales PIT CIT

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Figure 5 Absolute Value of Percentage Forecasting Errors by Tax Median for 1987-2013

Size of medianabsolute error

Sales PIT CITSum of Sales

PIT CITlt 2 15 42 to 3 18 8 133 to 4 8 10 1 184 to 5 2 9 85 to 10 3 15 17 610 to 20 26 1gt 20 3Total 46 42 47 50

Number of states by median absolute percentage error and tax 1987 2013

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

Table 4 Very Large Errors Are Much Rarer for the Sales Tax

Than for Personal Income or Corporate Income Taxes

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

more volatile econo-mies than othersand so on If forecasterrors are greater inone year or statethan in another itmay not mean theforecaster was lessadept but rathermay mean that theforecasting task wasmore difficult

Many research-ers who examinerevenue forecastingerrors attempt tocontrol for this im-plicitly by includingmeasures of eco-nomic conditions inregressions such as

unemployment rates and changes in personal income While thisis useful tax revenue depends on many things other than broadmeasures of the economy For example capital gains can varywidely even when the economy is stable leading to forecast errorsnot easily tied to measures of economic volatility It would be use-ful to have a measure of forecast difficulty that does a better job ofcapturing whether a particular forecast is difficult or not It is animportant technical issue because it is not possible to draw manyconclusions in regression analyses about factors affecting forecastaccuracy unless we take into account how difficult a particularforecast is Beyond that having a strong measure of forecast diffi-culty is useful in its own right because it allows much greater in-sights even when examining forecast errors in a descriptive senserather than in a statistical sense

Our contribution to this issue is to develop a more-sophisticatedmeasure of forecast difficulty9 The way we have done this is byconstructing a uniformly applied ldquonaiumlverdquo model that forecastseach tax in each state and each year using data that generallywould have been available to forecasters at the time We thencompute the error in the naiumlve modelrsquos forecast and use that asour measure of forecast difficulty If a model has trouble forecast-ing an observation that particular forecast probably is difficult

The particular model we use is a common and very usefulone an exponential smoothing model with a trend which usesten previous years of data to forecast each tax in each state andeach year The model uses Census Bureau data on taxes for eachstate and tax (so that it can use historical data for years before ourdata start and for other technical reasons) looks back ten yearsand forecasts ahead one To do this we need to estimate and

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 13 wwwrockinstorg

0

2

4

6

8

10

12

Forecastingpe

rcen

tage

error

Source Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys

1990 91 recession 2001 recession Great recession

Figure 6 Absolute Value of Percentage Forecasting Errors by Fiscal Year 1987-2013

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

forecast from a separate model for every single datapoint For example to forecast Massachusettsrsquos incometax in 1998 we would use data for the Massachusetts in-come tax from 1988 to 1997 estimate an exponentialsmoothing model and forecast one year ahead To fore-cast Massachusettsrsquos income tax in 1999 we would esti-mate a new model using data from 1990 to 1998 andthen forecast one year ahead Since we need a separatemodel for every data point we had to estimate and fore-cast from approximately 5000 models (one for eachstate tax and year and one for the sum of the taxes ineach state and year)

The great advantages of this approach are that it isuniformly applied to each data point it relies largely(but not perfectly) on data that forecasters would havehad available at the time and it works well The maindisadvantages are that it is a very naiumlve model mdash expo-nential smoothing models forecast a variable using dataonly from that variablersquos past and they cannot call turn-ing points and are likely to be particularly wrong whenthe economy changes direction Because revenue fore-casters read the newspapers talk to economic forecast-ers and make use of a wide variety of informationsources that cannot easily fit into uniform models theyare likely to be more accurate than our naiumlve model

Figure 7 on page 15 shows the percentage error fromour naiumlve model and the actual forecasting error bystates for the US as a whole over time It is clear thatthe model does a fairly good job of capturing the diffi-culty state forecasters face Wersquoll say more about this be-low10

Figure 8 on page 16 shows histograms with the dis-tribution of state forecastersrsquo errors (the top panel) andthe distribution of errors from our naiumlve model Notethat the forecastersrsquo errors are not clustered around zerobut rather tend to be positive on average As we havediscussed this is an indication of possible bias Of courseit is also possible that this is simply a function of ourdata and that for the years we have good actual revenuewas likely to be higher than estimates from unbiasedmodels However note that the errors from the naiumlve

models are clustered around zero and are quite symmetric (al-though not normal) Given that the naiumlve modelrsquos errors tend tobe clustered around zero but the naiumlve modelrsquos errors are not thissuggests that economic and revenue patterns were not the causeof positive errors further reinforcing the idea that state forecasterswere biased

Finally Figure 9 on page 17 shows the geographic distributionof the median absolute percentage forecast error from our naiumlvemodel Forecast difficulty appears to be greater in smaller states

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 14 wwwrockinstorg

State MeanStandarddeviation

25 Median 75

US 33 27 20 23 40AK 184 142 97 146 252AL 43 42 15 36 52AR 34 29 18 30 38AZ 52 81 18 27 54CA 59 43 27 43 82CO 59 62 14 38 77CT 54 45 26 44 68DE 53 43 26 47 70FL 37 38 13 22 47GA 41 26 22 37 52HI 51 41 21 39 67IA 34 24 16 29 48ID 50 49 12 27 86IL 39 35 20 27 47IN 37 33 11 24 54KS 31 38 10 17 32KY 42 31 24 34 50LA 52 39 23 41 65MA 58 47 26 51 81MD 46 30 22 37 70ME 35 34 13 22 46MI 42 35 13 36 56MN 42 33 18 34 62MO 40 37 13 31 54MS 44 29 23 38 62MT 99 64 60 80 130NC 46 36 22 31 61ND 117 175 27 48 146NE 41 32 16 32 62NH 109 116 22 66 126NJ 40 39 18 30 48NM 47 38 19 37 71NV 63 36 39 62 79NY 36 28 16 27 52OH 38 37 15 25 42OK 48 51 15 27 63OR 68 72 20 44 92PA 28 24 12 28 39RI 46 33 22 35 77SC 66 55 26 46 79SD 21 17 07 17 30TN 35 29 09 30 50TX 56 44 23 52 90UT 49 40 23 42 61VA 46 47 15 27 44VT 60 34 31 57 80WA 48 51 12 30 68WI 20 26 08 16 22WV 29 24 14 19 34WY 56 54 12 34 95Source Rockefeller Institute analysis of data from NASBONote States with relied on PIT CIT amp sales taxes for less than half oftheir tax revenue in a typical year

Table 5 Absolute Value of

Percentage Forecasting Errors By State

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

and in resource richstates suggestingthat the larger actualerrors seen in thosestates is not simplyan indication of badforecasting but thatthere is somethingabout those statesthat has made reve-nue harder to fore-cast in those states

Conclusions and

Policy Implications

From Descriptive

Analysis

The main con-clusions from ourdescriptive analysisof the data are

Corporate income tax forecasting errors are much largerthan for other taxes followed by the personal income taxThe median absolute percentage error was 118 percent forthe corporate income tax 44 percent for the personal in-come tax and 23 percent for the sales tax

Smaller states and states dependent on a few sectors of theeconomy (particularly states reliant on oil or natural gasor gambling) such as Alaska Montana Nevada NewHampshire and North Dakota tend to have larger errorsThose statesrsquo errors also tend to be more variable (Amongthese states the taxes reported to NASBO by AlaskaMontana and New Hampshire are a relatively small shareof total taxes as measured by the Census Bureau Their er-rors for total taxes in their budget which also includeother less volatile taxes appear to be smaller than thoseexamined here However these states also have large er-rors from naiumlve forecasting models using full Censusdata)

When taxes are particularly difficult to forecast states tendto be more likely to underforecast revenue suggesting thatthey may try to do so in an effort to avoid large shortfallsThus there is a pattern to the apparent bias in state reve-nue forecasts By contrast our naiumlve forecasting modeldoes not become more likely to underforecast when fore-casting becomes more difficult suggesting that this phe-nomenon may reflect the behavior of forecasters ratherthan underlying factors in the economy or tax revenuestructures

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 15 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

State tax revenue forecasting errors for United States as a wholeSum of personal income sales and corporate income taxes

Actual error Error from naive model

Sources Rockefeller Institute analysis of data from NASBO Fall Fiscal Surveys amp Census Bureau

Figure 7 Actual Forecasting Errors Follow Same Pattern as

Naiumlve Modelrsquos Errors But Are Smaller and Appear to be More Positive

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

State forecasters percent error

0

50

100

150

200

250

50 40 30 20 10 0 10 20 30 40 50

Naive model percent error

Figure 8 Errors from Naiumlve Forecasting Model Are More-Centered Around Zero

Than Forecastersrsquo Actual Errors and Tend to be Larger

Forecast errors of state forecasters and naiumlve model all statesSum of PIT CIT and sales taxes Cut off at [-50 +50]

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 16 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Errors becomeparticularlylarge in and afterrecessions

Variation in er-rors across statesalso increases inand afterrecessions

Errors near the2001 and 2007 re-cessions weremuch worse thanin the recession ofthe early 1990s

States have re-turned to ldquomorenormalrdquo forecast-ing errors withthe 2007 recessionnow behind us

One limitation of this study is that we do not have detailedand comprehensive data on total revenue forecasts andour analysis is limited to the sum of personal income cor-porate income and sales taxes Thus these data do not tellwhether and by what magnitude an overall state forecastwas missed

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 17 wwwrockinstorg

Forecast difficulty measure for sum of 3 major taxes 1987 2013

Percent changelt 33 to 4545 to 66 to 10gt 10

MI

FL

OH

WV

AZ

CA

WI

NV

HI

PA

TN

MN

AR

IN

MO

ME

IA

MS

IL

VA

AK

GA

NM

OR

NY

KY

AL

KS

ND

OKSC

SD

NE

NC

ID

CO

MT

WA

TX LA

UT

WY

VTNH

RI

MA

MD

NJ

CT

DE

Figure 9 Forecasting Appears to Have Been More Difficult

in Smaller States and Resource-Rich States

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Revenue Forecasting Accuracy and Revenue Volatility

Revenue volatility plays a significant role in revenue forecast-ing accuracy and as noted in the prior report tax revenue has be-come more volatile and difficult to forecast States do haveoptions for reducing volatility such as diversifying the set of taxesthey rely upon or broadening the bases of those taxes They alsohave options for strengthening their ability to manage volatilitythrough use of rainy day funds and other mechanisms

Prior Research on Revenue Volatility

and Revenue Forecasting

A number of factors influence the volatility of state tax reve-nues including the structure of each statersquos tax system theuniqueness of each statersquos economy and the severity of economiccycles (Felix 2008 Cornia and Nelson 2010)

Several studies have examined strategies that might reducebudget uncertainty caused by revenue volatility or that would al-low better management of the effects of volatility One such strat-egy is diversifying tax revenue to reduce volatility and another isestablishing a rainy day fund to manage the effects of volatility

Crain (2003) used panel data for all fifty states from 1968 to 1998to examine the volatility of sales and income tax revenues in eachstate as well as the impact of tax structure diversification on overallrevenue volatility Crain defined revenue volatility as ldquothe standarddeviation of the deviation in revenue from its long-run trend linerdquo(Crain 2003) According to the study results ldquoAmong the states thatlevy both types of taxes 62 percent (twenty-three out of thirty-seven states) experience less volatility in individual income tax rev-enues than in sales tax revenuesrdquo (Crain 2003) This conclusion mayseem inconsistent with the permanent income hypothesis and con-sumption smoothing where consumption patterns are based onlong-term expected income (not immediate income) and thereforemight be less volatile than annual income However state incomeand sales tax bases differ substantially from broader economic mea-sures of income and consumption and the conclusions need not beinconsistent (However unlike Crain we conclude in this reportthat income taxes are more volatile than sales taxes in part becausewe include more-recent data)

In addition Crain also examined the impact of diversified taxstructure on revenue volatility Crain found that in thirteen out ofthirty-seven states that have both sales and income taxes the vola-tility in one tax source is large enough to offset any potential ben-efits from tax diversification The author concludes ldquothat innearly two-thirds of the states tax diversification effectively re-duces revenue volatility as standard portfolio theory would pre-dict In one-third of the states relying on a single-tax instrumentwould predictably reduce revenue volatility relative to the statusquo level of tax diversificationrdquo (Crain 2003) Interestingly onlyhalf of the states levying both forms of taxes used the less volatiletax to collect the majority of revenue

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 18 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Crainrsquos work preceded dramatic growth in capital gains andother nonwage income and income tax volatility is a muchgreater problem now than when he wrote Two researchers at theChicago Federal Reserve Bank concluded in a 2008 paper ldquoOurfindings suggest that increasing income cyclicality in particular ofcapital gains have made state revenues more responsive to thebusiness cycle since the mid-1990srdquo (Mattoon and McGranahan2012)

Mattoon (2003) suggested creating a national rainy day fundas a strategy for improving state fiscal performance He arguesldquoStates seem reluctant to lsquofixrsquo their tax structures to better managevolatility In addition it is unclear that revenue volatility is neces-sarily a bad thing if states are willing to create budget stabilizationtools Efforts to broaden major tax bases such as subjecting ser-vices to sales taxation have seen little progressrdquo (Mattoon 2003)The author finds that if states had access to the national rainy dayfund he proposed for fiscal years 2002 to 2004 74 percent of theaggregate state budget deficit would have been covered by thefund where in practice only one of the fifty states avoided a bud-get shortfall over these three years (Wyoming) However the fundwould have required a 15 percent savings rate for each statewhich forty-eight of fifty states had not achieved already thus in-flating the rainy fundrsquos success rate (states could have performedbetter simply by saving 15 pecent)

Elder and Wagner (2013) also suggest establishing a nationalrainy day fund but argue a different rationale The authors con-tend that since state business cycles are not perfectly synchro-nized mdash economic booms and busts do not always occurconcurrently for all states mdash disparate state business cycles cancounteract each other as a collective insurance pool The studyfinds that states are synchronized with the national business cycleonly 836 percent of the time on average Ten states were synchro-nized with the national economy 90 percent of the time whileseven states were synchronized less than 75 percent of the timeGiven this lack of perfect synchronization a national rainy dayfund could be created for less than the cost of each state insuringdownturns independently as states with a surplus can supportthe states in a downturn (Elder and Wagner 2013)11

Thompson and Gates (2007) provide several recommendationsfor reducing state revenue volatility and improving state revenueforecasting Based on premises of modern financial economics theauthors emphasize the following set of tools that would help pol-icy and budget makers to reduce revenue volatility or manage itseffects and improve revenue forecasting accuracy substitutingmore progressive taxes with a less progressive tax having awell-designed portfolio of tax types investing in rainy day fundpools and balancing budgets in a present-value sense by usingsavings and debt to smooth consumption (Thompson and Gates2007) Thompson and Gates conclude that a state government can-not significantly reduce tax volatility by simply switching from

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 19 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

one tax-type to another unless also replacing a more progressivetax structure with a less progressive tax structure The authorsmaintain that a diversified portfolio of taxes can significantly re-duce revenue volatility but caveat that ldquoEven the best-designedtax portfolio would not eliminate all volatility in revenue growthrdquo(Thompson and Gates 2007)

In a recent study Cornia and Nelson (2010) explore the impactof state economies and tax portfolios on revenue growth and vola-tility According to their results states with a combination of taxeswould likely reduce the volatility of tax revenues By the time oftheir study the income tax had become more volatile than previ-ously and more volatile than the sales tax In contradiction toCrainrsquos study (and consistent with ours) the authors concludeldquoThe sales tax offers stability but at the cost of a lower growthrate The individual income tax offers growth but at the cost of in-creased volatilityrdquo (Cornia and Nelson 2010)

Trends in Revenue Volatility and

How It Relates to Forecasting Accuracy

Tax revenue volatility has increased in recent decades makingforecasting more difficult Figure 10 shows statesrsquo forecasting er-rors as a percentage of actual results the error from our naiumlveforecasting model and the year-over-year percentage change inreal gross domestic product in each case for the span of our dataie 1987-2013 and for the United States as a whole

The figure shows three periods in which revenue forecastingerrors were sharply negative with states overestimating howmuch would be available in and after 1990 in and after 2001 andin and after 2008 in each case associated with a national recession

The green lineshows growth inreal gross domesticproduct (GDP) andin each of those peri-ods it slowed sub-stantially or fell12

Several points areevident from thegraph

Revenue fore-casting errorswere more thantwice as large inthe two most re-cent recessionscompared to the1990 recession

Those errorswere muchgreater than the

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 20 wwwrockinstorg

18

15

12

9

6

3

0

3

6

9

Revenue forecasting errors amp change in the economy United States as a wholeSum of personal income sales and corporate income taxes

recession Actual error Error from naive model change in real GDP

Sources Rockefeller Institute analysis of data from NASBO and Bureau of Economic Analysis

Figure 10 Forecasting Errors of State Revenue Forecasters and of ldquoNaumliverdquo Forecasting

Models Increased Substantially in the Last Two Recessions

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

change in real GDP suggesting that even relatively smallchanges in a broad measure of the economy can lead tolarge tax revenue forecasting errors

Errors from our naiumlve forecasting model in the two mostrecent recessions also were more than twice as large as ear-lier errors suggesting that the task of revenue forecastingbecame more difficult rather than that forecasters becameless adept and

Revenue forecastersrsquo errors generally were smaller on thedownside than errors from our naiumlve model but about thesame as or even slightly more positive than those from ournaiumlve model when errors were positive This suggests thatrevenue forecasters have been more accurate than ournaiumlve model and perhaps biased toward positive errors(underestimates of revenue)

One common measure of volatility is the standard deviation ofthe periodic percentage change in a variable13 Applied to tax rev-enue it is the standard deviation of the annual percentage changein tax revenue14 It is always a positive number and the larger itis the more volatile the revenue The intuition is this suppose astatersquos tax revenue grew by 5 percent each year come rain orshine Its mean percentage change would be five and because itnever varies its standard deviation would be zero mdash revenuechanges from year to year but it is not volatile And it is easy topredict However suppose the mean remains 5 percent but reve-nue grows in some years by as much as 8 10 or even 20 percentwhile in other years it grows by 2 percent or even falls by 5 or 15percent In this case the standard deviation will be much largermdash the tax is more volatile And revenue probably will be difficult

to predict The stan-dard deviation ofpercent change alsocan be used to mea-sure the volatility ofeconomic variablessuch as real GDP Byexamining changesin the standard devi-ation over time wecan see whether avariable is becomingmore volatile or lessvolatile over time

Figure 11 showsthe rolling ten-yearstandard deviationof the percentagechange in the Cen-sus Bureaursquos data onstate tax revenue for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 21 wwwrockinstorg

0

2

4

6

8

10

12

14

16

18

Volatility

(rolling10

year

standard

deviationof

change)

Volatility of state tax revenue and of GDP

Total Tax PIT Sales CIT GDP

Sources Rockefeller Institute analysis of data from Census Bureau amp Bureau of Economic Analysis

Figure 11 Increasing Volatility of State Tax Revenue Has Been Driven by Personal and

Corporate Income Taxes and Not by Increased Volatility in the Broader Economy

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

the personal income tax sales tax and total taxes and of the per-centage change in nominal gross domestic product for the UnitedStates as a whole Each point on a line represents the standard de-viation of percentage changes over the ten previous years For ex-ample the first point for taxes is 1979 and it measures thestandard deviation of the annual percentage changes in each yearfrom 1970 through 1979 As we move forward in time we main-tain the ten-year window so that for example a data point for2009 is the standard deviation of annual percentage changes foreach year from 2000 through 2009 By comparing standard devia-tions across variables we can see whether one variable is morevolatile than another15 By examining how the standard deviationhas changed over time we can see whether volatility has beenincreasing or decreasing

From this figure we can see that

The rolling standard deviation of the percent change in to-tal taxes was below 35 percent in every year before 2001after which it rose sharply and exceeded 55 percent in2009-13 (Itrsquos important to remember that each point sum-marizes ten years of data so the increase in volatility seenbeginning in 2001 reflects percentage changes from 1992through 2001 and how they differed from previousperiods)

Corporate income taxes are by far the most volatile

Personal income tax volatility increased much more thandid sales tax volatility although the sales tax becamesomewhat more volatile

Throughout the period the income tax was more volatilethan the sales tax and volatility of total taxes generallywas between the two

Taxes are more volatile than gross domestic product abroad measure of the economy

While the economy became more volatile between 2001and 2010 the increase was much smaller than the increasein tax volatility16 Put differently the increase in tax reve-nue volatility appears to have been caused by forces otherthan just an increase in the volatility of the economybroadly measured

Increases in tax revenue volatility have been widespread Ouranalysis of Census tax data shows that forty-two states had an in-crease in revenue volatility between 2000 and 2013

We have noted elsewhere that the increase in tax revenue vol-atility is related to increasing volatility of the income tax in turninfluenced by increasing reliance on capital gains and othernonwage income and increasing volatility of those sources Fig-ure 12 illustrates this with data from California annual percent-age forecast errors and the year-over-year changes in incomefrom wages and from capital gains17 Capital gains clearly are

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 22 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

more volatile thanwages and forecast-ing errors generallywere much larger inyears when capitalgains changedsignificantly

In general vola-tility in the overalleconomy makes itharder to produceaccurate forecastsand states often re-visit their revenueestimates during thebusiness cycles Forexample an officialfrom Vermont saidldquoDuring periods ofeconomic volatilityespecially down-

turns updates and revisions have been made more frequently toallow timely executive and legislative response to fiscal eventsrdquoVolatility can be compounded by federal policies that have unin-tended and hard to measure consequences at the state level Forexample the fiscal cliff not only contributed to the volatility of in-come tax receipts but also made the job of the forecasters in manystates much harder

Reducing Volatility by Changing Tax Structure

How much could tax revenue volatility and associated reve-nue forecasting errors be reduced if states chose different portfo-lios of taxes We examined this by recomputing the total forecasterror that each states would have had in each year of our data ifthey had had different mixes of personal income sales and corpo-rate income taxes We did this for each state that used all threetaxes in our data for each year recomputing the percentage errorfor the sum of the three taxes while varying the percentage shareof each tax from zero to 100 percent in 10 percentage-point incre-ments We then summarized the error for each portfolio by com-puting the median absolute percentage error for the sum of thethree taxes across all states and years

Table 6 on the following page shows the results of this analy-sis Each cell represents a combination of personal income salesand corporate income taxes For example the fourth row and sec-ond column represents a portfolio in which 30 percent of the reve-nue is from the income tax and 10 percent of the revenue is fromthe sales tax and therefore by subtraction 60 percent of the reve-nue is from the corporate income tax It has a median absolutepercentage error across all states and years of 78 percent This is

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 23 wwwrockinstorg

80

60

40

20

0

20

40

60

80Income tax error in California and change in income components

Forecast error Wages change Capital gains change

Sources Rockefeller Institute analysis of data from NASBO amp Internal Revenue Service

Figure 12 Capital Gains Are Far More Volatile Than Wage Income and

Are Associated With Increased Forecast Error

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

considerably greater than the 32 absolute percentage error thatstates with all three taxes actually experienced (noted in the tablesubtitle) Cells in the table that are shaded pink have median er-rors that are larger than 9 percent cells that are shaded green arelower than the actual error of 32 percent and cells that have solidborder lines include no corporate income tax As we move to thenorthwest the portfolios have increasing corporate income taxshares

Several conclusions are evident

Increasing reliance on the corporate income tax increaseserrors dramatically

Increasing reliance on the personal income tax relative tothe sales tax tends to increase overall forecast errors

Relatively few combinations of revenue would reduce themedian absolute percent error from what states currentlyhave and they would not reduce forecast error by verymuch Only by virtually eliminating the corporate incometax and significantly increasing reliance on the sales taxrelative to the personal income tax could the typical statereduce revenue forecasting errors and even then most taxcombinations would not reduce forecast errors very much

In summary states are unlikely to solve the problem of reve-nue forecasting error by changing the mix of taxes that they have

Managing Volatility and Errors

States have many possible ways of managing the effects ofrevenue volatility and forecasting errors For example Delawarebudgets less than 100 percent of forecasted revenue Another

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 24 wwwrockinstorg

ps 0 10 20 30 40 50 60 70 80 90 1000 116 105 96 86 75 64 54 44 34 26 2110 106 97 87 75 65 54 46 35 27 2120 99 89 78 66 55 46 36 28 2330 89 78 68 58 47 37 30 2440 80 69 58 48 39 32 2550 71 60 50 41 33 2860 62 53 44 35 3070 55 46 38 3280 49 40 3590 43 38100 42

Sales tax as share of total taxes

Person

alincometaxas

share

oftotaltaxes

Notes (1) Corporate income tax is the residual For example when sales tax is 70 and income tax is 10 the corporateincome tax is 20(2) Limited to the 905 observations for which a state had all three taxes (personal income sales and corporate income)(3) Cells with lines around them have zero corporate income tax(4) Green shaded portfolios have lower total error than historical experience(5) Blue shaded portfolios have total error above 9 percent

Absolute percentage error median across states and years different tax portfoliosCompare to median absolute error of actual portfolios for these data 32

Table 6 States Have Relatively Little Opportunity to

Reduce Forecasting Error by Changing Their ldquoTax Portfoliosrdquo

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

major tool is rainyday funds whichcan help to smoothout volatility Wemight expect statesthat have the mostvolatile and difficultto predict tax reve-nue sources to havelarger rainy dayfunds to help themmanage the conse-quences of tax reve-nue shortfalls AsFigure 13 showsthat is not trueThere is little appar-ent relationship be-tween rainy dayfund size right be-fore the recession hitand the forecast er-

rors states had experienced to that point We are missing rainyday fund data for five states Alaska Idaho Nevada New Hamp-shire and Wyoming Thus those five states are not shown on Fig-ure 13 As mentioned previously we strongly urge caution wheninterpreting data for individual states due to various taxstructures and reliance on various sectors of economy

Conclusions and Policy Implications Regarding

Forecasting Accuracy and Revenue Volatility

We draw several conclusions from this analysis

Increases in forecasting errors have been driven by in-creases in revenue volatility which in turn have beendriven in large part by increased reliance on increasinglyvolatile capital gains

States have relatively little opportunity to reduce volatilitysimply by restructuring their tax portfolios Only by virtu-ally eliminating the corporate income tax and significantlyincreasing reliance on the sales tax relative to the personalincome tax could the typical state reduce revenue forecast-ing errors and even then most tax combinations wouldnot reduce forecast errors very much Changing the struc-ture of individual taxes may be more promising but it canraise difficult tax policy issues For example making in-come taxes less progressive might also make them less vol-atile and easier to forecast but that may run counter todistributional objectives that some policymakers hope for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 25 wwwrockinstorg

AL

AR

AZ

CACO

CT

DEFL

GA

HI

IA

IL

IN

KS KY

LAMA

MD

ME

MI

MN

MO

MSMT

NC

ND

NE

NJ

NM

NY

OH

OK

OR

PA RISC

SD

TN

TX

UTVA

VT

WA

WI

WV

(2)

0

2

4

6

8

10

12

14

16

18

20

0 1 2 3 4 5 6 7 8 9 10

RDFbalancein20

07

Median absolute forecast error 1987 2007

Forecast accuracy and rainy day funds right before the recession(red lines are means)

Figure 13 Size of Rainy Day Funds Does Not Appear

to be Related to Historical Forecast Error

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Because it is so hard to reduce forecasting errors by chang-ing tax structure it is especially important for states to beable to manage the effects of volatility Rainy day fundsare one important tool states can use However the datasuggest that states with larger forecast errors and moredifficult forecasting tasks do not have larger rainy dayfunds perhaps they should

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 26 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Revenue Forecasting Accuracy andthe Timing and Frequency of Forecasts

Prior Research on Forecasting Accuracy and

Forecast Timing and Frequency

Many factors affect forecasting accuracy including errors innational and state economic forecasts tax revenue volatility andother factors In this section we focus on the relationship betweenforecast accuracy and forecast timing and frequency

One important determinant of forecast accuracy is the lengthof time between forecast preparation and the start of the forecastperiod since in general the further ahead the forecast is the lessaccurate it is likely to be There are big differences across states inthis lead time Bretschneider (1989) noted that 68 percent of statesprepared their forecasts within six months of the start of the bud-get year 20 percent prepared forecasts more than six monthsahead and less than a year ahead and that 12 percent of statesprepared forecasts a year or more before start of a budget year asa consequence of biennial budgeting Mikesell and Ross (2014) ina working paper examined the implications of biennial budgetingfor forecasting error in Indiana According to the authors in Indi-ana the revenue forecast is prepared in December of each evenyear and there is an April update prepared in odd years

Because of widely varying differences across states and withinstates over time it is important to take the lead time between aforecast and its time period into account when examining differ-ences in forecast accuracy across states

Another important factor to take into consideration is the fre-quency of forecasts Some scholars argued that more frequent fore-casts actually decrease forecast accuracy For example Deschamps(2004) writes ldquoThough it seems intuitive that revising a forecastmore often would improve accuracy such a practice may increasethe risk of adjusting a forecast to random errorrdquo Voorhees (2004)found a negative correlation between forecasting frequency andforecast accuracy as the forecast frequency increases forecast accu-racy decreases According to Voorhees ldquoWhile frequent forecastswill provide an early warning of a downturn it may be difficult todetermine if the downturn is indeed a trend or just randomnessrdquo(Voorhees 2004) These conclusions might in part reflect reversecausality mdash states with harder revenue forecasting challenges maychoose to update those forecasts more frequently

Measuring Timing of Forecasts

We have developed two measures that allow us to examine theimplications of time lags between when a forecast is prepared andwhen it is used

The first such measure is a simple indicator variable signifyingwhen a statersquos forecast is for the second year of a biennium Theidea is that the lag between forecast preparation and forecast usegenerally is longer for the second year of a biennium This is a

Rockefeller Institute Page 27 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

crude measure because the forecast data states provide to NASBOmay not always reflect this lag mdash in some cases states may pro-vide updated estimates to NASBO mdash and because some second-year-of-biennium states will have longer lags than other states inthe same position On the other hand the variable has the greatadvantage that it has some variation within states over time be-cause several states have shifted from annual budgeting to bien-nial budgeting and others have shifted in the other direction Wecapture this variation in our measure

Our second measure is based on the survey we conductedwherein we asked states explicitly when they prepared their fore-casts The survey results indicate that there is a wide variationamong the states in terms of timing of the forecasts prepared forthe adopted budget For example officials in Arkansas reportedthat they prepare the preliminary estimates as early as in Octoberwhile officials in Pennsylvania indicated that they prepare prelim-inary estimates as late as in April or May

From these questions we developed an estimate of the numberof weeks between when a forecast was prepared and when the as-sociated fiscal year began This has the advantage of capturingmore subtle variation across states as it is measured in number ofweeks rather than by a single indicator It has the disadvantagehowever of not varying within states over time We only haveone number per state and we have to assume that the lag is con-stant within states over time which is not always the case

Accuracy and the Timing of Forecast

Descriptive Analysis

Figure 14 on the following page shows the median absolutepercentage error and the median percentage error for the sum ofthe three major taxes categorized by whether the forecast is for anannual budget the first year of a biennium or the second year ofa biennium It is not at all apparent from this that this measure oftiming matters although there is a hint that absolute errors maybe larger in the second year of a biennium However this does nottake into account the fact that forecast difficulty might have var-ied by these groups We need statistical analysis to take that intoaccount

Table 7 on the following page shows that the lag between fore-cast preparation and use (our second measure) tends to be greaterin the second year of a biennium (our first measure) and so wewould expect greater errors for those forecasts given that theymust be produced further ahead of time

The survey results indicate that in some states there are longerlags between forecast preparation and the start of the fiscal yearwhich means states provide forecasts further ahead Presumablyforecasting is a much harder task for those states that not onlydonrsquot prepare multiple forecasts but also have to prepare forecastsfar in advance Therefore forecasting is likely to be more error

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 28 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

prone This is partic-ularly relevant insome biennial bud-geting states wherethey have to forecastthe second year ofthe biennium beforethe first year haseven begun

The survey re-sults reveal thatthere is a wide vari-ation among thestates both in termsof timing of the fore-cast and frequencyof forecasts

In the survey weasked the states toindicate approxi-mately how manyweeks ahead of the

new state fiscal year the estimatesin the adopted budget are pre-pared in a typical year We tried tofill in the data for any missingstates based on publicly availableinformation on state governmentwebsites In some cases states in-dicated the number of weeks be-

tween the initial forecast and the start of the state fiscal year Insuch cases we have revised the number of weeks to include thelag between the latest forecast used in the adopted budget and thestart of the fiscal year There is a wide variation among the statesin terms of how far in advance each state prepares revenue esti-mates For example among the states with an annual budget cy-cle Alabama prepares revenue forecasts as early as thirty-fiveweeks before the start of the state fiscal year on October 1st Onthe other hand states such as Delaware Pennsylvania and Ten-nessee use the revenue estimates updated about two weeks beforethe start of the state fiscal year (see Figure 15 on the next page)

Among the states with a biennial budget cycle Texas preparesrevenue forecasts as early as thirty-four weeks before the start ofthe first year of the biennium while Ohio uses updated forecastsjust about two weeks before the start of the first year of the bien-nium (see Figure 16 on the next page)

Figure 17 on page 31 shows the average lag between forecastpreparation and use for the second year of a biennium In generalthe lag between forecast preparationupdate and the start of thesecond year of a biennium is greater than the lag between forecast

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 29 wwwrockinstorg

34

33

39

3

3

3

3

3

3

4

4

4

4

4

4

Annual budget Biennial budget 1st year Biennial budget 2nd year

Forecasting year and median absolute percent error

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 14 Forecast Errors Without Controlling for Forecast Difficulty

Do Not Appear Related to the Forecasting Year

25th percentile Median 75th percentileAnnual budget 31 75 135 205Biennial budget 1st year 19 80 120 190Biennial budget 2nd year 19 100 220 295Source Rockefeller Institute analysis of data from survey of state forecasting officials

of statesBudget cycle

Lag between when revenue forecast is prepared and when it is used of weeks from preparation to budget adoption

Table 7 Revenue Forecasts for the Second Year of a Biennium

Typically Are Prepared Well Before the Fiscal Year Starts

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

preparation and thestart of the first yearof the biennium Forexample in Ken-tucky the lag be-tween forecastpreparation and thefirst year of the bien-nium is thirteenweeks while the lagbetween forecastpreparation and thesecond year of thebiennium issixty-five weeks(Note Based on sur-vey results andbased on the infor-mation retrievedfrom the publiclyavailable docu-ments it appears

that Texas does not update the forecasts for the second year of thebiennium)

Regression Analysis

Table 8 on the next page reports the results of regression mod-els that examine how forecast accuracy (the absolute percentageerror) varies depending on whether the state is forecasting for the

second year of a bi-ennium after con-trolling for forecastdifficulty and otherimportant factorsThe different formu-lations of the modelare designed to en-sure that weexamine thisrobustly

The results ofthe analysis are veryclear After control-ling for forecast dif-ficulty forecasterrors for the secondyear of a bienniumare considerablylarger than for otherforecast periods

3530

2625

242424

212020

191818

1616

13513

1212

1010

88

76

544

222

0 3 6 9 12 15 18 21 24 27 30 33 36

ALWVGAID

MONMVTIAMIUTOKSCSDMAMDCOAKFLMSAZKSARNJLACARIILNYDEPATN

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 15 Average Number of Weeks the Forecast Is

Completed Before the Start of the Fiscal Year Annual States

3428

242020

1816

131212

88888

76

42

0 3 6 9 12 15 18 21 24 27 30 33 36

TXVAWYMNNDNCWAKYINMTCTMENENVNHWIORHIOH

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 16 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (First Year of Biennium)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 30 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

about 13 to 17 per-centage points de-pending on themodel

Table 9 on thenext page summa-rizes a similar analy-sis in which thetime-lag variable ofinterest is our mea-sure of the numberof weeks betweenforecast preparationand use from thesurvey Again wereport on severalmodels (we do notreport on a modelwith state fixed ef-fects because thismeasure does not

vary across years within states) Again the variable is statisticallysignificant at the 5 percent level The coefficients suggest that eachweek of lag is associated with increased error of about aone-twentieth of a percentage point (005) ndash for example tenweeks of lag would be associated with about a half a percentagepoint greater error which seems meaningful

The lag for an annual budgeting state at the 25th percentile isabout 75 weeks whereas at the 75th percentile it is 205 weeks

That thirteen weekdifference appearsto cost about 06 to07 percentagepoints in additionalerror on average

Accuracy and the

Frequency of

Forecast Updates

We also exam-ined the relationshipbetween forecast ac-curacy and the fre-quency of forecastupdates One mightargue that more fre-quent forecasts are asign of greater pro-fessionalism andperhaps greater

6564

6054

30282828

24202020

1688

744

0 10 20 30 40 50 60 70

TXKYMTNHOHNVVANDINWYMNWANEMENCCTWIORHI

Number of weeksSource Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 17 Average Number of Weeks the Forecast Is Completed

Before the Start of the Fiscal Year Biennial States (Second Year of Biennium)

Key variable of interest2nd year of biennium indicator variable 1337 1649 1676 1650 Controls that vary across modelsState fixed effects No No No YesYear fixed effects No No Yes YesBiennial budgeting NA (0396) (0409) (1985)Controls for forecast difficultyNaiumlve forecast error absolute value 0235 0235 0189 0185 Square of naiumlve forecast error (0001) (0001) (0001) 0000 change in tax revenue absolute value 0217 0217 0224 0213 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1539) (1546) (0461) (0398)Square of change in personal income 0130 0131 0075 0060Tax type controls (relative to CIT)PIT (6299) (6300) (6626) (6921) Sales Tax (7987) (7984) (8414) (8454)

Adjusted r squared 0398 0398 0423 0447Number of observations 3256 3256 3256 3256

Relationship between forecast accuracy and whether the forecast is for 2nd year of bienniumCoefficients and significance levels

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and tax

Model 1 Model 2 Model 3 Model 4Dependent Variable Forecast error as ofactual absolute value

Table 8 Forecasts for the Second Year of a Biennium

Are Less Accurate Than Other Forecasts

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 31 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

professionalism isassociated withsmaller errors How-ever as noted earlierthe research on thisis ambiguous

In the surveystate forecasting of-ficials were askedwhether they reviserevenue estimates orrelease multiple esti-mates throughoutthe fiscal year Moststate officials both inthe executive andlegislative branchesindicated that theydo usually providemultiple revenue es-timates throughout

the fiscal Figure 18 below captures the list of states where officialsin executive and legislative branches indicated absence of revisedor multiple revenue estimates

Officials in both executive and legislative branches of the gov-ernment indicated various timelines for the updates of the reve-nue estimates In some states the updates are done on an ad hocbasis while in others updates are done bimonthly quarterlysemiannually or annually In very few states revenue estimates

are updated morefrequently duringthe legislativesession

Table 10 on thenext page shows themedian absolutepercentage error foreach grouping ofnumber of forecastsin a year It does notlook from this likemore frequent fore-casts are associatedwith smaller errors

The table how-ever does not con-trol for forecastdifficulty and thatmay be differentacross the states We

Key variable of interest of weeks between forecast preparation amp use 0060 0058 0052 0055 Controls that vary across modelsYear fixed effects No No Yes YesBiennial budgeting NA 0226 0288 3196Controls for forecast difficultyNaiumlve forecast error absolute value 0240 0240 0193 0189 Square of naiumlve forecast error (0001) (0001) (0001) (0001) change in tax revenue absolute value 0216 0216 0222 0211 Square of change in tax revenue (0001) (0001) (0001) (0001) change in personal income absolute value (1583) (1572) (0509) (0431)Square of change in personal income 0133 0132 0077 0063Tax type controls (relative to CIT)PIT (6258) (6259) (6608) (6900) Sales Tax (8006) (8008) (8456) (8440)

Adjusted r squared 0401 0401 0425 0448Number of observations 3166 3166 3166 3166

Relationship between forecast accuracy and lag between forecast preparation and use

Significance codes 01 level 1 level 5 level 10 level Data NASBO forecast errors for states by year and taxNote Data for weeks between forecast preparation and use do not vary over time within states

Coefficients and significance levelsModel 1 Model 2 Model 3 Model 4

Dependent Variable Forecast error as ofactual absolute value

Table 9 Forecasts Become Less Accurate

the Longer the Lag Is Between Preparation and Use

42

37

3NV OH WV

9Al AK NVOH OK TNTX WI WY

0

5

10

15

20

25

30

35

40

45

Executive Legislature

Num

bero

fStates

Yes No

Source Rockefeller Institute analysis of data from survey of state forecasting officials

Figure 18 States That Revise Revenue Estimates or

Release Multiple Estimates Throughout the Year

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 32 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

ran regression models to examine thisquestion but none of them suggest thatforecast accuracy is associated with fre-quency of forecast updates This is consis-tent with past research

There are other good reasons for statesto update forecasts regularly Forecastersunderstand that forecasts will be wronghence the dictum cited by Paul Samuelsonand others ldquoIf you must forecast forecastoftenrdquo (Samuelson 1985) The GovernmentFinance Officers Association has long rec-ommended monitoring measuring and

evaluating budgetary performance and adjusting as necessary(National Advisory Council on State and Local Budgeting GFOA1998)

Conclusions and Policy Implications Regarding Forecasting

Accuracy and the Timing and Frequency of Forecasts

Further-ahead forecasts are more error prone whether we ex-amine the forecasts for the second year of a biennium which gen-erally are prepared further in advance or use a measure of the lagbetween forecast preparation and the start of the forecast periodthat we developed from our survey In both cases the effect ismeaningful The data suggest that accuracy worsens by well overa percentage point in the forecast for the second year of a bien-nium and worsens by nearly a half a percentage point for everyten weeks of lag between the time a forecast is prepared and thestart of the forecast period

There is no evidence from our limited data that frequent fore-cast updates lead to greater accuracy and this is consistent withpast research

The policy implications of the first part of our analysis areclear States should minimize unnecessary lags between forecastpreparation and the start of the fiscal year and should updatethose forecasts as close as possible to the start of the fiscal year asmany states do Even though there is no evidence that more fre-quent forecast updates during the forecast period will lead tomore accurate forecasting it is good management practice to up-date forecasts regularly as the year progresses so that finance offi-cials are managing the budget using the best availableinformation

PIT Sales tax CIT1 22 43 23 1182 17 47 24 1113 3 35 17 1084 6 40 27 1256 1 41 NA 17912 1 65 18 105

Median absolute percentage errorFrequency of forecast updates and forecast accuracy

of forecastsin a year

of states

Source Rockefeller Institute analysis of data from survey of stateforecasting officials and NASBO

Table 10 Frequency of Forecast Updates Does Not Appear Related

to Forecast Accuracy (Before Adjusting for Forecast Difficulty)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 33 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Revenue Forecasting Accuracyand Institutional Arrangements

Prior Research on Forecasting Accuracy

and Institutional Arrangements

Many factors contribute to revenue forecasting accuracy in-cluding forecast methodology (such as utilization of expert opin-ion nominal groups Delphi methods ARIMA exponentialsmoothing moving average simple regression multiple regres-sion multiple equation regression and simulation methods) po-litical factors (such as political party composition of thegovernment) economic factors (such as the economic condition ofthe state unemployment rate or per capita income) and institu-tional factors (such as tax and expenditure limits budget cyclesparties involved in revenue forecasting process frequency of theforecast whether the budget is bound by the forecast the use ofthe university faculty in the forecast preparation the presence ofan economic advisory council) (Rose and Smith 2011 Voorhees2004)

One institutional arrangement of great interest is consensusforecasting but it turns out that it is quite difficult to define andmeasure NASBO defines a consensus forecast as a ldquorevenue pro-jection developed in agreement through an official forecastinggroup representing both the executive and legislative branchesrdquo(NASBO 2008)

The definition used in the Government Performance Projectsurvey for 2005 was

A process that requires a panel of experts brought to-gether for purposes of generating the requested forecastExperts may include officials from the executive and leg-islative branches of the state as well as external research-ers or officials from universities private consultants orcitizens18

According to a study published by the Federation of Tax Ad-ministrators (FTA) ldquoThere are two types of consensus forecastingIn the first type members of the consensus group are composedof professionals from the different agencies responsible for reve-nue forecasting (ie Budget Office Revenue Department or Leg-islative Agencies) Each agency brings its own forecast andassumptions to the table By discussing its differences the groupcomes to an agreement on one single forecast for the statehellip Thesecond type of consensus forecasting includes independentgroups whose members are appointed by the executive and legis-lature These groups typically turn to the resources of otheragencies to make their economic and revenue forecastrdquo (FTA1993)

Before discussing the research on consensus forecasting it isimportant to touch on a related area combining separate forecasts(regardless of whether there is a formal consensus body or not)

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 34 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

This research goes well beyond revenue forecasting and pertainsto forecasts in general including business forecasts economicforecasts and other kinds of forecasts It concludes emphaticallythat combining multiple forecasts leads to increased forecast accu-racy (For summaries of this research see Batchelor and Dua 1988Clemen 1989 Voorhees 2004 Weltman 1995-96)

The research on combining economic and business forecasts isconvincing and certainly suggests that consensus state revenueforecasting mdash or at least combining forecasts as part of consensus-like institutional arrangements mdash could increase accuracy How-ever research bearing directly on institutional arrangements ofconsensus state revenue forecasting is less convincing althoughseveral researchers have argued that the data support the hypoth-esis that consensus forecasting is more accurate than other institu-tional arrangements The evidence for this so far is slim althoughthat does not necessarily mean the conclusion is wrong

Some analysts assume that consensus forecasting diminishesforecast bias and increases forecast accuracy as it ldquotakes the poli-tics out of the revenue forecast accuracyrdquo (Qiao 2008) But de-pending on how it is structured consensus revenue forecastingmay not remove politics from the process it faces its own chal-lenges such as being an extremely time-consuming process or in-volving genuine differences among the members of forecastinggroup (Qiao 2008) The time-consuming nature of consensus fore-casting can increase lags between forecast preparation and fore-cast use in turn potentially reducing accuracy a topic we discusslater in this report

Qiao (2008) used data from the Government Performance Pro-ject19 and analyzed revenue forecast errors for each state for fiscalyears 2002-04 using 5 percent as a threshold for forecast accuracyThe results indicated that eleven out of twenty-seven states thatutilize consensus forecasting made errors less than 5 percent ineach study year and only one state made errors greater than 5 per-cent in all the three years Qiao argues that while the consensusforecasting process certainly contributes to revenue forecast accu-racy there are other important variables that are essential for im-proving revenue forecasting accuracy Such variables include staterevenue structures the duration of time between the developmentof a revenue forecast and its implementation forecast frequencythe use of economic advisors and university consultation eco-nomic stability of the nation and the state political pressures andpolitical ideology (Qiao 2008)

Voorhees (2004) examined the impact of consensus forecastingon state revenue estimating error He argued that errors in staterevenue forecasts are not solely caused by the fluctuations in theeconomy but also are attributable to the state institutional struc-tures and the degree of consensus required for the state forecastVoorhees conducted a survey of forecasters in forty-four statesand collected institutional and methodology data pertaining tostate revenue forecasting for the years 1990-97 In addition

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 35 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Voorhees relied on secondary data from NASBOrsquos Fiscal Surveyof States for the years 1989-98 for calculating the forecast errorsVoorhees classified forecasting processes in the states into one ofthe following six categories (1) single executive agency (2) singlelegislative agency (3) two separate forecasts (executive and legis-lative) (4) consensual multiple executive agencies (5) consensualexecutive and legislative agencies and (6) consensual independ-ent conference Voorhees used the locus of forecast formulationand the degree of diversity in participation at that locus as a proxyfor consensus forecasting He concluded that an increase of con-sensus in the formulation of the forecast leads to decrease in fore-cast errors More specifically ldquofor each unit increase in theconsensus variable the forecast error is expected to decrease byapproximately 01075 and 01084 standard errors respectivelyrdquo(Voorhees 2004)

In a recent study Krause and Douglas (2013) examined the re-lationship between the organizational structure of the consensusgroups (both in terms of the size and the diversity of the consen-sus groups) and revenue forecasting accuracy The authors useddata for fiscal years 1987 to 2008 across twenty-five consensusgroup commissions The central theme of their study was to em-pirically test the following panel size-diversity trade-off hypothe-sis ldquoIncreases (Decreases) in CG [consensus group] commissiondiversity will improve revenue-forecast accuracy when CG com-mission size is decreasing (increasing)rdquo (Krause and Douglas2013) The authors operationalize the organizational diversitybased on the presence of the following three types of players ap-pointee (partisan nonpartisan) institutional (legislature gover-nor independent elected agency heads) and partisan (DemocratRepublican third party) Based on empirical results the authorsargue that organizational diversity in smaller consensus groups(with three or four members) yields a marginal improvement inrevenue forecasting accuracy However organizational diversityyields an increasingly adverse marginal impact on revenue fore-cast accuracy for larger consensus groups ldquoWhile organizationaldiversity has salutary benefits for collective decision making itcan reduce the accuracy of collective decisions in larger panelssince coordination of heterogeneous expertise becomes problem-atic This is because the marginal coordination losses from ineffi-cient pooling of expertise within the group will exceed itsmarginal information gains regarding accurate collective deci-sionsrdquo (Krause and Douglas 2013)

There are quite a few lessons from this literature review aboutthe impact of institutional arrangements on revenue forecasting

There is overwhelming evidence that combining forecastseven in simple ways tends to reduce the magnitudes of er-rors on average and reduce instances of extreme errorThis may be one of the reasons that some studies havefound consensus forecasting of state tax revenue to bemore effective than nonconsensus methods

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 36 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

Nonetheless the evidence supporting consensus forecast-ing of state revenue is decidedly less overwhelming thanthat in favor of combining forecasts (with combining beinga more mechanical task)

In evaluating consensus forecasting it is important to real-ize that there are important differences in how consensusforecasting is defined and important differences acrossstates in how consensus forecasting is implemented

Observations From the Survey

There is wide variation across states in terms of budgetingprocesses and rules and regulations governing revenue forecast-ing Some of these forecasts are prepared by a single entity alonesuch as either executive or legislative branch or both independ-ently and some are a joint effort of both executive and legislatureSome are prepared by an independent formal group generallyformed by a mix of executive legislature university economistsand experts from private or public sectors It is generally believedthat a depoliticized forecast is more objective

In the survey by the Rockefeller Institute state forecasting offi-cials were asked to indicate who is involved in the revenue esti-mating process The state forecasting officials were specificallyasked to specify which agencies are involved from the executiveand legislative branches as well as whether there are other partiesinvolved such as academicians experts from the private sector orany other parties According to the survey responses in moststates one or more agencies of the executive branch are involvedin the revenue estimating process The executive branch does nottake any role in producing revenue estimates only in very fewstates including Hawaii Montana Nevada and Washington

In Hawaii the revenue estimates are prepared by the Council

on Revenues which is attached to the Department of Taxation foradministrative purposes The Council on Revenues consists ofseven members three of whom are appointed by the governor forfour-year terms and two each of whom are appointed by the pres-ident of the Senate and the speaker of the House of Representa-tives for two-year terms

In Montana the preparation of the revenue estimates is underthe jurisdiction of the legislative branch and is carried out by theLegislative Fiscal Division The Legislative Fiscal Division preparesand presents the revenue estimates to the Revenue and

Transportation Interim Committee (RTIC) The RTIC which is atwelve-member joint bipartisan committee of the legislature usesthe information provided by the Legislative Fiscal Division to esti-mate the amount of revenue that the state will receive during thenext biennium

In Nevada the revenue forecast is provided by the statersquosEconomic Forum which is a panel of five economic and taxationexperts from the private sector all appointed by the governor Allagencies of the state including the governor and Nevada

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 37 wwwrockinstorg

legislature are required to use the Forumrsquos forecast which is pro-vided shortly before the beginning of a new legislative session

In Washington revenue estimates are provided by theEconomic and Revenue Forecast Council which is an independentagency and is not a part of the executive or legislative branchesThe Council was created in 1984 and consists of two members ap-pointed by the governor and four members from the legislature

There is some discrepancy in the answers provided by the offi-cials from the executive versus legislative branch for some statesin terms of the role of other parties such as the legislative branchacademicians and private sector experts in the revenues forecast-ing process Thirty of the forty-five respondents from the execu-tive branch and thirty-three of the forty-six respondents from thelegislative branch indicated that the legislative branch is involvedin the revenue estimating process in their respective states Ac-cording to survey responses only a few states involve academi-cians and private sector experts in the revenue estimating process

State officials were also asked to indicate if there is a formalgroup or organization tasked with developing or signing off onrevenue estimates Once again there is some discrepancy in theresponses provided by the officials from the executive branch ver-sus by the officials from the legislative branch Fifteen of forty-sixrespondents from the legislative branch indicated that there is noformal group and only nine respondents from the executivebranch indicated the absence of any formal group for revenue es-timates In most states where officials indicated existence of a for-mal group tasked with the preparation of revenue estimates theyalso indicated that those groups hold formal meetings throughoutthe year Again wide variation exists among the states in terms offrequency of the meetings by the groupsorganizations taskedwith the revenues estimation For example in some states such asAlaska Mississippi Nevada North Dakota or Virginia the groupmeets only once or twice in the given fiscal year In other statessuch as Washington the group meets more frequently eight timesper fiscal year

According to the survey responses in approximately two-thirdsof the states the participants in the revenue estimating processcome to an agreement on a single number Moreover in most ofthese states both the governor and the legislature are required touse the agreed upon estimates in the adopted budget

Due to ambiguity of the definition it is hard to identify the ex-act number of states that have a consensus forecasting processFor example in some states there is no official consensus forecast-ing group but there is an informal process for reaching agreementbetween the legislature and executive

The survey results indicate that states use various institutionalarrangements for revenue forecasting According to survey re-sponses there is a formal group tasked with the preparation ofrevenue estimates in at least two-thirds of the states Moreoversuch groups hold formal meetings throughout the year In some

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 38 wwwrockinstorg

states the revenue estimating groups have more clear and routineresponsibilities and processes in place For example a state officialfrom Connecticut indicated ldquoThe consensus process occurs threetimes per year in November January and April The January es-timate binds the Governorrsquos budget proposal and the April esti-mate binds the Legislaturersquos adopted budgetrdquo On the other handthe consensus revenue estimating process is more informal pro-cess in North Carolina ldquoThe consensus revenue forecast is an in-formal process that does not require either the executive orlegislative branch to use the revenue estimaterdquo said a state officialfrom North Carolina

In very few states not only there is no consensus revenue esti-mating process in place but also revenue estimating is under thejurisdiction of a sole entity For example in New Jersey officialrevenue estimates are prepared by the executive branch only andthere are no formal processes in place

Descriptive Analysis

We incorporated data on consensus forecasting and other fore-casting mechanisms into our analysis relying primarily on datacollected by Rose and Smith (2011) Table 11 shows the median

absolute percentage error under each ofseveral forecasting arrangements It is ap-parent that consensus forecasting as mea-sured by Rose and Smith was notassociated with more accurate forecasts(absent controls for forecast difficulty)Both expert forecasts and separate legisla-tive forecasts were associated with less ac-curate forecasts although the differencemay not be significant

We also examined the potential impactof these variables econometrically repeat-ing our earlier models that controlled forforecast difficulty In none of the specifica-tions did the variables indicating the type

of institutional arrangement have a statistically significant associa-tion with forecast accuracy at the 005 significance level We re-peated the analysis with the forecasting arrangement variables wecollected in the survey and none of them had a statisticallysignificant association with accuracy either

Conclusions and Policy Recommendations Regarding

Forecasting Accuracy and Institutional Arrangements

Our reading of the literature is that there is very little relation-ship between consensus forecasting and forecast accuracy That isconsistent with our examination of these data also However aswe have noted before the evidence in favor of examining andcombining forecasts is overwhelming Combining forecasts tends

States with thisprocess

States with adifferent process

Consensus forecast 34 35Executive forecast 34 35Expert forecast 37 33Separate legislative forecast 37 34

Median absolute percentage forecast error under different arrangementsto forecasting process

Sum of PIT sales taxes and CIT 1987 2007 by type of forecast

Sources Rockefeller Institute analysis of NASBO forecast error data anddata from Rose S amp Smith D L (2011) Budget Slack Institutions andTransparency Public Administration Review 72 (2) 187 195

Table 11 Summary Data Without Controlling for Forecast Difficulty

Do Not Suggest Consensus Forecasts Are More Accurate

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 39 wwwrockinstorg

to lead to lower errors Processes that encourage this to happenmay lead to lower errors

Beyond that it is good practice to try to insulate forecastingfrom the political process and consensus forecasting can helpachieve that

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 40 wwwrockinstorg

Appendix A Survey of State Officials

The Rockefeller Institute of Government conducted tworounds of surveys Both surveys were conducted online usingGoogle Docs

The first round of the survey was conducted in July-August of2013 The purpose of the first round of the survey was to ensurethat we understand as precisely as possible the information thatstate budget officials provide to NASBO on revenue forecasts andpreliminary actuals as published in the Fiscal Survey of States Thesurvey questions were designed to help us get a better under-standing of when estimates that underlie state budgets are devel-oped as that influences the difficulty of the forecasting job and tobe sure we understand how the forecasts are used in the budgetprocess For the first round of survey we sent out the surveyquestionnaire via online survey tool to state budget officers in allfifty states We developed two survey instruments one for thestates with an annual budget cycle and another one for the stateswith a biennial budget cycle For the biennial states we asked twoadditional questions to understand what estimates they provideto NASBO for the second year of the biennium We receivedthirty-four usable responses from the states of whichtwenty-three responding states were on annual budget cycleswhile eleven states were on biennial budget cycles

The second round of the study was conducted in the monthsof August and September of 2013 with a few follow-ups in Octo-ber 2013 The purpose of the second round of the survey was tocollect additional information from all the states to get further in-formation related to procedures and practices of revenue forecast-ing process in the states We sent out the survey questionnaire tostate budget officers representing both executive and legislativebranches of government in all fifty states We received fort-five re-sponses from the state officials representing the executive branchand forty-six responses from state officials representing the legis-lative branch On the executive side we did not receive responsesfrom Georgia Missouri Montana North Carolina and Oregonand on the legislative side no response were received fromMassachusetts Michigan New Hampshire and South Carolina

The survey questions asked during the first round of the sur-vey administration were targeted on getting better understandingof what is being reported to NASBO The most important ques-tions asked of the states were focused on whether the revenue es-timates being reported to NASBO are based on the final estimatesfor the adopted budget whether the revenue estimates take intoaccount legislative and administrative changes approximatelyhow many weeks ahead of the start of the new state fiscal year arethe estimates in the adopted budget prepared and whether the bi-ennial states report estimates for the first and second years ofbiennium separately or combined

Among the respondent states thirty-one states indicated thatthe estimates provided to NASBO are the final estimates used for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 41 wwwrockinstorg

the adopted budget Alabama Arizona and Idaho were the onlythree states indicating that the revenue estimates provided toNASBO are not the final estimates used for the adopted budgetTwenty-six states also indicated that the revenue estimates bindthe budget In most of the twenty-six states where the revenue es-timates are binding the budget such binding is defined either inthe constitution or by statute And only in a handful of states arethe revenue estimates binding based on customary practice

We also asked the states with the biennium budget cyclewhether they submit updated estimates to NASBO for the secondyear of the biennium Ten of the eleven responding states with thebiennial budget cycle indicated that they do provide updated esti-mates to NASBO for the second year of the biennium in order toaccount for legislative changes and updated economic forecastconditions

The main purpose of the second round of the survey was toget a better understanding about general procedures and practicesof revenue estimating processes in the states The survey ques-tions were targeted into getting more information about the tim-ing of forecasts frequency of forecast updates the long-termforecasting practices parties involved in the forecasting processesand whether there is a formal group that plays a major role in therevenue estimating process

Officials in at least half of the states indicated that they typi-cally generate revenue estimates for one or more years beyond theupcoming budget period Overall most states release revenue es-timates for another two to five years Only in very few states arethe forecasts provided for more than five years beyond the budgetcycle One example is Alaska where officials in the legislativebranch indicated generating revenue estimates for six years whileofficials in the executive branch said that the revenue estimatesare provided for nine additional years beyond the budget cycle

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 42 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 43 wwwrockinstorg

1 Statesrsquo Revenue Estimating Cracks in the Crystal Ball (Philadelphia PA and Albany NY Pew CharitableTrusts and Rockefeller Institute of Government March 2011)httpwwwrockinstorgpdfgovernment_finance2011-03-01-States_Revenue_Estimating_Reportpdf

2 Before the fall of 2013 NASBO conducted the Fiscal Survey of States jointly with the National Governors Association

3 These observations are not independent of those for the individual taxes of course

4 For more information please see Appendix A Survey of State Officials at the end of this report

5 Richard Mattoon and Leslie McGranahan Revenue Bubbles and Structural Deficits Whats a state to doWorking Paper (Chicago IL Federal Reserve Bank of Chicago April 2012)httpswwwchicagofedorgdigital_assetspublicationsworking_papers2008wp2008_15pdf

6 The upper whisker extends from the hinge (the 75th percentile) to the highest value that is within 15 timesthe interquartile range and the lower whisker extends similarly downward from the lower hinge Databeyond the end of the whiskers are outliers and plotted as points

7 This test involves regressing the forecast on the actual and testing the joint hypothesis that the intercept iszero and the slope is one See Jacob A Mincer and Victor Zarnowitz ldquoThe Evaluation of EconomicForecastsrdquo in Economic Forecasts and Expectations Analysis of Forecasting Behavior and Performance (New YorkNY National Bureau of Economic Research Distributed by Columbia University Press 1969) 1-46httpwwwnberorgchaptersc1214

8 The table shows forty-two states with a personal income tax There are only forty-one states with a broad-based personal income tax Tennessee which has a narrow tax on interest and dividends is included in theNASBO data This tax is a relatively small share of total Tennessee tax revenue

9 This is similar to an approach developed contemporaneously in a working paper by John Mikesell andJustin Ross (see Bibliography) regarding forecasting in Indiana

10 Our naiumlve forecasting model ends in 2013 because we do not yet have full data on actual tax revenue for 2014

11 Synchronization was calculated by using a monthly coincident index at the state level relying on(a) nonfarm payroll employment (b) average hours worked in manufacturing (c) the unemployment rateand (d) wage and salary disbursements Given the idiosyncrasies and data limitations between states thesemeasures are not a perfect barometer for each statersquos economy but are a good start Perhaps furthermeasures could be incorporated for example measures of tourism and travel as Hawaii was found tosynchronize with the national economy only 574 percent of the time

12 While recessions typically include one or more quarters in which real GDP declines these are annual dataand GDP declines on an annual basis are less common

13 For example analysts often use this to measure volatility of daily stock prices

14 This is a different measure of volatility than used by Crain (see Bibliography) discussed previously Fortechnical reasons we believe this is a better measure of volatility of tax revenue than his measure whichexamines deviations from a trend line

15 The tax data have not been adjusted for legislative changes because it is impractical to do so A smallportion of observed changes in state tax volatility may be attributable to legislative changes We do notbelieve this would affect conclusions materially

16 For several years before the 2007 recession a strand of economic research identified what came to be knownas the ldquoGreat Moderationrdquo mdash the idea that the United States economy was becoming less volatile mdash andexamined its potential causes While the data clearly supported declining economic volatility over longerperiods than shown in this figure the idea that we have entered a new period of stability is now in tatters

17 The data on wages and capital gains are from the Statistics of Income branch of the Internal RevenueService and were provided to us by Leslie McGranahan of the Chicago Federal Reserve Bank

18 States are listed in the spreadsheet ldquoGPP_Revenue Estimate Methods FY 2004xlsrdquoworksheet ldquoMeasurementMethods 2005rdquo

19 See ldquoArchived Project Government Performance Projectrdquo Pew Charitable Trustshttpwwwpewtrustsorgenarchived-projectsgovernment-performance-project

Endnotes

to lead to lower errors Processes that encourage this to happenmay lead to lower errors

Beyond that it is good practice to try to insulate forecastingfrom the political process and consensus forecasting can helpachieve that

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 40 wwwrockinstorg

Appendix A Survey of State Officials

The Rockefeller Institute of Government conducted tworounds of surveys Both surveys were conducted online usingGoogle Docs

The first round of the survey was conducted in July-August of2013 The purpose of the first round of the survey was to ensurethat we understand as precisely as possible the information thatstate budget officials provide to NASBO on revenue forecasts andpreliminary actuals as published in the Fiscal Survey of States Thesurvey questions were designed to help us get a better under-standing of when estimates that underlie state budgets are devel-oped as that influences the difficulty of the forecasting job and tobe sure we understand how the forecasts are used in the budgetprocess For the first round of survey we sent out the surveyquestionnaire via online survey tool to state budget officers in allfifty states We developed two survey instruments one for thestates with an annual budget cycle and another one for the stateswith a biennial budget cycle For the biennial states we asked twoadditional questions to understand what estimates they provideto NASBO for the second year of the biennium We receivedthirty-four usable responses from the states of whichtwenty-three responding states were on annual budget cycleswhile eleven states were on biennial budget cycles

The second round of the study was conducted in the monthsof August and September of 2013 with a few follow-ups in Octo-ber 2013 The purpose of the second round of the survey was tocollect additional information from all the states to get further in-formation related to procedures and practices of revenue forecast-ing process in the states We sent out the survey questionnaire tostate budget officers representing both executive and legislativebranches of government in all fifty states We received fort-five re-sponses from the state officials representing the executive branchand forty-six responses from state officials representing the legis-lative branch On the executive side we did not receive responsesfrom Georgia Missouri Montana North Carolina and Oregonand on the legislative side no response were received fromMassachusetts Michigan New Hampshire and South Carolina

The survey questions asked during the first round of the sur-vey administration were targeted on getting better understandingof what is being reported to NASBO The most important ques-tions asked of the states were focused on whether the revenue es-timates being reported to NASBO are based on the final estimatesfor the adopted budget whether the revenue estimates take intoaccount legislative and administrative changes approximatelyhow many weeks ahead of the start of the new state fiscal year arethe estimates in the adopted budget prepared and whether the bi-ennial states report estimates for the first and second years ofbiennium separately or combined

Among the respondent states thirty-one states indicated thatthe estimates provided to NASBO are the final estimates used for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 41 wwwrockinstorg

the adopted budget Alabama Arizona and Idaho were the onlythree states indicating that the revenue estimates provided toNASBO are not the final estimates used for the adopted budgetTwenty-six states also indicated that the revenue estimates bindthe budget In most of the twenty-six states where the revenue es-timates are binding the budget such binding is defined either inthe constitution or by statute And only in a handful of states arethe revenue estimates binding based on customary practice

We also asked the states with the biennium budget cyclewhether they submit updated estimates to NASBO for the secondyear of the biennium Ten of the eleven responding states with thebiennial budget cycle indicated that they do provide updated esti-mates to NASBO for the second year of the biennium in order toaccount for legislative changes and updated economic forecastconditions

The main purpose of the second round of the survey was toget a better understanding about general procedures and practicesof revenue estimating processes in the states The survey ques-tions were targeted into getting more information about the tim-ing of forecasts frequency of forecast updates the long-termforecasting practices parties involved in the forecasting processesand whether there is a formal group that plays a major role in therevenue estimating process

Officials in at least half of the states indicated that they typi-cally generate revenue estimates for one or more years beyond theupcoming budget period Overall most states release revenue es-timates for another two to five years Only in very few states arethe forecasts provided for more than five years beyond the budgetcycle One example is Alaska where officials in the legislativebranch indicated generating revenue estimates for six years whileofficials in the executive branch said that the revenue estimatesare provided for nine additional years beyond the budget cycle

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 42 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 43 wwwrockinstorg

1 Statesrsquo Revenue Estimating Cracks in the Crystal Ball (Philadelphia PA and Albany NY Pew CharitableTrusts and Rockefeller Institute of Government March 2011)httpwwwrockinstorgpdfgovernment_finance2011-03-01-States_Revenue_Estimating_Reportpdf

2 Before the fall of 2013 NASBO conducted the Fiscal Survey of States jointly with the National Governors Association

3 These observations are not independent of those for the individual taxes of course

4 For more information please see Appendix A Survey of State Officials at the end of this report

5 Richard Mattoon and Leslie McGranahan Revenue Bubbles and Structural Deficits Whats a state to doWorking Paper (Chicago IL Federal Reserve Bank of Chicago April 2012)httpswwwchicagofedorgdigital_assetspublicationsworking_papers2008wp2008_15pdf

6 The upper whisker extends from the hinge (the 75th percentile) to the highest value that is within 15 timesthe interquartile range and the lower whisker extends similarly downward from the lower hinge Databeyond the end of the whiskers are outliers and plotted as points

7 This test involves regressing the forecast on the actual and testing the joint hypothesis that the intercept iszero and the slope is one See Jacob A Mincer and Victor Zarnowitz ldquoThe Evaluation of EconomicForecastsrdquo in Economic Forecasts and Expectations Analysis of Forecasting Behavior and Performance (New YorkNY National Bureau of Economic Research Distributed by Columbia University Press 1969) 1-46httpwwwnberorgchaptersc1214

8 The table shows forty-two states with a personal income tax There are only forty-one states with a broad-based personal income tax Tennessee which has a narrow tax on interest and dividends is included in theNASBO data This tax is a relatively small share of total Tennessee tax revenue

9 This is similar to an approach developed contemporaneously in a working paper by John Mikesell andJustin Ross (see Bibliography) regarding forecasting in Indiana

10 Our naiumlve forecasting model ends in 2013 because we do not yet have full data on actual tax revenue for 2014

11 Synchronization was calculated by using a monthly coincident index at the state level relying on(a) nonfarm payroll employment (b) average hours worked in manufacturing (c) the unemployment rateand (d) wage and salary disbursements Given the idiosyncrasies and data limitations between states thesemeasures are not a perfect barometer for each statersquos economy but are a good start Perhaps furthermeasures could be incorporated for example measures of tourism and travel as Hawaii was found tosynchronize with the national economy only 574 percent of the time

12 While recessions typically include one or more quarters in which real GDP declines these are annual dataand GDP declines on an annual basis are less common

13 For example analysts often use this to measure volatility of daily stock prices

14 This is a different measure of volatility than used by Crain (see Bibliography) discussed previously Fortechnical reasons we believe this is a better measure of volatility of tax revenue than his measure whichexamines deviations from a trend line

15 The tax data have not been adjusted for legislative changes because it is impractical to do so A smallportion of observed changes in state tax volatility may be attributable to legislative changes We do notbelieve this would affect conclusions materially

16 For several years before the 2007 recession a strand of economic research identified what came to be knownas the ldquoGreat Moderationrdquo mdash the idea that the United States economy was becoming less volatile mdash andexamined its potential causes While the data clearly supported declining economic volatility over longerperiods than shown in this figure the idea that we have entered a new period of stability is now in tatters

17 The data on wages and capital gains are from the Statistics of Income branch of the Internal RevenueService and were provided to us by Leslie McGranahan of the Chicago Federal Reserve Bank

18 States are listed in the spreadsheet ldquoGPP_Revenue Estimate Methods FY 2004xlsrdquoworksheet ldquoMeasurementMethods 2005rdquo

19 See ldquoArchived Project Government Performance Projectrdquo Pew Charitable Trustshttpwwwpewtrustsorgenarchived-projectsgovernment-performance-project

Endnotes

Appendix A Survey of State Officials

The Rockefeller Institute of Government conducted tworounds of surveys Both surveys were conducted online usingGoogle Docs

The first round of the survey was conducted in July-August of2013 The purpose of the first round of the survey was to ensurethat we understand as precisely as possible the information thatstate budget officials provide to NASBO on revenue forecasts andpreliminary actuals as published in the Fiscal Survey of States Thesurvey questions were designed to help us get a better under-standing of when estimates that underlie state budgets are devel-oped as that influences the difficulty of the forecasting job and tobe sure we understand how the forecasts are used in the budgetprocess For the first round of survey we sent out the surveyquestionnaire via online survey tool to state budget officers in allfifty states We developed two survey instruments one for thestates with an annual budget cycle and another one for the stateswith a biennial budget cycle For the biennial states we asked twoadditional questions to understand what estimates they provideto NASBO for the second year of the biennium We receivedthirty-four usable responses from the states of whichtwenty-three responding states were on annual budget cycleswhile eleven states were on biennial budget cycles

The second round of the study was conducted in the monthsof August and September of 2013 with a few follow-ups in Octo-ber 2013 The purpose of the second round of the survey was tocollect additional information from all the states to get further in-formation related to procedures and practices of revenue forecast-ing process in the states We sent out the survey questionnaire tostate budget officers representing both executive and legislativebranches of government in all fifty states We received fort-five re-sponses from the state officials representing the executive branchand forty-six responses from state officials representing the legis-lative branch On the executive side we did not receive responsesfrom Georgia Missouri Montana North Carolina and Oregonand on the legislative side no response were received fromMassachusetts Michigan New Hampshire and South Carolina

The survey questions asked during the first round of the sur-vey administration were targeted on getting better understandingof what is being reported to NASBO The most important ques-tions asked of the states were focused on whether the revenue es-timates being reported to NASBO are based on the final estimatesfor the adopted budget whether the revenue estimates take intoaccount legislative and administrative changes approximatelyhow many weeks ahead of the start of the new state fiscal year arethe estimates in the adopted budget prepared and whether the bi-ennial states report estimates for the first and second years ofbiennium separately or combined

Among the respondent states thirty-one states indicated thatthe estimates provided to NASBO are the final estimates used for

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 41 wwwrockinstorg

the adopted budget Alabama Arizona and Idaho were the onlythree states indicating that the revenue estimates provided toNASBO are not the final estimates used for the adopted budgetTwenty-six states also indicated that the revenue estimates bindthe budget In most of the twenty-six states where the revenue es-timates are binding the budget such binding is defined either inthe constitution or by statute And only in a handful of states arethe revenue estimates binding based on customary practice

We also asked the states with the biennium budget cyclewhether they submit updated estimates to NASBO for the secondyear of the biennium Ten of the eleven responding states with thebiennial budget cycle indicated that they do provide updated esti-mates to NASBO for the second year of the biennium in order toaccount for legislative changes and updated economic forecastconditions

The main purpose of the second round of the survey was toget a better understanding about general procedures and practicesof revenue estimating processes in the states The survey ques-tions were targeted into getting more information about the tim-ing of forecasts frequency of forecast updates the long-termforecasting practices parties involved in the forecasting processesand whether there is a formal group that plays a major role in therevenue estimating process

Officials in at least half of the states indicated that they typi-cally generate revenue estimates for one or more years beyond theupcoming budget period Overall most states release revenue es-timates for another two to five years Only in very few states arethe forecasts provided for more than five years beyond the budgetcycle One example is Alaska where officials in the legislativebranch indicated generating revenue estimates for six years whileofficials in the executive branch said that the revenue estimatesare provided for nine additional years beyond the budget cycle

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 42 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 43 wwwrockinstorg

1 Statesrsquo Revenue Estimating Cracks in the Crystal Ball (Philadelphia PA and Albany NY Pew CharitableTrusts and Rockefeller Institute of Government March 2011)httpwwwrockinstorgpdfgovernment_finance2011-03-01-States_Revenue_Estimating_Reportpdf

2 Before the fall of 2013 NASBO conducted the Fiscal Survey of States jointly with the National Governors Association

3 These observations are not independent of those for the individual taxes of course

4 For more information please see Appendix A Survey of State Officials at the end of this report

5 Richard Mattoon and Leslie McGranahan Revenue Bubbles and Structural Deficits Whats a state to doWorking Paper (Chicago IL Federal Reserve Bank of Chicago April 2012)httpswwwchicagofedorgdigital_assetspublicationsworking_papers2008wp2008_15pdf

6 The upper whisker extends from the hinge (the 75th percentile) to the highest value that is within 15 timesthe interquartile range and the lower whisker extends similarly downward from the lower hinge Databeyond the end of the whiskers are outliers and plotted as points

7 This test involves regressing the forecast on the actual and testing the joint hypothesis that the intercept iszero and the slope is one See Jacob A Mincer and Victor Zarnowitz ldquoThe Evaluation of EconomicForecastsrdquo in Economic Forecasts and Expectations Analysis of Forecasting Behavior and Performance (New YorkNY National Bureau of Economic Research Distributed by Columbia University Press 1969) 1-46httpwwwnberorgchaptersc1214

8 The table shows forty-two states with a personal income tax There are only forty-one states with a broad-based personal income tax Tennessee which has a narrow tax on interest and dividends is included in theNASBO data This tax is a relatively small share of total Tennessee tax revenue

9 This is similar to an approach developed contemporaneously in a working paper by John Mikesell andJustin Ross (see Bibliography) regarding forecasting in Indiana

10 Our naiumlve forecasting model ends in 2013 because we do not yet have full data on actual tax revenue for 2014

11 Synchronization was calculated by using a monthly coincident index at the state level relying on(a) nonfarm payroll employment (b) average hours worked in manufacturing (c) the unemployment rateand (d) wage and salary disbursements Given the idiosyncrasies and data limitations between states thesemeasures are not a perfect barometer for each statersquos economy but are a good start Perhaps furthermeasures could be incorporated for example measures of tourism and travel as Hawaii was found tosynchronize with the national economy only 574 percent of the time

12 While recessions typically include one or more quarters in which real GDP declines these are annual dataand GDP declines on an annual basis are less common

13 For example analysts often use this to measure volatility of daily stock prices

14 This is a different measure of volatility than used by Crain (see Bibliography) discussed previously Fortechnical reasons we believe this is a better measure of volatility of tax revenue than his measure whichexamines deviations from a trend line

15 The tax data have not been adjusted for legislative changes because it is impractical to do so A smallportion of observed changes in state tax volatility may be attributable to legislative changes We do notbelieve this would affect conclusions materially

16 For several years before the 2007 recession a strand of economic research identified what came to be knownas the ldquoGreat Moderationrdquo mdash the idea that the United States economy was becoming less volatile mdash andexamined its potential causes While the data clearly supported declining economic volatility over longerperiods than shown in this figure the idea that we have entered a new period of stability is now in tatters

17 The data on wages and capital gains are from the Statistics of Income branch of the Internal RevenueService and were provided to us by Leslie McGranahan of the Chicago Federal Reserve Bank

18 States are listed in the spreadsheet ldquoGPP_Revenue Estimate Methods FY 2004xlsrdquoworksheet ldquoMeasurementMethods 2005rdquo

19 See ldquoArchived Project Government Performance Projectrdquo Pew Charitable Trustshttpwwwpewtrustsorgenarchived-projectsgovernment-performance-project

Endnotes

the adopted budget Alabama Arizona and Idaho were the onlythree states indicating that the revenue estimates provided toNASBO are not the final estimates used for the adopted budgetTwenty-six states also indicated that the revenue estimates bindthe budget In most of the twenty-six states where the revenue es-timates are binding the budget such binding is defined either inthe constitution or by statute And only in a handful of states arethe revenue estimates binding based on customary practice

We also asked the states with the biennium budget cyclewhether they submit updated estimates to NASBO for the secondyear of the biennium Ten of the eleven responding states with thebiennial budget cycle indicated that they do provide updated esti-mates to NASBO for the second year of the biennium in order toaccount for legislative changes and updated economic forecastconditions

The main purpose of the second round of the survey was toget a better understanding about general procedures and practicesof revenue estimating processes in the states The survey ques-tions were targeted into getting more information about the tim-ing of forecasts frequency of forecast updates the long-termforecasting practices parties involved in the forecasting processesand whether there is a formal group that plays a major role in therevenue estimating process

Officials in at least half of the states indicated that they typi-cally generate revenue estimates for one or more years beyond theupcoming budget period Overall most states release revenue es-timates for another two to five years Only in very few states arethe forecasts provided for more than five years beyond the budgetcycle One example is Alaska where officials in the legislativebranch indicated generating revenue estimates for six years whileofficials in the executive branch said that the revenue estimatesare provided for nine additional years beyond the budget cycle

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 42 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 43 wwwrockinstorg

1 Statesrsquo Revenue Estimating Cracks in the Crystal Ball (Philadelphia PA and Albany NY Pew CharitableTrusts and Rockefeller Institute of Government March 2011)httpwwwrockinstorgpdfgovernment_finance2011-03-01-States_Revenue_Estimating_Reportpdf

2 Before the fall of 2013 NASBO conducted the Fiscal Survey of States jointly with the National Governors Association

3 These observations are not independent of those for the individual taxes of course

4 For more information please see Appendix A Survey of State Officials at the end of this report

5 Richard Mattoon and Leslie McGranahan Revenue Bubbles and Structural Deficits Whats a state to doWorking Paper (Chicago IL Federal Reserve Bank of Chicago April 2012)httpswwwchicagofedorgdigital_assetspublicationsworking_papers2008wp2008_15pdf

6 The upper whisker extends from the hinge (the 75th percentile) to the highest value that is within 15 timesthe interquartile range and the lower whisker extends similarly downward from the lower hinge Databeyond the end of the whiskers are outliers and plotted as points

7 This test involves regressing the forecast on the actual and testing the joint hypothesis that the intercept iszero and the slope is one See Jacob A Mincer and Victor Zarnowitz ldquoThe Evaluation of EconomicForecastsrdquo in Economic Forecasts and Expectations Analysis of Forecasting Behavior and Performance (New YorkNY National Bureau of Economic Research Distributed by Columbia University Press 1969) 1-46httpwwwnberorgchaptersc1214

8 The table shows forty-two states with a personal income tax There are only forty-one states with a broad-based personal income tax Tennessee which has a narrow tax on interest and dividends is included in theNASBO data This tax is a relatively small share of total Tennessee tax revenue

9 This is similar to an approach developed contemporaneously in a working paper by John Mikesell andJustin Ross (see Bibliography) regarding forecasting in Indiana

10 Our naiumlve forecasting model ends in 2013 because we do not yet have full data on actual tax revenue for 2014

11 Synchronization was calculated by using a monthly coincident index at the state level relying on(a) nonfarm payroll employment (b) average hours worked in manufacturing (c) the unemployment rateand (d) wage and salary disbursements Given the idiosyncrasies and data limitations between states thesemeasures are not a perfect barometer for each statersquos economy but are a good start Perhaps furthermeasures could be incorporated for example measures of tourism and travel as Hawaii was found tosynchronize with the national economy only 574 percent of the time

12 While recessions typically include one or more quarters in which real GDP declines these are annual dataand GDP declines on an annual basis are less common

13 For example analysts often use this to measure volatility of daily stock prices

14 This is a different measure of volatility than used by Crain (see Bibliography) discussed previously Fortechnical reasons we believe this is a better measure of volatility of tax revenue than his measure whichexamines deviations from a trend line

15 The tax data have not been adjusted for legislative changes because it is impractical to do so A smallportion of observed changes in state tax volatility may be attributable to legislative changes We do notbelieve this would affect conclusions materially

16 For several years before the 2007 recession a strand of economic research identified what came to be knownas the ldquoGreat Moderationrdquo mdash the idea that the United States economy was becoming less volatile mdash andexamined its potential causes While the data clearly supported declining economic volatility over longerperiods than shown in this figure the idea that we have entered a new period of stability is now in tatters

17 The data on wages and capital gains are from the Statistics of Income branch of the Internal RevenueService and were provided to us by Leslie McGranahan of the Chicago Federal Reserve Bank

18 States are listed in the spreadsheet ldquoGPP_Revenue Estimate Methods FY 2004xlsrdquoworksheet ldquoMeasurementMethods 2005rdquo

19 See ldquoArchived Project Government Performance Projectrdquo Pew Charitable Trustshttpwwwpewtrustsorgenarchived-projectsgovernment-performance-project

Endnotes

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Rockefeller Institute Page 43 wwwrockinstorg

1 Statesrsquo Revenue Estimating Cracks in the Crystal Ball (Philadelphia PA and Albany NY Pew CharitableTrusts and Rockefeller Institute of Government March 2011)httpwwwrockinstorgpdfgovernment_finance2011-03-01-States_Revenue_Estimating_Reportpdf

2 Before the fall of 2013 NASBO conducted the Fiscal Survey of States jointly with the National Governors Association

3 These observations are not independent of those for the individual taxes of course

4 For more information please see Appendix A Survey of State Officials at the end of this report

5 Richard Mattoon and Leslie McGranahan Revenue Bubbles and Structural Deficits Whats a state to doWorking Paper (Chicago IL Federal Reserve Bank of Chicago April 2012)httpswwwchicagofedorgdigital_assetspublicationsworking_papers2008wp2008_15pdf

6 The upper whisker extends from the hinge (the 75th percentile) to the highest value that is within 15 timesthe interquartile range and the lower whisker extends similarly downward from the lower hinge Databeyond the end of the whiskers are outliers and plotted as points

7 This test involves regressing the forecast on the actual and testing the joint hypothesis that the intercept iszero and the slope is one See Jacob A Mincer and Victor Zarnowitz ldquoThe Evaluation of EconomicForecastsrdquo in Economic Forecasts and Expectations Analysis of Forecasting Behavior and Performance (New YorkNY National Bureau of Economic Research Distributed by Columbia University Press 1969) 1-46httpwwwnberorgchaptersc1214

8 The table shows forty-two states with a personal income tax There are only forty-one states with a broad-based personal income tax Tennessee which has a narrow tax on interest and dividends is included in theNASBO data This tax is a relatively small share of total Tennessee tax revenue

9 This is similar to an approach developed contemporaneously in a working paper by John Mikesell andJustin Ross (see Bibliography) regarding forecasting in Indiana

10 Our naiumlve forecasting model ends in 2013 because we do not yet have full data on actual tax revenue for 2014

11 Synchronization was calculated by using a monthly coincident index at the state level relying on(a) nonfarm payroll employment (b) average hours worked in manufacturing (c) the unemployment rateand (d) wage and salary disbursements Given the idiosyncrasies and data limitations between states thesemeasures are not a perfect barometer for each statersquos economy but are a good start Perhaps furthermeasures could be incorporated for example measures of tourism and travel as Hawaii was found tosynchronize with the national economy only 574 percent of the time

12 While recessions typically include one or more quarters in which real GDP declines these are annual dataand GDP declines on an annual basis are less common

13 For example analysts often use this to measure volatility of daily stock prices

14 This is a different measure of volatility than used by Crain (see Bibliography) discussed previously Fortechnical reasons we believe this is a better measure of volatility of tax revenue than his measure whichexamines deviations from a trend line

15 The tax data have not been adjusted for legislative changes because it is impractical to do so A smallportion of observed changes in state tax volatility may be attributable to legislative changes We do notbelieve this would affect conclusions materially

16 For several years before the 2007 recession a strand of economic research identified what came to be knownas the ldquoGreat Moderationrdquo mdash the idea that the United States economy was becoming less volatile mdash andexamined its potential causes While the data clearly supported declining economic volatility over longerperiods than shown in this figure the idea that we have entered a new period of stability is now in tatters

17 The data on wages and capital gains are from the Statistics of Income branch of the Internal RevenueService and were provided to us by Leslie McGranahan of the Chicago Federal Reserve Bank

18 States are listed in the spreadsheet ldquoGPP_Revenue Estimate Methods FY 2004xlsrdquoworksheet ldquoMeasurementMethods 2005rdquo

19 See ldquoArchived Project Government Performance Projectrdquo Pew Charitable Trustshttpwwwpewtrustsorgenarchived-projectsgovernment-performance-project

Endnotes

Rockefeller Institute Page 44 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Bibliography

Batchelor Roy and Pam Dua ldquoForecaster Ideology Forecasting Technique and the Accuracy of Eco-nomic Forecastsrdquo Eighth International Symposium on Forecasting Amsterdam 1988

Bretschneider Stuart I Wilpen L Gorr Gloria Grizzle and Earle Klay (1989) ldquoPolitical and organiza-tional influences on the accuracy of forecasting state government revenuesrdquo International Journal ofForecasting 5 3 (1989) 307-19

Clemen Robert T ldquoCombining forecasts A review and annotated bibliographyrdquo International Journal ofForecasting 5 4 (1989) 559-83

Cornia Gary C and Ray D Nelson ldquoState Tax Revenue Growth and Volatilityrdquo Regional Economic De-velopment 6 1 (2010) 23-58

Crain William Mark ldquoReliability of Revenues from Alternative Tax Instrumentsrdquo In Volatile States In-stitutions Policy and the Performance of American State Economies Ann Arbor MI University ofMichigan Press 2003 73-81

Deschamps Elaine ldquoThe impact of institutional change on forecast accuracy A case study of budgetforecasting in Washington Staterdquo International Journal of Forecasting 20 4 (2004) 647ndash 57

Elder Erick M and Gary A Wagner ldquoRevenue Cycles and Risk-Sharing in Local Governments AnAnalysis of State Rainy Day Fundsrdquo National Tax Journal 66 4 (2013) 939-54

Felix R Alison ldquoThe Growth and Volatility of State Tax Revenue Sources in the Tenth Districtrdquo Eco-nomic Review 93 3 (2008) 63-88

FTA State Revenue Forecasting and Estimation Practices Research Report No 139 Washington DC Fed-eration of Tax Administrators March 1993

Krause George A and James W Douglas ldquoOrganizational Structure and the Optimal Design ofPolicymaking Panels Evidence from Consensus Group Commissionsrsquo Revenue Forecasts in theAmerican Statesrdquo American Journal of Political Science 57 1 (2013) 135-49

Mattoon Rick Creating a National State Rainy Day Fund A Modest Proposal to Improve State Fiscal Perfor-mance Working Paper No 2003-20 Chicago IL Federal Reserve Bank of Chicago 2003

Mattoon Rick and Leslie McGranahan Revenue Bubbles and Structural Deficits Whatrsquos a State to DoWorking Paper 2008-15 Chicago IL Federal Reserve Bank of Chicago 2008

Mikesell John L and Justin M Ross ldquoState Revenue Forecasts and Political Acceptance The Value ofConsensus Forecasting in the Budget Processrdquo Public Administration Review 74 2 (2014) 188-203

NASBO Budget Processes in the States Washington DC National Association of State Budget Officers2008

National Advisory Council on State and Local Budgeting GFOA Recommended Budget Practices AFramework for Improved State and Local Government Budgeting Chicago IL Government Finance Of-ficers Association (GFOA) 1998

Penner Rudolph G ldquoFederal Revenue Forecastingrdquo In Government Budget Forecasting Theory and Prac-tice ed Jinping Sun and Thomas D Lynch 11-25 Boca Raton FL Auerbach Publications 2008

Pew Center on the States and Rockefeller Institute of Government Statesrsquo Revenue Estimating Cracks inthe Crystal Ball Philadelphia PA and Albany NY Pew Charitable Trusts and Rockefeller Instituteof Government March 2011

Rockefeller Institute Page 45 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Qiao Yuhua ldquoUse of Consensus Revenue Forecasting in US State Governmentrdquo In Government BudgetForecasting Theory and Practice ed Jinping Sun and Thomas D Lynch 393-413 Boca Raton FLAuerbach Publications 2008

Rose Shanna and Daniel L Smith ldquoBudget Slack Institutions and Transparencyrdquo Public Administra-tion Review 72 2 (2011) 187-195

Samuelson Paul rdquoQuotes from Paul Samuelsonrsquos February 1985 lecture at Trinity University February6 1985 httpwwwtrinityedunobelSamuelson_filesSamuelson20web20quoteshtm

Thompson Fred and Bruce L Gates ldquoBetting on the Future with a Cloudy Crystal Ball How FinancialTheory Can Improve Revenue Forecasting and Budgets in the Statesrdquo Public Administration Review67 5 (2007) 825-36

Voorhees William R ldquoMore Is Better Consensual Forecasting and State Revenue Forecast Errorrdquo Inter-national Journal of Public Administration 27 8-9 (2004) 651-71

Weltman Jeremy C ldquoUsing Consensus Forecasts in Business Planningrdquo The Journal of Business Forecast-ing 14 4 (Winter 1995-96) 13-6

Willoughby Katherine G and Hai Guo ldquoThe State of the Art Revenue Forecasting in US State Gov-ernmentsrdquo In Government Budget Forecasting Theory and Practice ed Jinping Sun and Thomas DLynch 393-413 Boca Raton FL Auerbach Publications 2008

Rockefeller Institute Page 45 wwwrockinstorg

Revenue Forecasting State Tax Revenue Forecasting Accuracy Technical Report

Qiao Yuhua ldquoUse of Consensus Revenue Forecasting in US State Governmentrdquo In Government BudgetForecasting Theory and Practice ed Jinping Sun and Thomas D Lynch 393-413 Boca Raton FLAuerbach Publications 2008

Rose Shanna and Daniel L Smith ldquoBudget Slack Institutions and Transparencyrdquo Public Administra-tion Review 72 2 (2011) 187-195

Samuelson Paul rdquoQuotes from Paul Samuelsonrsquos February 1985 lecture at Trinity University February6 1985 httpwwwtrinityedunobelSamuelson_filesSamuelson20web20quoteshtm

Thompson Fred and Bruce L Gates ldquoBetting on the Future with a Cloudy Crystal Ball How FinancialTheory Can Improve Revenue Forecasting and Budgets in the Statesrdquo Public Administration Review67 5 (2007) 825-36

Voorhees William R ldquoMore Is Better Consensual Forecasting and State Revenue Forecast Errorrdquo Inter-national Journal of Public Administration 27 8-9 (2004) 651-71

Weltman Jeremy C ldquoUsing Consensus Forecasts in Business Planningrdquo The Journal of Business Forecast-ing 14 4 (Winter 1995-96) 13-6

Willoughby Katherine G and Hai Guo ldquoThe State of the Art Revenue Forecasting in US State Gov-ernmentsrdquo In Government Budget Forecasting Theory and Practice ed Jinping Sun and Thomas DLynch 393-413 Boca Raton FL Auerbach Publications 2008