evidence review 7 transport · 2019. 1. 16. · evidence review: transport - july 2015 7 to...
TRANSCRIPT
![Page 1: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/1.jpg)
Evidence Review 7
Transport
July 2015
![Page 2: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/2.jpg)
Contents
Preface 3
ExecutiveSummary 4
Introduction 8
Impactevaluation 11
Methodology 14
Definition 17
Findings 20
Summaryoffindings 31
References 38
AppendixA:Findingsbyoutcome 39
AppendixB:EvidenceReviewed 40
00
![Page 3: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/3.jpg)
Evidence Review: Transport - July 2015 3
01
Preface
Thisreportpresentsfindingsfromasystematicreviewofevaluationsofthelocaleconomicimpactoftransport.Itcoversevidenceonroads,rail(includinglightrailandsubways),trams,buses,cyclingandwalking–areasofexpenditurewhichaccountforthemajorityoftransportschemesconsideredbylocaldecisionmakers.Evidenceonportsandairportswillbecoveredinafurtherreport.
ThisreportistheseventhreviewproducedbytheWhatWorksCentreforLocalEconomicGrowth.Ourreviewsconsideraspecifictypeofevidence–impact evaluation–thatseekstounderstandthecausaleffectofpolicyinterventionsandtoestablishtheircost-effectiveness.Toputitanotherwaytheyask‘didthepolicywork’and‘diditrepresentgoodvalueformoney’?Withthisreviewweareparticularlyinterestedindemonstratingthatthelocaleconomicimpactsoftransportcanberigorouslyevaluatedandindrawingoutthewiderlessonsforpolicy–includingquestionsofschemeappraisalandprioritisation.
Evidenceonimpactandeffectivenessisacrucialinputtogoodpolicymaking.Inthecaseoftransportthemainaimisnotnecessarilytoimprovethelocaleconomy.However,policymakersoftenclaimeconomicbenefitsfortheseinterventions,andsoitisimportanttoundertakeeconomicimpactevaluationtounderstandiftheseclaimsarejustified.Otherwaysofconsideringtheimpactoftransport(e.g.casestudies)provideavaluablecomplementtoimpactevaluation,butwedonotfocusontheseinthisreport.
We see these impact-focused reviews as an essential part of more effective policy making.Weoftensimplydonotknowtheanswerstomanyofthequestionsthatmightreasonablybeaskedwhenimplementinganewpolicy–notleast,doesitwork?Figuringoutwhatwedoknowallowsustomakebetterdecisionsandtostartfillingthegapsinourknowledge.This also helps us to have more informed discussions and to improve policy making.
Thesereviewsthereforerepresentafirststepinimprovingourunderstandingofwhatworksforlocaleconomicgrowth.Inthemonthsahead,wewillbeworkingwithlocaldecisionmakersandpractitioners,usingthesefindingstohelpthemgeneratebetterpolicy.
HenryOverman;Director,WhatWorksCentreforLocalEconomicGrowth
![Page 4: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/4.jpg)
Evidence Review: Transport - July 2015 4
Executive Summary
Thisreportpresentsfindingsfromasystematicreviewofevaluationsofthelocaleconomicimpactoftransportprojects.Itcoversevidenceonroads,rail(includinglightrailandsubways),trams,buses,cyclingandwalking.Evidenceonportsandairportswillbeconsideredinafurtherreport.ThisreviewistheseventhproducedbytheWhatWorksCentreforLocalEconomicGrowth.
Thereviewconsideredmorethan2,300policyevaluationsandevidencereviewsfromtheUKandotherOECDcountries.Itfound29impactevaluationsthatmettheCentre’sminimumstandards.
ApproachTheCentreseekstoestablishcausalimpact–anestimateofthedifferencethatcanbeexpectedbetweentheoutcomeforareasthatbenefitfromtransportinvestmentandtheaverageoutcometheywouldhaveexperiencedwithoutinvestment(seeFigure1).OurmethodologyforproducingourreviewsisoutlinedinFigure2.
02
Figure 1: Evaluating impactEvaluating impact
VS
Change inoutcome for those
with improved transport
Change inoutcome for those without improved
transport
![Page 5: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/5.jpg)
Evidence Review: Transport - July 2015 5
Findings
Thissectionsummarisesthedetailedfindings.Weemphasisethatmanyofthesefindingsdependonasmallnumberofstudies.Theyare,however,consistentwithotherresearchontheeconomicimpactoftransportimprovements.
What the evidence shows
• Roadprojectscanpositivelyimpactlocalemployment.Buteffectsarenotalwayspositiveandamajorityofevaluationsshowno(ormixed)effectsonemployment.
• Roadprojectsmayincreasefirmentry(eitherthroughnewfirmsstartingup,orexistingfirmsrelocating).However,thisdoesnotnecessarilyincreasetheoverallnumberofbusinesses(sincenewarrivalsmaydisplaceexistingfirms).
• Roadprojectstendtohaveapositiveeffectonpropertyprices,althougheffectsdependondistancetotheproject(andtheeffectscanalsovaryovertime).
• Theimpactofroadsprojectsonthesizeofthelocalpopulationmayvarydependingonwhethertheprojectisurban,suburbanorrural.
• Thereissomeevidencethatroadprojectshavepositiveeffectsonwagesorincomes.
• Thereissomeevidencethatroadprojectshaveapositiveeffectonproductivity.
• Railprojectstendtohaveapositiveeffectonpropertyprices,althougheffectsdependondistancetotheproject(andtheeffectscanalsovaryovertime).
Where there is a lack of evidence
• Wefoundnohighqualityevaluationsthatprovideevidenceontheimpactofrailinfrastructureonemployment,andonlyalimitednumberofevaluationsshowingthatroadprojectshaveapositiveeffect.
Figure 2: Methodology
government
34
5
1 & 2
user panelacademic panel
1scope
2search
3sift
4score
5
synthesis
To identify what works, each policy review finds and evaluates the evidence which is robust and demonstrates clear outcomes in a 5 stage process
Evaluation evidence is collected using a wide range of sources
Each study is scored based on the quality of
method and quality of implementation
The full set of evidence is refined based on its relevance and the robustness of the research method
Conclusions drawn are based on a combination of these findings and existing literature
academiathinktanks
call forevidence
Existing literature and evidence is reviewed on the basis of an agreed review question,
specific search terms, and a set of inclusion criteria
![Page 6: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/6.jpg)
Evidence Review: Transport - July 2015 6
• Wefoundnohighqualityevaluationsthatprovideevidenceontheimpactsoftrams,buses,cyclingandwalkingschemesonanyeconomicoutcomes.
• Evenwhenstudiesareabletoidentifyapositiveimpactonemployment,theextenttowhichthisisasaresultofdisplacementfromothernearbylocationsisstillunresolved.Moregenerally,thespatialscaleofanyemploymenteffectsvariesandwedonothaveenoughevidencetobeabletogeneraliseaboutthespatialdistributionofeffectsiftheyoccur.Thesameistrueforotheroutcomes.ThescaleatwhichthestudiesevaluateimpactvariesfromadjacentneighbourhoodstomuchlargerUScounties.
• Surprisingly,veryfewevaluationsconsidertheimpactoftransportinvestmentonproductivity(wefoundjustthreestudies,twoforroadsandoneforrail).Althoughtheuseofsuchproductivityeffectstocalculate‘widereconomicbenefits’intransportappraisalisunderpinnedbyalargerevidencebase,itisstillworryingthatsofewevaluationscandemonstratethattheseeffectsoccurinpractice.
• Wehavelittleevidencethatwouldallowustodrawconclusionsonwhetherlarge-scaleprojects(e.g.highspeedrailormotorwayconstruction)havelargereconomicgrowthimpactsthanspendingsimilaramountsonacollectionofsmall-scaleprojects(e.g.lightrailorjunctionimprovements).
• Moregenerally,wedonotknowhowdifferencesinthenatureofimprovements(e.g.journeytimesavedornumberofadditionaljourneys)affectlocaleconomicoutcomes.
• Thereissomeevidencethatcontextmatters.Forexample,propertypriceeffectsmaydependonthetypeofproperty,whilewageeffectsmaydifferbetweenlowskilledandhighskilledworkers.But,onceagainwedonothaveenoughevidencetobeabletogeneralise.
How to use these reviewsTheevidencereviewhighlightsanumberoffactorsforpolicymakerstobeawareofwhenconsideringtransportpolicy:
• Muchmoreempiricalworkremainstobedoneonunderstandingtheimpactofinfrastructureimprovementsonlocaleconomicgrowth.Theeconomicbenefitsoftransportinfrastructurespending–particularlyasamechanismforgeneratinglocaleconomicgrowth–arenotasclear-cutastheymightseemonfacevalue.
• WhileitisunderstandablethatpoliticaldebatefocusesonexpenditurefiguresacrossdifferentpartsoftheUK,theydonothelpanswerthequestionofwhatwouldhappenifexpenditurewasdistributeddifferently.Argumentsforspendingmoreinareasthatarelesseconomicallysuccessfulhingeonthehopethatnewtransportisacost-effectivewaytostimulateneweconomicactivity.Asthisreviewshows,wedonotyethaveclearanddefinitiveevidencetosupportthatclaim.
• Thesefindingsraisefundamentalquestionsaboutschemeappraisalandprioritisation,andabouttheroleofimpactevaluationinimprovingdecision-makingaroundtransportinvestment.SomepreliminaryrecommendationsbaseduponourworkwithDfTandLEPsareoutlinedinsection8ofthefullreport.
![Page 7: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/7.jpg)
Evidence Review: Transport - July 2015 7
To determine policy priorities
TheCentre’sreviewsconsideraspecifictypeofevidence–impactevaluation–thatseekstounderstandthecausaleffectofpolicyinterventionsandtoestablishtheircost-effectiveness.Inthelongerterm,theCentrewillproducearangeofevidencereviewsthatwillhelplocaldecisionmakersdecidethebroadpolicyareasonwhichtospendlimitedresources.Figure3illustrateshowthereviewsrelatetotheotherworkstreamsoftheCentre.
HelpingtofilltheevidencegapsAsshouldbeclearfromthisreview,therearemanythingsthatwedonotknowaboutthelocaleconomicimpactofinfrastructure.Tohelpfilltheseevidencegaps,thefinalpartofthereviewprovidesanumberofrecommendationsaimedatimprovingtheevaluationandappraisaloftransportschemes.
TheCentre’slongertermobjectivesaretoensurethatrobustevidenceisembeddedinthedevelopmentofpolicy,thatthesepolicesareeffectivelyevaluatedandthatfeedbackisusedtoimprovethem.Toachievetheseobjectiveswewantto:
• workwithlocaldecisionmakerstoimproveevaluationstandardssothatwecanlearnmoreaboutwhatpolicieswork,where.
• setupaseriesof‘demonstrationprojects’toshowhoweffectiveevaluationcanworkinpractice.
Interestedpolicymakerspleasegetintouch.
Evidence reviews
Demonstrationprojects
You are here
Capacitybuilding
Understanding what works
More effective policy
Capacitybuilding
Capacitybuilding
Figure 3: What Works Centre work programme
![Page 8: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/8.jpg)
Evidence Review: Transport - July 2015 8
Introduction
Thisreviewlooksatthelocaleconomicimpactsoftransportinvestment.Itcoversevidenceonroads,rail(includinglightrailandsubways),trams,buses,cyclingandwalking–areasofexpenditurewhichaccountforthemajorityoftransportspendingthatwillbeconsideredbylocaldecisionmakers.
Transportinfrastructureandservicesarefundamentalpublicgoodsthataffectthewaysocietiesandeconomiesfunction.Localdecisionmakerswillwanttotakemanyfactorsintoaccountwhendecidinglocaltransportpolicy,butourfocusisonthenarrowerissueofunderstandingtheeconomicimpact.
Therearetwomaineconomicaimsoftransportspending.First,toreducetransportcoststobusinessesandcommuters(forexamplebyreducingcongestion–andthussavingtime-orbyreducingfares).Second,andrelated,tostimulatetheUKandlocaleconomies,forexample,byraisingtheproductivityofexistingfirmsandworkersorbyattractingnewfirmsandprivatesectorinvestment.Tomeetthesepolicyaimsrequiresanunderstandingofwhetherwearespendingenoughandontherightthings.
Tohelpanswerthisquestion,thisreviewsummarisessomeofthekeytheoriesandevidenceregardingtheimpactoftransportontheeconomy–withaparticularfocusonthelessonsthatwecandrawfromthelimitednumberofavailableimpactevaluations.
Thebasicmessagethatemergesfromthisreviewisthattheeconomicbenefitsoftransportinfrastructurespending–particularlyasamechanismforgeneratinglocaleconomicgrowth-arenotasclear-cutastheymightseemonfacevalue.Inturn,thisraisesfundamentalquestionsaboutschemeappraisalandprioritisationandabouttheroleofimpactevaluationinimprovingdecisionmakingaroundtransportinvestment.Thelatterpartofthisreviewaddressessomeoftheserelatedquestions.
The economic aims of transport spendingForacountryliketheUKwithawell-developedtransportnetwork,wecanidentifytwokeypolicyaims(Gibbons,2015).Thefirstistorespondtogrowingdemandsothatincreasedcongestion,longertraveltimesandhighercoststoproducersandconsumers,donotconstraingrowth.Onthebasisofthiskindof“ameliorative”argument,weshouldinvestmoreinplaceswheretheeconomyandtransportdemandisgrowing.
03
![Page 9: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/9.jpg)
Evidence Review: Transport - July 2015 9
Oneconcernwiththisapproachisthatmakingtraveleasierinthiswaysimplyencouragesmoretravel.Ifthishappens,itmaydivertresourcesfromotherplacesandsectors,withlittleeconomicgainandbigenvironmentalcosts.Anotherconcernisthatthiskindofpolicymayexacerbatespatialinequalitiesbytargetingresourcesatplaceswhicharealreadyprosperousandgrowing.
Asecondaimoftransportspendingistostimulatelocaleconomies.Thatis,todrivegrowthinthelocaleconomy,ratherthanjustrespondtoit.Argumentsforgreaterinvestmenttomeetthisobjectivearebasedontheideathatlowertransportcostsallowforthemoreefficientallocationofexistingresources.Forexample,aconsiderablebodyofevidencesuggeststhatconnectingpeople,firmsandplacesmorecloselygenerates“agglomerationeconomies”,whichincreaseproductivity.Loweringtransportcostsalsoincreasesprivatesectorreturnsandthismaystimulateinvestment.
Buildingontheseideas,anumberofrecentreportshavearguedforgreaterinvestmenttostimulatenationalgrowth,andalsototacklespatialdisparitieswithintheUK(e.g.CityGrowthCommission2014).Tomeetthelatterobjective,suchreportsarguethatweshouldtargetmoreresourcestoplaceswhereeconomicperformanceislagging,inordertostimulategrowth.
ThehighprofileEddingtonReviewoftheUK’stransportnetworkfocusedmoreonthefirstoftheseissues.Ithighlightedtheproblemsofcongestionandthepotentialeconomicbenefitsofanimprovedsystemestimatingthata5%reductionintraveltimesnationallywouldbewortharound0.2%ofGDPannually(Eddington2006).ThereportarguedthattheUKwasalreadywellinterconnected,andrecommendedthatimprovementsshouldfocusonincreasingtheperformanceoftheexistingnetworkthroughmanagementandpricing.
ThekeypolicyprioritiestheEddingtonReviewidentifiedweregrowingandcongestedareas,urbanareas,andmajorcongestedinter-citylinks.Accordingtothisanalysis,transportinfrastructureinvestmentshouldaimtorelaxtheconstraintsthatacongestedsystemimposesontravelandbusinesscosts.Investmentshouldbetargetedtoplaceswherethereisgrowingdemandfortransport,implyingthatinvestmentshouldflowtothefastestgrowingcitiesandregions.
TheLSEGrowthCommission(Aghionetal2013)echoedmanyoftheseconclusions,andproposedasetofnewindependentinstitutionstounblockmajortransportinfrastructureplanningdecisions–includingaStrategyBoardtodeterminelong-terminfrastructureplans(thenratifiedbyParliament),aCommissiontodeliverthisplan(includinggenerouscompensationforloserstodeflectNimbyism)andanInfrastructureBanktohelpwithbothfinanceandprivateexpertise.
AstheLSEGrowthCommissionreportdemonstrates,littlehaschangedaboutourunderstandingoftheinteractionsbetweentransportandtheeconomysincetheEddingtonreportwaswritten.However,since2007theGreatRecessionhasledtoarenewedfocusondisparitiesbetweenmajorcities(Londoninparticular)andtherestofthecountry.Inturn,thishasraisedquestionsabouttheextenttowhichtransportinvestmentcouldhelpnarrowthesedisparities.Forexample,arecentreportbyIPPR(CoxandDavies2013)onregionalinfrastructureissueshighlightedstarkdifferencesinplannedspendingperpersonindifferentregions,andarguedforgreaterspendinginlaggingareasintheNorthofEngland.1Recentreportssuchasthesehaveonceagainraisedthequestionofwhetherwecanstimulateeconomicactivity–locally,regionallyornationally–throughinfrastructure
1 Theextentofdisparitiesdependscruciallyonwhethertheexpenditurefiguresusedforcomparisonincludeonlypublicinvestmentorarebasedontotalinvestmentwherethereissomeelementofpublicsupport.Disparitiesinthelatterlookmuchlargerthandisparitiesinpublicinvestmentalone.Itisalsoimportanttonotethathistoricaldisparities–whichunderpintoday’sdifferencesineconomicperformance-aremuchsmallerthandisparitiesinplannedspending.Finally,differentwaysofpresentingthesefigureseliminateorevenreversethesedisparities.SeeGibbons(2015)forfurtherdiscussion.
![Page 10: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/10.jpg)
Evidence Review: Transport - July 2015 10
investment,ratherthansimplytargetingittomeetunderlyingdemand.Thisquestioniscentraltoourunderstandingoftheroleoftransportinvestmentinimprovinglocaleconomicgrowthandisthemainfocusoftheremainderofthisreview.
The effect of transport investment on local economic growthTherearetwowaysofstructuringourthinkingaboutthelikelyeconomicimpactofinfrastructureinvestments.Thefirstviewspublicsectorinfrastructureinvestmentasprovidingacapitalstockthatiscomplementarytoprivatesectorphysicalcapital(i.e.machinesandbuildings)andtohumancapital(i.e.skills).Thesecondthinksofinfrastructureasprovidinganetworkthatconnectsdifferentplacessothatpublicsectorinvestmentreducesthetransportcostsbetweenplaces.
Thefirstwayofthinkingsuggeststhatprovidingmoreinfrastructurewillalwaysimprovearealevelproductivity(Jones,2013).Ofcourse,infrastructurecanbeveryexpensivesotheseproductivitybenefitsmightbeoutweighedbythecostsofprovision.Thisdisparitybetweenproductivitybenefitsandcostsmaybeparticularlyacutewheninfrastructureisusedtotrytoturnaroundstrugglinglocaleconomies.Becauseinfrastructureisdurable,placesthathaveseenslowgrowthwilltendtohaverelativelylargeamountsofinfrastructureperperson.Theconcretemanifestationofthisarerelativelylowcongestionlevelsinpoorlyperformingcities.Economictheory–supportedbyempiricalevidence-suggeststhataddingfurthertransportinvestmentinthoseplacesmaynotdomuchtoimproveproductivity.2Incontrast,investingincongestedplaceswilltendtodeliverhigherreturnsbecausethecongestionreflectsthefactthattheseplaceshavelowinfrastructureperperson.Ofcourse,thesearegeneraltendencieswhichdon’truleoutthepossibilitythatspecificprojectsmayhavelargerimpactsinpoorlyperformingcities(andvice-versa).
Thesecondwayofthinkingaboutinfrastructure–asanetworkthatconnectsdifferentplaces–providesmoremixedmessages;particularlywhenitcomestobetterconnectingrichandpoorregionsregions(Baldwin,etal;2005).Onewaytothinkaboutthesetypesoftransportinvestmentistoviewenhancedintegrationasawayofincreasingtheeffectivesizeofthelocaleconomies.Asalargerlocaleconomymeanshigheragglomerationeconomiesthisshouldhelpfirmsbemoreproductive.
Therearetwoimportantcaveatsconcerningthislineofreasoning.First,theavailableempiricalevidencesuggeststhatagglomerationeconomiesmayattenuatequitequicklywithdistance.Itisnotclear,therefore,whetherconnectingdifferentcitieswillalwaysgeneratesignificantagglomerationbenefits.
Second,loweringtransportcostsmayencouragefirmstomoveintotherichermarketandservetheircustomersfromthere.This‘twowayroadsproblem’ispoorlyunderstood,leadingsomepolicymakerstofocussolelyonthebenefitstothepoorermarket–ratherthanthinkingthroughthe‘threats’fromgreatercompetition.
Aswillbecomeclearfromtheevidencereviewedbelow,muchmoreempiricalworkremainstobedoneonunderstandingtheimpactofinfrastructureimprovementsonlocaleconomicgrowth.Theoreticalanalysiscertainlyurgescautioninassumingthatinfrastructureinvestmentcanstimulategrowthinpoorlyperformingareas.Inshort,whileinfrastructureinvestmentmaybevitallyimportantforgrowingcities,itsroleinstimulatinggrowthisnotasclear-cutasassumedbymanydecisionmakers.
2 Thisisbecauseinvestmentsinphysicalcapitalarelikelytobesubjectto‘diminishingmarginalreturns’.Thismeansthat,whenaplacehaslotsofcapitalperpersonaddingextracapitalwillnotdomuchtoincreaseproductivity.See,forexample,Solow(1956)andthelargeeconomicgrowthliteraturethatbuildsonthiswork.
![Page 11: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/11.jpg)
Evidence Review: Transport - July 2015 11
Impact evaluation
Governmentsaroundtheworldincreasinglyhavestrongsystemstomonitorpolicyinputs(suchasspendingoninfrastructureprovision)andoutputs(suchasthenumberandspeedofjourneysmadeonanewroad).However,theyarelessgoodatidentifyingpolicyoutcomes(suchasthewidereffectoftransportonlocalemployment).Inparticular,manygovernment-sponsoredevaluationsthatlookatoutcomesdonotusecrediblestrategiestoassessthecausalimpactofinfrastructureinvestment(henceforth,werefertotheseas‘projects’).
Bycausalimpact,theevaluationliteraturemeansanestimateofthedifferencethatcanbeexpectedbetweentheoutcomeforareasundertakingaproject(inthiscase,improvingtransportprovision)andtheaverageoutcometheywouldhaveexperiencedwithouttheproject.Pinningdowncausalityisacruciallyimportantpartofimpactevaluation.Estimates of the benefits of a project are of limited use to policy makers unless those benefits can be attributed, with a reasonable degree of certainty, to that project.
Thecredibilitywithwhichevaluationsestablishcausalityisthecriteriononwhichthisreviewassessestheliterature.
Using CounterfactualsEstablishing causality requires the construction of a valid counterfactual–i.e.whatwouldhavehappenedtoanarea(orpartofanarea)iftheprojecthadn’thappened.Thatoutcomeisfundamentallyunobservable,soresearchersspendagreatdealoftimetryingtorebuildit.Thewayinwhichthiscounterfactualis(re)constructedisthekeyelementofimpactevaluationdesign.
A standard approach is to create a counterfactual group of similar places not undertaking the kind of project being evaluated. Changesinoutcomescanthenbecomparedbetweenthe‘treatmentgroup’(locationsaffectedbyimprovedtransportation)andthe‘controlgroup’(locationsnotaffected).Aswediscussbelow,inthecaseoftransportprovision,suchtreatmentandcontrolgroupsarenotalwayseasytoidentify.
04
![Page 12: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/12.jpg)
Evidence Review: Transport - July 2015 12
A key issue in creating the counterfactual group is dealing with the ‘selection into treatment’ problem.Selectionintotreatmentoccurswhenlocationsthatundergotransportimprovementsdifferfromthosewhodonotdoso.
Anexampleofthisproblemfortransportprojectswouldbewhenagovernmentfocusestransportinvestmentonitsbestperformingcities.Ifthishappens,estimatesofpolicyimpactmaybebiasedupwardsbecauseweincorrectlyattributebettereconomicoutcomestotheproject,ratherthantothefactthatthecityisalreadyperformingbetterthanaverage.
Selectionproblemsmayalsoleadtodownwardbias.Forexample,ifalocalauthorityprojectexplicitlytargetsslowgrowingareasfortransportimprovementsthenwemaymistakenlyattributepooreconomicperformancetotheprojectratherthantounderlyingconditionsinthearea.
Thesefactorsareoftenunobservabletoresearchers.So the challenge for good programme evaluation is to deal with these issues, and to demonstrate that the control group is plausible.Iftheconstructionofplausiblecounterfactualsiscentraltogoodpolicyevaluation,thenthecrucialquestionbecomes:how do we design counterfactuals? Box1providessomeexamples.
Box 1: Impact evaluation techniques
Onewaytoidentifycausalimpactsofaprojectistorandomlyassignparticipantstotreatmentandcontrolgroups.Forresearchers,suchRandomised Control Trials(RCTs)areoftenconsideredthe‘goldstandard’ofevaluation.Properlyimplemented,randomisationensuresthattreatmentandcontrolgroupsarecomparablebothintermsofobservedandunobservedattributes,thusidentifyingthecausalimpactoftheproject.However, implementation of these ‘real world’ experiments is challenging and can be problematic. RCTsmaynotalwaysbefeasibleforlocaleconomicgrowthpolicies–forexample,policymakersmayunderstandablybeunwillingtorandomisethelocationofprojects.3
Whererandomisedcontroltrialsarenotanoption,‘quasi-experimental’approachesofrandomisationcanhelp.Thesestrategiescandealwithselectiononunobservables,by(say)exploitinginstitutionalrulesandprocessesthatresultinsomelocationsquasi-randomlyundertakingprojects.
Evenusingthesestrategies,though,thetreatmentandcontrolgroupsmaynotbefullycomparableintermsofobservables.StatisticaltechniquessuchasOrdinary Least Squares(OLS)andmatching canbeusedtoaddressthisproblem.
Notethathigherqualityimpactevaluationfirstusesidentificationstrategiestoconstructacontrolgroupanddealwithselectiononunobservables.Thenittriestocontrolforremainingdifferencesinobservablecharacteristics.Itisthecombinationthatisparticularlypowerful:OLSormatchingaloneraiseconcernsabouttheextenttowhichunobservablecharacteristicsdeterminebothtreatmentandoutcomesandthusbiastheevaluation.
3 Gibbons,NathanandOverman(2014).
![Page 13: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/13.jpg)
Evidence Review: Transport - July 2015 13
Evidence included in the review
Weincludeanyevaluationthatcomparesoutcomesforareasimprovingtransportprovision(thetreatedgroup)aftertheprojectwithoutcomesinthetreatedgroupbeforetheproject;relativetoacomparisongroupusedtoprovideacounterfactualofwhatwouldhavehappenedtotheseoutcomesintheabsenceoftheproject.
Thismeanswelookatevaluationsthatdoareasonablejobofestimatingtheimpactoftheprojectusingeitherrandomisedcontroltrials,quasi-randomvariationorstatisticaltechniques(suchasOLSandmatching)thathelpmaketreatmentandcontrolgroupscomparable.Weviewtheseevaluationsasprovidingcredibleimpactevaluationinthesensethattheyidentifyeffectsthatcanbeattributed,withareasonabledegreeofcertainty,totheprojectinquestion.AfulllistofshortlistedstudiesisgiveninAppendixB.
Evidence excluded from the review
Weexcludeevaluationsthatprovideasimplebeforeandaftercomparisononlyforthoseplacesundertakingtransportprojectsbecausewecannotbereasonablysurethatchangesforthetreatedgroupcanbeattributedtotheeffectoftheproject.
Wealsoexcludecasestudiesorevaluationsthatfocusonprocess(howtheprojectisimplemented)ratherthanimpact(whatwastheeffectoftheproject).Suchstudieshavearoletoplayinhelpingformulatebetterpolicybuttheyarenotthefocusofourevidencereviews.
![Page 14: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/14.jpg)
Evidence Review: Transport - July 2015 14
Methodology
ToidentifyrobustevaluationevidenceonthecausalimpactoftransportweconductedasystematicreviewoftheevidencefromtheUKandacrosstheworld.Ourreviewfollowedafive-stageprocess:scope,search,sift,scoreandsynthesise.
Stage 1: Scope of Review WorkingwithourUserPanelandamemberofourAcademicPanel,weagreedthereviewquestion,keytermsandinclusioncriteria.Wealsousedexistingliteraturereviewsandmeta-analysestoinformourthinking.
05
Figure 1: Methodology
government
34
5
1 & 2
user panelacademic panel
1scope
2search
3sift
4score
5
synthesis
To identify what works, each policy review finds and evaluates the evidence which is robust and demonstrates clear outcomes in a 5 stage process
Evaluation evidence is collected using a wide range of sources
Each study is scored based on the quality of
method and quality of implementation
The full set of evidence is refined based on its relevance and the robustness of the research method
Conclusions drawn are based on a combination of these findings and existing literature
academiathinktanks
call forevidence
Existing literature and evidence is reviewed on the basis of an agreed review question,
specific search terms, and a set of inclusion criteria
![Page 15: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/15.jpg)
Evidence Review: Transport - July 2015 15
Stage 2: Searching for EvaluationsWesearchedforevaluationevidenceacrossawiderangeofsources,frompeer-reviewedacademicresearchtogovernmentevaluationsandthinktankreports.Specifically,welookedatacademicdatabases(suchasEconLit,WebofScienceandGoogleScholar),specialistresearchinstitutes(suchasCEPRandIZA),UKcentralandlocalgovernmentdepartments,andworkdonebythinktanks(suchastheOECD,ILO,IPPRandPolicyExchange.)Wealsoissuedacallforevidenceviaourmailinglistandsocialmedia.Thissearchfoundjustover2,300books,articlesandreports(thefulllistofsearchtermscanbefoundonlinehere:whatworksgrowth.org/policies/transport/search-terms).
Stage 3: Sifting EvaluationsWescreenedourlong-listonrelevance,geography,languageandmethods,keepingimpactevaluationsfromtheUKandotherOECDcountries,withnotimerestrictionsonwhentheevaluationwasdone.WefocusedonEnglish-languagestudies,butwouldconsiderkeyevidenceifitwasinotherlanguages.Wethenscreenedtheremainingevaluationsontherobustnessoftheirresearchmethods,keepingonlythemorerobustimpactevaluations.WeusedanadjustedversionoftheMarylandScientificMethodsScale(SMS)todothis.4TheSMSisafive-pointscalerangingfrom1,forevaluationsbasedonsimplecrosssectionalcorrelations,to5forrandomisedcontroltrials(seeBox2).WeshortlistedallthoseimpactevaluationsthatcouldpotentiallyscorethreeoraboveontheSMS5.Inthiscasewefoundnoevaluationsscoringfive:forexamplesofimpactevaluationsthatscorethreeorfourontheSMSscaleseethecasestudiesandourscoringguideavailableat:www.whatworksgrowth.org/resources/scoring-guide.
Stage 4: Scoring EvaluationsWeconductedafullappraisalofeachevaluationontheshortlist,collectingkeyresultsandusingtheSMStogiveafinalscoreforevaluationsthatreflectedboththequalityofmethodschosenandqualityofimplementation(whichcanbelowerthanclaimedbysomeauthors).Scoringandshortlistingdecisionswerecross-checkedwiththeacademicpanelmembersandthecoreteamatLSE.Thefinallistofincludedstudiesandtheirreferencenumbers(usedintherestofthisreport)canbefoundinAppendixB.
Stage 5: Synthesising EvaluationsWedrewtogetherourfindings,combiningmaterialfromourevaluationsandtheexistingliterature.
4 Sherman,Gottfredson,MacKenzie,Eck,Reuter,andBushway(1998).5 Shermanetal.(1998)alsosuggestthatlevel3istheminimumlevelrequiredforareasonableaccuracyofresults.
![Page 16: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/16.jpg)
Evidence Review: Transport - July 2015 16
Box 2: Our robustness scores (based on adjusted Maryland Scientific Methods Scale)
Level 1:Either (a) across-sectional comparison of treated groups with untreated groups, or (b) a before-and-after comparison of treated group, without an untreated comparison group.Nouseofcontrolvariablesinstatisticalanalysistoadjustfordifferencesbetweentreatedanduntreatedgroupsorperiods.
Level 2:Use of adequate control variables and either (a) across-sectional comparison of treated groups with untreated groups, or (b) a before-and-after comparison of treated group, without an untreated comparison group. In(a), controlvariablesormatchingtechniquesusedtoaccountforcross-sectionaldifferencesbetweentreatedandcontrolsgroups.In(b),controlvariablesareusedtoaccountforbefore-and-afterchangesinmacrolevelfactors.
Level 3:Comparison of outcomes in treated group after an intervention, with outcomes in the treated group before the intervention, and a comparison group used to provide a counterfactual (e.g. difference in difference). Justificationgiventochoiceofcomparatorgroupthatisarguedtobesimilartothetreatmentgroup.Evidencepresentedoncomparabilityoftreatmentandcontrolgroups.Techniquessuchasregressionand(propensityscore)matchingmaybeusedtoadjustfordifferencebetweentreatedanduntreatedgroups,buttherearelikelytobeimportantunobserveddifferencesremaining.
Level 4:Quasi-randomness in treatment is exploited, so that it can be credibly held that treatment and control groups differ only in their exposure to the random allocation of treatment.Thisoftenentailstheuseofaninstrumentordiscontinuityintreatment,thesuitabilityofwhichshouldbeadequatelydemonstratedanddefended.
Level 5: Reserved for research designs that involve explicit randomisation into treatment and control groups, with Randomised Control Trials (RCTs) providing the definitive example. Extensiveevidenceprovidedoncomparabilityoftreatmentandcontrolgroups,showingnosignificantdifferencesintermsoflevelsortrends.Controlvariablesmaybeusedtoadjustfortreatmentandcontrolgroupdifferences,butthisadjustmentshouldnothavealargeimpactonthemainresults.Attentionpaidtoproblemsofselectiveattritionfromrandomlyassignedgroups,whichisshowntobeofnegligibleimportance.Thereshouldbelimitedor,ideally,nooccurrenceof‘contamination’ofthecontrolgroupwiththetreatment.
Note:TheselevelsarebasedonbutnotidenticaltotheoriginalMarylandSMS.Thelevelsherearegenerallyalittlestricterthantheoriginalscaletohelptoclearlyseparatelevels3,4and5whichformthebasisforourevidencereviews.
![Page 17: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/17.jpg)
Evidence Review: Transport - July 2015 17
Definition
Transportimprovementprojectsarebroadinscope,notonlyintermsofthetransportmode(seebelow),butalsointermsofthetypeofinterventions.Threebroadtypesofinterventionswereconsideredaspartofthisreview:
• Physicalintervention–i.e.theexpansionandimprovementoftransportinfrastructure.Thiscouldeitherincludethebuildingofnewroutesandfacilities,orthroughmakingcapitalimprovementstoexistingones(e.g.increasinghighwaycapacitythroughjunctionupgradesorextralanes).
• Serviceenhancement–i.e.wherethephysicallayoutofthetransportinfrastructureremainsunchangedbutwhereitsqualityisincreased(e.g.improvementstoreliability,increasingservicefrequency).
• Revenueprojects–i.e.changestothewayexistingtransportinfrastructureissuppliedandconsumed.Thiscanbesplitintotwofurthergroups:
• Pricinginterventions/subsidies–e.g.faresubsidies,car-poollanes,congestionchargesetc.
• Sectoralservicechange–changingtheownershiporoperationoftransportservices,e.g.privatisationornationalisation.
Whilstevaluationsfromallthreegroupswereincludedduringthesearchphaseofthereview,ultimatelythemajorityofthearticlesmeetingtheCentre’sstandardsfocusonphysicalinterventionstoexpand/improveinfrastructure.
Tohelporderthelargeamountofliterature(around2,300policyevaluationsandevidencereviews),studiesweresplitbymodeasfollows:
• Road.
• Rail–coveringarangeoftypes,includinghighspeed,regional,urban,andlight(e.g.subway)railinfrastructure.
• Non-railpublictransport–e.g.tramsandbuses.
• Walkingandcycling.
06
![Page 18: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/18.jpg)
Evidence Review: Transport - July 2015 18
• Ports.
• Airports.
• Multi-modal.
Thisreportcoversevidenceonroads,rail(includinglightrailandsubways),trams,buses,cyclingandwalking–areasofexpenditurewhichaccountforthemajorityoftransportspendingthatwillbeconsideredbylocaldecisionmakers.Evidenceonportsandairportswillbeconsideredinafurtherreview.
Impact evaluation for infrastructure projectsAsdiscussedabove,evaluatingtheeconomiceffectsoftransportprojectsischallenging:transportwillaffectmultipleeconomicoutcomesinwaysthatarehardforresearcherstodisentangle.
Therearealsospecificchallengesinundertakinghighqualityimpactevaluation.Itisfairlyeasytounderstandhowwemightconstructcontrolgroupsandundertakeevaluationforpoliciestargetedatindividuals,householdsorfirms.Itishardertothinkabouthowwemightdothisforpolicies–suchasrailandroad–thattargetareas.Inadditiontooursubstantiveinterestintheimpactsofpolicy,oneofourmotivationsinconsideringtransportistohelpconvincedecisionmakersthatbetterevaluationispossible.Thissectionprovidesabriefexplanationofhowthereportsweconsideredhavetriedtodothis.Furtherdetailsonspecificexamplescanbefoundinourscoringguideavailablefromwww.whatworksgrowth.org/resources/scoring-guide.
Evaluationofthelocaleconomicgrowtheffectsoftransportisparticularlychallenging.Theuseofcost-benefitanalysismeansthatmuchinfrastructurespendingoccursinareaswherethereisexpectedtobestrongandgrowingdemand.Oftentheselocationswillalreadybeexperiencingeconomicgrowthandincreasesinjobsandwages–underlyingfactorsthataredrivingthegrowthindemand.Theeffectsoftheseunderlyingfactors(‘selectioneffects’)mustbeaccountedforifwewanttounderstandtheextenttowhichtransportspendingactuallyincreasesgrowth.
Selectionislikelytobeamuchbiggerproblemfortransportprojectsthanforsomeofourpreviousreviewsthatconsideredsimilarareabasedpolicies.Forexample,whenreviewingtheeffectsofsportsandculturalfacilitiesorofestaterenewal,economicfactorsmayoftenbeoneconsiderationamongmanywhenmakingdecisionsonprojects.However,fortransportprojects,economicfactorsarelikelytobeacoreconsideration.Forthisreason,treatedareasarealmostalwayslikelytobedifferenttountreatedareas.Someofthesedifferenceswillbehardtoobserveinavailabledata,makingitverydifficulttoconstructanappropriatecontrolgroup.Furthermore,itisunlikelythattheseunderlyingdifferenceswillbeconstantovertime.
Inmanycircumstancesevaluationscould,inprinciple,userandomisedcontroltrialstoaddresstheseconcernsoverselection.Forcapitalexpenditure,whereinvestmentsaredurable,itishardtoimaginesituationsinwhichtruerandomisationofprojectplacementwouldbeeitherfeasibleordesirable.Thismeansthatweneedtorelyonalternativeevaluationapproachestotrytoaddresstheproblemofselectionandthusidentifythecausalimpactoftransportinvestment.
Manystudiesinthisreviewattempttoaddressthese‘selectionproblems’usingvariationsondifference-in-differenceorpanelfixedeffectsmethods.Inthesemethods,thechangeinoutcomeinthe‘treatment’areas(thosethatundertakeprojects)iscomparedwiththechangeinoutcomeinagroupofsimilarcontrolareas(whichdonot).Thecontrolgroupisconstructedtobesimilartothetreatmentgroupeitherbymatchingonobservedcharacteristicsorbyusingcontrolvariables.By
![Page 19: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/19.jpg)
Evidence Review: Transport - July 2015 19
takingabefore-and-afterdifference,thismethodeliminatesallfixedunobservabledifferencesbetweenthetreatmentandcontrolgroups.However,asalreadydiscussed,therearealsolikelytobetime-varyingunobservabledifferencesthatleadtoinvestmentintransportinfrastructure.Thesemethodscannotaccountfortheseunderlyingfactors.
Inordertoallowfortheseunobservablefactors,andthusmorereliablyassesstheimpactoftransportprojectsitisimportanttoexploitsomesourceofrandomnessinthewaytransportinfrastructureisdelivered.Althoughtheoverallnumberofevaluationswehaveavailableissmall,aroundonethirdofthemhaveattemptedtousemethodsthatexploitsomesourceofrandomness.Thisisalargersharethanformanyofourotherreviews(itisaboutthesameasforbroadband),reflectingtheimportanceofsuchmethodsforevaluationinthispolicyarea.
Forexample,study1067looksattheeffectofhighwaysonemploymentandwagesofskilledlabourusinganinstrumentbasedonthefactthattheUShighwaysystemwasplannedalongagridpattern.6Thismeansthathighwaysaremorelikelytorunthroughruralcountriesthataredirectlynorth,south,eastorwestofthenearestmajorcity.Thisarbitraryfeatureofthesystemprovidesquasi-randomvariationinthedeliveryofroadstotheruralcountiesthatcanbeexploitedtoestimateacausaleffect.Onaverage,countieslyingdirectlynorthofamajorcityarenotexpectedtobedifferentfromcountieslyingsaynorth-eastofamajorcityapartfromthefacttheyaremorelikelytoreceiveahighway.Therefore,anydifferenceintheemploymentandwagesofskilledlabourmaymoreconfidentlybeattributedtotheeffectsofthehighwayinfrastructure.
Inasecondexample,paper1017examinestheeffectofroadsonfirmperformance,exploitingthefactthatthefirstfewsectionsofaninter-citymotorwayprovideimprovedaccessbetweenlocationswithinalocalarea.Sincetheintercityconnectionisprovidedtoincreaseaccessbetween,ratherthanwithinlocalareas,thislocalimprovementisconsideredquasi-random.Thereforeanyimprovementsinfirmperformancefortheimprovedareascomparedwithsimilarunimprovedareasinthesamelocalareacanbeattributedtotheeffectoftheroad.
Thesemethodsarepotentiallytheonlywaytoachievereliableestimatesoftheimpactoftransportinvestmentonlocaleconomicgrowthoutcomes.Futuretransportevaluationsshouldpaycloseattentiontotechniquesusedinstudiessuchasthese,anissuetowhichwereturnbelow.
6 AllstudynumbersrefertospecificevaluationsaslistedinAppendixB:EvidenceReviewed.
![Page 20: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/20.jpg)
Evidence Review: Transport - July 2015 20
Findings
Thissectionsetsoutthereview’sfindings.Webeginwithadiscussionoftheevidencebase,andthenexploretheoverallpatternofresults.Afterthisweconsiderspecificoutcomesinmoredetail.
Thereviewinitiallyconsidered2,300policyevaluationsandevidencereviewsfromtheUKandotherOECDcountries,identifiedduringtheinitialkeywordsearch.Thisisasignificantlylargerstartingevidencebasethanourearlierreviews.
Followingafurtherhighlevelreview,over1,800weresiftedoutasnotrelevant(e.g.becausetheyweretheoreticalratherthandata-based;reviewednon-OECDcountries;orbecauseofsubjectrelevance).Fromtheremainingevaluations,wediscardedover250furtherevaluationsastheywerefoundnottobeeconometricallyrobust.Finally,232studieswereshortlistedfordetailedreview.Theresultsofthatdetailedreviewareoutlinedinthefollowingsections,whichsplittheevaluationsbymode.
ThescaleatwhichthestudiesevaluateimpactvariesfromadjacentneighbourhoodstomuchlargerUScounties.
Roads
Quantity and quality of the evidence base
Ofthe232shortlistedstudiesreviewedindetail,80consideredtheimpactofroadsprojects.
Ofthese80studies,anadditional62studieswerediscounted:Eightongroundsofrelevance,and54ongroundsofnotmeetingtheCentre’sminimumstandardofevidence(i.e.scored2orbelowontheSMSscale).Theremaining17studieshavebeenincludedinthisreview.
Thisisasmallerevidencebasethanmostofourreviewstodate(onemploymenttraining,businessadvice,sportsandcultureprojects,accesstofinanceandestaterenewal)butroughlyonparwithourreviewofbroadband.Asdiscussedabove,thispartlyreflectsthedifficultiesinevaluatingtransportprojectsbutisalsoindicativeofafailuretocarefullyevaluateexistingpolicyinterventions.Table1showsthedistributionofthestudiesrankedbySMSscore.
07
![Page 21: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/21.jpg)
Evidence Review: Transport - July 2015 21
Table 1: Implementation Quality Scores
SMS Score No. of studiesEvaluation reference
numbers
3
12 1009,1011,1015,1016,1027,1031,1035,1050,1052,1055,1061,1062
4 5 1005,1017,1037,1063,1067
Total 17
Wefoundnostudiesthatusedrandomisedcontroltrials,butfivestudiesthatusedcrediblerandomsourcesofvariation.Asdiscussedintheprevioussection,thisisnotthatsurprisinggiventhenatureoftheseprojects.Theremaining12studiesusedvariationsofdifference-in-differenceandpanelmethods(scoring3ontheSMS).Thetechniquesappliedinthesestudiesmeanthatwecanbereasonablyconfidentthattheyhavedoneagoodjobofcontrollingforobservablecharacteristicsofareas,individualhouseholdsandfirmsaffectedbytheprojects.However,itislikelythatunobservablecharacteristicsmaystillbeaffectingtheresults.
Type and Focus of Support
Inmostofourpreviousevidencereviewswehavefocusedonspecificpolicyinterventionsaimedatdeliveringparticularobjectives(e.g.governmentfundedemploymenttraininginourfirstreview).Incontrast,thevastmajorityofstudiesinthisreviewfocusonevaluatingtheimpactofspecificinvestmentprojectsoroverallspendingratherthanevaluatingaspecificpolicywithexplicitobjectivesandrationales.Thisisunsurprisinggiventhenatureofmosttransportinvestment.
Themajorityofprogrammeswerepubliclyfunded(eitheratanational,localorEUlevel).Oftheevaluations:
• Twostudiesevaluatenamedpolicieswhichprovidedfundsforroadbuildingandimprovementprojects:
• EuropeanRegionalDevelopmentFund–Trans-EuropeanNetworks(TENs),EU.7
• TheuseofEuropeanStructuralFundstoupgraderoadsinSpain.8
• Tenevaluationsexaminedconstructionandimprovementworksrelatedtoroadnetworksgenerally:
• TheexpansionofPortugal’smotorwaynetworkintoeconomicallylaggingregions.9
• TheexpansionoftheUnitedStatesInterstateHighwaySystem.10
• InvestmentintheSpanishroadsnetworkgenerally.11
• RoadconstructionandimprovementprojectsintheUnitedKingdomgenerally.12
• DevelopmentofroadsintheUnitedStatesgenerally.13
7 Study1055.8 Study1016.9 Study1061.10 Study1067.11 Study1062.12 Study1017.13 Study1037.
![Page 22: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/22.jpg)
Evidence Review: Transport - July 2015 22
• ThegrowthofroadnetworksinthestateofMinnesota,USA.14
• RoadwideningprojectsinthestateofNorthCarolina,USA.15
• OnestudyreferstotheUnitedStates1947highwayplanasaproxyfordatasetconstruction.16
• Anotherstudyfocussesonfederalroad-buildingintheUnitedStatesfollowingtheHighwayAct1944andtheInterstateHighwayAct1956.17
• Onestudyfocussesontollroads,constructedinOrangeCountyinCalifornia,USAandoperatedbyTransportationCorridorAgencies.18
• Fivestudiesfocusontheconstructionofspecificroadsorroadnetworks:
• TheopeningofInterstate105inCalifornia,USA.19
• TheconstructionandopeningoftheInterstate210extensioninCalifornia,USA.20
• TheconstructionandopeningofStateRoutes87,85and237inLosAngeles,USA.21
• TheextensionofPresidentGeorgeBushTurnpikeandDallasNorthTollwaytollroadsinDallas,USA.22
• TheconstructionoftheM6andM60highwaysinHungary.23
Oftheseventeenstudiesontheroadsfinalshortlist,onlyonefocussesontheeffectsofroadconstructionandimprovementsintheUK.Themajorityofstudies(eleven),examineprogrammesintheUSA.TheremainingstudiesevaluateprogrammesinSpain(two),PortugalandHungarywithonestudyexaminingprogrammesthroughouttheEU.
Findings by outcomeAbreakdownofthestudiesbyoutcomeandoverallfindingisprovidedinthetablesinAppendixA.
Employment
Road projects can positively impact local employment. But effects are not always positive and a majority of evaluations show no (or mixed) effects on employment.
Table 2: Road investment evaluations by outcome on employment
Outcome No. of studiesEvaluation reference
numbersPositive 2 1011,1017
Zero 3 1027,1031,1067
Mixed 1 1015
14 Study1027.15 Study1031.16 Study1063.17 Study1005.18 Study1009.19 Study1011.20 Study1035.21 Study1015.22 Study1050.23 Study1052.
![Page 23: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/23.jpg)
Evidence Review: Transport - July 2015 23
Ifweareinterestedintherolethattransportmayplayindrivingeconomicgrowth,thenacentralquestionistheextenttowhichprojectscausechangesinemployment.Sixevaluationsconsidertheimpactofroadprojectsonlocalemployment.Ofthese,twoevaluationsfindpositiveimpacts,threefindnoimpactandoneevaluationshowsmixedresults.
Ofthetwoevaluationsreportingpositiveeffects,onefindsimpactsthatarerelativelylarge:employmentinthetreatmentgroupincreasedby200%against10%inthecontrolgroupoverthestudyperiod,between1980and1997.24Theauthorssuggestthatthestrengthoftheseeffectsmayreflectnegativespillovers–i.e.positiveeffectsonareasalongthehighwaycorridor(the‘treatment’group),accompaniedbylossesforareasatagreaterdistance(the‘control’group).Movingjobsaroundisnotthesameascreatingjobs,therefore,thisissueofdisplacementshouldbeamajorconcernforlocaldecisionmakersinterestedindistinguishingbetweentotalandadditionaleconomicgrowth.
Forthe‘noimpact’evaluations,onestudyfoundtherelationshipbetweenroadnetworkexpansionandemploymentgrowthdisappearedaftercontrollingforlocationalfactorssuchashumancapitallevelsandtaxrates.25Similarlyonestudyfoundthatevenaftercapacityenhancements,thehighwaynetworkquicklybecamesaturatedwithtraffic.26Employmentbetween1985and1997remainedunchangedin‘treated’countieswithanincreaseindensityofhighwaylane-milesduringthatperiod.
Themixedresultlookedatseveralcasestudieswhichshowedincreasing,staticanddecreasingtotalemploymentintheareasaroundhighwayexpansionorimprovementprogrammesinthreeneighbouringCaliforniacounties.27
Firm Entry and Number of Businesses
Road projects may increase firm entry, although not necessarily the overall number of businesses (as new entrants may displace existing firms).
Table 3: Road investment evaluations by outcome on firm entry
Outcome No. of studiesEvaluation reference
numbersPositive 2 1017,1061
Zero 1 1016
Whenemploymenteffectsarepositive,thismaybedrivenbybothexpansionofexistingfirmsandentryofnewfirms.Evenintheabsenceofemploymentgrowth,effectsonfirmentry,exitandtheoverallnumberofbusinessmaybeofinteresttolocaldecisionmakers.
Threeevaluationsconsidertheseeffects,withtwofindingpositiveeffectsandonefindingzeroeffect.Thetwostudiesreportingpositiveeffectsbothlookatfirmentry.28Inoneofthesestudies,plantbirthwasaffectedpositivelyacrossmostsectorswithin10kmofnewmotorways,althoughtheeffectwaslargestforsectorsrequiringproximitytomarketsandclientssuchasprimaryindustries.29Theshareofindustrialsectorplantbirthsinmunicipalitieswithin10kmofmotorwaysincreasedatamuchhigherratethantheshareofservicesectorfirmbirthsinthesamemunicipalitiesbetween1986and1997.The
24 Study1011.25 Study1027.26 Study1031.27 Study1015.28 Studies1017and1061.29 Study1061.
![Page 24: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/24.jpg)
Evidence Review: Transport - July 2015 24
authorsattributethisdifferenceineffectacrosssectorstoapossiblegreaterrelianceontransportbymanufacturingfirms.Thepositiveeffectstendedtodecreasebeyond10km,withzeroeffectsbeyond50km.Again,thisraisesthepossibilitythatthereissomedisplacementtoareasnexttotheproject,fromareasclosetobutnotnexttotheproject.Thesecondpositivestudy30consideredbusinessaccessibilitytoroadimprovementsbyUKelectoralwardsandfoundthatwardsincloseproximitytoroadimprovementsrecordedanincreaseinnewplantsoverthestudyperiod.Inthisevaluationitwasfoundthata10%improvementinaccessibilityleadstoa3%increaseinthenumberofbusinessesandemploymentupto30kmfromthesiteoftheimprovement.Incontrasttothesetwostudies,thethirdstudy,foundthatnationalroadcapacityinSpainhadnoeffectonthetotalnumberoffirms.31
Property Prices
Road projects tend to have a positive effect on property prices, although the effect in prices may depend on distance to the project (and the effects can vary over time).
Table 4: Road investment evaluations by outcome on property prices
Outcome No. of studiesEvaluation reference
numbersPositive 3 1009,1035,1052
Mixed 1 1050
Fourevaluationsconsiderpropertyprices.Threefindpositiveimpactswithoneshowingmixedresults.
Twoofthestudieslookataveragehousepricesin‘treated’areasandfindpositiveimpactsonhousepricesrelativeto‘untreated’areas.
Theothertwostudiessuggestthatpriceeffectsdependondistancetotheroadproject(consistentwiththehedonicpricingliteraturethatlooksatthelinkfrompropertycharacteristicstoprices).Housesclosetotheprojectdonotexperiencethesamepositivepricerisesasthoseclose,butnotimmediatelyadjacentto,theproject.Theymayevendepreciate.Thesizeofthe‘buffer’zoneinwhichthesenon-positiveeffectsoccurvaries:inonestudynegativeeffectsarepresentupto0.2miles,butpositivefrom0.25milesaway32;whileasecondstudyshowsoverallpositiveeffectsonpropertypricesinalltreatmentareasalbeitwithslightlysmallerincreasesupto0.4milesfromtheintervention.33
Oneofthetwoevaluations(study1035)thatconsideredthespatialpatternofdistanceeffectsalsolookedatwhetherthesechangedovertime.Itfoundlittleevidenceofannouncementeffects(2yearspriortoconstruction)butpriceswerealreadyincreasingclosetotheprojectinthefirstthreeyearsofthefiveyearconstructionperiod.34Afterprojectcompletion,thegreatestpriceappreciationwas0.4-0.8milesawayduringthefirst3yearsfollowingcompletion,0.8-1.2milesaway4yearsfollowingcompletionwitheffectsdisappearinginthefifthyear.
30 Study1017.31 Study1016.32 Study1050showsnegativeeffectsofaround10%upto0.2milesfromthetoll-roadcorridor,whilepricesincrease13%
at0.25-1miledistantand19%at1-2milesdistant.33 Study1035showsthathouses0.4milesfromthehighwayarebetween$22,000and$33,000moreexpensivethan
thoseadjacent,withthesepositiveeffectsdiminishingaspropertiesarelocatedfurtherfromthehighway.34 Theseeffectsdisappearedforyears4and5ofconstruction,whichtheauthorsspeculatethiscanbeattributedto
increasednoiseexternalityduringthatperiod.
![Page 25: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/25.jpg)
Evidence Review: Transport - July 2015 25
Thegeneralfindingthatpriceeffectsvarybydistancefromtheprojectandtimefromconstructionmayapplytoothercontexts(andasalreadynotedthedistanceeffectisconsistentwiththewiderhedonicliterature).35Thiscouldhaveimportantimplicationsforpredictingtheimpactofschemesandforincorporatinglandpriceupliftinappraisalandevaluation.Wereturntothisissueintheconclusions.
Population
The impact of roads projects on local population may vary depending on whether the project is urban, suburban or rural.
Threeevaluationsconsideredtheimpactofroadsonlocalpopulation.Allthreeevaluationslookedattheeffectofoverallroadinvestmentsratherthanaspecificproject.
Study1005foundthatanewhighwaypassingthroughacitycentreleadstoan18%fallinpopulation,whileeach‘ray’(ahighwaysegmentconnectingtheCentralBusinessDistrictwiththesuburbs)causesa9%drop.Thisimpliesthattheconstructionofaroadmayleadtosuburbanisation.
Consistentwiththis,study1015foundpositiveeffectsonhousingdevelopment(andhencepopulation)fornon-urbanareaswithin0-0.75milesofnewroads.36
Study1027comestoasimilarconclusionthatthelinkagebetweenhighwayinfrastructureandgrowthpatternsvariesdependingonthetypeofimprovementandcharacteristicsofthelocation.Inurbansettings,highwaysmayleadtopopulationdecline37whileinasuburbanorruralcontext,populationincreasesclosetothehighway38.
Aswiththepropertypriceeffects,itishardtoknowwhethertheseresultsgeneralise.However,aswiththeemploymentresults,theyemphasisethefactthatlocaleffectsofroadprojectsneednotnecessarilybepositive.
Income/Wages
There is some evidence that road projects have positive effects on wages/income.
Onlytwoevaluationsconsideredtheimpactofroadconstructiononincomeand/orwageswithonestudyfindingpositiveeffects,theotherreportingmixedfindings.39
Thepositiveeffectsinstudy1017varywiththeextentofchangesinaccessibility.Within20kmofnewroadconstructionprojects(includingnewjunctions,dualling,widening,upgradesandroadconstruction)afirmexperiencingthemeanincreaseinaccessibilitysawa0.2%averageincreaseinwages(calculatedastotalwagebillperworker).
Study1067reportsmoremixedfindingsforruralcountiesinthevicinityoftheUnitedStatesInterstateHighwaySystem.Countieswithahighendowmentofskilledworkerssawincreasesinwages,whilethosewithalowproportionofskilledworkerssawdecreases.
35 Howeveritisunlikelythattheexactpatternofeffectsreportedinthesetwoevaluationswillgeneralise:indeed,study1050showthattheyvarydependingonlocalcontextandtheparticularstretchofroadevaluated.
36 Theseeffectscanbequitelarge.Inonearea(inMercedCounty),therewas65,501moresquarefeetofhousingconstructedperkilometersquaredwithin0-0.75milesfromthehighway.Note,however,thatthiswaspartlyoffsetbynegativeeffects2.7-3milesfromthehighway.
37 Study1005.38 Study1015.39 Studies1017and1067.
![Page 26: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/26.jpg)
Evidence Review: Transport - July 2015 26
Itishardtogeneralisefromtheseresults–althoughstudy1017istheonlystudythatfocusesonroadprojectsintheUK.Theydoprovidesome,albeitlimited,evidencethatproductivityeffects(whichunderpinUKWebTAGcalculationsofwiderbenefits)occurinpractice.Theseproductivityeffectsarealsoconsidereddirectlyintwofurtherstudieswhichweconsidernext.
Productivity
There is some evidence that road projects have a positive effect on productivity.
Twoevaluationsconsidertheimpactofroadsonproductivity.Onestudyfindsthat,ingeneral,theconstructionoforimprovementtomajorroadsleadstoa0.4%upliftinGVAperworker(thougha0.2%increaseinworkers’wagessuggeststhatpartofthisproductivitygainispaidoutinincreasedsalaries–thesefindingsarecomplementary).40
Thesecondstudyalsofindspositiveeffectsonprovincialproductivitywithstrongereffectsforareasintensiveinsectorsthataremoredependentonroads(e.g.manufacturingandlogistics).41Roadusebyprovincialindustriesisproxiedbytheseindustries’vehicleintensitywiththestudyfindingaparticularincreaseinuseofroadsbyindustrialsectors.
Alongwiththeresultsonwages,thisprovidesmoredirectevidencethattheproductivityeffectsthatunderpinWebTAGappraisalguidanceoccurinpractice.Although,asshouldbeclearfromthediscussionsofar,thenumberofevaluationsthatcandemonstrateacausallinkfromroadprojectstoproductivityisextremelylimited.
Other Outcomes
Twoevaluationsconsideredimpactonbusinessandtradevolume42.Bothfoundpositiveimpacts.Thefirstoftheseobserveda1.4%upliftinthevalueoftradeanda1.9%increaseinthevolumeoftradeforevery1%reductionintraveldistancebetweentradingpartners.43Thesamestudyalsofounda10%increaseinthestockofurbanhighwaysincreasedexportweightby5%(butdidnotinduceanupliftinvalue).Thesecondstudyfoundthattruckingactivityincreasedby7-10percentagepointspercapitainruralcountiescrossedbyhighways.44
Twoevaluationsconsideredtheimpactoninnovation(oneintermsofoutputs,theotherintermsofinputs).ThefirstoftheseconsideredtheimpactofInterstatehighwaysintheUSAonpatentingactivity45.Thefindingssuggestthata10%increaseinaregion’shighwaystockcauseda1.7%increaseinregionalinnovationgrowthoverafiveyearperiod.ThesecondstudyconsideredtheeffectofroadinfrastructureonGDPinNUTS1andNUTS2EUregions.46Theevaluationfoundthattheeconomicperformanceofregionswithagoodendowmentofmotorwayinfrastructureisenhancedwhentheyhave–andaresurroundedbyregionswith–highlevelsofR&Dinvestment.
40 Study1017.41 Study1062.42 Studies1037and1067.43 Study1037.44 Study1067.45 Study1063.46 Study1055.
![Page 27: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/27.jpg)
Evidence Review: Transport - July 2015 27
Rail
Quantity and quality of the evidence base
Ofthe232shortlistedstudiesreviewedindetail,95consideredtheimpactofrailprojects.
Ofthese95studies,anadditional83studieswerediscounted:Eightongroundsofrelevance(e.g.theylookedatanoutcomenotrelatedtolocaleconomicgrowth,suchasridershiplevels),47and65ongroundsofnotmeetingtheCentre’sminimumstandardofevidence(i.e.scored2orbelowontheSMSscale).Theremaining12studieshavebeenincludedinthisreview.
Thisisasmallerevidencebasethanourreviewstodate(onemploymenttraining,businessadvice,sportsandcultureprojects,accesstofinanceandestaterenewalandbroadband)aswellasbeingsmallerthanthatavailableforroads.Aswithroads,thispartlyreflectsthedifficultiesinevaluatingtransportprojectsbutisalsoindicativeofafailuretocarefullyevaluateexistingpolicyinterventions.Table5showsthedistributionofthestudiesrankedbySMSscore.
Table 5: Ranking Studies by Quality of Implementation
SMS ScoreNumber of
studies Evaluation reference
numbers
3 10
1070,1071,1074,1083,1107,1108,1111,1112,
1114,1116
4 2 1075,1109
Total 12
Wefoundnostudiesthatusedrandomisedcontroltrials,buttwostudiesthatusedcrediblerandomsourcesofvariation.Asdiscussedforroads,thisisnotthatsurprisinggiventhenatureoftheseprojects.Theremainingtenstudiesusedvariationsofdifference-in-differenceandpaneltechniques(scoring3ontheSMS).Thetechniquesappliedinthesestudiesmeanthatwecanbereasonablyconfidentthattheyhavedoneagoodjobofcontrollingforobservablecharacteristicsofareasandindividualhouseholdsandfirmsaffectedbytheprojects.However,itislikelythatunobservablecharacteristicsmaystillbeaffectingtheresults.
Types and Focus of Support
Aswithroads,thestudiesincludedinthefinalshortlistdidnotgenerallyevaluatespecificpolicies(e.g.nationalisation/privatisation,co-ordinatedrollingstockimprovementschemesetc.).Instead,theyeither:
• Focussedonindividualnewrailprojectsinspecificlocations.
• Evaluatedtheimpactofaccesstorailmoregenerally.
Theevaluatedprojectsvariedbybothscaleandtype:
• Twoevaluationslookedathighspeedrail.ThefirstlookedattheimpactofnewrailservicesbetweenCologneandFrankfurt,ontwosmalltownsthatgotnewstationsontheline.48The
47 Thisincludedonehighqualitystudy(SMSlevel4)whichevaluatedtheimpactofrailexpansioninnineteenthcenturyAmericaon‘farmimprovement’showingthatcountiesthatgainedaccesstorailwaysbetween1850and1860experiencedsignificantlygreaterincreasesinthepercentageoffarmsthatwereimproved.
48 Study1075.
![Page 28: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/28.jpg)
Evidence Review: Transport - July 2015 28
secondevaluatedtheopeningofahighspeed(Shinkansen)raillineinJapan.49
• Threeevaluationslookedatregionalrail(i.e.connectingdifferentcitiesorwiderregions).50Allthreeofthesestudieslookedatrailaccessgenerally,ratherthanspecificprojects.
• Sevenevaluationslookedatlightrail:
• Study1108and1114bothfocusedonanewlightraillineinCharlotte,NorthCarolina.
• Study1071evaluatedtheHudson-BergenLightRailsysteminNewJersey.
• Study1070evaluatedtheRiverLineraillineinNewJersey.
• Study1083focusedontheMetrorailinMiami.
• Study1111evaluatedlightrailtransitinMinneapolis.
• Study1112studiedtheimpactofextensionstotheDocklandsLightRailwayandJubileeLineinLondon.
Fundinganddeliveryfortheprogrammesarenotstatedinmanyoftheevaluations,butwherethisismentionedtheyareoverwhelminglypubliclyfunded.Similarly,theobjectiveoforrationalefortheinterventionisoftennotreported,thoughinsomecasesitisimpliedthatatleastpartoftherationalewastoboosteconomicgrowth.
Findings by outcomeAbreakdownofthestudiesbyoutcomeandoverallfindingisprovidedinthetablesinAppendixA.
Property Values
Rail projects tend to have a positive effect on property prices, although the size of the effect varies considerably.
Table 6: Rail investment evaluations by outcome on property values
SMS ScoreNumber of
studies Evaluation reference
numbersResidentialPositive 5 1070,1071,1107,1108,1112
Zero 2 1074,1083
CommercialZero 1 1108
Sevenevaluationsconsiderpropertyprices.51Allsevenstudiesconsidertheeffectofproximitytonewrailstationsonresidentialpropertyprices,withstudy1108alsolookingatcommercialpropertyprices.Fiveoutofthesevenstudiesthatconsideredresidentialpropertyfoundpositiveeffectsofproximitytostations,whiletwostudiesfoundnoeffectofproximity.Theonestudythatconsideredcommercialpropertypricesfoundnoeffectsofproximitytostations.
49 Study1117.50 Studies1107,1109,1074.51 Studies1071,1107,1074,1070,1083,1108and1112.Mostevaluationsmeasuredchangesinpropertyvaluesusing
hedonicmodelsappliedtorepeatedsalesdatasets.
![Page 29: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/29.jpg)
Evidence Review: Transport - July 2015 29
Forevaluationsshowingpositiveeffects,thedegreeofpriceappreciationrangedfromextremelysmalltoquitesubstantial.Forexample,Study1108(whichlookedattheimpactoflightrailinCharlotte,NorthCarolina)foundeffectsthatrangedfromnearzerouptoaround13%,dependingon:thetypeofproperty(forexample,condominiumsseeagreaterincreasethansingle-familyproperties);andproximityfromthestation(forexample,single-familyhomeswithinhalfamileofthestationseenoimpact,whilstcondominiumswithinhalfamilearesubjecttoagreaterincreasethanthosefurtheraway).Study1071foundeffectsashighas18.4%.Thishighvariationinpriceeffectsacrossstudiesimpliesasimilarlyhighvariationintheimpliedvalueofimprovementinrailaccessintermsofwillingnesstopayforresidentialhousing.
Incontrast,Study1074foundthataccesstointercityrailconnectionsinpost-unificationBerlinhadnoimpactonpropertyprices.Interestingly,theevaluationalsofoundthatthenewmainlinenetworkhadonaverageanadverseimpactonmainlineaccessibilityatthecitylevel(asaresultoftheallocationoftransportcapacityfavouringsomelinesoverothers,includingthecompletedisconnectionofastationinanareawhichhadservedastheCBDofWestBerlinfordecades),whichmayexplainthelackofeconomicbenefit.Study1083,whichlookedattheimpactoftheMiamiMetrorailonthevalueofhousesnearstationlocations,alsofoundnostatisticallysignificanteffectsonresidentialpropertyprices.
Aswithroads,anumberofevaluationssuggestthatthepriceeffectsdependondistancefromtheproject(consistentwiththehedonicpricingliteraturethatlooksatthelinkfrompropertycharacteristicstoprices).Study1071findspositiveeffectsuptoaquartermilefromthestation,witheffectsdecreasingwithdistance.Study1107foundsimilarevidencethateffectsdecayedwithdistance,buteffectswerestillpositiveupto2.2milesfromthestation.Resultsweresimilarforstudy1070,althoughhereeffectswerepositiveupto4miles.Finally,study1108suggeststhattheeffectofdistancemaydifferbypropertytypewiththelargesteffectsforcondominiumsathalfamile,butthelargesteffectsforone-familyhomesatamile.Incontrasttothefindingsforroads,noneoftheevaluationsreportsmallereffectsforpropertiesveryclosetostations.52
Anumberofevaluationsalsoconsideredthetimingofpricechanges.Forexample,Study1107(intheNetherlands)and1083(inMiami)foundweakevidenceofanannouncementeffect–i.e.theappreciationofpropertypricespost-announcement,butpre-completionoftheproject.Thelengthoftimeoverwhichpriceeffectsareobservedalsovariesbystudy.Somestudiesonlylookateffectsoneortwoyearsaftercompletionofastation(forexample,Studies1071,1047);othersconsideredchangesoveramuchlongertimeperiod(forexample,Studies1070,1112).
Other outcomes
Asidefromtheeffectonpropertyprices,theevaluationevidenceonotherlocaleconomicimpactsisextremelylimited.Wefoundnoevaluationsthatconsideredtheimpactonemployment–whichisstartlinggiventhedegreeofinterestintheexistenceandsizeoftheseeffects.
Oneevaluation(Study1116)lookedattheimpactofanewJapanesehighspeed(Shinkansen)passengerraillineonbusinessperformanceandbusinessproductivity,findingpositiveeffectsinbothareas.Businessperformanceismeasuredbysalesrevenueandbusinessproductivityismeasuredbysalesrevenueperemployee.Theauthorssuggestthatthesebenefitsoccurredasaresultofincreasedaccessandlowersearchcoststoothermarkets,resultinginfirmsbeingabletofindbettersuppliers.Thesebenefitsoccurreddespitethefactthattheinterventiononlyloweredthecost/timeof
52 AlthoughanumberoftheSMS2levelstudiesdidreportsmaller,ornegative,priceeffectsforpropertiesveryclosetoimprovements.
![Page 30: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/30.jpg)
Evidence Review: Transport - July 2015 30
passengertravelanddidnotaffectfreighttransportationcosts.Overall,theresultsindicatedthatforinput-intensivefirmssalesperemployeeincreasedrelativetotheindustryby42%.
Asecondhighspeedrailstudy53lookedattheimpactonGDPfortwosmalltownsthatlieonthenewhighspeedlineconnectingCologneandFrankfurt.Thestudyreportsquitelargeeffectsamountingtoa2.7%increaseintotalGDP.Unfortunately,thecontextthatmakesthisstudyattractivefromananalyticalpointofview(thisstudyscoreslevel4ontheSMSscale)alsomakesthefindingshardtogeneralise.ThesmalltownsofMontabaurandLimburg(populationsof12,571and33,84354)happenedtogetstationsfollowingcomplexnegotiations,despitetheirsmallsizeandperipherality.Thishelpsaddressconcernsabout‘selectionintotreatment’,55butmakesitimpossibletoknowwhethertheseGDPeffectswouldextrapolatetothelargercities(e.g.Birmingham,LondonandManchester)thatwouldtypicallybehometonewhighspeedrailstations.
Theresultsfromtheonlyevaluationthatconsiderspopulation(study1109)aresimilarlyhardtoextrapolate.ThisstudylookedattheimpactofrailwayconstructioninSwedeninthenineteenthcentury,evaluatingitsimpactonthegrowthofcitiessincethattime.Thestudyfoundthatcitieswithearlyaccesstothenetworkcontinuedtogrowfasteroverthefirsthalfofthetwentiethcentury,andthatthetreatmentgroupcitiesareonaverage51%largertodaycomparedtocitiesthatdidnotgainaccesstotherailroadnetworkinthefirstwaveofexpansion.Allofthissuggeststhateffectsmayplayoutovertheverylongterm,butitishardtoknowwhetherthesegeneralisetoadditionstoanalreadyexistingwell-developedrailnetwork.
Perhapsofmoreinteresttolocaldecisionmakersisstudy1111whichexaminedtheimpactofametrolineinMinneapolisonlandusechanges,findingonlysmallandverylocalisedimpact.Specifically,single-familyandindustrialpropertieswithinhalfamileofoperationalstationsexperiencedasmallincreaseinthelikelihoodoflandusechangeawayfromtheseuses.However,onalargerscaletheintroductionofthenewlinedidnotincreasethelikelihoodofchangesinlanduseabovenormallevels,nordidithaveanyeffectonthelikelihoodofchangesinlanduseawayfromvacantland,commercialpropertiesandmulti-purposefamilies.
Afinalstudy56lookedattheimpactofrailonlevelsofcrime.Study1114,lookingattheimpactofnewlightrailtransitinCharlotte,NorthCarolina,foundthattheannouncementoftherailtransitledtoadecreaseinpropertycrimes,whichwasmaintainedevenafterthestationswereopened.Thiswasattributedtopublicandprivatedecisionstoinvestalongtransportcorridors,whichgentrifiedsurroundingneighbourhoodsanddecreasedcriminalactivity.Oncethestationsopened,thedecreaseincrimewasmaintainedanddidnotreturntopre-announcementlevels.Aswithanumberofpreviousreviews,thisfindingservestohighlightthefactthatinfrastructureinvestmentcandeliveramenitybenefitsthatareimportant,butseparateto,theeffectonlocaleconomicgrowth.
Other modes: Trams, Buses, Cycling and Walking Ofthe232shortlistedstudiesreviewedindetail,10consideredtheimpactofbuses,1theimpactoftrams,1cycling;1walking.Unfortunately,fromthistotalof13studiescoveringthesefourareas,wefoundnohighqualityevaluationsthatprovideevidenceontheimpactsoftrams,buses,cyclingandwalkingschemesonanyeconomicoutcomes.
53 Study1075.54 PopulationstatisticsfromStatistischesBundesamt(2014).55 SeetheImpactEvaluationforTransportInfrastructuresectionforfurtherdiscussionofthisselectionproblem.56 Study1114.
![Page 31: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/31.jpg)
Evidence Review: Transport - July 2015 31
Summary of findings
Thissectionsummarisesthedetailedfindings.Weemphasisethatmanyofthesefindingsdependonasmallnumberofstudies.Theyare,however,consistentwithotherresearchonthebroaderimpactoftransportimprovements.
What the evidence shows• Roadprojectscanpositivelyimpactlocalemployment.Buteffectsarenotalwayspositive
andamajorityofevaluationsshowno(ormixed)effectsonemployment.
• Roadprojectsmayincreasefirmentry(eitherthroughnewfirmsstartingup,orexistingfirmsrelocating).However,thisdoesnotnecessarilyincreasetheoverallnumberofbusinesses(sincenewarrivalsmaydisplaceexistingfirms).
• Roadprojectstendtohaveapositiveeffectonpropertyprices,althougheffectsdependondistancetotheproject(andtheeffectscanvaryovertime).
• Theimpactofroadsprojectsonthesizeofthelocalpopulationmayvarydependingonwhethertheprojectisurban,suburbanorrural.
• Thereissomeevidencethatroadprojectshavepositiveeffectsonwagesorincomes.
• Thereissomeevidencethatroadprojectshaveapositiveeffectonproductivity.
• Railprojectstendtohaveapositiveeffectonpropertyprices,athougheffectsdependondistancetotheproject(andthesecanalsovaryovertime).
Where there is a lack of evidence • Wefoundnohighqualityevaluationsthatprovideevidenceontheimpactofrailinfrastructure
onemployment,andonlyalimitednumberofevaluationsshowingthatroadprojectshaveapositiveeffect.
• Wefoundnohighqualityevaluationsthatprovideevidenceontheimpactsoftrams,buses,cyclingandwalkingschemesonanyeconomicoutcomes.
• Evenwhenstudiesareabletoidentifyapositiveimpactonemployment,theextenttowhichthisisaresultofdisplacementfromothernearbylocationsisstillunresolved.Moregenerally,
08
![Page 32: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/32.jpg)
Evidence Review: Transport - July 2015 32
thespatialscaleofanyemploymenteffectsvariesandwedonothaveenoughevidencetobeabletogeneraliseaboutthespatialdistributionofeffectsiftheyoccur.Thesameistrueforotheroutcomes.ThescaleatwhichthestudiesevaluateimpactvariesfromadjacentneighbourhoodstomuchlargerUScounties.
• Surprisingly,veryfewevaluationsconsidertheimpactoftransportinvestmentonproductivity(wefoundjustthreestudies,twoforroadsandoneforrail).Althoughtheuseofsuchproductivityeffectstocalculate‘widereconomicbenefits’intransportappraisalisunderpinnedbyalargerevidencebase,itisstillworryingthatsofewevaluationscandemonstratethattheseeffectsoccurinpractice.
• Wehavelittleevidencethatwouldallowustodrawconclusionsonwhetherlarge-scaleprojects(e.g.highspeedrailormotorwayconstruction)havelargereconomicgrowthimpactsthanspendingsimilaramountsonacollectionofsmall-scaleprojects(e.g.lightrailorjunctionimprovements).
• Moregenerally,wedonotknowhowdifferencesinthenatureofimprovements(e.g.journeytimesavedornumberofadditionaljourneys)affectanylocaleconomicoutcomes.
• Thereissomeevidencethatcontextmatters.Forexamplepropertypriceeffectsmaydependonthetypeofproperty,whilewageeffectsmaydifferbetweenlowskilledandhighskilledworkers.But,onceagainwedonothaveenoughevidencetobeabletogeneralise.
How to use these reviewsTheevidencereviewhighlightsanumberoffactorsforpolicymakerstobeawareofwhenconsideringtransportpolicy:
• Muchmoreempiricalworkremainstobedoneonunderstandingtheimpactofinfrastructureimprovementsonlocaleconomicgrowth.Theeconomicbenefitsoftransportinfrastructurespending–particularlyasamechanismforgeneratinglocaleconomicgrowth–arenotasclear-cutastheymightseemonfacevalue.
• WhileitisunderstandablethatpoliticaldebatefocusesonexpenditurefiguresacrossdifferentpartsoftheUK,theydonothelpanswerthequestionofwhatwouldhappenifexpenditurewasdistributeddifferently.Argumentsforspendingmoreinareasthatarelesseconomicallysuccessfulhingeonthehopethatnewtransportisacost-effectivewaytostimulateneweconomicactivity.Asthisreviewshows,wedonotyethaveclearanddefinitiveevidencetosupportthatclaim.
• Thisraisesfundamentalquestionsaboutschemeappraisalandprioritisation,andabouttheroleofimpactevaluationinimprovingdecision-makingaroundtransportinvestment.
Helping to fill the evidence gaps: improving evaluation and appraisalInmanyinstanceslocaleconomicimpactisanimportantpartofthecasefortransportinvestment.Suchinvestmentalsoformsacentralcomponentofmanygovernmentalpolicyinitiativesaimedatincreasinglocaleconomicdevelopment(e.g.theUKgovernment’sLocalGrowthDealprocess).Itisthereforevitalthatprogressismadeinfillingtheevidencegapsandinimprovingourunderstandingoftheeffectoftransportimprovementonlocalgrowth.
Inthisfinalsection,wemakesomepreliminaryrecommendationsbuildingonrecentworkwiththeDepartmentforTransport,aswellasthediscussionsofaLEPworkinggroupconvenedbytheWhatWorksCentreforLocalEconomicGrowth.Ourrecommendationsfocusontheneedformore,andbetter,ex-postimpactevaluationandtheneedtoembedsuchevaluationintotheappraisalprocess.
![Page 33: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/33.jpg)
Evidence Review: Transport - July 2015 33
Considerableresourcesarealreadydevotedtotheex-anteappraisaloftransportschemesaspartofthedecisionmakingprocess.Cost-benefitanalysisplaysacentralroleinsuchappraisals.Theincreasedinterestintheeffectoftransportinvestmentonthelocaleconomyhasalsobeenaccompaniedbygrowingcriticismoftheappraisalapproachesusedtohelpfacilitateschemeprioritisation.Inparticular,thereisgrowingcriticismthatthecurrentapproachtocost-benefitanalysisdoesnotcaptureallofthebenefitsthatmaybeassociatedwithtransportinvestment.
Ifthemainaimofnewtransportinfrastructureisfasterjourneys,thenbenefitstotheeconomymaterialisebecausetimesavedcanbeusedonproductiveorotherwisevaluableactivities(eitherinbusinessorleisure).Thisiswhythemostfundamentalinputintotransportinfrastructurecost–benefitsanalysishastraditionallybeenthesocalled‘valueoftraveltimesavings’.Thisistraveltimesaved,convertedintomonetaryunits.Thesemonetisedtimesavingsareacrucialmeasureoftheeconomicbenefitfromtransportinvestment(andcanbesupplementedbymonetisedestimatesofthebenefitsofreductionsinothercostslikeaccidentsandunreliability).
Butoverthepast15yearstherehasbeengreaterinterestinthepotentialfortransporttogenerate‘widereconomicbenefits’thatgobeyondthesetraveltimessavings(inadditiontoarangeofotherwidersocialandenvironmentalbenefits).InthecontextofUKappraisalaparticularfocus,intermsofwidereconomicbenefits,hasbeenonthosethatcomefromeffectivelybringingpeopleandbusinessesclosertogethertoformagglomerationsofeconomicactivity.Thelogicfollowsfromtheobservationthatcitiesaremoreproductivethanruralplacesandbigcitiesaremoreproductivethansmallercities.Solinkingplacestogethermayhelpgenerateproductivityimprovements.
Despitetheseimprovementstoappraisalpractice,therecontinuetobeconcernsthatappraisalmissesimportantbenefitsoftransportinvestment–particularlyintermsoftheimpactonlocaleconomicgrowth.SomeoftheseissueswererecentlyconsideredinanindependentreportfortheDepartmentforTransport(Laird,OvermanandVenables,2015).Thereportconcludesthat,insomecases,traditionalcost-benefitanalysismayindeedmissimportantbenefitsthatshouldbeincludedintheanalysis(althoughtheDepartment’sWebTAGguidanceincludesalmostallofthem).
Thereportalsoarguesthattheremaybeinstancesinwhichlocaldecisionmakersareinterestedinthelocaleconomiceffectsoftransport–e.g.onemploymentandinvestment–evenwhentheseshouldnotbeincludedinacost-benefitanalysiswhichseekstoevaluatetheoverall(i.e.national)gainsfromanewproject.Aconcreteexamplewouldoccurwhenemploymentgrowthneartonewtransportinvestmentispurelydrivenbydisplacementfromelsewhereintheeconomy.Atraditionalcost-benefitanalysis–whichtriestoassesstheoverallgainstosociety–wouldignoresuchdisplacement.Buttheseeffectsmaybeoflegitimateinteresttolocaldecisionmakers.
DfTisplanningtorefreshWebTAG(thesetofprocedureswhichoutlinehowappraisalsshouldbeconducted)torespondtotheseobservations.Whilesuchanexercisewillbewelcomedbymany,thefindingsinthisreviewalsohighlighttheimportanceofcomplementinganyfurtherworkontheex-anteappraisalframework(i.e.analysistopredictwhatmighthappen)withadditionalworktostrengthentheex-postevaluationoftransportinvestment(i.e.analysisofwhatactuallyhappened).Notleastbecause,asthisreportmakesclear,thereislimitedevidencethattheemployment(andothereffects)thatwouldunderpinanychangestotheguidanceactuallyoccurinpractice.
Whatformshouldsuchex-postevaluationtake?Asdiscussedabove,forcapitalexpenditure,whereinvestmentsaredurable,itishardtoimaginesituationsinwhichtruerandomisationofprojectplacementwouldbeeitherfeasibleordesirable.Thismeansthatweneedtorelyonalternativeevaluationapproaches
![Page 34: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/34.jpg)
Evidence Review: Transport - July 2015 34
totrytoidentifythecausalimpactoftransportinvestment.Onthebasisofourreviewwork,plusourwiderworkontheissueoftransportevaluationwethinkthatworktodevelopanewapproachisurgentlyneeded.Anysuchapproachneedstobebothfeasibleandproportional.Italsoneedstoproduceevidencethatishelpfulinimprovingfuturedecisionmaking.Unfortunately,manyexistingstudiesappeartohavecostmuchbutwitharguablylittlebenefitinimprovingdecisionmaking.Thereareanumberofpossibleavenuesthatcanbeexploredandissuesthatwillneedtobeconsidered.Forexample:
1. AtpresenttheHighwaysAgencyundertakesPostOpeningProjectEvaluation(POPE)ofalargenumberofschemes.Thistakestwoforms–one,lightertouch,forsmallerprojects(LocalNetworkManagementSchemes);andamoreextensiveevaluationforlargerprojects(MajorSchemes).POPEaimstodeterminehowschemeshaveperformedintheiropeningyearand,formajorprojects,fiveyearsafteropening.Findingsarecomparedtoex-anteappraisaltoassessaccuracyandanannualmeta-studypullstogetherfindingsfromallPOPEundertakenthatyear.POPEusesbeforeandafteranalysisofschemedata–anapproachwhichwouldscorelevel2ontheSMSscale.Morerecently,theDfThasissuedaMonitoringandEvaluationFrameworkforLocalAuthorityMajorSchemes(LAMS)thatprovidesguidancethatoutlinesaPOPEstyleapproachforthoseschemes.Thisguidanceoutlinesthreeapproaches:standardandenhancedmonitoringwhichparallelthesmallerprojectsapproachinPOPE;andfullerevaluationwhichisclosertothePOPEguidanceforMajorSchemes.AswithPOPE,thereisastrongemphasisonbeforeandaftercomparisons.57
2. Itwouldbehelpfultoconsiderhowtheuseofappropriatecontrolgroupscouldrefinetheseprocesses.Interestingly,themostrecentPOPEguidanceformajorschemeshasrecognisedtheimportanceofcontrollingforthebackgroundreductioninthenumberofcollisionswhenassessingbenefitsfromaccidentreduction.LAMSalsoplacesmoreemphasisontheuseofcontrolgroups–atleastforthefullerevaluations(see,forexample,thediscussionofcontrolgroupsintheassessmentofchangesintravelbehavior).Itwouldbeusefultoundertakefurtherworktoidentifyappropriatecontrolgroupsandtoencouragetheiruseforbenefitswhichmightbemostaffectedbyother‘background’changes.Controlgroupscouldbeconstructedinanumberofways:Forexamplea)forsimilarpartsofthenetworkthathavenotbeensubjecttoimprovement58b)fromschemesthatarelikelytobefundedinthefuturebuthavenotyetbeenfunded;c)fromschemesthathavesimilarbenefit-costratiosbutweredeclinedfunding;d)forareasclosetofundedschemesthatarenotdirectlyaffectedbythescheme.Moresimply,appropriateareawideaverages(whichwouldideallyexcludenewschemes)couldbeusedtoprovideaverybasiccontrolgroup.Similarapproachescouldbeusedtoidentifysuitablecontrolgroupswhenusingindividualleveldata(e.g.ontravelbehavior).Therewill,ofcourse,beprosandconstoalloftheseapproachesandfurtherworkwouldbeneededtoconsiderthealternatives(andwhetherthebenefitsintermsofimprovedPOPEandLAMSanalysis,outweightheadditionalcosts).
3. AlthoughPOPEconsidersperformanceofeachschemeagainsttheGovernment’sfourWebTAGobjectives(economy,environment,societyandpublicaccounts)theanalysisofeconomicimpactstendstofocusondirectbenefits–particularlyintheformofreducedaccidentsandimprovedjourneytimes.ThisisunsurprisinggiventhecurrentfocusofPOPEonperformance.Interestingly,whilethelightertouchapproachesinLAMSgivelessconsideration
57 AthirdsetofguidelinesconsiderthemonitoringandevaluationframeworkfortheLocalSustainableTransportFund.ManyofthepointswemakeherecouldalsoapplytodevelopmentoftheLSTFframeworkbutthetextfocusesonPOPEandLAMSwhichhaveclearerparallelsandastrongerimpactevaluationfocusthanLSTF.
58 Thisistheapproachissimilartothatusedforcollisions–atleastforkeylinks–whereadjustmentisbasedonnationaltrendsforthattypeofroad(althoughsomeproportionofthenetworkwillalsohavebenefitedfromimprovements).
![Page 35: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/35.jpg)
Evidence Review: Transport - July 2015 35
tosomeWebTAGobjectives59theyplacemoreemphasisoneconomicbenefits–particularlyintermsofeffectsonemploymentandrentalvalues.POPEformajorschemesdoesallowfortheassessmentofwidereconomicbenefitswiththeconsiderationgiventothesebenefitsvaryingaccordingtothelevelofPOPEthatisundertaken.Atitsmostextensive,thiswillinvolveasurveyoflocalbusinessatoneyearto‘identifyemergingconcernsorpositiveoutcomesassociatedwiththescheme’andatfiveyearsa‘focusedsurveyofbusinessestoidentifywidereconomicimpacts’.Giventheincreasedinterestinwidereconomiceffects,wethinkthatthisprocesscouldbeimprovedtobetteralignthePOPEandLAMSprocessestoensurethatbothcarefullyassesstheseeconomiceffects.60AsismadeclearinLAMS,notallschemeswouldwarrantsuchananalysis,butthisshouldbeconsideredwhenemployment,orotherlocaleconomyeffects,areanimportantcomponentofthestrategicoreconomiccaseformajorschemes.Resultsfromthisanalysisareunlikelytobeusefulinisolation.BothLAMSandPOPEhighlighttheimportanceofcomparingoutcomestokeyappraisalassumptions.61Butwewouldalsohighlighttheimportanceofbroadercomparisonstoboththestrategicandeconomiccasesthatformpartoftheappraisalprocess.
4. Onceagain,itwillbeimportanttoconsiderhowtheuseofsuitablecontrolgroupscouldplayapartintheanalysisoftheseeconomiceffects.AsLAMSrecognises,therearestrongargumentsinfavourofdevelopingsuchanapproach–atleastforlargerschemes.Someoftheoptionsforconstructingsuchcontrolgroupswerediscussedunderpoint(2).Itwouldalsobeusefultoconsiderwhetheralighttouchapproachcouldbedevelopedforsmallerschemes.
5. Thereneedstobeamuchcloserlinkbetweentheex-anteappraisalandex-postevaluationofschemes.Ourreviewoftheliteraturediscoveredalargenumberofex-postevaluationsthatappeartoliveinavacuum,62withnoattemptmadetolinkthefindingsfromthesereportsbacktoschemeappraisals.Higherqualityimpactevaluations–i.e.thosethatseektoidentifythecausalimpactofinvestmentsusingchangesinoutcomescomparedtoacontrolgroup(i.e.arescoredSMS3andaboveandincludedinourreview)–arestillhelpfulevenintheabsenceofsuchcomparisons.Thesearethestudiesthatwehaveusedinthisreview.Thisis,unfortunately,notsotrueforlessrobustevaluations(e.g.thoseinvolvingsimplebeforeandaftercomparisons).Embeddingevaluationintotheschemeprioritisationprocessisanimportantstepinensuringthatmoneyspentonex-postevaluationsiscost-effectiveinimprovingprioritisationforfuturespending.OneoftheadvantagesofincorporatingtheevaluationofwidereconomicimpactswithinanimprovedPOPEmethodologyisthatthesecomparisonsarealreadypartofthePOPE‘meta-analysis’process.Itwillbeimportanttodevelopasimilar‘meta-analysis’forLAMSthatparallelsthePOPEprocess.Thesecomparisonsacrossevaluationsshouldallowfindingsonschemeeffectsandthecomparisontoappraisalassumptionstobeusedtoimproveschemeprioritisation(forexample,throughtheuseofoptimismbiastoadjustpredictedemploymenteffects).GiventheinterestintheeconomicimpactsofinvestmentinothertransportmodesweshouldconsiderhowandwhenasimilarapproachcouldbeextendedtosuchschemesnotcoveredbyPOPEorLAMS.
6. InlinewiththerecommendationsoftheDfTTIEPreport,thereshouldbefargreaterattentionpaidtothecriticalanalysisofboththeeconomicandstrategiccasesforsupport.This
59 Forexample,inLAMSenvironmentisconsideredinenhancedmonitoring,butnotinstandardmonitoring.60 InPOPE,inparticular,thiswouldinvolvebringingtheapproachtowidereconomiceffectsinlinewiththoseusedtomore
carefullymeasurechangesinjourneytimes,etc.61 Forexample,POPEsystematicallycomparesthemonetisedvalueofchangesinaccidentsandjourneytimestotheex-
anteappraisalpredictions.62 Indeed,anumberofevaluationsarenot(easily)accessibleevenwhentheyhavebeenpublicallyfunded.Whileconcerns
overcommercialconfidentialitymaybeproblematicforsomeaspectsoftheex-postevaluation(asforappraisal)routinepublicationofpublicallyfundedevaluationsshould,arguably,bethenorm.
![Page 36: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/36.jpg)
Evidence Review: Transport - July 2015 36
criticalanalysisshouldoccurbothex-ante(onthebasisofavailableevidence–includingthatcoveredinthisreview)andex-post(onthebasisofappropriateevaluation–includinganalysisdevelopedaccordingtotherecommendationsabove).
7. Ex-postevaluationneedstoincludedueconsiderationoftheextenttowhichanyemploymenteffectsarelikelytoresultfromdisplacement(theshiftingofjobsfromoneplacetoanother).Addressingconcernsoverdisplacementwillneedtobeakeyquestioninunderstandingthenetimpactofinvestments.Therearesimilarconcernsoverspillovereffectswherebyemploymentgrowthresultingfromtransportimprovementsdrivesgrowthinareasnotdirectlyaffectedbythescheme.However,forboththesequestionscarefullyidentifyinganydirectemploymentimpactisafirststepinunderstandingtheseotherfactors.Concernsoverdisplacementandspilloversshouldnotpreventprogressinaskingthesimplerquestionastowhetheranychangesinemploymentoccurdirectlyasaresultofthescheme.Workisneededtotackleallthreequestions–whatistheemploymenteffect;areanylocalemploymentchangesadditional;dothesespillovertowiderareas?Asimilarpointholdswithregardtovariationsineffectsacrossschemes.Worktoidentifytheaverageeffectshouldbeafirststepinunderstandinghowvariationineffectsdependsoncontext.Again,concernsoverheterogeneityofeffectsshouldnotpreventprogressonthesimplerquestionofidentifyingaverageeffects.
8. ThecurrentLAMSguidance,includingacomparisontoappraisalassumptions,wouldappeartoprovideanappropriateframeworkforundertakingandimprovingevaluationandschemeprioritisationforindividualLAs/LEPs.However,thereisaroleforDfTinhelpingdeveloptheguidelinesforhowthisanalysiscouldbeconductedandimprovedalongthelinesofpoints(2)to(7).Thiswillensure,aswithPOPE,thatresultsforspecificLAs/LEPsaretransferableacrossareas.Thedevolutionagendaraisesquestionsabouttheextenttowhichsuchanapproachcould(orshould)bemandatory.Regardlessoftheoutcomeofthatdebate,manyLAs/LEPswouldstillwelcomeguidanceonhowbesttoproceed–especiallygivenlocalconstraintsonanalyticalcapacity.
9. ConsiderationneedstobegivenastohowtoensureLAs/LEPshavetheincentive(andtheresources)tocollectdataincontrol/comparisonareas.Itispossiblethatcentralgovernmentdepartmentscouldprovideappropriateareadata(andtheuseofsuchsecondarydatawouldsubstantiallyreducethecostimplicationsofundertakingevaluations).Wheredataiscollectedatthelocallevelitwillbeimportanttoensurethatsuchdataareavailabletoresearchersforuseinaggregated/multi-interventionanalysis.
10. Giventhecomplexityofmanyoftheissuesraisedabove,andtheneedforcomparisonacrossareas,itislikelythatDfTwillneedtoplayacoordinatingroleinaddressingmanyoftheseevaluationchallenges.ThereisalsoaroleforDfTinundertakingmulti-interventionex-postanalysisusingthekindofapproachesusedbythehigherqualitystudiesconsideredaspartofthisreport.
Furtherworkwouldbeneededtodeveloptheissuesdiscussedhereandtoconsiderappropriatesolutions.Itiscrucialthatfurtherworkrecognisestheimportanceofembeddingevaluationintotheschemeprioritisationprocess(DfTiscurrentlyundertakingworkonthisissue).Thismeansbetteraligningappraisalandevaluation,particularlyiftheobjectiveistoimproveschemeprioritisation.Withoutcloserintegration,thereisadangerthatweundertakerefinementstotheappraisalprocess–e.g.toincludeemploymentandinvestmenteffects–withoutknowingthelikelymagnitudeofeffects,whethertheyareadditional,etc.Similarly,whileex-postevaluationcanservesomeroleintermsofmonitoringandaccountability,itsmainaimshouldbeinimprovingfuturedecisionmaking.Thismeansthinkingaboutwaysinwhichevaluationcanfeedbackintotheschemeprioritisationprocess–bothintermsofdevelopingex-anteappraisal,butalsoinprovidingameansofscrutinisingstrategiccasesforfutureinvestment(again,anareainwhichDfTiscurrentlyundertakingwork).
![Page 37: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/37.jpg)
Evidence Review: Transport - July 2015 37
References
Aghion,P.,Besley,T.,Browne,J.,CaselliF.,Lambert,R.,LomaxR.,Pissarides,C.,Stern,N.,VanReenen,J.(2013)InvestingforProsperity,Skills,InfrastructureandInnovation,ReportoftheLSEGrowthCommissionLondon:LSEGrowthCommissionhttp://www.lse.ac.uk/researchAndExpertise/units/growthCommission/documents/pdf/LSEGC-Report.pdf
Baldwin,R.,Forslid,R.,Martin,P.,Ottaviano,G.,andRobert-Nicoud,F.(2005)EconomicGeographyandPublicPolicy,PrincetonUniversityPress
CityGrowthCommission(2014)UnleashingMetroGrowth,London:RSAhttp://www.citygrowthcommission.com/publication/final-report-unleashing-metro-growth
Cox,E.andDaviesB.(2013)StillontheWrongTrack:AnUpdatedAnalysisofTransportInfrastructureSpending,IPPRNorth
http://www.ippr.org/publications/still-on-the-wrong-track-an-updated-analysis-of-transport-infrastructure-spending
Eddington,R.(2006)TheEddingtonTransportStudy,Thecaseforaction:SirRodEddington’sadvicetoGovernment,London:HMSOhttp://www.thepep.org/ClearingHouse/docfiles/Eddington.Transport.Study%20-%20Rod.pdf
Gibbons,S.(2015)Planes,TrainsandAutomobiles:TheEconomicImpactofTransportInfrastructure.SERCPolicyPaper#13
Gibbons,S.,Nathan,M.,andOverman,H.G.(2014)Evaluatingspatialpolicies.TownPlanningReview,85(4).pp.427-432.
Jones,C.andVollrathD.(2013)IntroductiontoEconomicGrowthW.W.Norton&Co.;3editionSherman,L.W.,Gottfredson,D.C.,MacKenzie,D.L.,Eck,J.,Reuter,P.,&Bushway,S.D.(1998).PreventingCrime:WhatWorks,WhatDoesn’t,What’sPromising.WashingtonDC:USDepartmentofJustice.https://www.ncjrs.gov/pdffiles/171676.PDF
Solow,R.M.(1956)AContributiontotheTheoryofEconomicGrowth,TheQuarterlyJournalofEconomics,Vol.70,No.1.(Feb.,1956),pp.65-94
StatistischesBundesamt(2014)AllepolitischselbständigenGemeindenmitausgewähltenMerkmalenam31.12.2013.https://www.destatis.de/DE/ZahlenFakten/LaenderRegionen/Regionales/Gemeindeverzeichnis/Administrativ/Archiv/GVAuszugJ/31122013_Auszug_GV.html
![Page 38: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/38.jpg)
Evidence Review: Transport - July 2015 38
09
Appendix A: Findings by outcome
Table A1: Roads
Outcome +ve Zero -ve Mixed No. +ve
Employment 1011,1017, 1027,1031,1067
- 1015 2/6
PropertyValues/Rents 1052,1009,1035
- - 1050 3/4
BusinessVolume/Sales 1037,1067 - - - 2/2
Firmentryandnumberofbusinesses
1017,1061 1016 - - 2/3
Productivity 1017,1062 - - 2/2
Innovation 1063,1055 - - - 2/2
Income/Wages 1017 - - 1067 1/2
Population - - 1005 1015,1027 0/3
Table A2: Rail
Outcome Total evaluated
+ve Zero -ve Mixed No. +ve
EconomicPropertyValues
61070,1071,1107,1108,
11121074 - - 5/6
GDP 1 1075 - - - 1/1
BusinessProductivity 1 1116 - - - 1/1
Non-economicPopulation 1 1109 - - - 1/1
Crime-reduction 1 1114 - - - 1/1
Note:Inadditiontooutcomesreportedhere,Study1111showedevidenceofchangesinlanduseasdiscussedinthetext.
![Page 39: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/39.jpg)
Evidence Review: Transport - July 2015 39
Appendix B: Evidence Reviewed
Roads
Ref No. Reference1005 Baum-Snow,N.(2007).Didhighwayscausesuburbanization?TheQuarterlyJournalof
Economics,775-805.
1009 Boarnet,M.G.&Chalermpong,S.(2001)NewHighways,HousePricesandUrbanDevelopment:ACaseStudyofTollRoadsinOrangeCounty,CA.HousingPolicyDebate,12(3),575-605
1011 Chalermpong,S.(2002).EconomicSpilloversofHighwayInvestment:ACaseStudyoftheEmploymentImpactsofInterstate105inLosAngelesCounty.UniversityofCaliforniaTransportationCenter.
1015 Funderburg,R.G.,Nixon,H.,&Boarnet,M.G.(2010)NewHighwaysandLandUseChange:ResultsfromaQuasi-experimentalResearchDesign.TransportationResearch:PartA:PolicyandPractice,44(2),76-98
1016 Garcia-Mila,T.&Montalvo,J.G.(2013)ANewApproachtoMeasuretheImpactofHighwaysonBusinessLocationwithanApplicationtoSpain,”UniversitatPompeuFabra,BarcelonaGSEandIVIE.
1017 Gibbons,S.,Lyytikainen,T.,Overman,H.G.,&Sanchis-Guarner,R.(2012).Newroadinfrastructure:theeffectsonfirms.
1027 Iacono,M.,&Levinson,D.(2013).CausalityintheLinkBetweenRoadNetworkGrowthandRegionalDevelopment(No.000112)
1031 Jiwattanakulpaisarn,P.,Noland,R.B.,Graham,D.J.,&Polak,J.W.(2009).Highwayinfrastructureinvestmentandcountyemploymentgrowth:Adynamicpanelregressionanalysis*.JournalofRegionalScience,49(2),263-286.
1035 Chernobai,E.,Reibel,M.,&Carney,M.(2011).Nonlinearspatialandtemporaleffectsofhighwayconstructiononhouseprices.TheJournalofRealEstateFinanceandEconomics,42(3),348-370.
1037 Duranton,G.,Morrow,P.M.,&Turner,M.A.(2014).RoadsandTrade:EvidencefromtheUS.TheReviewofEconomicStudies,81(2),681-724.
1050 Vadali,S.(2008).Tollroadsandeconomicdevelopment:exploringeffectsonpropertyvalues.TheAnnalsofRegionalScience,42(3),591-620.
1052 Márk,L.(2013).TheEffectofHighwaysonNearbyResidentialPropertyPricesinHungary(Doctoraldissertation,CentralEuropeanUniversity).
1055 Crescenzi,R.,&Rodríguez-Pose,A.(2012).InfrastructureandregionalgrowthintheEuropeanUnion*.Papersinregionalscience,91(3),487-513.
1061 Holl,A.(2004).TransportInfrastructure,AgglomerationEconomies,andFirmBirth:EmpiricalEvidencefromPortugal*.JournalofRegionalScience,44(4),693-712.
1062 Montolio,D.,&Solé-Ollé,A.(2009).Roadinvestmentandregionalproductivitygrowth:theeffectsofvehicleintensityandcongestion*.PapersinRegionalScience,88(1),99-118.
1063 Agrawal,A.,Galasso,A.,&Oettl,A.(2014).RoadsandInnovation.RotmanSchoolofManagementWorkingPaper,(2478752).
1067 Michaels,G.(2008).Theeffectoftradeonthedemandforskill:Evidencefromtheinterstatehighwaysystem.TheReviewofEconomicsandStatistics,90(4),683-701.
![Page 40: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/40.jpg)
Evidence Review: Transport - July 2015 40
Rail
Ref No. Reference1070 Chatman,D.G.,Tulach,N.K.,&Kim,K.(2012).Evaluatingtheeconomicimpactsoflightrail
bymeasuringhomeappreciationafirstlookatNewJersey’sRiverLine.Urbanstudies,49(3),467-487.
1071 Kim,K.,&Lahr,M.L.(2014).TheimpactofHudson-BergenLightRailonresidentialpropertyappreciation.PapersinRegionalScience,93(S1),S79-S97.
1074 Ahlfeldt,G.M.(2011).Thetrainhasleftthestation:domarketsvalueintracityaccesstointercityrailconnections?.Germaneconomicreview,12(3),312-335.
1075 Ahlfeldt,G.M.,&Feddersen,A.(2010).Fromperipherytocore:economicadjustmentstohighspeedrail.DocumentsdetreballIEB,(38),1.
1083 Gatzlaff,D.H.,&Smith,M.T.(1993).TheimpactoftheMiamiMetrorailonthevalueofresidencesnearstationlocations.LandEconomics,54-66.
1107 Koster,H.R.A.,Ommeren,J.N.&Rietveld,P.(2010).Estimatingthebenefitsofimprovedrailaccess;geographicalrangeandanticipationeffects.TinbergenInstituteDiscussionPaper(TI2010-094/3).
1108 Billings,S.B.(2011).Estimatingthevalueofanewtransitoption.RegionalScienceandUrbanEconomics,41(6),525-536.
1109 Enflo,K.,Berger,T.(2013)LocomotivesofLocalGrowth:TheShortandLong-TermimpactofRailroadsinSweden.EHESWorkingPapersinEconomicHistory.No.42.
1111 Hurst,N.B.,&West,S.E.(2014).Publictransitandurbanredevelopment:TheeffectoflightrailtransitonlanduseinMinneapolis,Minnesota.RegionalScienceandUrbanEconomics,46,57-72.
1112 Ahlfeldt,G.M.(2011).Ifwebuild,willtheypay?:predictingpropertypriceeffectsoftransportinnovations.
1114 Billings,S.B.,Leland,S.,&Swindell,D.(2011).Theeffectsoftheannouncementandopeningoflightrailtransitstationsonneighborhoodcrime.JournalofUrbanAffairs,33(5),549-566.
1116 Bernard,A.B.,Moxnes,A.,&Saito,Y.U.(2014).ProductionNetworks,GeographyandFirmPerformance.TuckSchoolofBusinessWorkingPaper.
Findthefulllistofsearchtermsweusedtosearchforevaluationsonourwebsitehere:whatworksgrowth.org/policies/transport/search-terms.
![Page 41: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/41.jpg)
The What Works Centre for Local Economic Growth is a collaboration between the London School of Economics and Political Science (LSE), Centre for Cities and Arup.
www.whatworksgrowth.org
![Page 42: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence](https://reader035.vdocuments.net/reader035/viewer/2022071004/5fc1c50c5dd07b65531a4e8e/html5/thumbnails/42.jpg)
ThisworkispublishedbytheWhatWorksCentreforLocalEconomicGrowth,whichisfundedbyagrantfromtheEconomicandSocialResearchCouncil,theDepartmentforBusiness,InnovationandSkillsandtheDepartmentofCommunitiesandLocalGovernment.ThesupportoftheFundersisacknowledged.TheviewsexpressedarethoseoftheCentreanddonotrepresenttheviewsoftheFunders.
Everyefforthasbeenmadetoensuretheaccuracyofthereport,butnolegalresponsibilityisacceptedforanyerrorsomissionsormisleadingstatements.
Thereportincludesreferencetoresearchandpublicationsofthirdparties;thewhatworkscentreisnotresponsiblefor,andcannotguaranteetheaccuracyof,thosethirdpartymaterialsoranyrelatedmaterial.
Photocredit:“AdelaideTracks”byLesHaines
July2015
WhatWorksCentreforLocalEconomicGrowth
[email protected]@whatworksgrowth
www.whatworksgrowth.org
©WhatWorksCentreforLocalEconomicGrowth2015