1

MACROECONOMIC MODELLING ANDSIMULATION APPROACHES

1 . Methodological background

The evaluation of S&T policies has generally been focused on technologicalobjectives, e.g. additionality of public subsidies on private R&D, leaving out the crucialissue of the socio-economic impacts of implemented policies. However, welfareimprovements, which are often concretised through socio-economic impacts, can beexpected to be the ultimate goal of S&T policies.

Macroeconometric models based on sound economic theory principles areuseful to assess the effects of R&D and technology on economic performance. Thismethodology looks at the impact of using different policy instruments on the relevantvariables of the model. In general, the impact is measured in terms of changes ofrelevant variables against a reference base scenario.

The use of simulations is justified due to the non-linearities, complexities andfeedback mechanisms linking R&D, productivity and the economy. Recent studies havebegan to develop workable models applying the type of concepts related to knowledgegeneration and diffusion developed in new growth theory and in evolutionaryeconomics. These models can provide useful insights for understanding how theinnovation process takes place and increase the efficiency under which policy decisionsare taken.

Employment generation is an objective to all areas of public intervention.Several quantitative methodologies have been applied to evaluate the impact of policyinstruments on employment, including shift-share, multiplier effects, input-outputanalysis, social accounting matrices and macroeconomic simulation (see CEC, 1999).

The measure of the impact of S&T policy based on reduced forms, as it is thecase for the analysis of the impact of R&D expenditures on total factor productivity,only gives a limited view of the linkages between S&T variables and macroeconomics.There is a need to clarify the chain of causal effects between technological change andthe main macroeconomic variables such as productivity, production, employment,investment, profitability, and exports.

2

2. General description

In reviewing the methodologies used for modelling the impact of technologicalchange on economic performance two broad types of models can be distinguished (seeBradley and Whelan, 1992): Neo-Keynesian Macroeconomic Models (e.g.HERMES/HERMIT and INTERLINK) and Computable General Equilibrium models(e.g. IMF Multimod model, European Commission Quest Model). The former type ofmodel consist in disequilibrium models where in the absence of well defined marketclearing conditions, long run properties are difficult to rationalise and interpret. Thelatter approach is built from microeconomic foundations which determine the fulldynamics of the model. In this respect, O'Sullivan and Röeger (1992) provide an earlyattempt to analyse the cross-country diffusion of R&D1, however one of the limitationsof this model is R&D spending is exogenous. Giorno, Richardson et al. (1995) use theOECD INTERLINK model to examine under different scenarios the consequencesand sensitivity of the macroeconomic model to changes in trend factor productivitygrowth. Their results support that a rise in trend factor productivity increases the level ofproduction and real income. The level of unemployment could initially increase, but thedegree of labour market flexibility will undoubtedly affect the adjustment process.

The empirical evidence at the macro level supports the large contribution ofR&D to productivity and economic growth complementing the microeconomicevidence on the positive contribution of R&D to economic performance (large payoffsto society, lower returns to the innovator, relevance of commercialisation of research).The estimated private rate of return to R&D is 20-30% on average. The estimated socialrates of return to private R&D are even larger varying from 70-100%.

Macroeconomic modelling and simulation exercises signal the non-linearities,complexity, and feedback mechanisms characterising R&D and innovation process. Theexistence of labour supply rigidities indicate the relevance of integrating the functioningof the labour markets in R&D models. Simulation exercises permit to study andmeasure the impact of international spillovers in RTD.

The returns to large investments in ICT begin to be observed in countries at thetechnological frontier specially the US. The productivity increase tends to beconcentrated and not equally spread across particular industries. In particular, theincrease in productivity has concentrated in the durable goods manufacturing sector.

Direct government funding and tax incentives have a positive impact on privateR&D. The stimulation of government funding is non-linear. There are substitutioneffects between direct government funding and R&D tax incentives. The stability ofpolicy tools over time is important. Public R&D support influence TFP. Subsidies toprivate R&D have a positive effect on targeted firms, but do not reduce market failure.Evidence of under-investment in R&D in US (optimal level of R&D investment fourtimes larger than current investment).

1 The simulations are performed using the European Commission macroeconomic model QUEST.

3

Knowledge spillovers are geographically localised and are broader betweenregions with similar or complementary technology specialisation patterns. The measuredsocial rates of return to R&D differ for different levels of geographic aggregation.Important technology barriers remain in Europe, e.g. country borders significantlyhinder knowledge spillovers. There are strong positive impacts of university research onprivate R&D and patents. The impact of the R&D performed by multinationalcorporations is important for the development of local markets. Differences acrossregions in learning capacity and the exogenous rate of growth of the knowledge stockpersist over time.

Accessibility of industrial R&D laboratories to university research increases theimpact of national level innovative activities. Innovation and a potential fortechnological diffusion contribute to regional growth. The efficiency of innovationefforts is larger in advanced regions. Business services and related industry presencefacilitate information spillovers, which regionally lower the costs of developing newinnovations. Income level of Southern EU regions is largely determined byemployment/educational levels and past public investments.

3. Policy instruments and interventions to evaluate with the method

As a matter of fact, each policy, even if it is not directly targeted towards S&Tdevelopment, can have strong implications on the innovative activities of economicagents. Figure 1 illustrates the scheme generally in use for the evaluation of public actionin the field of economic policy.

As a whole, the goal of government action in public life is to upgrade socialwelfare. Its achievement refers to the satisfaction of needs defined as constituting thequality of live. The development of S&T is one of the means that allows the society toimprove its social welfare. Yet, it is now recognised that the broader concept ofknowledge is more appropriate to understand the innovation process. It is the efficiencyand the effectiveness in producing, diffusing and exploiting useful knowledge, whichallows improving the well being.

With reference to that, generating, acquiring and diffusing knowledge can beidentified as the main objectives of government action. The creative, transfer andabsorptive capacities have been viewed as the main characteristics which allow one toappreciate the efficiency in the production and exploitation of technology flows at thesource of knowledge accumulation. These technology flows are themselves influencedby the characteristics of knowledge (its degrees of codification and disclosure, itsownership status…). Yet, they are not invariable but depend on the social organisationand the incentive structure of existing institutions and depend on institutionaladjustments. The process according to which the objectives can be achieved isdetermined by the policy mix implemented.

The observation of market failures gives the economic foundation of technologypolicy. However, the market failure approach does not provide a sufficient foundation

4

for implementing strategic technology policy. It is the comparative effectiveness betweengovernment intervention and market mechanisms which must guide policy choices.

GOALS: Improving living standards

OBJECTIVES: Improving innovative effectiveness

TARGETS : S&T priorities & orientations Designing the new policy

Adaptation of the policy

INSTRUMENTS: public S&T policies Choice of instruments

INTERVENTIONS: implementation of the policy

exogenous impulses

SOCIOECONOMIC SYSTEM MODELING: EVALUATION PROCESS: - macroeconometric models Ex ante: appreciating policy - general equilibrium models alternatives

In itinere : needs of policy adjustments

Ex post: measuring the realimpact

Figure 1. Impact Assessment of S&T Policy

This implementation has to be based upon a diagnosis of the socio-economicsituation and the technological position of the country in order to define on whichcomponents of the innovation system the policy stress will be put. On the basis of theseelements, private as well as public policy-makers identify targets, which define thetechnology orientations and priorities through which technology flows should mainlyoperate and new knowledge stimulated.

The targeting of public intervention leads them to implement instruments whichare mainly concretised through the institutional device aimed at improving theeffectiveness of the knowledge-producing mechanisms. Yet, all the instruments are notsystematically under the control of public authorities. Put differently, the effectiveness ofpublic actions can be expected to be influenced by the degree of control of instrumentsby governments. So, it is useful to distinguish between control instruments which areunder the command of governments (e.g. education budget) and influencing instrumentswhich depend on the relative attractiveness of government actions for the targetedeconomic agents (e.g. firms’ reactions to R&D incentives). Governments, in fact, selectthe instruments and mix them according to the targeted policy.

5

The intervention is at the core of the instrumentation of policy mix and bears on thequantitative values given to instruments. The intervention is the channel through whichinstruments are made operational. Interventions as well as instruments are notindependent and vertical devices but are intertwined by a game of hierarchical and/orcausal relationships and interdependencies. It is the mix of instruments which areinvolved and their interactions which shape the whole process of economic and socialgrowth, including technological change.

The choice of instruments will lead to the design of quantitative and qualitativeinterventions. The implementation of the policy-making mix so obtained leads to thetechnology as well as the economic performance which is a result obtained as aconsequence of the socioeconomic mechanisms. The latter are approached bysocioeconomic system modelling. So far two methods can be used: macroeconometricmodels which are based on a set of econometrically estimated structural equations andcomputational general equilibrium models which are a mix of the input-output modelsand the Walrasian approach.

The effectiveness of the S&T policy can be measured by the improvements observedin the economic effectiveness, the social equity and environmental sustainability. Theperformance observed leads decision-makers to adjust their policy, i.e. the instruments,in order to take into account the new socio-economic environment and improve theeffectiveness of the S&T policy. These adjustments can be viewed as the fine-tuning ofthe S&T policy.

In fact, we face a complex problem because there are strong interdependencies andinteractions within each category of instrument. A main question of S&T policy is torealise the fine-tuning between the instruments available: How to allocate R&Dresources among the different types of research? What is the most appropriate policy-mix to promote an efficient distribution of knowledge?…

4. Good practices examples

David and Hall (2000) construct a structural model to study the impact of publicR&D spending on private R&D expenditures. The model permits to identify the shortrun and long run impact effects between public and private R&D observed in theregression analyses.

Bayoumi, Coe et al. (1999) use an aggregated structural model2 of endogenousgrowth to simulate the effects of spillovers on productivity growth and consumption. Inthese model R&D expenditures, R&D spillovers and trade endogenously determineTFP. In their simulation exercise an increase of R&D investment in US of 0.5% ofGDP raises real output in the long run by more than 9%. At the same time domesticR&D spending generates significant spillovers to output in other countries. An increase

2 They use a modified version of MULTIMOD, the model used for making the projections of the World EconomicOutlook publication of the IMF.

6

of in R&D spending of 0.5% of GDP, the increase in output in terms of US GDP is50% larger than the case in which only US R&D spending augments.

Eaton and Kortum (1997) perform a simulation exercise using a multicountry modelof international technology diffusion to study productivity growth differences inmanufacturing. The state variables used are the productivity levels the national andforeign pool of ideas that countries have to adopt.

Jones and Williams (1999) propose an endogenous growth model incorporating fourimportant distortions to R&D to investigate whether a decentralized economyundertakes too little or too much R&D. The distortions are the surplus appropriabilityproblem, knowledge spillovers, creative destruction, and congestion externalities. Arobust result achieved after model calibration is that the decentralized economy generallyunderinvests in R&D relative to what is socially optimal. The only exceptions to thisconclusions occur when both the congestion externality is extremely strong and theequilibrium real interest rate is very high.

Several studies (Goolsbee, 1998; David and Hall, 2000) have criticised the R&D andtechnology literature for concentrating on R&D spending assuming the private R&Dperforming sector is price and wage-taking. Public support can increase the average andmarginal cost of private R&D performance by driving up the prices of R&D inputs.Therefore studying the responses of increases in public support to R&D on theflexibility of the scientific labour supply constitutes a highly relevant issue for policymaking.

4.1. S&T Policies, R&D, and Economic Growth (Guellec and van Pottelsberghe)

Guellec and van Pottelsberghe (1999, 2000, 2001) estimate the contribution ofvarious sources of knowledge (R&D capital stocks performed by the business sector, byforeign firms, and by public institutions) to productivity growth as well as thedeterminants of privately funded and performed R&D.

Goal variable(s)

Private R&D and productivity growth.

Econometric model

The contribution of various sources of knowledge to productivity performance isquantified by means of a Cobb-Douglas production function. The dependent variable isthe multifactor productivity growth (MFP) of the industrial sector (computed under thehypotheses of perfect competition and constant returns to scale).

[ ] GUhegovfrrp GUSRHEGOVSFRSRPexpMFP it2it1it1itittiitσσβ

−β

−β− ⋅⋅⋅⋅µ+ϕ+φ=

where: SRP is the stock of business performed R&D;SFR is the stock of foreign business performed R&D;

7

SRHEGOV is the stock of publicly performed R&D;i refers to the country and t to the period.

Country dummies, time dummies, employment rate (U, controlling for business cycleeffects), and a dummy for the German unification (G) are included as control variables.

In order to assess the determinants of privately funded and performed R&D, a R&Dinvestment model that considers business-funded R&D (RP) as a function of output(proxied by value added, VA) and several policy instruments, i.e. government funding ofR&D performed by business (RG), tax incentives, government intramural expenditureon R&D (GOV), research performed by universities (or higher education, HE), timedummies, and country-specific fixed effects is estimated.

t,it1t,iHE1t,iGOV1t,iB1t,iRGt,iVA1t,it,i eHEGOVBRGVARPRP +τ+∆β+∆β+∆β+∆β+∆β+∆λ=∆ −−−−−

Data base

The data consist of a panel of 16 and 17 OECD Member countries over the period1980-1998.

Econometric method

Both equations are estimated through an error correction model that allows toseparate short term and long term effect of the right-hand side variable. Theeconometric method was a three stages instrumental variable least squares that takes intoaccount the presence of the lagged dependent variable among the explanatory variablesand corrects for contemporaneous correlation of the error term.

Main findings

• Public R&D and productivity: public R&D includes R&D performed both ingovernment laboratories and in universities. The elasticity of government anduniversity performed research on productivity is 0.17. This tends to show thatoverall public R&D is very valuable to the economy. The effect of public R&D onproductivity is also larger in countries where the share of universities (as opposed togovernment laboratories) in public research is higher.

• Public funding of business R&D investments: the first policy instrument aiming atstimulating business R&D is direct financial support of research performed by thebusiness sector. These subsidies are targeted to specific goals chosen by the fundere.g. “generic technologies”, “pre-competitive research”, health, defence.Government-funded R&D has a positive and significant effect on business R&D asthe long term elasticity is 0.08. An alternative specification of the equation allows toapproximate the average optimal subsidisation rate of business R&D. The resultssuggests that the effectiveness of government funding increases up to a particularthreshold and decreases after that, which can be represented by an inverted U-shaped curve.

8

• Fiscal incentives and private R&D investments: Government can also help firmsthrough tax breaks. Most OECD countries allow for a full write-off of current R&Dexpenditures, which implies that depreciation allowances are deducted from taxableincome. The long term elasticity of business R&D with respect to tax breaks isnegative (-0.33). The estimates also suggest that the effect of tax breaks is quickerthan the effect of government funding, as business spending reacts immediately to achange in taxes.

• Public research and business R&D investments: government and university researchhave both a negative and significant impact on business funded R&D. Long termelasticities are respectively –0.08 and –0.05. Moreover, this negative impact is spreadover several years (although there is no contemporaneous impact). The crowding-out effect – which is due either to an induced increase in the cost of R&D or todirect displacement – appears to dominate the stimulating effect. Public laboratoriesare supposed to meet public goals, however, not those of business; spillovers mayoccur but they are not instantaneous and are not the primary goal. The negativeimpact of university research on business funded R&D may also point to thedifficulties in transferring basic knowledge to firms.

• Interaction between the various policy tools: the estimates show that governmentfunding of business R&D is substitute to fiscal incentives, complementary touniversity research, and does not interact with government research. In other words,increasing the direct funding (tax incentives) of business research reduces thestimulating effect of tax incentives (direct government funding). In addition,increased government funding of business research appears to reduce the negativeeffect of university research on business funding, possibly because governmentfunding helps firms to absorb knowledge from universities that may be poorly used.

• Defence oriented public support to business R&D: Defence technologies are lesslikely to be characterised by spillovers, as they are often specific, with little emphasison cost but primarily on extreme performance in extreme conditions. Secrecyconstraints may also imply that the results will only diffuse slowly to civilianapplications. Furthermore, because defence contracting is attractive - it generateshigh rewards at low risk - firms might allocate resources that would otherwise havebeen used for civilian research. The estimates show that the higher the share ofdefence, the lower the positive effect of government funding on business R&D.

4.2. Macro-Economic Evaluation of the Effects of Community Structural Funds(CSF) Interventions with QUEST II (Röeger, 1996)

Goal variable(s)

This model analyses the short, medium and long run macroeconomic effects bothon the demnd and the supply sides of CSF on key macroeconomic variables such asGDP and its components, employment, real wages as well as government deficit andpublic debt. Since prices, interest rates, exchange rates and wages are allowed to respondto the CSF induced public investment in our analysis, the simulation results allow toshed some light on the question to what extent public investment adds to total GDP,

9

rather than at least partly displacing investment activities of the private sector. Themodel can therefore be used to look at the major determinants of displacement effectson a macroeconomic level.

Econometric model

The paper traces the macroeconomic impact by using the respective countrymodules of DG IIs macro econometric model QUEST II. The current version ofQUEST bases behavioural relationships on principles of dynamic optimising behaviourof households and firms. Since the model has a supply block based on a neoclassicalproduction function, it is possible to model explicitly the supply side effects ofinfrastructure and human capital investments. The model is also closed with respect tostock flow interactions. Those stock variables which are identifiable on amacroeconomic level, such as physical capital, net foreign assets, money and thegovernment debt, are endogenously determined and wealth effects are allowed toinfluence savings, production and investment decisions of households, firms andgovernments. Moreover trade and financial linkages of each country to the rest of theworld are explicitly modelled as well, which allows for an endogenous determination ofinterest and exchange rates. The labour market is based on a standard bargaining model.

Data base

The analysis is based on payments data for Greece, Ireland, Portugal and Spain overthe period 1989-93 as well as data on planned CSF spending for the same countries overthe period 1994-99.

Econometric method

Wthe model solution method, which solves a forward looking model with rationalexpectations is based on a linearisation of the model around the steady state and appliesclosed form solution algorithms to the linearised model3.

4.3. Dynamic Input-Output Model to Evaluate the Economic Impacts of the CSF(Beutel, 1996)

This input-output model has been developed for the Directorate-General forRegional Policies and Cohesion to evaluate the economic impacts of the CSF.

Goal variable(s)

Evaluation of the economic impacts of the CSF interventions on economic growth,structural change, foreign trade and employment and induced changes in in technology,imports, labour and capital use.

Econometric model

3 The model is estimated using the TROLL software. See Roeger and Veld (1997) for more details as well as thesimulations methods.

10

In the previous studies for the periods 1989-1993 and 1994-1999 the main issue wasto identify the short-term supply and demand effects of the Community SupportFrameworks for the objective 1 regions. The impact analysis system was designed as acomparative static input-output model to assess the quantitative impacts of theStructural Funds on economic growth, structural change, foreign trade and employment.Based of these former studies, a dynamic input-output model was developed which iscapable to evaluate the long-term supply and demand effects of the Communitystructural policies. Expenditures of the Structural Funds will affect the structure andlevel of final demand but will also induce changes in technology, imports, labour andcapital use. In particular the long-term effects on capital and labour, output andproductivity are in the focus of interest and will be covered by the dynamic input-outputapproach.

Data base

A set of harmonised input-output tables with labour and capital stock data is usedwhich has been established by Eurostat in cooperation with the author. The projectedinput-output tables are based on harmonised National Accounts of Eurostat and thelatest official economic forecasts of the Directorate General for Economic and FinancialAffairs.

Econometric method

The dynamic input-output model is designed in line with the macroeconomicmultiplier-accelerator theory. According to this theory it is expected that new capacitiesare required if final demand components are growing. Therefore, induced investment isestimated which can be related to the activities of the Structural Funds. The first part ofthe model estimates how an increase of gross fixed capital formation will affect theeconomy which is financed by the Structural Funds to improve the infrastructure ofpublic and private institutions. The second part analyses how the contributions ofCommunity interventions affect value added. A dynamic version of the input-outputmodel is used with the third element (induced investment) of the impact analysis systemto evaluate the long-term supply effects of the CSF.

Main findings

In the previous study the impact of Structural Funds expenditure was analysed forindividual years assuming that the Funds were still active in the previous year. The short-term impact of the Structural Funds activities revealed that the growth potential of theeconomy would be substantially reduced in individual years if the Structural Funds werenot in existence. The dynamic version of the model estimates the impact for a sequenceof years and consequently the supply effects are more profound. The results of thedynamic input-output model reflect a different growth path of the economy whichwould be realised in the absence of the Structural Funds.

So far, the dynamic impact analysis was conducted for four countries EUR4(Greece, Spain, Ireland, Portugal) on a national scale and for two countries (EastGermany, Mezzogiorno) on a regional basis. For 1994-1999 it is expected that in EUR4

11

the gross domestic product will grow at an average annual rate of 3.2 %. Withoutinterventions of the CSF the annual GDP growth rate would be reduced to 2.5 %. Thelabour force is projected to grow at an annual rate of 1.4 % versus 0.6 % withoutCommunity interventions. The capital stock is expected to grow in the same period at anaverage annual rate of 3.7 % versus 2.7 % without Community interventions. It isestimated that in 1999 approximately 35 percent of Community interventions is leakinginto the rest of the Union and another 10 percent into the rest of the world.

4.4. “An empirical model for endogenous technology in the Netherlands” (denButter and Wollmer 1996, the Netherlands)

The authors develop an empirical simulation model for the Dutch production sectorwhich is inspired by modern endogenous growth theory. The model is used to simulatevarious technological impulses. The parameter of the model are determined byconsidering empirical results from the literature as well own estimates and by calibratingthe model over the reference period 1972-1987.

Goal variable(s)

Economic growth, long term industrial output.

Econometric model

At the core of the model is a production block of nested CES functions, whereinvestments in technology capital and in human capital play a major role. The externaleffects of R&D are modelled in such a way that R&D investments not only lead to moretechnology capital, but also have a positive impact on human capital through learning bydoing and learning by designing. Technology capital enters into the production block intwo related way: firms accumulate knowledge by either undertaking R&D or importingknowledge. Technology capital is then assumed to augment the human capital ofworkers as they work with new technology. Human capital is assumed to be a substitutefor raw labour. Raw capital and technology capital, considered as complementary arecombined to produce efficient units of physical capital. The demand for domestic andimported R&D is positively related to output. The production block is extended byadding output demand and monetary equations.

Data base

Macroeconomic aggregates.

Results of policy evaluation

The simulations show the importance of incorporating elements of new growththeory into macroeconomic policy models. An impulse in R&D investments leads tohigher labour productivity and consequently increase the long term demand for allinputs, except labour, and increases final output. In order to avoid negative employmenteffects, any public policy of enhancing economic growth through impulses to privateR&D should be accompanied by a strong appeal to the social partners not to translatethe rise in labour productivity fully into wage demands.

12

4.5. “Endogenizing technological progress: The MESEMET Model” (vanBergeijk, van Hagen and al. 1997, The Netherlands)

The model aims at illustrating the macroeconomic consequences of technologypolicies in different institutional settings. Technological progress and knowledgeformation are endogenised in the model. Results are presented for three simulations:public R&D expenditures, public education expenditures and tax allowances for privateR&D. The paper puts forward that endogenising technological progress in anempirically relevant context turns out to be important.

Goal variable(s)

Macroeconomic aggregates.

Econometric model

The model is a macroeconomic semi equilibrium one that attempts to bridge the gapbetween applied general equilibrium models with a microeconomic foundation and themacroeconometric models typically applied in Dutch policy-making. The productionstructure contains various nested constant elasticity of substitution functionsrepresenting the relationships between the inputs of a representative firm and thecorresponding output. Human capital is considered to be a public good so thatindividual firms as well as households have no incentive to invest in it. Neverthelessinvestment in physical capital exerts a positive external learning-by-doing effect on thestock of total capital. Both private and public R&D expenditures are considered to havea similar effect. Conversely, the taxation is expected to have a negative effect on thestock. Private R&D is assumed to be a continuous variable input. Both the relativestocks of human capital and technology capital exert a positive effect on exports.

Data base

The model is parameterized for the Dutch economy in 1992.

Results of policy evaluation

Tax allowances for private R&D expenditures and public expenditures on botheducation and R&D are effective instruments to stimulate economic growth through theaccumulation of knowledge. Technology policies have a positive impact on overallemployment. The spillover effects from R&D on human capital seem to be crucial forthe economic consequences of public R&D and tax-free allowances on private R&D.Furthermore, the degree of complementarity between physical and technology capital onthe one hand and human capital on the other hand is important for the degree in whichpublic expenditures crowd out private investments in physical and technology capital.

4.6. The Hainaut economic-lead-in model (DULBEA-CERT 1998 and 2001,Belgium)

In the framework of Objective 1 programmes, the Belgian Hainaut has benefited ofthe Community intervention over the period 1994-1999. For the programming period

13

2000-2006, it will continue to benefit of structural interventions. In order to evaluate theimpact of the policy intervention, a small-scale macroeconometric model has beenimplemented. In 1993, some part of the model was initially used to evaluate the expectedimpact of public funds (ex ante evaluation exercise). The results of the model have alsobeen used to appreciate the expected impact over the period 2000-2006.

Goal variable(s)

Value added, employment, investment, labour productivity, R&D.

Econometric model

The structure of the model is based on the Kaldorian theory according to which themanufacturing industry is the leading sector of economic growth. The dynamics oftechnical change in the manufacturing industry is a main factor of the developmentprocess thanks to the productivity gains that can be accumulated. Hainaut being an oldindustrial region, the approach is well-suited to its economic situation. The provincebeing a small open economy, these productivity gains explain to a large extent itscompetitiveness. The trade balance of the region is mainly composed withmanufacturing products. It is the increases in competitiveness that determine the growthpotential inside the region thanks to direct or indirect impulse to other economicactivities.

The equations of the model dealing with the public support to R&D are specified asfollows:

• Business R&D

log RDPRH(t) = α + β*log SRDPUH(t-4) + γ*log VMANH(t-1) + δ*logRDPRH(t-1) - φ*∆{1/AIDRD(t)}

• Manufacturing employment

∆ log EMANH(t) = α + β*∆log SFORMH(t) + γ*∆log SRDPRH(t-3) - δ*∆logCTRA(t) + φ*∆log VEXH(t) + ε*log IAIDR(t)+ η*TX(t)

• Manufacturing production

log PRODH(t) = α + β*log SRDPRH(t-1) + γ*log INTCAPH(t) + δ*∆logDUC(t) + φ*log SADMH(t-1) + ε*log TX(t)

• External demand

log LIVEXTH(t) = α + β*log PRODH(t) +γ*log SMANH(t) + δ*log {IRDH(t-3)/IRDE(t-3)} + φ*log ECU(t-1) + ε*log DEFLEN(t)

where: RDPRH = private R&D expendituresSRDPRH = private R&D stockSRDPUH = public R&D stock

14

VMANH = manufacturing value addedAIDRD = R&D subsidiesEMANH = manufacturing employmentSFORMH = human capital stockCTRA = labour costVEXH = mining value addedIAIDR = investment subsidiesTX= investment subsidies/subsidised investmentPRODH = labor productivityINTCAPH = capital intensity per manufacturing workerDUC = degree of use of Belgian production capacitySADMH = public physical capital stockLIVEXTH = exportsSMANH = manunfacturing physical capital stockIRDH = Hainaut R&D intensityIRDE = European R&D intensityECU = value of the Belgian franc in ECUDEFLEN = deflator of the energy value added

Data base

Macrosectoral annual data were collected for the period 1964-1993. The six sectorscovered by the model are respectively: agriculture, manufacturing, energy, construction,market services and non-market services. Four groups of variables were considered: totalinvestment, employment, production and demand. It is made use of the concept of totalcapital, which leads to make a distinction between physical capital (with a distinctionbetween agriculture, tourism, government and other private expenditures) human capital(based upon the “weighted” number of persons following a formation cycle in thehigher education system as well as in the on-the-job training system) and knowledgecapital (based on a distinction between private and public R&D). The model iscomposed with 50 variables, 28 econometric equations and 12 definition equations.

Results of policy evaluation

The simulation results put forward that the impact of interventions will be lagged(first significant impacts can only be expected in 1999) and that it cannot be expectedthe intervention will be enough to reverse the divergence trajectory of the province.Regarding the impact of public support to R&D on the growth process, it has a highlong term effect on output, employment and physical investment. Nevertheless,investment in human capital appears to be crucial for the recovery process of the region,they are characterised by indirect important positive effects on physical investment andemployment. Conversely to investment in R&D and human capital which have longterm effects on the growth process, physical investment subsidies have only a temporaryeffect on economy growth.

15

5. Conditions for methodology application

To apply satisfactorily the macroeconometric methodology, it is worth keepingin mind that this approach is mainly adapted to measure the global expected economicand social impacts of a programme or a set of programmes. If its main advantage is togive an evaluation of direct, indirect and induced effects of a policy in a structured andsystematic way based on the economy theory, it is not suited to appreciate the effects ofsmall scale programmes, to estimate the benefit of programmes whose expectedeconomic outcomes are only very marginal or to select projects. As this approach allowstaking into account a large variety of effects, it requires:

• the availability of a large scale socioeconomic dataset;

• a high degree of expertise;

• enough time to build the model prior to any evaluation exercise;

• the implementation of policies which do not lead to a high dilution of economiceffects.

Regarding the first constraint, the data requirement bears not only on data linkedto the public policy but also on the quantitative information necessary to model with asufficient degree of reliability the functioning of the socioeconomic system. It is a littlebit trivial to say that the power and the potential of a large scale econometric model willbe higher than what can be evaluated from a small scale econometric one. Given thehigh complexity of socioeconomic systems, not any econometric model can be used toevaluate the impacts of a public policy. Generally, econometric models are calibrated toassist decision makers to deal with specific economic questions as monetary policy,employment policy, public finance… or to formalise economic mechanisms restrictedalong some time periods (short terms, medium term and long term). Yet, a large part ofpresent macromodels, at the least their structural basis, could be adapted to measure theimpact of S&T policy instruments.

The design of macroeconometric models implies a high level of expertise, notonly in the field of economic theory but also in the field of applied economics(understanding of hypotheses underlying the model, translation of theoretical conceptsinto empirical terms, identification of appropriate variables…), econometrics (estimationprocedures, calibration of the model, simulation…), as well as in the building ofeconomic databases (sources of information, collection, transformation and limits ofdata, data analysis...). Once estimated and well calibrated for the needs of evaluation, amacromodel allows one to simulate a scenario with the public intervention and ascenario without the public intervention. The difference between both scenarios gives anestimation of the impact of the public intervention on the modelled macroeconomicvariables.

The building of macromodels is time and resources expensive. The elaborationof a small scale macromodel takes a minimum of one year and its usefulness is often

16

limited to the study of a preliminary defined economic problem (for example,employment policy or fiscal policy). The construction of large scale macromodels oftenrequire a team composed of four to five persons for several years. Once the model built,a permanent team is necessary to ensure its utilisation and its periodical updating. It iswhy macromodels are rarely built for one-shot evaluations. It is generally made use ofexisting macromodels to which some adaptations or extensions are brought to satisfythe needs of evaluation. In the present state-of-the-art, macromodels should be adaptedto improve the relationships between S&T indicators and macroeconomic aggregates.

A last word of caution bears on the use of macromodels for the evaluation ofS&T policy. Indeed, the use of a model makes only sense if the intervention (theprogramme or the set of programmes) has a sufficient critical mass compared to theweight of the macroeconomic aggregates, or at the least, by comparison with the maineconomic aggregates on which is calibrated the policy.

6. Operational steps for method implementation

The construction of a macroeconometric model is composed with nine operationalsteps, which are in fact strongly interconnected. Nevertheless, these different steps canbe schematised as follows:

1. Defining the objectives of the model and the feasibility of the experiment: what arethe policy instruments to evaluate and do they interact with macroeconomic variables? Isthe intervention directed towards the economy as a whole or is it limited to some agentsor sectors? What do we want to measure? Is the macroeconometric approach suited tohighlight policy makers given the questions asked?

2. Investigating data availability: what are the data requirements? Are all the datanecessary available? How to solve the problem of missing or deficient data? Is it possibleto use proxies?

3. Specification of the model: what is the economic canvas to formalise (causality links,conceptualisation of the different blocs of the model, theoretical foundations of themodel, empirical background)? Can we adapt an existing model and if yes what are themodifications to make?

4. Collecting, analysing and transforming the data: besides its economic foundations, thequality of an econometric model depends to a large extent on the reliability of thedataset. It is only rarely that raw data can be used. Some transformations are oftennecessary as it is the case to obtain deflated data or capital stocks. A good knowledge ofdata (their content, their limits, their sources, the influence of exogenous shocks…) isindispensable before estimating the equations of the model.

5. Econometric estimations of the equations of the model: it is the most exciting as wellas frustrating step of modelling. Exciting because it is at this level that the theoretical

17

canvas takes its empirical content and that economic interdependences take shape. It isalso frustrating because the economic reality is more complex than the economic theoryand leads to some adaptations to the methodological canvas.

6. Testing and calibrating the model: despite good estimates, a model can give a poorperformance due to exogenous shocks, the omission of some phenomena or the limitedquality of some data. This implies to go back to the estimation procedure or to improvethe quality of data in order to reproduce with a sufficient degree of reliability theeconomic dynamics. It is the fine tuning of the model. A macroeconometric model canbe considered to be fine tuned when it is able to reproduce correctly the economicdynamics over the past ten or fifteen years.

7. Simulating the reference scenarios: to measure the impact of public policy it isnecessary to appreciate how should have performed the economy in the absence of thepolicy intervention. This step of the modelling will give the reference situation fromwhich will be estimated the impact of the public policy. This situation is called thecounterfactual situation because it will not be observed if effectively the policyinstruments are implemented.

8. Simulating the policy options: in this simulation exercise, exogenous quantitativevalues of instruments (the extent of the intervention) are injected into the model tomeasure how much they influence the aggregates.

9. Interpreting the results: the comparison of results obtained from simulation exercises(the reference scenarios against the policy scenarios) allows one to give estimations ofthe global impact of the policy upon the economy as well as upon main macroeconomicaggregates.

Once the model is operational, it can be used to evaluate ex ante the expected impact ofa policy. It can also be implemented to evaluate in itinere the effects of a policy and so togive some guidelines for a revision of the policy. Ex post, the model allows to measureto what extent the expected effects have been achieved, to appreciate how efficient hasbeen the policy and to suggest improvement for the new policy to implement.

7. Data requirements /indicators

The studies surveyed in section 4 use macro and meso level aggregated data ingeneral for several industrialised countries and long time periods (more than 20 years).Among these variables we can distinguish between socio-economic variables, e.g. GDP,GDP per capita, multifactor productivity growth, employment, real wages, governmentdeficit, public debt, prices, interest rates, exchange rates,… and S&T variables, e.g.public and private R&D expenditures and stocks, human capital stock, ICT stock,knowlegde spillover stocks, … Data referring to the policy instruments whose impact isto be estimated concerns among other multiannual RTD programmes budgets,(planned) CSF spending, R&D tax breaks, R&D subsidies… More and more macro

18

evaluation studies use panel data sets, which as compared to purely cross section or timeseries data, provide more information, variability, efficiency and less collinearity acrossaggregates and finally a higher flexibility in the modelling of behavioural differencesacross individual units, e.g. countries or regions. Despite the development of new andimproved indicators, more detailed indicators regarding S&T activities, e.g. absortivecapacity, channels of knoweledge exchanges, spatial and technological proximities,quality embodied deflators are needed as well as data at the regional level.

8. General assessment of the scope and limits of the methodology

The evaluation toolbox consists of a large variety of complementary rather thansubstitutable methods (Capron, 1992a, 1992b; Capron and van Pottelsberghe de laPotterie, 1997). Globally, a distinction can be made between qualitative, e.g. peer review,semi-quantitative, e.g. matrix and systemic approaches, and quantitative methods, e.g.cost/benefit and econometric analysis. There is no perfect (or complete) assessmentmethod: each method has its own advantages and drawbacks. The choice of a methoddepends of the issues that are to be investigated, the data availability and the level ofanalysis, e.g. the macro-level. Answers and highlights obtained from a single method willalways be partial and will often give rise to new questions. Taken individually, each ofthem is able to provide relevant additional piece of information in the evaluationprocess. To increase the credibility of evaluation results, alternative methods should beideally used to consolidate the foundations of policy recommendations. Despite theirdrawbacks and the apparent mistrust from which they suffer, and given the absence ofany firmly established substitutes at the time being, econometric methods appear to bethe most appropriate to quantitatively assess the socio-economic impacts of RTDprogrammes. Yet, the lack of relevant and sufficiently detailed S&T indicators iscertainly a main bottleneck preventing a more intensive use of such quantitativemethods.

The impact of S&T policies shows itself at different levels of aggregation ofeconomic activities. For instance, evaluations at the micro level give a good insight intothe direct impact of a policy but do not generally provide any information on theindirect ones. Only inter-industry studies (at the meso level) and studies at the macro-level can provide information on the global impact, i.e. direct and indirect effects, ofS&T policies. Yet, the global impact is a net effect since the indirect effects may eithercounterbalance or reinforce the effects of the direct ones. Hence, it is not easy todisentangle, at the macro-level, between the variety of direct, indirect and induced effectsthat contribute to specific outcomes.

The evaluation at the macro-level measures the global socio-economic impact in thelong-term (global impact of a system of programmes or benefits to the society as awhole) rather than the short-term partial impact (benefits to the participants of anyspecific RTD project or programme). The timing itself needs to be addressed since thesocio-economic impacts are not immediate.

19

Macro-economic effects are in general more difficult to assess than micro-economicmechanisms. Given the range of S&T public interventions and of public policies ingeneral, it is not easy to isolate the impact of a particular RTD programme, whichaccounts for only a small part of total public intervention on macro-aggregates such asGDP growth, exports, and quality of life. More generally, socio-economic performanceis conditioned by many other factors that influence the broader economic, institutionaland social context of innovation. Despite these issues, it seems useful before anyevaluation, to identify the different types and nature of the socio-economic impacts ofRTD programmes.

9. Commented bibliography

• Klein L., A. Welfe and W. Welfe, (1999), Principles of Macroeconometric Modeling,Advancs Textbooks in Economtrics, C. Bugg and M. Intriligator (eds), North-Holland, 366 pages.

State of the art of macroeconometric modeling : what is a macroeconomic model ?,demand and supply determined models, disequilibrium macromodels, use ofmacroeconometric models, production functions, price and wage equations,financial flows, some LINK applications, simulation, dynamic analysis, VAR models,stationarity and non-stationarity, equilibrium relationships and cointegration analysis,rational expectations, multiplier and policy analysis, forecasting, ex ante prediction,forecast preparation.

• Romer D., (2000), Advanced Macroeconomics, 2nd edition, Irwin-McGraw-Hill,672 pages.

Topics covered: the Solow growth model, infinite-horizon and overlapping-generations models, new growth theory, real-business-cycle theory, traditionalkeynesian theories of fluctuations, microeconomic foundations of incompletenominal adjustment, consumption, investment, unemployment and the labormarket, inflation and monetary policy, budget deficits and fiscal policy.

• Research Policy, Volume 29, Issue 4-5, 01-April-2000

Special issue on the evaluation of S&T policies : Stephen Martin, John T. Scott, Thenature of innovation market failure and the design of public support for privateinnovation, Research Policy (29)4-5 (2000) pp. 437-447 ; Bronwyn Hall, John VanReenen, How effective are fiscal incentives for R&D? A review of the evidence,Research Policy (29)4-5 (2000) pp. 449-469 ; Tor Jakob Klette, Jarle Møen, ZviGriliches, Do subsidies to commercial R&D reduce market failures?Microeconometric evaluation studies, Research Policy (29)4-5 (2000) pp. 471-495 ;Paul A. David, Bronwyn H. Hall, Andrew A. Toole, Is public R&D a complementor substitute for private R&D? A review of the econometric evidence, ResearchPolicy (29)4-5 (2000) pp. 497-529 ; F.M. Scherer, Dietmar Harhoff, Technology

20

policy for a world of skew-distributed outcomes, Research Policy (29)4-5 (2000) pp.559-566 ; John Hagedoorn, Albert N. Link, Nicholas S. Vonortas, Researchpartnerships, Research Policy (29)4-5 (2000) pp. 567-586 ; P.A. Geroski, Models oftechnology diffusion, Research Policy (29)4-5 (2000) pp. 603-625 ; Barry Bozeman,Technology transfer and public policy: a review of research and theory, ResearchPolicy (29)4-5 (2000) pp. 627-655 ; Luke Georghiou, David Roessner, Evaluatingtechnology programs: tools and methods, Research Policy (29)4-5 (2000) pp. 657-678.

References

Bayoumi, T., D. Coe, et al. (1999). “R&D Spillovers and Global Growth.” Journal ofInternational Economics, 47(2): 399-428.Eaton and Kortum (1997)

Beutel J. (1996). “Dynamic Input-Output Model to Evaluate the Economic Impacts ofthe Structural Funds. European Commission DGII.

Bradley, J. and K. Whelan (1992). "Econometric Modelling and Technological Change aReview of Issues" in Quantitative Evaluation of the Impact of R&D Programmes :Brussels, European Commission.

Capron H. (1992b) “A State of the Art of Quantitative Methods for the Assessment ofR&D Programmes”, in M.T. Khalil and B. Bayraktar (eds.), Management ofTechnology III, Industrial Engineering and Management Press, Georgia, 1195-1204.

Capron H. and M. Cincera (1998) “Exploring the spillover impact on productivity ofWorld-wide manufacturing firms", Annales d’économie et de statistiques, 2ièmetrimestre: 566-87.

Capron H.(1992a) Economic Quantitative Methods for the Evaluation of the Impactof R&D Programmes. A State of the Art, Monitor-Spear Series, EuropeanCommission, Brussels.

Capron, H. and B. Van Pottelsberghe de la Potterie (1997). "Public Support toBusiness R&D: A Survey and some quantitative evidence" in Policy Evaluationin Innovation and Technology: Towards Best Practices : Paris, OECD.

Cincera M. (1997) “Patents, R&D and Technological Spillovers at the Firm Level:Some Evidence from Econometric Count Models for Panel Data”, Journal ofApplied Econometrics 12(3): 265-280

David, P. and B. H. Hall (2000). “Heart of Darkness: Modelling Public-PrivateFunding Interactions inside the R&D Black Box.” NBER Working Paper No.7538

David, P., B. H. Hall, et al. (2000). “Is Public R&D a Complement or Substitute forPrivate R&D? A Review of the Econometric Evidence.” Research Policy (29):497-529.

21

den Butter F. and Wollmer F., 1996, “An empirical model for endogenous technologyin the Netherlands”, Economic Modelling, 13(1), pp.15-40.

DULBEA-CERT, 1997, “Evaluation des impacts du DOCUP” in SEMA GROUP,Evaluation intermédiaire du DOCUP Objectif 1 Hainaut, Rapport au Ministère dela Région wallonne.

DULBEA-CERT, 2001, Evolution de la situation économique hennuyère, Tableau debord et Modèle économétrique HELM, Rapport au Ministère de la Régionwallonne.

European Commission (1999). Evaluating Socio-Economic Programmes, MEANSCollection (6 Vol.). Luxembourg.

Giorno, C., P. Richardson, et al. (1995). “Technical Progress, Factor Productivity andMacroeconomic Performance in the Medium Term.” OECD EconomicsDepartment Working Paper No. 157, Paris

Goolsbee, A. (1998). “Does Government R&D Policy Mainly Benefit Scientists andEngineers?” American Economic Review 88(2): 298-302.

Guellec D. and van Pottelsberghe B., 2000, The Impact of Public R&D Expenditure onBusiness R&D; STI Working Paper 2000/4.

Guellec D. and B. van Pottelsberghe, 1999, Does government support stimulates privateR&D ?, OECD Economic Studies, 29, 1997/II, pp. 298-122.

Guellec D. and B. van Pottelsberghe, 2000, The effectiveness of public R&D policies inOECD countries, STI Working Papers, DSTI/DOC(2000)4, OECD.

Guellec D. and B. van Pottelsberghe, 2001, R&D and productivity growth – A paneldata analysis of 16 OECD countries, STI Working Papers, OECD, forthcoming.

Jones, C. and J. Williams (1999). “Measuring the Social Returns to R&D.” QuarterlyJournal of Economics 113: 1119-1135.

O'Sullivan, L. and W. Röeger (1992). "An Exploration of the Interrelationships of R&DExpenditures and Technological Progress in major OECD Economies." inQuantitative Evaluation of the Impact of R&D Programmes : Brussels, EuropeanCommission.

Röeger W. (1996). “Macro-Economic Evaluation of the Effects of CommunityStructural Funds Interventions with QUEST II”. European CommissionDGII.

Roeger W. and Veld J., (1997), “QUEST II. A Multi-Country Business Cycle andGrowth Model”, Economic Papers, No. 123. European Commission. Brussels

van Bergeijk., van Hagen G. and al., 1997, “Endogenizing technological progress: TheMESEMET Model” Economic Modelling, 14(3), pp.341-68.