public r&d investment comparison and performance evaluation framework: an...

Public R&D Investment Comparison and Performance Evaluation Framework: An Input-Output-Econometric (IOE)

Approach

FTEval Conference, Vienna, Session D April 25th, 2006

Frederick A. KijekReza Ghazal

DeCallière Research Associates

Frederick A. Kijek, Reza Ghazal 2

Content

Introduction

Overview

Objectives

Approach & Methodology

Demonstration Model Results

Results Conclusions

Expanding the Model


Introduction

R&D is a great catalyst to economic growth

Canada spent roughly $13 billion (Cdn)[in 2005 on publicly funded R&D programs and initiatives[ (not including R&D tax credits) – no one knows what the return on investment is

Historically public R&D was justified based on qualitative “motherhood” statements of socio-economic impacts

increasing number of alternative emerging technologies and science based growth opportunities for public sector to invest in

the initiatives for which the claims can be substantiated and differentiated in quantitative terms – at executive (cabinet) and political levels – will receive public funding

Input-Output and Econometrics analysis can provide rich information as to the economic returns on investment – have been insufficiently adopted in comparing public R&D investment options and undertaking economic performance evaluations


Overview

Our paper was based on the design and development of a quantitative R&D economic impact assessment framework to meet Canada’s National Research Council’s strategic planning and performance evaluation needs

The resulting model compared seven “pre-defined” R&D investment options and ranked them according to their relative economic impacts and contributions

Two separate indicators were used to compare the relative rankings

— Economic Impact Index

— R&D Benefit-Cost ratio.

The reliability of the comparison rankings are strengthened by the similarities in the results of the two indicators - and the results of an independent bibliometric analysis

Recommendations are provided to further enhance the capabilities of the model. We are presently expanding and improving the framework.


Objectives

The overall objective was to develop a multi purpose tool to:

— strategically compare research funding options based on their relative economic return on investment

— perform economic performance evaluation of public R&D funding projects and/or programs

To quickly develop and customize a quantitative R&D economic analysis model to demonstrate/validate its capabilities by assessing the economic impacts and contributions of “pre-defined” investment options as part of its “renewal” exercise.


Approach and Methodology

The demonstration model has been developed by adapting from a number of known (trusted) quantitative methods including economic impact analysis, input-output matrix analysis, econometric analysis and benefit-cost analysis.

The results of the analysis provided the components and data on which two separate comparison indexes were created.

— The R&D Economic Impact Index includes eleven separate coefficients and multipliers including a total factor productivity coefficient, four R&D multipliers (for Investment, Consumption, Export, and Intermediate inputs) as well as six general economic impact multipliers (including Output, GDP, Exports, Imports, Employment and Labour Income.)

— The Benefit-Cost ratio was developed as a result of comparing the present value of forecasted benefit streams, using the dynamic productivity data and the R&D and economic impact multipliers.

Primary data was collected both from Canada’s National Accounts and Input-Output Matrices as compiled by Statistics Canada as well as data from the OECD.



Estimating Total Factor Productivity

For calculating the effects of R&D on TFP, a two-staged approach was adopted. This approach for estimating the impact of R&D on TFP borrowed from recent work done by the Congressional Budget Office[as well as work done by the OECD.

A summary of this two stages approach is as follows:

.

First stage: )exp( tttt

tt MLKAeY or in logarithmic form,

ttttt MLKtaY loglogloglog ,

Where, tttt MLKY ,,, are measures of output, capital stock, labor input and

intermediate input, respectively, and ,, represents the elasticity of output to capital stock, labor input and intermediate input, respectively. a 1 is a proxy for TFP that we are looking for. Second stage: ttt DRca )&log(. where represents the effect of R&D

on TFP growth, a is the TFP growth proxy.

1 TFP is defined as TFP= Technical Change * Technical Efficiency Change. Although, the change rate of TFP is equal to rate of technical change (TC) or technical efficiency change just in the case of CRS, but generally using TC as a proxy for TFP is not an appropriate method in calculating TFP. Because of time constraint and limited availability of detailed R&D data, we assumed CRS.



Estimating R&D Multipliers

We estimated both the backward and forward R&D multipliers for consumption, investment and export for each of the sectors for which data was available.

The methodology used was borrowed from recent work undertaken at the SOM Research Institute[and relies heavily on Input-Output analysis and matrix manipulation.

A summary of this methodology is as follows:

.

M a t r ix o f In t e r m e d ia t e D e liv e r ie s : Z V e c to r o f G ro ss O u tp u t s : X

I n p u t C o e ff ic ie n t s M a t r ix : 1ˆ XZA L e o n t ie f In v e r se M a t r ix : 1)1( AL V e c to r o f R & D E x p e n d itu re : r

R & D In t e n s it y M a t r ix : 1ˆ Xr

E m b o d im e n t M a t r ix H fo r e x p o r t : eLH e ˆ̂

E m b o d im e n t M a t r ix H fo r c o n su m p tio n : cLH c ˆ̂

E m b o d im e n t M a t r ix H fo r in v e s tm e n t : iLH i ˆ̂ T h e su m m a t io n o f r e sp e c t iv e c o lu m n s o f H p ro v id e s t h e to t a l a m o u n t o f R & D e m b o d ie d in t h e e x p o rt ( co n su m p t io n , in v e s t m e n t ) o f in d u s t r y j. T h e ro w su m o f H re p re se n t s t h e a m o u n t o f in d u s t r y i’s R & D a s e m b o d ie d in a ll in d u s t r ie s ’ e x p o rt s ( co n su m p t io n s , in v e s tm e n t s ) .



The table on the following page provides the ranking of the R&D investment opportunities based on the analysis results.

The R&D opportunities with the highest ranking provide the highest economic return on investment in terms of economic impacts and benefits.

As can be seen from tables and charts provided on the following pages the ranking results based on both indicators are very similar.

The results indicate that investments in the Resource and Energy (I.e. opportunities #4 and #1) related sectors provide the higher returns.



As can be seen from the table below the indicators show consistent however slightly different ranking results (specifically with respect to R&D investment opportunities #2 and #3).

This is due primarily to the dynamic nature of the benefit-cost analysis relative to the economic impact index (I.e. timing considerations of the impacts).

Opp # R&D Investment OpportunitiesR&D Benefit-

Cost RatioR&D Economic Impact Index

1 Enhanced energy technologies and alternatives sources 2 2

2 Sustainable advanced technologies and bioproducts for industrial applications. 4 33 Environmental technologies 3 44 Natural Resources 1 1

5 Chronic Diseases of the ageing population 6 66 Pandemics and Infectious Diseases 7 77 Water 5 5

Opportunities Ranking based on R&D Economic Impact and Benefit-Cost Analysis Model Results



0.00 1.00 2.00 3.00 4.00 5.00 6.00

1

2

3

4

5

6

7

R&

D In

vestm

ent O

pport

unitie

s

Comparison of the Economic Impact Index and Benefit-Cost Ratio of Seven R&D Investment Opportunities

Economic Impact Index

Benefit-Cost Ratio

Economic Impact Index 3.86 3.20 2.82 5.26 0.88 0.88 2.28

Benefit-Cost Ratio 2.04 1.87 1.90 2.40 1.44 1.43 1.75

1 2 3 4 5 6 7



1 2 3 4 5 6 7

Investment

Consumption

Exports

Labour IncomeGDP

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

90,000

$ T

housa

nds

R&D Investment Opportunities

Sample of Economic Impacts from $100 Million R&D Infusion (Both ranking indicators are based on an amalgamation of these values or the multipliers used in their estimation)

Investment

Consumption

Exports

Labour Income

GDP


Results Conclusions

The reliability of the comparison rankings are strengthened by the similarities in the results provided by the two separate indicators and the results of an independent bibliometric analysis of the same investment options.

The “pre-identified” investment options were broadly defined.

— to facilitate an iterative process for formulating gradually more specific strategic planning options.

— proprietary nature of the details

These results can be interpreted as an acknowledgement that the economic sectors in which Canada presently has strong productive capabilities, existing infrastructure, available domestic inputs and strong established comparative advantages in will generate relatively higher economic impacts.

The higher ranked opportunities include sectors for which Canada has been able to generate historically higher productivity and economic impacts from R&D expenditures.


Expanding the Framework

Though successful, our approach is not without its weaknesses

1) Data Availability and Aggregation

— NAICS sub-sectors used to identify investment option sectors

— Data availability problems (due to aggregation, historical sector reconciliation, data confidentiality, and time constraints)

TFP coefficients - 39 sectors R&D multipliers - 118 sectors general impact multipliers - 300 sectors

— weakness inherent in matching NAICS identified sub-sectors with R&D investment option sectors

Even with full contingent of NAICS sub-sectors - still an objective element in aligning the sector data with investment options - more structured expert panel approach will be adopted

2) Consideration of More Difficult to Quantify Impacts

— Wealth of recent work in quantifying environmental and health impacts (such as QALY)

3) Dynamic Forecasting & Risk Analysis

— In addition to the lags estimated as part of TFO and used within our R&D BC Ratio - a more structure forecasting capability to estimate the magnitude of impacts within dynamic, future, economic structure using computable general equilibrium (CGE) modeling – also to estimate spillover effects

— Risk analysis to account for the variability and uncertainty in the data and forecasted results


Authors

This presentation was prepared by:

Frederick KijekDirector & Senior EconomistDeCallière Research Associates34 Glen Avenue, Suite 100Ottawa, ON K2P 1P1C: 613-314-6625

This document is protected under the copyright laws of Canada and other countries as an unpublished work. This document contains information that is proprietary and confidential to DeCallière Research Associates or its partners, which shall not be disclosed outside or

duplicated, used, or disclosed in whole or in part for any purpose other than to evaluate DeCallière Research Associates. Any use or disclosure in whole or in part of this information without the express written permission of DeCallière Research Associates is prohibited.

public r&d investment comparison and performance evaluation framework: an...

Documents

economic impact analysis

rd economic impact index

public rd investment

reza ghazal introduction

economic returns

relative economic impacts

model slide

relative economic return