patent statistics - working paper methods for nowcasting ...€¦ · 5 3. potential econometric...
TRANSCRIPT
Final Version - 21 December 2010
Patent Statistics - Working Paper: Methods for Nowcasting Patent Data
1
Abstract
Patent indicators provide a measure of the innovative performance at country, firm or region
level. Nevertheless, indicators are criticized as being “outdated”. This is due to the fact that
information on patent applications is disclosed to the public 18 months or more after priority
date. This issue is know as “timeliness”. In order to overcome this, nowcasting methods have
been discussed and developed.
The main purpose of this paper is the presentation of the existing methods for nowcasting of
patent data to the European Patent Office (EPO) and the proposal of improved methods. An
evaluation of these methods has been performed and conclusions regarding the most
adequate methods for most of the countries have been drawn.
Keywords: tineliness, nowcasting methods, regional phase PCT, transfer rate models, trend
models, econometric models
2
1. Introduction
Within the framework of project “Patent Statistics” with contract no 73101.2007.003-2009.535
there have been specified particular actions and tasks for the implementation of the project.
The current report is the final report under the scope of Action 3: “Patent Statistics” and the
task “Methods for Nowcasting Patent Data”.
The main objectives of this particular task is to present the existing methods for nowcasting of
patent applications. Moreover, an attempt to apply econometric models for nowcasting patent
applications to the EPO was attempted as well as a comparison analysis.
In the second section of the report, we mention the reason for Nowcasting and briefly present
the existing methods (deterministic and stochastic) for nowcasting patent data.
The third section deals with econometric models for nowcasting patent data. In particular, the
need for the use of econometric models is emphasized. Following, the existing econometric
models are presented and 6 new models are presented. Lastly, a comparison analysis is
performed and the strengths and weaknesses of each model is outlined.
2. The issue of Nowcasting in patent indicators
2.1 Why Nowcasting
Patent indicators are valuable in providing a measure of the innovative performance and
technology outputs at country, firm or region level. Nevertheless, due to legal rules imposed
by the patent application process, information on patent applications is disclosed to the public
18 months or more after priority date.
As a result, patent indicators are faced with the “timeliness” issue, which can extend to more
then five years depending on the computational method used to develop indicators. In order
to overcome this issue, nowcasting methods have been discussed and developed.
2.2 Existing Nowcasting Methods
The objective in nowcasting is to obtain the total number of EPO patents filings.
Total EPO Filings = Direct EPO filings + EPO Regional phase PCT
Euro-PCT filings that entered the EPO regional phase have a time lag of approximately 18
months in relation to the Direct EPO filings. Thus, in order to nowcast total EPO filings we can
estimate EPO regional phase PCT filings.
3
The methods developed in order to deal with the nowcasting issue can be classified into
deterministic and stochastic methods.
Deterministic Methods
Transfer Rate Models: The idea behind the transfer rate models is that the ratio of the EPO
regional phase PCT applications to the EPO designated PCT applications (Transfer Rate) is
stable from year to year. So, if we know EPO designated PCT applications for the year we
want to estimate total EPO filings, we can estimate EPO regional filings for this year
assuming that the ratio (Transfer Rate) of the estimated year is the same as the previous.
A summary of the transfer rate models developed is presented in the following table.
MODEL Transfer Rate Regional Phase PCT filings
Source
TR_R(1)
tPCTtEPCT
tt
PCTPCT
1
1-tt
EPCT =EPCT Khan and Dernis (2005)
TR_R(4)
tPCTtEPCT
ttt
tt PCTPCTPCT
EPCTEPCT
21
21t =EPCT Khan and Dernis (2005)
TR_R(2)
1
tEPCT
tPCT 1
2
1-tt
EPCT =EPCT
tt
PCTPCT
Methods for Nowcasting
Patent Data, European
commission Eurostat,
30/8/20008
TR_R(3)
tPCTtEPCT
tt
ttt PCT
RATETR
RATETRRATETREPCT
2
11 _
__
Methods for Nowcasting
Patent Data, European
commission Eurostat,
30/8/20008
TR_R(5) 1
1tEPCT
tt
t
PCTPCT
EPCT
1121
21t )( =EPCT
ttttt
tt EPCTPCTPCTPCTPCT
EPCTEPCT
Methods for Nowcasting
Patent Data, European
commission Eurostat,
30/8/20008
TR_R(6)
1
tEPCT
tt PCTPCT )( =EPCT 1
21
1t
tt
tt
t PCTPCTPCTPCT
EPCT
Methods for Nowcasting
Patent Data, European
commission Eurostat,
30/8/20008
TR_R(7)
21
tEPCT
tt PCTPCT )( =EPCT 21
32
1t
tt
tt
t PCTPCTPCTPCT
EPCT
Methods for Nowcasting
Patent Data, European
commission Eurostat,
30/8/20008
Stochastic Methods
Trend Models: Trend models consist of simple extrapolation of the trends over various time
periods.
Autoregressive Integrated Moving Average (ARIMA) models are used to nowcast the EPO
regional phase PCT data as well as total EPO patent applications. Also, linear, quadratic and
logistic (S shaped) models are systematically tested on series of various length.
A summary of the trend models developed is presented in the following table.
4
MODEL Model
Description Regional Phase PCT filings
Source
TREND(4)
Exponential form or,
Logarithmic transformation
tt
t εαβ=EPCT
tt )t(loglog=)log(EPCT
Khan and Dernis (2005)
TREND(5)
AR(1) (with
logarithmic transformation)
ttEPCT 1t ΔΕPCT
, where
)log()log( ΔΕPCT 1t tt EPCTEPCT
Khan and Dernis (2005)
TREND(6)
AR(2) (with
logarithmic transformation)
ttt EPCTEPCT 21t ΔΕPCT
, where )log()log( ΔΕPCT 1t tt EPCTEPCT
Khan and Dernis (2005)
TREND(7)
MA(1) (with
logarithmic transformation)
1t ΔΕPCT tt Khan and Dernis (2005)
TREND(8)
MA(2) (with
logarithmic transformation)
21t ΔΕPCT ttt
Khan and Dernis (2005)
TREND(1) Random Walk with Drift (α) 1tt EPCTEPCT
Methods for Nowcasting
Patent Data, European
commission Eurostat,
30/8/20008
TREND(2) Double(Brown)
Exponential Smoothing
tEPCT ttt
Methods for Nowcasting
Patent Data, European
commission Eurostat,
30/8/20008
TREND(3) Linear
Regression tt εβtα=EPCT
Methods for Nowcasting
Patent Data, European
commission Eurostat,
30/8/20008
Econometric models:
Another method of improving the models described above, is to make use of additional
information concerning the countries. This information can be:
economic indicators such as R&D expenditures by sectors, and source of funds
GDP
number of researchers
indicators of technological opportunities
indicators based on specific information from patent office (budget, number of patent
examiners, patent fees), etc.
Econometric models are then constructed based on the above additional information. (see
van Pottelsberghe and Dehon, 2003 ; Hausman, Hall and Griliches, 1981).
An attempt to create econometric models is presented in the following chapter.
5
3. Potential econometric models
In the previous section were mentioned the main methodological approaches for nowcasting
analysis in patent indicators. Taking into account the available papers on nowcasting
techniques, one notices that the majority of the attempts / proposals for nowcasting patent
indicators are related to the application of transfer rate modes and trend analysis. References
on the use of econometric models are also available but the efforts are not extended at the
same level as in the other methods of nowcasting.
The current section examines the application of specific econometric models for
nowcasting patent applications to the EPO.
3.1 Why Econometric Models
Patents constitute the main outcome of an inventive procedure and are directly related to
major sectors of the modern market. Patenting activity is related to the evolution of
technological development, affects the financial development and also is affected by the
financial conditions of the market at national or international level.
Apart from the close relationship of patenting activity with the market, the main motivations for
working on econometric models arose from the following conditions:
Modern markets are currently affected by the domino of economic crisis. Although the current
conditions do not allow the mature drawn of conclusions for the effect of the economic crisis
on patenting activity, in this paper we try to investigate if it is feasible to answer the following
questions:
Q1.Can estimates of the total number of patents for the most recent years be calculated?
Q2. Can those estimates reveal the effect of economic crisis?
In the following sub section are presented the econometric models that were tested for
nowcasting at annual level the total number of patent applications to the EPO.
3.2 Existing econometric methods for nowcasting patents
The relationship among patenting activities and the research and technological development
sector is proven in previous studies(van Pottelsberghe and Dehon, 2003 ; Hausman, Hall and
Griliches, 1984). Besides, during recent years papers concerning the use of econometric
models for measuring patenting activity have been published ( Marek Szajt, Technical
University of Czestochowa, 2009; Walter G. Park and Peter Hingley, 2009).
Each one of the above mentioned papers presents methodological approaches that are
based on the use of econometric models. Besides, there is a vast plurality in the use of
independent variables, the predicting variables and the predicting models that are used.
Econometric models can be used for estimating PCT applications, EPO PCT applications at
regional phase or for Triadic Patents. The common factor among all the proposed models is
6
the use of independent variables that reflect R&D activity or reflect the general financial status
of a country. Hence the most common variables that are used in econometric models are:
GDP (Gross Domestic Product), GERD (Gross Expenditure in Research and Development)
and Researchers.
3.3 Econometric models for nowcasting
The present paper does not intend to present the most complex models for nowcasting patent
indicators. The main aim is to combine econometric variables that reflect patenting activity in
order to estimate counts of patents for 3 or 4 years (2007,2008,2009 and 2010).
Under the study for nowcasting patents with econometric models we designed and calculated
estimates for the Total Number of Patent Applications to the EPO, with 6 models. In the
present research the following assumptions were made:
The estimates concern the total number for patent applications to the EPO in
year t by country, according to the inventor’s country of residence. The data set that was used in the analysis with the proposed models, is constructed
from the available statistics database in Eurostat’s web portal The calculations were applied at regional level for 27 EU member states, 2 EFTA
countries and 3 candidate countries. The main reason for applying the tests only at
these specific regional levels is that for those countries data were available for the
variables used in the econometric models. The data that were used in the tests of the models constitute the official
statistics provided by Eurostat. The statistics were downloaded from http://epp.eurostat.ec.europa.eu/portal/page/portal/statistics/themes
The required data by statistical domain were the following:
Domain Variable
Code
Description of
Variable
Unit of
measure
Economy and finance GDP
Gross Domestic
Product
(Current Prices)
Millions of
Euros
Science and technology
.......Research and development GERD
Total intramural R&D
expenditures
Millions of
Euros
Science and technology
.......Research and development RES
Total number of
Researchers Headcounts
Science and technology
...... Human Resources in Science & Technology
............ Stocks of HRST
HRST
Human recourses in
Science and
Technology
Headcounts
(Thousands)
In this paper were tested 6 different econometric models. The method that was used for the
proposed models is simple regression analysis. All the models have the same dependent
variable, which is the total number of patent applications that were submitted to the EPO by
inventor(s) of country c at priority year t. The main difference among the models concerns the
use of the independent variables and the application of transformations for achieving better fit
of the models through linearity. Besides, another difference concerns the availability of data at
7
annual level for each one of the independent variables used in the models. The main
differences will be presented in latter sections of the document.
Finally, a new independent variable was tested in the econometric model. This variable
concerns the human resources in Science and Technology and constitutes a new statistic in
the domain of Research and Technological development. The specific data provide a deeper
insight in the stock and mobility of human resources within the field of Science and
Technology among the countries of European Research Area.
In this part of the paper is presented the “identity” of each of the tested models.
3.3.1 Model 1 Total number of patent applications that were submitted to the EPO by inventor(s) of country
c at priority year t as a linear function of Gross Domestic Product of country c at year t and
the Gross Expenditures of country c in the field of Research and Technology during year t.
ctctcctccct eGERDGDPaEPO
where α,b and c are unknown parameters and e is the error term.
3.3.2 Model 2 Total number of patent applications that were submitted to the EPO by inventor(s) of country
c at priority year t as a linear logarithmic function of Gross Domestic Product of country c at
year t and the Gross Expenditures of country c in the field of Research and Technology
during year t.
ctctctcct eGERDGDPaEPO )ln()ln()ln( This transformation was applied in order to apply better fit of the model, since in some cases
the dependent variables and the independent follow an exponential distribution over time.
3.3.3 Model 3 Total number of patent applications that were submitted to the EPO by inventor(s) of country
c at priority year t as a linear function of Gross Expenditures of country c in the field of
Research and Technology during year t.
ctctcct eGERDaEPO
3.3.4 Model 4 Total number of patent applications that were submitted to the EPO by inventor(s) of country
c at priority year t as a linear function of Gross Expenditures of country c in the field of
Research and Technology during year t and the Total Number of Researchers that were
occupied in country c during year t in the domain of Research and Technology.
ctctctcct eRECHGERDaEPO
3.3.5 Model 5 Total number of patent applications that were submitted to the EPO by inventor(s) of country
c at priority year t as a linear function of Gross Expenditures of country c in the field of
Research and Technology during year t and the Total Number of People who fulfil the
8
definition of Human Resources in the domain of Research and Technology for country c at
year t.
ctctctct eHRSTGERDaEPO The term Human Resources in Research and Development is different from the term
“Researchers” in R&D sector. Science and Technology is a broader term than
Research and Development. Below are presented the definitions for stock in HRST that are
provided by Eurostat and the definition of HRST provided by OECD.
“For HRST statistics, stock data relate to the employment status as well as the occupational and educational
profiles of individuals in any given year. An HRST stock is "the number of people at a particular point in time
who fulfil the conditions of the definition of HRST".
SOURCE: Eurostat http://epp.eurostat.ec.europa.eu/cache/ITY_SDDS/EN/hrst_st_esms.htm
Reference Metadata in Euro SDMX Metadata Structure (ESMS) ,Compiling agency: Eurostat, the statistical office of
the European Union
“The term "Human Resources in Science and Technology" (HRST) has been coined for use in this manual to
describe this special skilled labor force. At its widest, it extends to everyone who has successfully
completed post-secondary education (or is working in an associated S&T occupation); at its narrowest it
covers only those with at least university-level qualifications in natural sciences or engineering (or working
in an associated S&T occupation). "Human resources" is a synonym for personnel" or the now obsolete
"manpower", and has been chosen in order to avoid confusion with other methodologies and statistical
sources.”
SOURCE: OECD http://www.oecd.org/dataoecd/34/0/2096025.pdf
The measurement of scientific and technological activities, manual on the measurement of human resources devoted
to S&T "Canberra Manual"
3.3.6 Model 6 Total number of patent applications that were submitted to the EPO by inventor(s) of country
c at priority year t as a linear logarithmic function of Gross Expenditures of country c in the
field of Research and Technology during year t and the Total Number of People who fulfil the
definition of Human Resources in the domain of Research and Technology for country c at
year t.
ctctctcct eHRSTGERDaEPO )ln()ln(
9
3.4 Compare Models
The econometric analysis that was performed with the 6 models revealed that the total
number of patent applications to the EPO can be predicted for the following years: 2007,
2008,2009 and 2010 in some cases. However, there is no clear conclusion for the model that
applies very well to all the countries that were included in the study. For that reason is
considered appropriate to create a set of specific criteria that will be used for the assessment
of the 6 econometric models. Taking into account statistical and empirical criteria, we
formulated the following list of criteria for comparing the econometric models:
Criterion 1: A set of diagnostics for best model fitting
The diagnostics that were used for deterring the goodness of fir are: R2,
Square Root of Mean Square Error (Square root of MSE)
Mallow’s C(p)
Akaike's information criterion
Criterion 2: The precision of the estimates in relation to the actual observations
The precision of the estimates was tested towards observed values that are already
available for the total number of patents to EPO. The models were tested for providing
estimates for priority years 2005, 2006 and 2007.
Criterion 3: The prediction period that each model supports. This factor is closed related
to the time series for which data are available for each dependent variable of the model,
by country.
Criterion 4: The total number of countries for which predictions can be provided with
each model
Criterion 5: Can they reveal the economic crisis affect?
In the following table are summarized the main conclusions by criterion
Model
Criterion 1
Model
fitting
Diagnostics
Criterion 2
Precision of
Estimates
Criterion 3
Data availability
Criterion 3
Prediction
period
Criterion 4
Coverage of countries with
predictions
Criterion 5
Reveal economic crisis
1 High in
most cases Good
GERD : 1981‐2010 (*
for some countries)
GDP: 1981‐2010
2007,2008,20
09 and some
for countries
2010
2007: 31/32 Yes
(in some cases between
2007 ‐2009)
2008: 31/32
2009: 28/32
2010: 5/32
2 High in
most cases Very good
GERD : 1981‐2010 (*
for some countries)
GDP: 1981‐2010
2007,2008,20
09 and some
for countries
2010
2007: 31/32 Yes
(in some cases between
2007 ‐2009)
2008: 31/32
2009: 28/32
2010: 5/32
3 High in
most cases Very good
GERD : 1981‐2010 (*
for some countries)
2007,2008,20
09 and some
for countries
2010
2007: 31/32 Yes
(in some cases between
2007 ‐2009)
2008: 31/32
2009: 28/32
2010: 5/32
10
4 High in
most cases Good
GERD : 1981‐2010 (*
for some countries)
RECH: (1981‐2008(* for
some countries)
2007,2008,20
09 and 2010
for
2countries
2010
2007: 29/32 Yes
(in some cases between
2007 ‐2009)
2008: 29/32
2009: 5/32
2010: 2/32
5 High in
most cases Good
GERD : 1981‐2010 (*
for some countries)
HRST: (1981‐2008(*
most countries have for
1994‐2009)
2007: 31/32 Yes
(in some cases between 2007 ‐2009)
2008: 31/32
2009: 28/32
2010: 0/32
6 High in
most cases Good
GERD : 1981‐2010 (*
for some countries)
HRST: (1981‐2008(*
most countries have for
1994‐2009)
2007: 31/32 Yes
(in some cases between 2007 ‐2009)
2008: 31/32
2009: 28/32
2010: 0/32
3.5 Strengths & Weaknesses
The main strengths and weaknesses for each one of the 6 econometric models can be
summarized as follows:
Model 1: ctctcctccct eGERDGDPaEPO Provides estimates mostly for 2009 2008 and 2007 and 2010 (5 countries)
The goodness of fit statistics are good in general
Reveals decrease in patenting activity for some countries
- Provides good estimates only for 13/96 testing cases regarding 32 countries and 3
years (2005,2006 and 2007)
Model 2: ctctctcct eGERDGDPaEPO )ln()ln()ln( Provides estimates mostly for 2009 2008 and 2007 and 2010 (5 countries)
The goodness of fit statistics are good in general
Reveals decrease in patenting activity for some countries
Provides good estimates for 20/96 testing cases regarding 32 countries and 3 years
(2005,2006 and 2007)
Model 3: ctctcct eGERDaEPO Provides estimates mostly for 2009 2008 and 2007 and 2010 (5 countries)
The goodness of fit statistics are good in general
Provides good estimates for 22/96 testing cases regarding 32 countries and 3 years
(2005,2006 and 2007)
Reveals decrease in patenting activity for some countries
Model 4: ctctctcct eRECHGERDaEPO Provides estimates mostly for 2009 2008 and 2007 and 2010 (2 countries)
The goodness of fit statistics are good in general
Reveals decrease in patenting activity for some countries
11
- Provides good estimates for 17/96 testing cases regarding 32 countries and 3 years
(2005,2006 and 2007)
- The absence of data availability for many countries regarding 2009 and 2010 reduces
the predictive ability of the model and the goodness of fit for some countries
Model 5: ctctctct eHRSTGERDaEPO Provides estimates mostly for 2009 2008 and 2007
The goodness of fit statistics are good in general
The variable HRST seems that constitutes a good explanatory variable for the model,
despite the fact that the time series mostly provides data between 1994-2009
Reveals decrease in patenting activity for some countries
HRST data are of better quality than RECH regarding the time series (longer and
most recent data availability)
- Provides good estimates only for 10/96 testing cases regarding 32 countries and 3
years (2005,2006 and 2007)
- The fit of the model is poor because of the short time series for many countries,
regarding mostly the historical data. The statistics from 1994 onwards seem to be
available for almost all the countries of ESS.
Model 6: ctctctcct eHRSTGERDaEPO )ln()ln( Provides estimates mostly for 2009 2008 and 2007
The goodness of fit statistics are good in general
The variable HRST seems that constitutes a good explanatory variable for the model,
despite the fact that the time series mostly provides data between 1994-2009
HRST data are of better quality than RECH regarding the time series (longer and
most recent data availability)
Reveals decrease in patenting activity for some countries
- Provides good estimates only for 12/96 testing cases regarding 32 countries and 3
years (2005,2006 and 2007)
4. Conclusions
Regarding the issue of timeliness in patent statistics it seems that through the application of
proper nowcasting techniques this can be eliminated. The recent researches have all came to
the same conclusion that “there is no a unique nowcasting method that can provide the most
accurate nowcasts for a whole set of countries”. This happens because:
Each country has its own profile in patenting activity
For each country the availability of data for the auxiliary variables that are used in
Trend Models, Transfer Rate Models or even in Econometric models differs.
Focusing on the use of the econometric models, it seems that:
Predictions can be derived for some countries even for 2010
12
Estimates for 2008,2009 can be obtained relatively ease for the majority of the
countries
The econometric models can reveal the decrease in patenting activity between 2007
and 2009. However, it will be proved in the future if those predictions are accurate
and if the patenting activity was reduced due to the economic criss.
When using econometric variables that reflect GDP, GERD, researchers and human
resources in science and technology, econometric models can be produced only for
countries that belong to ESS1 and for some candidate countries, due to the
fact that data are available for those countries in Eurobase.
It seems that each one of the tested model works well for specific countries
The prediction ability of the model is directly related to the availability of data for each
country in relation to the independent variables that are used
The variable HRST seems that can support well the prediction ability of an
econometric model. The time series for this variable is still short, however taking
into account the fact that it is planned to be regularly available at annual level, it
seems that HRST can be used in future econometric models.
HRST works better than RECH because it has better timeliness and also covers that
availability of human resources within the broader area of Research and Technology
Of course further work on nowcasting models using information from R&D statistics can be
produced. Apart from exploring the contribution of variable HRST in future nowcasts could
also be explored the relationship between R&D expenditure by type (theoretical, applied, and
technical and patent activity at national level. Then could be used in the econometric models
the expenditures of R&D that are (if they) related mostly to patenting activity.
1 European Statistical System (EU27, EFTA countries)
13
Annex
Diagnostics for regression models 1 to 6 Diagnostics for Goodness of Fit (Models 1 to 6)
Model Member of Country
Root of Mean Square Error
Number of Parameters
in the Model
Degrees of Freedom
R‐squared Mallows C(p)
Akaike's information criterion
1 EU27 AT 72.05 3 21 0.95289 3 208.108
2 EU27 AT 0.11 3 21 0.93248 3 ‐102.481
3 EU27 AT 77.32 2 22 0.94316 2 210.617
4 EU27 AT 3 0 1
5 EU27 AT 70.26 3 7 0.93589 3 87.479
6 EU27 AT 0.08 3 7 0.91611 3 ‐48.065
1 EU27 BE 83.64 3 17 0.95688 3 179.812
2 EU27 BE 0.08 3 17 0.97479 3 ‐97.647
3 EU27 BE 81.29 2 18 0.95688 2 177.813
4 EU27 BE 3 0 1
5 EU27 BE 109.49 3 8 0.81988 3 105.805
6 EU27 BE 0.1 3 8 0.82931 3 ‐48.117
1 EU27 BG 2.6 3 12 0.81777 3 31.331
2 EU27 BG 0.33 3 12 0.74055 3 ‐30.618
3 EU27 BG 5.78 2 16 0.17833 2 65.068
4 EU27 BG 5.24 3 15 0.36722 3 62.366
5 EU27 BG 3.1 3 2 0.82836 3 12.74
6 EU27 BG 0.17 3 2 0.91421 3 ‐16.29
1 EFTA CH 260.51 3 5 0.88503 3 91.242
2 EFTA CH 0.13 3 5 0.8913 3 ‐30.743
3 EFTA CH 248.33 2 6 0.87464 2 89.935
4 EFTA CH 3 0 1
5 EFTA CH 3 0 1
6 EFTA CH 3 0 1
1 EU27 CY 2.54 3 4 0.75849 3 15.14
2 EU27 CY 0.36 3 4 0.76591 3 ‐12.053
3 EU27 CY 4.05 2 7 0.21325 2 26.9
4 EU27 CY 3.61 3 4 0.51345 3 20.043
5 EU27 CY 3.49 3 3 0.51328 3 16.837
6 EU27 CY 0.29 3 3 0.67377 3 ‐13.121
1 EU27 CZ 7.49 3 7 0.95635 3 42.709
2 EU27 CZ 0.15 3 7 0.94656 3 ‐35.474
3 EU27 CZ 6.98 2 10 0.96041 2 48.458
4 EU27 CZ 8.15 3 7 0.94833 3 44.395
5 EU27 CZ 9.36 3 4 0.88549 3 33.398
6 EU27 CZ 0.11 3 4 0.88492 3 ‐28.938
1 EU27 DE 1640.55 3 11 0.8924 3 209.902
2 EU27 DE 0.1 3 11 0.88933 3 ‐61.306
3 EU27 DE 1846.13 2 22 0.88849 2 362.912
4 EU27 DE 1 0
5 EU27 DE 1302.91 3 7 0.91615 3 145.88
6 EU27 DE 0.08 3 7 0.91385 3 ‐48.907
14
1 EU27 DK 47.35 3 20 0.97888 3 180.231
2 EU27 DK 0.09 3 20 0.98067 3 ‐105.962
3 EU27 DK 46.3 2 21 0.9788 2 178.321
4 EU27 DK 39.64 3 9 0.98696 3 90.863
5 EU27 DK 62.14 3 7 0.91392 3 85.021
6 EU27 DK 0.08 3 7 0.9153 3 ‐47.521
1 EU27 EE 2.08 3 4 0.39556 3 12.339
2 EU27 EE 0.24 3 4 0.53107 3 ‐17.747
3 EU27 EE 1.87 2 5 0.3897 2 10.406
4 EU27 EE 2.08 3 4 0.3955 3 12.339
5 EU27 EE 2.05 3 4 0.41216 3 12.144
6 EU27 EE 0.26 3 4 0.44943 3 ‐16.624
1 EU27 EL 12.61 3 3 0.7679 3 32.253
2 EU27 EL 0.21 3 3 0.82304 3 ‐16.657
3 EU27 EL 8.43 2 10 0.90342 2 52.975
4 EU27 EL 8.29 3 3 0.9388 3 27.228
5 EU27 EL 11.9 3 3 0.79337 3 31.555
6 EU27 EL 0.21 3 3 0.82523 3 ‐16.731
1 EU27 FI 142.55 3 20 0.91959 3 230.932
2 EU27 FI 0.17 3 20 0.96562 3 ‐79.756
3 EU27 FI 148.62 2 21 0.90822 2 231.971
4 EU27 FI 1 0
5 EU27 FI 59.73 3 5 0.86654 3 67.678
6 EU27 FI 0.05 3 5 0.87882 3 ‐46.871
1 EU27 FR 312.2 3 21 0.96716 3 278.491
2 EU27 FR 0.07 3 21 0.95968 3 ‐126.304
3 EU27 FR 539.54 2 22 0.89726 2 303.866
4 EU27 FR 487.17 3 9 0.88272 3 151.075
5 EU27 FR 489.53 3 8 0.84233 3 138.753
6 EU27 FR 0.08 3 8 0.81345 3 ‐52.215
1 CANDIDATE HR 3 0 1 ‐67.397
2 CANDIDATE HR 3 0 1 ‐78.819
3 CANDIDATE HR 5.43 2 1 0.51013 2 10.852
4 CANDIDATE HR 3 0 1
5 CANDIDATE HR 3 0 1 ‐62.025
1 EU27 HU 18.46 3 11 0.79226 3 84.257
2 EU27 HU 0.26 3 11 0.70089 3 ‐35.121
3 EU27 HU 19.3 2 16 0.69117 2 108.441
4 EU27 HU 18.31 3 15 0.73924 3 107.396
5 EU27 HU 15.35 3 5 0.82508 3 45.937
6 EU27 HU 0.19 3 5 0.74703 3 ‐24.282
1 EU27 IE 29.24 3 7 0.79862 3 69.941
2 EU27 IE 0.15 3 7 0.86324 3 ‐36.153
3 EU27 IE 21.23 2 22 0.93332 2 148.561
4 EU27 IE 3 0 1
5 EU27 IE 21.94 3 7 0.91596 3 64.2
6 EU27 IE 0.09 3 7 0.96138 3 ‐44.77
1 CANDIDATE IS 4.32 3 17 0.89549 3 61.281
2 CANDIDATE IS 0.33 3 17 0.91368 3 ‐41.439
3 CANDIDATE IS 4.6 2 18 0.87443 2 62.952
4 CANDIDATE IS 7.69 3 5 0.63278 3 34.883
15
5 CANDIDATE IS 8.65 3 2 0.117 3 22.992
6 CANDIDATE IS 0.32 3 2 0.10033 3 ‐10.023
1 EU27 IT 227.16 3 21 0.9634 3 263.226
2 EU27 IT 0.09 3 21 0.97029 3 ‐112.292
3 EU27 IT 399.41 2 22 0.88146 2 289.431
4 EU27 IT 189.96 3 5 0.88908 3 86.189
5 EU27 IT 206.32 3 8 0.94094 3 119.744
6 EU27 IT 0.06 3 8 0.93658 3 ‐57.862
1 EU27 LT 3.52 3 7 0.60967 3 27.62
2 EU27 LT 0.87 3 7 0.36069 3 ‐0.307
3 EU27 LT 3.37 2 8 0.59135 2 26.078
4 EU27 LT 3.01 3 6 0.75102 3 22.213
5 EU27 LT 3.45 3 4 0.74329 3 19.44
6 EU27 LT 0.73 3 4 0.68298 3 ‐2.393
1 EU27 LU 3 0 1
3 EU27 LU 13.91 2 1 0.66602 2 16.498
4 EU27 LU 1 0
5 EU27 LU 3 0 1
6 EU27 LU 3 0 1
1 EU27 LV 1.69 3 6 0.78057 3 11.83
2 EU27 LV 0.41 3 6 0.86027 3 ‐13.744
3 EU27 LV 1.59 2 7 0.77511 2 10.051
4 EU27 LV 1.71 3 6 0.7758 3 12.023
5 EU27 LV 1.78 3 4 0.72592 3 10.177
6 EU27 LV 0.41 3 4 0.68099 3 ‐10.395
1 EU27 MT 3 0 1
3 EU27 MT 1.11 2 1 0.05529 2 1.33
4 EU27 MT 3 0 1
5 EU27 MT 3 0 1
6 EU27 MT 3 0 1
1 EU27 NL 274.11 3 21 0.92613 3 272.245
2 EU27 NL 0.13 3 21 0.92536 3 ‐93.622
3 EU27 NL 324.41 2 22 0.8916 2 279.449
4 EU27 NL 2 0 1
5 EU27 NL 414.63 3 6 0.65297 3 110.844
6 EU27 NL 0.13 3 6 0.6998 3 ‐33.96
1 EFTA NO 36.02 3 13 0.9247 3 117.369
2 EFTA NO 0.16 3 13 0.94837 3 ‐56.848
3 EFTA NO 34.9 2 14 0.92386 2 115.546
4 EFTA NO 33.29 3 3 0.88856 3 43.904
5 EFTA NO 25.6 3 3 0.53722 3 40.752
6 EFTA NO 0.07 3 3 0.58619 3 ‐30.192
1 EU27 PL 14.78 3 7 0.8816 3 56.294
2 EU27 PL 0.23 3 7 0.9173 3 ‐27.025
3 EU27 PL 25.95 2 16 0.42562 2 119.098
4 EU27 PL 19.83 3 8 0.77649 3 68.214
5 EU27 PL 15.22 3 5 0.88 3 45.807
6 EU27 PL 0.25 3 5 0.86175 3 ‐19.669
1 EU27 PT 5.02 3 7 0.92475 3 34.716
2 EU27 PT 0.13 3 7 0.94607 3 ‐38.769
3 EU27 PT 5.77 2 21 0.90158 2 82.559
16
4 EU27 PT 4.99 3 20 0.92991 3 76.749
5 EU27 PT 6.33 3 8 0.88622 3 43.083
6 EU27 PT 0.14 3 8 0.94521 3 ‐41.017
1 EU27 RO 2.59 3 4 0.88466 3 15.429
2 EU27 RO 0.27 3 4 0.82907 3 ‐16.444
3 EU27 RO 4.69 2 12 0.37773 2 45.102
4 EU27 RO 3.37 3 9 0.69299 3 31.734
5 EU27 RO 3.75 3 5 0.71836 3 23.403
6 EU27 RO 0.4 3 5 0.54706 3 ‐12.387
1 EU27 SE 219.05 3 10 0.88758 3 142.711
2 EU27 SE 0.15 3 10 0.90281 3 ‐47.331
3 EU27 SE 222.43 2 11 0.8725 2 142.348
5 EU27 SE 114.03 3 2 0.35318 3 48.783
6 EU27 SE 0.06 3 2 0.31961 3 ‐27.456
1 EU27 SI 14.98 3 9 0.79899 3 67.507
2 EU27 SI 0.28 3 9 0.83898 3 ‐28.259
3 EU27 SI 14.29 2 10 0.79671 2 65.642
4 EU27 SI 8.97 3 9 0.92796 3 55.194
5 EU27 SI 10.11 3 6 0.92199 3 43.991
6 EU27 SI 0.2 3 6 0.92526 3 ‐26.992
1 EU27 SK 4.99 3 8 0.66689 3 37.869
2 EU27 SK 0.26 3 8 0.7736 3 ‐27.001
3 EU27 SK 7.6 2 9 0.1317 2 46.408
4 EU27 SK 7.89 3 8 0.16846 3 47.932
5 EU27 SK 7.26 3 4 0.39845 3 29.839
6 EU27 SK 0.37 3 4 0.42083 3 ‐11.73
1 EU27 SP 77.59 3 21 0.94937 3 211.665
2 EU27 SP 0.18 3 21 0.96503 3 ‐80.492
3 EU27 SP 80.87 2 22 0.94237 2 212.771
4 EU27 SP 68.92 3 12 0.96619 3 129.641
5 EU27 SP 56.61 3 8 0.96291 3 91.293
6 EU27 SP 0.08 3 8 0.96366 3 ‐52.481
1 CANDIDATE TR 17.44 3 12 0.80048 3 88.416
2 CANDIDATE TR 0.73 3 12 0.80855 3 ‐6.802
3 CANDIDATE TR 16.8 2 13 0.79936 2 86.5
4 CANDIDATE TR 17.83 3 6 0.82175 3 54.203
1 EU27 UK 346.12 3 19 0.89984 3 260.033
2 EU27 UK 0.08 3 19 0.89711 3 ‐106.428
3 EU27 UK 339.19 2 20 0.89874 2 258.272
5 EU27 UK 346.08 3 7 0.87159 3 119.367
6 EU27 UK 0.07 3 7 0.89226 3 ‐51.579
17
Predictions by Model for 2005 2006 2007
Year Country
Actual Values
Estimations of Total Patents to EPO Absolute Difference (Observed‐Estimated value)
EPO Total
Model 1
Model 2
Model 3
Model 4
Model 5
Model 6
Obs‐Est1
Obs‐Est2
Obs‐Est3
Obs‐Est4
Obs‐Est5
Obs‐Est6
2007 BE 1471.53 1645 1630 1645 1518 1718 1769 173.64 158.93 173.15 46.42 246.88 297.47
2006 BE 1433.69 1530 1509 1529 1479 1641 1656 96.06 74.89 95.45 45.38 207.3 222.25
2005 BE 1416.17 1429 1404 1428 1539 1560 1549 13.01 12.52 12.31 123.18 144.21 132.46
2007 BG 29.02 29.8 26.61 10.16 12.44 34.78 47.29 0.78 2.41 18.86 16.58 5.76 18.27
2006 BG 27.13 25.27 22.4 9.99 12.34 27.08 30.01 1.86 4.73 17.14 14.79 0.05 2.88
2005 BG 23.82 21.88 19.29 9.85 12.09 23.52 25.48 1.94 4.53 13.97 11.73 0.3 1.66
2007 CZ 162.31 256.7 335.3 223.4 223.5 211.7 184.2 94.36 173.03 61.12 61.23 49.35 21.91
2006 CZ 150.21 233.3 313.1 198.6 197.9 192.3 176 83.09 162.85 48.41 47.72 42.08 25.79
2005 CZ 106.42 171.1 201.9 154.8 155.9 144.6 133.3 64.66 95.43 48.33 49.52 38.2 26.92
2007 DK 1057.03 1255 1147 1256 1221 1275 1265 198.25 90.01 199.03 163.56 218.19 208.17
2006 DK 1051.46 1175 1077 1175 1114 1100 1085 123.36 25.55 123.05 62.07 48.84 33.59
2005 DK 1093.78 1104 1037 1105 1035 1034 1027 10.51 56.84 10.74 58.55 60 66.51
2007 DE 23929.2 27844 29506 24230 21735 28712 31520 3914.6 5577.1 300.33 2194.4 4782.6 7590.6
2006 DE 23380.6 26017 27104 23134 21735 26762 28556 2636.8 3723.5 246.13 1645.8 3381.5 5175.3
2005 DE 23409.2 24053 24636 21903 21735 26391 28354 644 1226.9 1506.4 1674.4 2981.4 4944.3
2007 EE 23.38 17.67 14.47 16.32 19.24 16.38 13.6 5.71 8.91 7.06 4.14 7 9.78
2006 EE 20.22 16.29 15.74 14.7 17.76 14.56 12.66 3.93 4.48 5.52 2.46 5.66 7.56
2005 EE 6.37 11.75 11.42 11.35 12.67 10.55 10.34 5.38 5.05 4.98 6.3 4.18 3.97
2007 IE 288.2 292 287 375.9 437.3 360.8 319 3.83 1.23 87.7 149.08 72.6 30.82
2006 IE 271.02 284.1 280.6 343.4 397.1 326.3 299.4 13.04 9.61 72.36 126.06 55.27 28.39
2005 IE 261.64 264.6 262.6 315.4 332.5 311 294.2 2.91 0.92 53.77 70.83 49.32 32.56
2007 EL 109.38 101.7 108.7 97.45 98.2 95.47 7.69 0.68 11.93 11.18 13.91
2006 EL 103.7 94.38 99.39 91.08 90.3 88.66 9.32 4.31 12.62 13.4 15.04
2005 EL 109.74 88.23 91.41 86.12 100.8 87.27 85.82 21.51 18.33 23.62 8.99 22.47 23.92
2007 SP 1450.9 1483 1856 1753 1487 1523 1594 32.44 404.89 302.09 36.13 72.59 142.9
2006 SP 1322.35 1355 1641 1544 1355 1438 1519 33.1 319 221.56 32.2 115.88 196.28
2005 SP 1332.94 1217 1422 1322 1225 1298 1355 115.93 88.93 10.58 107.54 35.05 22.02
2007 FR 8421.49 9874 9649 8432 9762 9478 9699 1452.3 1227.7 10.54 1340.1 1056.7 1277.2
2006 FR 8274.99 9299 9122 8149 9532 9501 9686 1024.3 847.22 125.63 1257.2 1225.6 1410.9
2005 FR 8206.34 8887 8718 7810 8656 9214 9341 680.3 511.31 395.86 449.61 1007.5 1135
2007 IT 5107.1 4834 5151 5226 5546 5640 273.54 44.16 118.83 438.96 533.39
2006 IT 4909.13 4710 4908 4790 4470 5122 5175 199.53 0.86 119.37 438.86 212.87 265.98
2005 IT 4811.88 4588 4686 4406 4296 4718 4749 224.28 125.48 406.08 516.06 93.61 62.62
2007 CY 8.92 8.78 32.6 12.84 ‐5.71 2.74 8.33 0.14 23.68 3.92 14.63 6.18 0.59
2006 CY 7.33 6.43 16.8 11.63 2.92 0.75 5.28 0.9 9.47 4.3 4.41 6.58 2.05
2005 CY 16.04 5.57 10.56 10.56 7.91 1.04 4.38 10.47 5.48 5.48 8.13 15 11.66
2007 LV 19.17 31.34 69.8 26.97 27.24 30.98 40.74 12.17 50.63 7.8 8.07 11.81 21.57
2006 LV 16.53 29.27 64.58 23.93 24.19 28.69 45.19 12.74 48.05 7.4 7.66 12.16 28.66
2005 LV 18.49 17.06 25.94 14.85 15.03 17.09 22.23 1.43 7.45 3.64 3.46 1.4 3.74
2007 LT 8.17 22.93 11.44 21.48 29.54 38.83 185.6 14.76 3.27 13.31 21.37 30.66 177.47
2006 LT 9.67 18.36 9.5 17.11 21.23 28.23 71.09 8.69 0.17 7.44 11.56 18.56 61.42
2005 LT 8.93 14.12 7.83 13.62 17.55 20.46 33.71 5.19 1.1 4.69 8.62 11.53 24.78
2007 LU 109.6 227.8 113.8 150.6 86.67 140 143.6 118.2 4.18 40.99 22.93 30.43 33.97
2006 LU 103.87 60.05 107.7 141.6 86.67 129.4 131.3 43.82 3.8 37.74 17.2 25.48 27.4
18
2005 LU 96.47 234.2 100.4 112.4 86.67 119.4 120.1 137.73 3.93 15.91 9.8 22.95 23.58
2007 HU 172.67 174 162.3 181.9 176.5 180.3 196.7 1.3 10.34 9.22 3.79 7.62 24.07
2006 HU 161.43 155.2 150 169.5 165.6 175.8 189.8 6.23 11.45 8.08 4.21 14.35 28.33
2005 HU 135.13 154.3 145.5 159.4 154.8 161 169 19.19 10.39 24.27 19.65 25.84 33.86
2007 MT 8.34 ‐17.9 11.32 4.27 3.04 13.1 22.21 26.23 2.98 4.07 5.3 4.76 13.87
2006 MT 7.65 ‐9.59 9.2 4.28 7.1 10.83 15.22 17.24 1.55 3.37 0.55 3.18 7.57
2005 MT 11.25 ‐1.87 7.22 4.39 6.37 8.09 9.17 13.12 4.03 6.86 4.88 3.16 2.08
2007 NL 3655.83 4377 4185 4117 3442 4270 4480 721.23 529.6 461.47 213.69 614.41 823.83
2006 NL 3602.4 4044 3961 4043 3442 4084 4279 441.19 359.02 440.65 160.26 481.83 676.16
2005 NL 3395.03 3808 3747 3864 3442 4020 4160 413.4 352.02 468.84 47.09 624.93 764.97
2007 AT 1797.12 1816 1674 1724 1783 1851 1950 19.13 123.47 73.06 14.3 53.45 152.5
2006 AT 1680.11 1685 1593 1605 1640 1712 1775 4.42 87.14 75.42 40.05 31.85 94.68
2005 AT 1475.93 1611 1566 1529 1623 1669 135.1 89.67 53.45 146.65 192.98
2007 PL 145.52 153 206.8 103.4 61.85 267 1087 7.49 61.3 42.17 83.67 121.43 941.4
2006 PL 137.76 140.3 171.2 86.62 75.41 226.3 611.4 2.52 33.39 51.14 62.35 88.51 473.68
2005 PL 122.03 117.5 130.1 78.17 94.11 185.1 347.7 4.5 8.06 43.86 27.92 63.11 225.62
2007 PT 121.22 25.96 78.45 93.03 70.2 114.3 142.2 95.26 42.77 28.19 51.02 6.92 20.98
2006 PT 106.72 41.34 70.06 74.16 61.59 90.61 102.5 65.38 36.66 32.56 45.13 16.11 4.21
2005 PT 115.33 59.69 65.05 55.3 52.98 66 66.86 55.64 50.28 60.03 62.35 49.33 48.47
2007 RO 21.08 60.99 82.51 47.63 83.6 88.84 158.7 39.91 61.43 26.55 62.52 67.76 137.64
2006 RO 19.32 44.01 52.27 30.38 50.21 53.79 67.27 24.69 32.95 11.06 30.89 34.47 47.95
2005 RO 28.68 32.74 35.46 20.7 31.59 34.95 36.1 4.06 6.78 7.98 2.91 6.27 7.42
2007 SI 103.47 135.7 164.2 137.3 103.4 151.8 232.6 32.26 60.75 33.83 0.07 48.3 129.15
2006 SI 96.51 126.8 154.1 130.4 104.8 147.1 214.7 30.3 57.6 33.91 8.28 50.55 118.16
2005 SI 106.58 100.3 103.6 101.2 82.37 122.6 142.5 6.28 2.97 5.38 24.21 15.99 35.95
2007 SK 42.25 47.84 55.2 26.59 21.45 41.29 48.35 5.59 12.95 15.66 20.8 0.96 6.1
2006 SK 39.56 37.59 40.72 22.37 17.4 39.02 46.69 1.97 1.16 17.19 22.16 0.54 7.13
2005 SK 30.7 31.57 32.89 19.73 17.48 33.75 37.31 0.87 2.19 10.97 13.22 3.05 6.61
2007 FI 1323.25 1814 1714 1862 1378 1422 1430 490.29 390.38 539.19 54.7 98.31 107.08
2006 FI 1306.53 1695 1705 1713 1378 1390 1395 388.93 397.98 406.77 71.42 83.89 88.4
2005 FI 1293.83 1625 1697 1624 1378 1366 1369 330.74 403.61 330.41 84.12 72.26 74.91
2007 SE 2719.05 2306 1922 2473 2273 2278 412.67 797.21 245.78 446.24 441.31
2006 SE 2533.7 2350 2048 2432 2220 2226 183.74 485.47 102.13 313.76 307.78
2005 SE 2343.55 2127 1896 2222 2265 2260 216.65 447.46 121.94 78.19 83.23
2007 UK 5422.41 6699 6419 6779 6427 6361 1276.4 996.65 1356.1 1004.4 938.8
2006 UK 5426.41 6375 6192 6401 6091 6077 948.38 765.93 974.13 664.76 650.77
2005 UK 5312.7 6056 5953 6047 5810 5825 743.54 640.7 734.38 497.03 512.74
2007 IS 27.87 60.06 66.5 50.98 43.03 23.94 24.08 32.19 38.63 23.11 15.16 3.93 3.79
2006 IS 29.17 53.45 62.07 50.67 43.94 31.16 28.75 24.28 32.9 21.5 14.77 1.99 0.42
2005 IS 29.84 51.45 56.63 46.21 39.82 29.56 28.01 21.61 26.79 16.37 9.98 0.28 1.83
2007 NO 514.69 559.5 601.1 554.2 380 356.6 359.8 44.84 86.44 39.5 134.74 158.14 154.94
2006 NO 471.47 497.6 515.8 481.5 . 362.5 362.7 26.1 44.34 9.99 108.98 108.73
2005 NO 480.5 449.4 460.8 436 353.8 362.3 361.5 31.14 19.69 44.48 126.71 118.22 119
2007 CH 3223.79
2006 CH 3099.37
2005 CH 3095.94
2007 HR 32.02 149.3 477.3 31.99 89.98 135.3 117.32 445.31 0.03 57.96 103.28
2006 HR 34.52 146.5 579.7 37.15 74.86 220.7 111.99 545.17 2.63 40.34 186.14
2005 HR 32.88 93.73 158.5 35.63 86.62 144.5 60.85 125.63 2.75 53.74 111.58
2007 TR 220.13 228.7 451.4 238.6 199.6 8.59 231.24 18.47 20.54
2006 TR 186.38 155.5 314.6 157.9 148.1 30.85 128.25 28.47 38.3
20
References
Dernis, H., D. Guellec and B. van Pottelsberghe (2001). .Using Patent Counts for Cross-
country Comparisons of Technology Output., STI Review No. 27, OECD, Paris
Dehon, C. and B. van Pottelsberghe (2003).Implementing a Forecasting Methodology for
PCT Applications at WIPO, paper presented at WIPO-OECD Workshop on Statistics in the
Patent Field, Geneva.
Joutz, F. (2003). Forecasting USPTO Patent Application Filings, paper presented at WIPO
OECD Workshop on Statistics in the Patent Field, Geneva.
Dernis, H. and M. Khan (2004), "Triadic Patent Families Methodology", STI Working Paper 2004/2, OECD, Paris Nicolas, M. and P. Hingley (2003). .Forecasting the Number of European Patent Applications
at the European Patent Office., paper presented at WIPO-OECD Workshop on Statistics in
the Patent Field, Geneva.
Khan, M. And H. Dernis (2005).Impact of Patent Cooperation Treaty Data on EPO Patent
Statistics and Improving the Timeliness of EPO Indicators., OECD, STI Working Paper
2005/9.
Marek Szajt, DYNAMIC ECONOMETRIC MODELS, Vol. 9 – Nicolaus Copernicus University
– Toruń – 2009Technical University of Częstochowa, 2009
OECD (2007), “Compendium of Patent Statistics”, OECD
OECD (2005),“Compendium of Patent Statistics”, OECD, Paris.
Hélène Dernis (2007), NOWCASTING PATENT INDICATORS, Statistical Analysis of
Science, Technology and Industry, OECD, STI WORKING PAPER, 2007/3
Harhoff, D. (2001). .Research Program Announced by the European Patent Office:
Improvements of Methods for Forecasting Patent Filings., background information document,
EPO.
Hausman, J.A., B.H. Hall, and Z. Griliches (1984). .Econometric Models for Count Data with
an Application to the Patents-R&D Relationship,. Econometrica 52: 909-938.