yneasurement of output of research and experimental...

yneasurement of output of research and experimental

development :

a review paper.

by Christopher Treeman

Director, Science Policy Research Unit University of Sussex

United Kingdom

unesco

I ’

Printed in 1969 in the Workshops of the United Nations Educational, Scientific and Cultural Organization,

Place de Fontenoy, Paris 7e, France

s-r/s/ 16 COM.69/XVI- 16 A Printed in France

0 Unesco 1970

.

PREVIOUS TITLES IN THIS SERIES

No. 1 - Film and cinema statistics. (with resumk in French and Spanish) 0 ut of print.

No. 2 - Book production 1937-1954 and translations 1950-1954. (Paris, 1958) Production de livres 1937-1954 et traductions 1950-1954. (Paris, 1958)

No. 3 - Statistics on libraries. (Paris, 1959) Statistiques SW les biblioth&ques. (Paris, 1959)

No. 4 - Statistics of newspapers and other periodicals. (Paris, 1959) Statistiques sut les journaux et autres gtiodiques. (Paris, 1959)

No. 5 - Statistics on special education. (Paris, 1960) Rapport statistique sur I’enseignement spicial. (Paris, 1960)

No. 6 - Requirements and resources of scientific and technical personnel in ten Asian countries- (Paris. 1960) Besoins et ressources de dix pays d’Asie en personnel scientifique et technique. (Paris, 1961)

No. 7 - Pre-school education. (Paris, 1963) L’iducation prescolaire. (Paris, 1963)

No. 8 -Statistics on radio and television 1950-1960. (Paris, 1963) Statistiques de la radiodiffusion et de la tClivision 1950- 1960. (Paris, 1963)

No. 9 - Methods of estimating the demand for specialists and of planning specialized training within the USSR. (Paris, 1964) M.Gthodes d’Pvaluation des besoins en sp&zialistes et de planification de la formation spicialisCe en URSS. (Paris, 1964)

No. 10 - Estimating future school enrolment in developing countries,

No.

No.

No.

a manual if methodology. (Paris, 1966) Estimation des effectifs scolaires futurs dans les pays en voie de diveloppement - manuel de mCthodologie. (Paris, 1967)

1 - Methods of analysing educational outlay. (Paris. 1966) Methodes d’analyse des depenses d’enseignement. (Paris, 1967)

2 - Methods of long-term projection of requirements for and supply of qualified manpower. (Paris, 1967) MCthodes de projection B long terme de l’offre et de la demande de maiwd’ceuvre qualifiCe. (Paris, 1968)

1 3 - Statistics of educ‘ation in developing countries. An introduction to their collection and presentation. (Paris, 1968) Les statistiques de I’iducation dans les pays en voie de d.+veloDDement - Comment les rassembler et les prisenter. (Paris,.i968) . ̂ _^

No. 14 - International developments of education expenditure 1Y5U- 1965. (Paris, 1969) L’Cvolution internationale des dPpenses d’iducation entre 1950-1965. (Paris, 1969)

No. 15 - The measurement of scientific and technological activities. (Paris, 1969) La mesure des activitCs scientifiques et techniques. (Paris, 1969)

In the press

No. 17 - World Summary of Statistics on Science and Technology/ Statistiques de la science et de la technologie : Apetgu mondial. (Bilingual publication/ Cdition bilingue) (Paris, 1969)

PREFACE

This study on the Measurement of output of research and experimental development was originally prepared as a working document for the first meeting of the Unesco/Economic Commission for Europe Working Group on Statistics of Science and Tech- nology which met in Geneva in June 1969. The favourable reception given to this paper, prepared by Mr. C. Freeman, Director of the Science Policy Research Unit of the University of Sussex, United Kingdom, has led Unesco to make it available to a wider public by publishing it in the series Statistical Reports and Studies.

In the same series Unesco previously presented a document on The measurement of scientific and technological activities (Unesco, Statistical Reports and Studies, ST/S/15, Paris 1969)bythe same author, designed to add to the literature on the classification and measurement of the inputs of human and financial resources into scientific and technological activities. The present study, a complement to the first, reviews the far more complicated subject of the measurement of the output of research and experimental development (R and D) .

Statistical measures of the output of R and D are needed in order to reach a fullerunderstanding

,of the innovation process anil its impact on the economy and to allow for amorertitional allocation of limited resources to compet;ingRsnBD activities. However, many theoretic& and :practical problems involved in the compilation of such statistics remain to be solved and it cannot be expected that solutions will be reached in the near future in view of the many difficulties still existing in the far more advanced area of statistics of R and D inputs. Although no immediate solution may be found, a review of the results achieved so far in measuring the output of R and D can nevertheless contribute to a better understanding of the problem and stimulate further discussion.

The ideas expressed in the present paper are those of the author and do not necessarily represent the views of Unesco.

The designations employed and the presentation of the material in this publication do not imply the expressionof any opinion whatsoever on the part of the Unesco Secreiariat concerning the legal status of any country or territory, or of its authorities, or c.oncerning the delimitations of the frontiers of any country or territory.

CONTENTS

Chapter I

Chapter II

Chapter III

Chapter IV

Chapter V

Chapter VI

Tables 1.

2.

INTRODUCTION . . . . . . . . . . . . . . The growth of research activities . . . . . - . . Professionalization and specialization of scientific research The measurement of inputs into research and experimental development activities . . . . . . . . . . . . Input measurement and output measurement . . . . .

SOME PROBLEMS OF OUTPUT MEASUREMENT . . . . Theoretical objections to any scheme of output measurement R and D output as a flow of information and innovation . . “Accidental” factors affecting R and D output . _ . . . Use of input measures as a surrogate for output . . . .

THE USE OF SCIENTIFIC PAPERS IN MEASUREMENT OF RESEARCH OUTPUT . . . . . . . . . . . Numbers of papers and other output measures . . . . _ The use of output measures in the sociology of science . . The need to relate “output” to “input” measures . . . . The need to relate output measures to institutional factors and social changes . . . . . . . . . . . . .

THE USE OF PATENT STATISTICS FOR MEASUREmNT . Patents, inventive activity and “applied” research . . . Limitations of patent statistics . . . . . . . . . Applications of patent statistics . . . . . . . . . “Science” and “technology” in relation to papers and patents

INNOVATIONS AND THE R AND D SYSTEM . . . . . Invention and innovation . . * . - .- . - . - . C&t-benefit analysis of innovkions . ‘.; . . . . . . Indirect measures of the output of innovations . . . _ . Output budgeting . . . . . . . . . . . . . .

SUMMARY AND CONCLUSIONS . . . . . . . . .

APPENDIX

.

Page

7 7 7

8 8

10 10 10 11 12

14 14 15 16

16

20 20 21 22 23

25 25 25 27 28

31

The flow of ideas through the stages of research, invention and development to application . . . . . . . . . . . . . . . . . 34

Schematic table showing numbers of authors of various degrees of productivity (in papers per lifetime) and numbers of papers so produced . . . . . . . . . . . . . . . . . . . . 35

Tables 3.

4.

5.

6.

7.

a.

9.

10.

Origin of science abstracts . . . . . . . . . . . . . . . . 36

Percentage of women on the professional staffs of higher educational institutions in 1947 compared with percentage of articles in professional journals contributed by women in1940 and 1960 . . . ~ . . . . . . . . . . . . . . . 37

Distribution of research publications and their origin . . . . . . . - 38

Institutions from which more than 100 papers were published during the year under analysis . . . . . . . . . . . . . . . 39

Number of discoveries in the medical sciences by nations, 1800-1926 . . . . . . . . . . . . . . . . . . . . . 39

Patents granted to foreigners as a percentage of total patents, 195’7-1961 . . . . . . . . . . . . . , . . . . . . . 40

Approved enterprise-to-enterprise agreements for the transfer of technology in Japan . . . . . . . . . . . . . . 41

Outline of a proposed output budget covering Home Office responsibilities . . . . . . . . , . . . . . . . . . . 42

-. - -- _--- ___... -_~.. _--..

Chapter I

INTRODUCTION

THE GROWTH OF RESEARCH ACTIVITIES

One of the outstanding features of the Twentieth century has been the rapid growth of scientific and technological research and a range of associated activities. The growth of these activities and their impact on economic, military and social policy has been variously described as the “Research revolution”, the “Scientific revolution” or the “Techno- logical revolution”. One result of this “revolution”, however it may be defined or described, has been a demand for statistical measurement of the resources devoted to the generation of new science and technology and of the efficiency with which they are being used.

It may well be true, as Derek J. de Solla Price has suggested, (1) that scientific activities were growing very rapidly already in the Eighteenth and Nineteenth centuries. In this perspective, the Twentieth century growth may be regarded simply as the continuation of a long-term trend. But the absolute scale of the resources committed before 1900 was so small, even in the United States, that it amounted to much less than 0.1% of GNP. Today almost all European countries devote resources to research and experimental development which are the equivalent of between 1% and 3% of GNP, (3) while many developing countries are already spending the equivalent of 0. 1% or more of their GNP.

In these circumstances it was inevitable that there should be increasing concern with the efficiency of the research-innovation system. In some sense, this implies measurement of inputs and outputs of the process. As long as governments or enterprises were spending only very small sums on scientific research, they could afford to regard this outlay in a very similar way to patronage of the arts, using “prestige” criteria rather than attempting to assess “efficiency”. But it is one thing to endow an occasional eminent scientist; it is quite another to maintain laboratories regularly employ- ing thousands of scientists and technicians on a continuous basis. The increased scale of scientific activities led inexorably to an increased concern with their effectiveness.

PROFESSIONALIZATION AND SPECIALIZATION OF SCIENTIFIC RESEARCH

The larger scale of scientific research was associated with its increased professionalization. A high proportion of scientific and inventive work in the Eighteenth and Nineteenth centuries was conducted on a part-time or amateur basis. This proportion is, of course, still significant today, particularly with respect to inventive activity, but more characteristic of the second half of the Twen- tieth century is the full-time professional research scientist or engineer and the specialized research institute. It is sometimes forgotten that, even in the Nineteenth century, the elite scientific organizations, such as the Royal Society, found it difficult to accept the idea of professional status for scientists. Ben-David has pointed out that: “Aca- demic appointments . . . were regarded as honours rather than careers and turning science into an occupation would have seemed something like a sacrilege. “(3) The very word “scientist” only came into general use in the middle of the century. Even thenthe main awards to scientists were seldom on the professional basis of full costs, includin instrumentation, overheads and supporting staff. ( 47 It was only towards the end of the century that a few industrial firms began to set up small professional research laboratories on a permanent footing. Thus, Whitehead was justified in describing the greatest invention of the Nineteenth century as the “method of invention itself”, (5) in the sense of a network of full-time research organizations.

Although the professionalization of science proceeded rapidly in the industrialized countries in the early part of the Twentieth century, it was not until the 1930’s that the number of patents taken out by corporations in the United States exceeded those taken out by individuals, (6) and it was not until 1953 that the first official government survey was made of the total resources employed in professional research and experimental development in the country.

It would indeed have been difficult to survey and measure scientific research activities before

7

they reached this fairly advanced level of specialization and professionalization. Once it had been reached, however, it became possible toundertake reasonably accurate surveys of the numbers of professional research and experimental development scientists and engineers in industry and in government and of the expenditures necessary to finance their activities. But, even now, one of the biggest difficulties in surveys of research and experimental development inputs is the problem of part-time research workers, and no country has satisfactorily

’ resolved this question with respect to university research.

THE MEASUREMENT OF INPUTS INTO RESEARCH AND EXPERIMENTAL DEVELOP- MENT ACTIVITIES

The first official government statistics were those published by the Soviet Union since the early 1930’s. (7) These related, however, to a range of scientific and technological activities somewhat wider than those now commonly defined as “research and experimental development”. @) Most other socialist countries now also publish annual statistics of scientific services, although without comparability in their coverage and definitions.

The first experimental attempt in a market economy to measure R and D inputs in all sectors (industry, government and universities) was made by J. D. Bernal in 1938-1939. (g) But Bernalhad to make use of very poor data for industry and an un- satisfactory breakdown of government expenditure. Industrial organizations had begun to publish figures of expenditure on industrial R and D in the United States and in the United Kingdom in the 1930’s, (16) but these were incomplete in their coverage and in- consistent in their definitions. It was not until the 1950’s that the National Science Foundation (NSF) in the United States resolved these problems by systematic comprehensive surveys in industrial and government organizations on consistent definitions.

Since the NSF began their regular annual surveys of R and D expenditures hnd manpower in 1953, many other OECD countries have followed suit. Unfortunately, they often did so on the basis of varying national definitions and concepts, so that international comparability was difficult to attain. The Directorate for Scientific Affairs of OECD took the initiative in attempting to standardize definitions and systems of measurement. The first Frascati Conference in 1963 agreed on a standard system of measurement, (11) and as a result of this OECD was able to undertake an experimental international comparison of a few countries(l3) and later a more

8

systematic comparison - the first international statistical year for research and development. (13)

Within the United Nations, Unesco has taken the lead in encouraging and systematizing the measurement of R and D inputs. (14) It has recently initiated attempts to reconcile definitions and concepts as between East and West European countries. (15) As with other similar statistical series, institutional differences between socialist and capitalist economies make this especially difficult and progress so far has been small. Unesco has also stimulated measurement of scientific and technological services in many of the developing countries, but here too there are major problems of scope and comparability of national statistics.

Nevertheless, it is not unreasonable to suppose that as an increasing number of countries gain experience of regular statistics of Rand D manpower and expenditure, and as international organizations become more familiar with the problems of comparison, it will be possible to use a fairly wide range of moderately accurate and comparable statistics of R and D inputs. Persistent efforts will be necessary to improve their accuracy and range.

INPUT MEASUREMENT AND OUTPUT MEASUREMENT

The position with regard to measurement of R and D outputs is completely different. There is no nationally agreed system of output measurement, still less any international system. Nor does it seem likely that there will be any such system for some time to come. At the most, it maybe hoped that more systematic statistics might become possible in a decade or two.

This paper, therefore, is not concerned to make proposals for international systems of R and D output measurement. It has the much more limited objective of reviewing briefly some of the experimental attempts at output measurement which have been made, of selecting those which appear to offer the hest future prospects, and of indicating ways in which international organizations might stimulate further experimental developments to the point where regular national and international series become feasible.

First it will be necessary, in Chapter II of this paper, to deal with some theoretical objections to the whole idea of output measurement and with the present treatment of R and D inputs in systems of national accounts. The next three sections review various attempts to develop systematic output measures for part of the R and D spectrum of activities, and in the final section suggestions are made for further experimental work.

NOTES

(1) Derek J. de Solla Price, Little Science, Columbia University Press, 1963

(2) OECD, International Statistical Year for Re- search and Development, Vols. 1 and 2, Paris 1967-1968

(3) J. Ben-David, “Scientific Productivity and Academic Organization in Nineteenth Century Medicine”, American Sociological Review, Decem- ber 1960, page 836

(4) R. MacLeod, The Institutionalization of Basic Research: the Government Grant Committee ofthe Royal Society, 1850-1914, Science Policy Research Unit, University of Sussex, to be published.

(5) A. N. Whitehead. Science and the Modern World, Pelican, 1937, page 120

(6) Jacob Schmookler, Invention and Economic Growth, Harvard University Press, 1966, page 26

(7) J-M Collette. “Recherche-developpement en URSS”, Cahiers de lIZSEA, Institute de Science Bconomique appliquge, August 196 2

(8) Unesco, The Measurement of Scientific and Technological Activities, 1969

(9) J. D. Bernal, The Social Function of Science, Routledge, 1939

(10) N. E. Terleckj, Research and Development: its Growth and Composition, National Industrial Conference Board, (NICB), 1963 Federation of British Industries, Surveys of In- dustrial Research

ill) OECD, Proposed Standard Practice for Sur- veys of Research and Development: The Measure- ment of Scientific and Technical Activities, DASIPDI62.47

(12) C. Freeman and A. Young, The Research and Development Effort in Western Europe, North America and the Soviet Union, OECD, Paris, 1965

(13) OECD, op. cit. (2)

(14) Unesco, Provisional Guide to the Collection of Science Statistics, COM/MD/3, Paris, 1968 Unesco, op. cit. (8)

(15) Unesco, op. cit. (8) idem, Questionnaire on statistics of research and experimental development effort, 1967 ( Unesco, STS/Q/SSl), Paris, July 1968

Chapter II

SOME PROBLEMS OF OUTPUT MEASUREMENT

THEORETICAL OBJECTIONS TO ANY SCHEME OF OUTPUT MEASUREMENT

The need for output measurement is seldom disputed by those actively engaged in the management of research and experimental development, whether in government, industry or universities. But, however desirable such measurement may appear to policy-makers, it is sometimes maintained that output measurements are either unattainable or useless.

It may be that the satisfactory measurement of part or all of R and D output will prove unattainable on purely practical grounds. This can only be established by attempting the measurements with skill, determination and ample resources over a considerable period of time. The measurement of Gross National Product or of R and D inputs at one time also appeared extremely difficult on purely practical grounds. However, there would be no point in making even the attempt to measure R and D output if it could be clearly demonstrated that the objective was in principle unattainable or unnecessary.

Two such arguments are therefore briefly considered here, as summarized by Machlup:(lG)

“One might take the position and defend it on good grounds that it is impossible even to define invention, let alone to identify, count and weight inventions, and if it is meaningless to quantify the output it must be meaningless to assert or posit the existence of a quantitative relationship between input and output . . . ”

“Or one might take a less negative position and grant the possibility of quantifying input and output at least roughly or for the purpose of con- structional reasoning but at the same time hold that the incidence of accidents in making inventions is too great to legitimize even provision- ally the assumption of a production function. ”

Although “invention” is discussed here, essentially similar arguments may be applied to other types of R and D activity. Take first the argument that it is

10

impossible to define and measure inventive output (or scientific output). It may be conceded that this is extremely difficult, but it cannot be denied that there is an output of some kind from all types of research and experimental development activity. An attempted representation of these outputs and the corresponding inputs for various stages of research and experimental development work is shown schematically in Table 1. (The differences between this scheme and the usual “input” classification of R and D are discussed in Chapter IV.) From this it can be seen that the output of all stages of R and D activity is a flow of information and the final output of the whole system is “innovations” - new products, processes and systems. This information is conveyed in various forms and through various media, with varying degrees of secrecy or freedom. Some of it is “intermediate” or “feedback” output. But there can be no doubt that such a flow of information exists and that it is valuable.

R AND D OUTPUT AS A FLOW OF INFORMATION AND INNOVATION

The problem is therefore one of defining and measuring the flow of certain types of information and the efficiency with which this flow is utilized by various organizations to make innovations. The main criterion to distinguish this flow from all otherinfor- mation flows is the criterion of novelty. It maybe readily admitted that this is not an easy criterion, either in definitions or in practice. But it is not an impossible one. It is a criterion which, although difficult to apply, is constantly used. It has for centuries been the foundation of patent law in many countries and it is a criterion commonly applied for scientific publication. Whilst it is true that individual cases may be disputed, it would be hard to deny that there is an essential difference between repeating information which is already known and imparting something new, and that there are new products and processes. Therefore, the argument that the whole output of research andexperimental

----_--___ -._- --

development is in principle not definable is unaccept- able. The problem is reduced to the practical one of senaratine: the R and D information flow from other types of information flow, of trying to measure all or part of this flow, and of assessing efficiency in using new information generated by R and D activities.

If we cannot measure all of it because of a variety of practical difficulties, this does not mean that it may not be useful to measure part of it. The GNP does not measure the whole of the production activity of any country, largely because of the practical difficulties of measuring certain types of work. The measurements of R and D inputs omit important areas of research and inventive activity. But this does not mean that GNP or R and D input measures are useless.

Some parts of the information flow are captured and embodied in well-established, accessible forms. The best-known examples are published scientific papers and patents. It can scarcely be denied that these do represent a part of the output of research and experimental development activity, although it may be (as Machlup maintains) (16) that they do not represent the most important part of the output of fundamental research or of inventive work, or that they are not representative of the whole. Neverthe- less, it must be conceded that if we are able to measure that part of the information flow which is embodied in scientific papers and in patents, then we would in principle be able to measure at least a part of the output of R and D activity.

It may still be argued that there is as yet no satisfactory way of reducing scientific papers or patents to comparable standards as yardsticks of measurement. This is a question on which there has been a certain amount of empirical research and on which there are some important findings. The third and fourth sections of this paper are, therefore, largely devoted to the use of scientific papers and patents, as a possible means of measuring part of the information flow of research and experimental development. It is argued that the results of empirical work already justify the use of scientific papers and patents for some output measurement purposes, despite the severe difficulties and limitations involved.

The measurement of that part of the output of research and experimental development work which is embodied neither in published papers nor in patents, presents greater difficulties and evenless empirical work has been done. Nevertheless, there are some possibilities of measurement, although largely indirect. These are discussed in Chapter V of this paper.

For many policy purposes the information flow, which is generated during the R and D process, is only a means to an end. Theultimate aimisusually a flow of innovations, which may be considered as the final output of the system, while the information flow is an intermediate output. Chapter V therefore considers the measurement of the “final output” in this sense.

“ACCIDENTAL” FACTORS AFFECTING RANDDOUTPUT

We may now briefly c0nside.r the second main line of theoretical criticism which, if valid, might be sufficient reason to discontinue attempts at output measurement. This is the argument that the input/ output relationship is too arbitrary and uncertain in research and experimental development activity to ,justify any attempts to improve efficiency or effectiveness. It rests largely on the view that unpredictable accidents are so characteristic of the process that rationality in management is impossible to attain. This argument need not detain us long.

It is evident immediately that the argument itself assumes some knowledge of both inputs and outputs in at least part of the range of R and D activities. Otherwise the la&of relationship could not be presumed. The view is usually largely based on some well-publicized anecdotes of supposedly accidental factors in scientific work, such as Fleming’s penicillin mould. (17)

The straightforward answer to this type of argument was given by Cottrell when he said:(18)

“If . . . you accept an invitation by a pharma- ceutical firm to investigate the medical effects of chemicals, you are distinctly more likely, to put it mildly, to turn up a new drug than a new alloy or a new radio-star. ”

The logical fallacy lies in assuming that, because accidental features are present inindividualcases, it is therefore impossible to make useful statistical generalizations about a class of phenomena, whether natural or social. Those concerned with am individual street accident are always impressed by the peculiar features of the occurrence - if X had not postponed his journey by 15 minutes, it would never have happened; if Y had not been worried by his wife’s illness, if the street sign had been lOyards further down etc., etc. All these factors are undoubtedly extremely important in determining the specific form of each accident, which individuals are involved in it, the nature of their injuries and so forth. But they in no way prevent the statisti- cian from forecasting with a high degree of accuracy the number of street accidents which will occur in a given month to a given country, andfrom classi- fying many features of the “accidents”. Similarly those involved in any individual scientific discovery or invention are always impressed by the number of apparently accidental features, and often they may be right to think that but for these accidents this particular discovery or invention would not have been made. But this need not prevent the social scientist from making useful generalizations about a class of discoveries or inventions.

This is not to deny the presence of accidental factors in research, as in many other human activities. Nor is it to deny the existence of very wide variations in the relationship between input and output. All industrial production functions involve

11

a statistical distribution with fairly wide deviations from the norm. (lg) In agriculture, for example, when the farmer uses certain inputs such as seed, fertiliser, land and labour, he knows verywell that the actual output per acre, or per hour of labour, may vary enormously from season to season. These variations in output due to “accidental” factors, such as weather, or to factors over which he has limited control, such as pests, do not mean that he cannot take rational decisions about the use of inputs, or that he is rendered incapable of improving average yields.

By analogy, many R and D managers or scientists act “as if” they were farmers. They know that there are unpredictable and accidental factors present in their work. But they also lmow that, if they apply their labour with ingenuity and appropriate equipment over a sufficiently long period, they will probably achieve some useful results. This attitude has been justified in practice by the whole growth of science and technology over the past hundred years. The existence of commercial contract research institutes and the steady increase of company- financed R and D operations are evidence of the economic viability of a large range of R and D activities, which can be managed with some degree of rationality, despite the unpredictability ofparticular experiments.

USE OF INPUT NIEASURES AS A SURROGATE FOR OUTPUT

The converse of the argument on the role of accidental factors in research and experimental development is the view that variations in output are so slight that they can be disregarded or that they average out over alarge enough sample. Some such assumption is in fact involved in the use of input measures as a surrogate for output measures. We already know enough about output variations to know that for many purposes input measures are not enough although they are better than nothing. We cannot compare relative efficiency unless we have some direct or indirect measure of output as well as input.

In market economies the use of R and D input measures in national accounts may nevertheless be justified in the absence of any output measures, as in the case of many other service activities. But it must be remembered that the actual treatment of R and D in nationalincome statistics today is complex, depending upon the type of economy, the sector of performance and method of finance. (20) In many capitalist economies, when a piece of research is both performed and financed by government, it will normally be treated as part of GNP - final output - and measured by its input cost. This would also be true if the work were performed extra-murally in industry but paid for by government. But if the work is “company-financed” it will normally be treated by the firm as a cost of production and will not be measured as a final product. If the research

12

is financed by a private individual donor or a non- profit institute, it may be treated in national ac-, counts as private consumption expenditure and included in GNP. Despite this variety of treatment by the national income statisticians, it is quite reasonable for the economist to treat all R and D expenditures as a form of social investment in both capitalist and socialist economies.

The treatment of the public sector in many social accounts systems involves frequent use of cost of input measures in lieu of output measures. As with R and D, efforts at output measurement are still at a very primitive stage in areas such as education, health services and so forth. Some direct indicators have been used, for example, numbers of patients in hospital or children in school, ratios such as patients per doctor or pupils per teachers, or indirect indicators such as mortality rates. But it is generally agreed that none of these measures yet provides any satisfactory general scheme of “output” or quality measurement for these services.

It may be that a general system of output measurement suitable for incorporation in national accounts will never be attained and that for this purpose we shall have to continue to use input measures. In the socialist countries measurements for some service activities, whether of “inputs”or “Outputs” are in any case often excluded from the national accounts system. But this need not prevent the development of output and efficiency measures, which are specific to each activity and which can be used to compare the performance of organizations, of individuals and of countries in that activity, and with the financial outlays for each activity.

One of the greatest difficulties in representing research output in a form suitable for a national accounts scheme is that so much of it is:

(a) Feedback output to other parts of the system. (b) Output which is used only after long and

unpredictable time lags. (c) Output which can be “consumed” an infinite

number of times. This applies above all to basic research, whose function is to generate and maintain a “multi-purpose knowledge base”. (21) It is clear that the results of basic research are, by definition, not intended to serve any specific practical aim, but to provide a flow of general scientific information which maybe used in a great variety of applications. This “output” cannot be assessed in relation to the policy goal of any particular government department or industrial enterprise, but only in a much wider context. Even “applied research” which does have specific practical objectives may find very wide applications far outside the context of the original research. For this reason it is both more practical and more logical to attempt measurement of research output initially by the -- flow of published information, rather than indirectlv through the ultimate applications. This becomes progressively less true as we move across the spectrum to experimental development.

.__-_-.__ ~ .------- - -..

NOTES

and Direction of Inventive Activity, University Press, 196

(16) F. Machlup, paper contributed to The Rate Princeton

2

(19) F. Machlup, op. cit. (16) C. Freeman, paper in Problems of Science Policy, OECD, Paris 1968

(17) R. Taton, Reason and Chance in Scientific Discovery, Trans. A. J. Pomerans, New York, 1957

(18) A. H. Cottrell, Science and Economic Growth in the United Kingdom, British Association, 1966

(20) F. Machlup, The Production and Distribution of Knowledge, Princeton, 1962, pages 184-7

J. P. Lamouche, Recherche Scientifique et Comp- tabilite Nationale, EEC, Brussels, 1968

(21) J. Schmookler, op. cit. (6)

13

Chapter III

THE USE OF SCIENTIFIC PAPERS IN MEASUREMENT OF RESEARCH OUTPUT

NUMBERS OF PAPERS AND OTHER OUTPUT MEASURES

Three main yardsticks have been used for the measurement of output of basic research: scientific publications (usually “papers”), “discoveries” or other major contributions to the advance of knowledge, and colleague evaluation or peer judgements. The last two methods often depend upon some qualitative evaluation of the first. Moreover, a count of papers, whether weighted or otherwise, is the only method which lends itself readily to large-scale statistical application. Whilst allthese methods can easily be used on a smallscale simultaneously or combined for ranking purposes in a field which is well-known to the investigators, it is difficult to extend such combined indices of performance across a wide range of disciplines and countries or over an extended time period.

The promotions board, or appointments committee for research or university teaching posts will normally try to take into account all three methods of evaluating the output of candidates. This is quite feasible, since they are usually familiar with the individuals concerned, as well as the subject. But despite the very widespread practical application of such rough and ready output assessment, little success has been attained in generalizing this experience across a wider frame of reference. Most historians of science and sociologists have also tended to work mainly on a “micro” scale, using the second and third techniques of output measurement. As in the case of the appointments board, this is quite reasonable procedure at the micro level. To everyone who is familiar with research, it is obvious that it may be dangerous to rely on a simple count of numbers of papers in assessing the output of any particular individual or small group. But it does not necessarily follow that such quantitative techniques cannot be applied to much larger aggre- gates. A great deal of statistical analysis is based upon the knowledge that in a sufficiently large population many individual variations can be ignored, even though they cannot be ignored at the level of

the small group. For example, Pareto’s law is generally valid for a country but not necessarily for the individual firm or for a village. It may be legitimate to use quantitative measures as a sub- stitute for a qualitative assessment or a combined quantity-weighted-by-quality index, if it can be shown for any field of investigation that at the selected level of aggregation the quantity of scientific papers does not vary greatly from the combined quantity/quality index. Thus, if the national origin of a list of major medical discoveries in the twentieth century conformed almost exactly to the national origin of the “key papers” as assessed by experts, and the pattern of both in turn conformed closely to the national origin of the gross number of medical research papers appearing in a selected range of journals, it might be legitimate touse the third measure for some purposes as a proxy for the other two.

In practice the difficulty is that no one has yet established the range of applications, or the limits within which such surrogate quantitative measures may confidently be used. Some empirical workhas shown a degree of correlation between the three types of measurement in a few areas of application. But the errors and difficulties associated with each type of measurement, as well as the restricted area of validation, do not yet give sufficient grounds for confidence in widespread application. Reasonable caution dictates that, wherever possible, several methods of assessment should continue to be used simultaneously as with the interviewing boards. Experimental work should be continued, as some of the results thrown up by straightforward quantitative analysis are of great interest for science policy.

For example, Rangarao has estimated that the average output of papers by Indian researchers is approximately e uivalent to one paper every lo-12 scientist-years. 7 22) This may be compared with Price’s rough estimate for world science of an output of one paper every two scientist-years. (23) Careful examination of both estimates would be necessary to ascertain the degree of comparability

14

in definition of input (numbers of full-time equivalent research scientists) and output (range and proce- dures of abstracting services) and dates of measurement (which differ slightly). At first sight the difference in output is very great and at variance with Price’s own hypothesis of a roughly similar input/ output ratio for world basic research in general. But if valid it provides important supporting evidence for the views of Dedijer(24) and others on the research environment in Indian research institutes and universities. Kapitza has also estimated very roughly that the output of Russian research scientists in terms of papers is only half that of their United States colleagues. (25) He emphasizes the severe statistical difficulties in making such estimates and all these comparisons need very considerable care in their interpretation. It would be unwise to jump to policy conclusions without serious critical analysis of the data and methods. It seems likely that the Indian definition is much wider than Price’s, including scientists who donot publish at all.

THE USE CF OUTPUT MEASURES IN THE SOCIOLOGY OF SCIENCE

Several important contributions to the sociology of science have already been made by studies based on the use of scientific papers as a method of measurement. The broad scope of these contributions has covered such questions as the following: the age at which scientists are most productive in various disciplines; the relative contribution to science of industrial, government and university scientists; the long-term rate of growth of the output of various scientific disciplines and sub-disciplines; the relative contribution of various countries to world science in particular disciplines; the relative contribution of male and female scientists to research output; the growth of multiple authorship of scientific papers and its implications; the relative contribution of outstanding and lesser scientists to research; the institutional environment most conducive to high research productivity. Whilst most of these results must be regarded as in need of further validation and testing, they already constitute an important body of knowledge. Here it is only possible to indicate very briefly some of the most important findings and some of the hypotheses which these have generated or tested.

Alfred J. Lotka, in a pioneering article in 1926 on “The frequency distribution of scientific productivity , ” (26) demonstrated for some branches of natural science that for every 100 authors whopro- duce only one paper in a particular period, the number of people.producing “n” papers is approximately “l/n2 ” (Table 2) . Derek Price has provided additional evidence supporting Lotka’s observations, has reformulated the “law” governing distribution of productivity and has pointed to some of its implications for the long-term growth of the scientific community, (27) Price emphasizes the importance

of the findings of Wayne Dennis (28) and others on the output of the most eminent men of science. These show that the most outstanding scientists have usually been prolific in the volume of their output. Price is, of course, well aware that one paper by Einstein cannot be compar,ed with one or even 100 papers by “John Doe”; (2g) and that there is “no guarantee that the small producer is a nonen- tity and the big producer a distinguished scientist’! He argues, nevertheless, that in spite of obvious exceptions and variations, “on the whole there is, whether we like it or not, a reasonably good correlation between the eminence of a scientist and his productivity of papers. It takes persistence and perseverence to be a. good scientist and these are frequently reflected in a sustained production of scholarly writing”. (30)

Price has used the output of scientific papers and the number of scientific journals to generalize about the long-term growth rate of the scientific community, (31) in the United States and elsewhere. In broad terms he has deduced an input measure from an output measure (the opposite procedure from some national income statistics), arguing that the number of scientists, the number of papers and the number of journals have allbeenincreasing at an exponential rate of 5% to 7% per annum (i. e. doubling every 10 to 15 years).

These observations provide extremelyinterest- ing hypotheses on the long-term trends in science and likely future trends, but it should be notedthat they are not entirely consistent with Price’smodi- fied version of “Lotka’s Law” and his generalizations about the much slower rate of increase in numbers of “good scientists”. He argues that the “total number of scientists goes up as the square, more or less, of the number of good ones”. (32) But if “good” scientists are much more prolific than lesser ones, then not only would there be diminishing returns, in terms of average quality of output per scientist, as Price postulates, but there would also be a continuous slowing of the rate of increase in quantity of papers, by comparison with number of scientists. The point is by no means academic, since the period under consideration is three centuries and “Lotka’s Law” suggests that the “good” scientists account for a high proportion of total output. ( 33) The tendency might, of course, be offset by other long-run changes, such asvary- ing pressures to publish, the changing pattern of scientific careers, “professionalization” of research, the growth of post-graduate scientific research degrees and so forth. The variations which apparently exist between Indian, Soviet and United States’ output of papers per “scientist” provide grounds for considerable caution in making generalizations about “world science” over long periods. Not only are publication practices different but the amount of secondary material in journals varies significantly.

15

THE NEED TO RELATE “OUTPUT” TO “INPUT” MEASURES

From this it is evident that Price’s daring generalizations and first approximations will benefit from detailed national surveys, relating institutional and other sociological factors to the broad long-term international trends which he has discerned. One of the most valuable lines of research would be to relate his “output” series to an equivalent long- term “input” series in each country. Unfortunately, this is very difficult because satisfactory manpower “input” statistics are available onlyforveryrecent periods. The proportion of the total stock of scientists engaged in research and experimental development has been changing fairly rapidly, so that figures for total numbers of scientists are insufficient for this purpose. Moreover, the distribution of research scientists between industry, universities, and other laboratories will affect the output as measured in terms of papers. Even for the last few decades where we have figures for the total numbers of R and D scientists and engineers for a few countries, there are major methodological problems to be resolved. The “input” manpower statistics for Rand D do not distinguish between those who produce papers and those who do not, nor by the type of papers. Price draws a very sharp distinctionbetween “science”, which leads to an output of “papers” and “technology” which does not, in terms of his definitions. Many engineers and scientists and particularly social scientists would not accept this rigid distinction and would argue that an important part of the “output” of “technology” or “applied research” is also embodied in the form of research papers. Price concedes this for electronics, industrial chemistry and computer science. (34) This question is discussed further in later chapters.

Perhaps even more controversial are Price’s first generalizations on the distribution of scientific papers by country of publication (Table 3). He is, of course, well aware that country of publication is not the same as country of origin (whether in the sense of where the work was carried out or of citizenship of the author(s), and suggests some adjustments to allow for well-imown cases of international journals, such as “Nuclear physics” (Netherlands) and “Nuovo cimente” (Italy). But even if it is assumed for purposes of argument that country of publication is equivalent to country of origin, few economists would accept the conclusions which he draws from his analysis. He argues that the “cost” of producing a research paper is equivalent to $20,000 per paper. It is not clear how this figure is derived but he deduces from it that all countries contributing to world science are spending the equivalent of 0.7% of GNPonbasic research. (35) These estimates disre- gard the substantial data on international variations in research costs, which show, for example, that the input cost per research scientist is much lower in Europe than in the United States of America, (33) and lower still in Japan and the developing countries. The

16

“input” data on R and D expenditures also suggest that there are major variations between countries in the proportion of GNP devoted to basic and appliedre- search, as with R and D as a whole. (37) It would be surprising if these were not reflected in variations in the proportion of GNP devoted to that part of science concerned with the output of papers (this is not necessarily the same as basic research).

Price accepts, of course (and it is evident from his table) that a large number of countries in the under-developed world do not spend 0.7% of GNP on basic research. But he argues that 0. 7% is the minimum for a “genuine scientific effort”. It could be argued that this figure should be an absolute rather than a ratio if it is a threshold. But Price states that “with few exceptions, countries either participate in basic science to this extent or they stay out of the race completely”. (33) But anumber of developing countries are trying to enter the race and some have only joined it recently, so that it would seem a more reasonable hypothesis that countries will tend to spend a slowly rising proportion of GNP on R and D in general and on basic research in particular. This would also be consistent with the observed phenomenon of a rising proportion of expenditure on higher education, which closely affects the output of scientific papers. It would also be more consistent with Price’s own long-term historical observations on the growth of scientific activities.

These criticisms are made not in order tobe- little the major contribution which Price has made to social studies of science but in order to bring out the need for studies relating “input” and “output” measures and for refinement of his first herioic generalizations on the output of scientific papers.

THE NEED TO RELATE OUTPUT MEASURES TO INSTITUTIONAL FACTORS AND SOCIAL CHANGES

The general justification for Price’s original attempts to quantify aspects of scientific activity was wellputby Dobrov andMackay in “Nature”:( 3g)

“Science, in its material aspects as a complex developing system involving people, information, materials and money, invites study in terms of statistics, cybernetics, systems analysis and other appropriate modern techniques., The aim of the science of science is the construction of theoretical foundations for the organization, planning, management and progno- sis of science. These foundations must be built from a solid corpus of observations and experiment which will satisfy the criteria of science. . . The interesting preliminary correlations found by Price indicate that as in the case of lung cancer there is something worth detailed study . . . Similarly in the study of science itself statistics indicate that there may be things to be explained by detailed study of

the actual mechanisms . . . Needless to say, as in all branches of science, the science of science can only advance . . . by the growth of a corpus of knowledge which after criticism is accepted by the scientific community. ”

Whilst not everyone might be happy with the expression “science of science”, few workers in this field of studies would dissent from these observations. There are many studies which suggest that auseful “corpus of knowledge” can be built up provided the “actual mechanisms” are subjected to critical analy- .sis. Some of these have been contributed by Dobrov and Mackay themselves.

G. M. Dobrov (Director of the Kiev Institute for the History of Science and Technology) has made important studies of the relative rate of growth of various disciplines in the USSR(40) using not only published papers but also all those unpublished research reports which are accessible through the Soviet scientific and technical information service. He has also made interesting observations on the lags in citation of foreign scientific literature, as compared with “home-produced” scientific papers. It would be valuable to extend such studies in order to assess such factors as the relative efficiency of information services, the extent to which scientists read foreign language literature, the extent to which foreign language results have to be “r-e-interpreted” or repeated to be absorbed in the localmainstream of scientific advance and into technology, and the “lags” in the world basic research system. It is often assumed by economists on an over-simplified model that “basic research”results areuniversally and freely available and that therefore from a strictly selfish economic (as opposed to a cultural) point of view, there is no need for everycountryto perform basic research. Such grossly over- simplified models may be dangerously misleading and this type of research could lead to empirical testing of their assumptions.

Dobrov has also made interesting comparisons of the relative output of male and female scientists in the USSR. Additional results were demonstrated by Dodge, who showed that the productivity of women scientists in terms of numbers of papers was about half that of men for employees of higher educational institutions, and that the discrepancy between the sexes was greatest at the highest levels of achievement(41) .( Table 4).

In another important research area Dobrov has contributed to studies of the age composition of scientific teams in relation to laboratory recruitment.(42) H. C. Lehman used the output of outstanding creative works to relate creativity to age. (43) (44) He found that scientific creativitytypically reached a peak in the late 30’s and then declined, but the peak was earlier in highly abstract disciplines such as maths and physics, and later in more empirical disciplines. Such findings have been used by research managers to justify transfer of older men from research to other activities, and by others to

explain the relatively poor achievements of some research institutes which are not able to transfer their elderly researchers.

But here again Lehman’s findings need to be interpreted with sociological insight and have been criticized on the grounds that they ignore the extent to which environmental factors, rather thanpsycho- logical factors, affect creativity. Some psychologists have dismissed them as a statistical artefact. (45) Wayne Dennis argues that historians and anthologists show a bias in favour of citing works done in earlier historical periods and that the apparent decline in major contributions with age reflects the behaviour of historians rather than that of scientists. He demonstrates that, because of the exponential growth of scientific publications, the proportion of the total which are cited in histories and anthologies is constantly declining:

“Since there is a general reduction over time in the likelihood of papers being cited, the later papers of men . . . would be expected to be cited less often than their earlier papers, simply because of changes in citation practices quite apart from any relationship between a e and quantity of high quality creative works 11 (f5) .

Dennis himself demonstrated that 100 eminent nineteenth century scientists maintained a fairly uniform rate of output between age 30 and age 70. (46) His criticisms suggest that for some purposes a simple count of papers may be a less misleading index than “major contributions” culled from anthologies, biographies or science histories. Schmook- ler came to avery similar conclusion with respect to indices based on a simple count of patent numbers, as opposed to “major inventions” (see Chapter IV).

In any case, this controversy once again emphasizes the care, discrimination and insight with which these research results must be interpreted. The institutional factors affecting output of papers are not only of the greatest importance in compar- ing age groups in various environments but alsoin comparisons of the total contribution to science of researchers in various sectors of the economy. Several inquiries have shown that university scientists make a relatively greater contribution to the output of scientific papers than industrial research scientists (for example, Table 5). But it would obviously be absurd to assume from this that uni- VeTSity researchers are “more productive” than industrial researchers. Any such comparison would have to take into account the restrictions on publication confronting many industrial scientists, the extent to which the output of industrial research scientists is embodied in other written forms (internal memoranda, patents, etc. ) and the differing objectives and motivation of the two groups.

An example of careful assessment of suchfac- tors is the paper by Cotgrove and Box on “The productivity of industrial scientists”. (47) They found that 56% of a sample of 400 scientists considered that company policies hindered publication and 42%

17

_.---- .

thought that company security restrictions lowered their publication rate. This is in line with the findings of earlier surveys in Britain and the United States based upon inquiries to companies rather than to individuals. These showed that only 140/0of 174 major United States industrial firms conducting basic research published “substantially all” of their research finding.s and a further 26% “most”of their findings. (48) An FBI survey in Britain showed that 23% of firms followed a “liberal” policy on publication, while 31% indicated “nil” or “very limited” publication. (4g) Among other reasons given for believing that publication rate was lower than it could be, Cotgrove and Box reported that 25% of their sample thought the company did not allow adequate time for scientists to write up their research results for publication and 23% thought that patents diminished the need for publication.

In addition to demonstrating the source of scientific papers by sector of origin (Table 5), Rangarao provides interesting data on the degree of concen- tration in research output in India; for example, eight universities (out of 68) accounted for 500/O of university research papers and 44 institutions (out of 2, 000) contributed 50% of all papers.. It would be useful to have supplementary information on the relative quality of output from various institutions and in various disciplines. Rangarao shows that

NOTES

(22) B. V. Rangarao, “Scientific Research in India: an analysis of Publications”, Journal of Scientific and Industrial Research, Vol. 26, No. 4, 1967, pages 166-176

(23) Derek J. de Solla Price, “Nations must Pub- iish or Perish”, Science and Technology, October 1967, page 87

(24) S. Dedijer, “Under developed science in under-developed countries”, Minerva, Vol. 2, No. 1, Autumn 1963

(25) P. L.> Kapitza, in Pravda, 20 January 1966

(26) A. J. Lotka, “The Frequency Distribution of Scientific Productivity”, Journal of the Washington Academy of Sciences, 16 (1926), page 317

(27) Derek J. de Solla Price, op. cit.- (1)

(28) Wayne Dennis, “Bibliographies of Eminent Scientists”, The Scientific Monthly, September 1954, pages 180-183

(29) Derek J. de Solla Price, op. cit. (1)) page 45

(30) Derek J. de Solla Price, op. cit. (1)) page 41

(31) Derek J. de Solla Price, Science Since Baby- *, Yale University Press, 1961, page 107

(32) Derek J. de Solla Price, op. cit. (l), page 53

the 17 institutes with the largest output of papers contributed 300/O of all papers to Indian journals, but 50% of all papers appearing in foreign journals (Table 6) . To interpret these findings in a way which would be useful for policy conclusions would require a great deal of additional sociological and economic research. But Rangarao’s findings provide an essential and stimulating first step.

An example of the sophisticated use of an output measure based on numbers of medical discoveries with penetrating sociological analysis of institutional changes is Ben-David’s study of “Scientific productivity and academic organization in nineteenth century medicine”. (50) Having demonstrated the outstanding German contribution to medical discovery in the second half of the century (Table 7), he sought to explain this nineteenth century lead in terms of the earlier professionalization of medical research in Germany and related organizational factors. A. Zloczower followed up this study with further independent measures of the structure of German research in the nineteenth century. (51) Neither of these studies has been related to a count of numbers of papers by national origin and this would be a valuable experiment to see how far the results were consistent with Ben-David’s and Zloczower’s findings; but Ben-David did make (;;p of publication data in his study of pyschology.

(33) Derek J. de Solla Price, op. cit. (l), page 45

(34) Derek J. de Solla Price, “Historical Rela- tions of Science and Technology”, Technology and Culture, Fall 1965, page 562 and The difference between science and technology, Edison Foundation, 1968

(35) Derek J. de Solla Price, “Research on Re- search”, from Ed. D. L. Arm, Journeys in Sci- ence, University of New Mexico Press, 1967

(36) E. D. Brunner, The Cost of Basic Scientific Research in Europe: Department of Defence Ex- perience, 1956-1966, Rand, April 1967

C. Freeman and A. Young, op. cit. (12), page 91-98

(37) OECD, op. cit. (2)

(38) Derek J. de Solla Price, “Nations must Pub- lish or Perish”, Science and Technology, October 1967, page 87

(39) G. M. Dobrov and A. L. Mackay, “Not the Mysticism but the Science of Numbers”, Nature, Vol. 219, 1968, page 662

(40) G. M. Dobrov, Nauka o Nauke, Kiev, 1966

(41) N. T. Dodge, Women in the Soviet Economy, Johns Hopkins, 1966, Ch. 10

18

(42) G. M. Dobrov, “Scientific Potential as an Ob- ject of Investigation and Control in the Soviet Union”, Decision-making in National Science Policy, Church- hill, 1968, pages 189-201

(43) H. C. Lehman, Age and Achievement, Prince- ton University Press, 1953 and Science, 1958, pages 1213-1222

(44) H. C. Lehman, “Men’s Creative Production Rate at Different Ages and in Different Countries”, Scientific Monthly, May 1954, pages 321-325

(45) Wayne Dennis, “The Age Decrement in Out- standing Scientific Contributions: Fact or Artefact? I’, American Psychologist, 1958, pages 457-460

(46) Wayne Dennis, “Age and Productivity Among Scientists”, Science, 1956, page 714

(47) S. Cotgrove and S. Box, “The Productivity of Industrial Scientists”, The Technologist, 1967, pages 98- 107

(48) National Science Foundation, Publication of Basic Research Findings in Industry, 1957-1959, NSF 61-62, Washington 1961

(49) Federation of British Industries, Scientific and Technical Research in British Industry, Table 7, London, 1947

(50) Joseph Ben-David, op. cit. (3) and J. Ben- David and R. Collins, “Social Factors in the Ori- gin of a New Science”, American Sociological Review, 1966

(51) A. Zloczower, Career Opportunities and the Growth of Scientific Discovery in Nineteenth Cen- tury Germany, Occasional papers in sociology, Hebrew University of Jerusalem

19

Chapter IV

THE USE OF PATENT STATISTICS FOR MEASUREMENT

PATENTS, INVENTIVE ACTIVITY AND “APPLIED” RESEARCH

As with papers, SO with patents, there are severe difficulties inusing crude numbers as output measures. Both the publication of scientific papers and the registration of patents have become well-established international systems, but there are still important national variations in custom, in laws and in cultural patterns which necessitate great care in attempting international comparisons and comparisons over time. For example, in some countries it is difficult to obtain a patent because of rigorous tests for originality; in others there is no such test. In some countries there are long delays in processing patents; in others, little delay, and so forth. There is consequently the same need for sociological insight and historical sense in the interpretation of patent statistics as in the interpretation of output of scientific papers, and similar difficulties in setting up any general scheme of measurement.

There is some degree of overlap between the two forms of output. The inventor or inventors who take out a patent may often afterwards publish articles in technological journals relating to their work. In many countries prior publication of the idea ina journal constitutes valid grounds for the refusal of patent protection. But whereas most published papers (even in technological journals) provide information which is not patentable because of its generality, patents always refer to quite specific applications.

Because they are so specific some economists prefer to use the category of “inventive work” rather than ” applied research”, or “applied research and experimental development”, and to regard patents mainly as a measure of :he output of “inventive work”. Thus, in Machlup’s scheme (Table 1) “inventive work”refers to a stage preced- x development, but not corresponding to applied research. In his scheme ” applied research” dis- appears as a separate category and would be divided between “basic research” and “inventive activity”. He uses “development” exclusively in the sense of taking a ready-made invention (from outside) and

developing it to the point of commercial suitability. Patents do not arise from development work in this scheme.

In the conventional NSF or OECD scheme of classification “inventive work” does not appear as a separate category, but its content would be divided between “applied research” and “experimental development ‘I, and sources outside the R and D system. Whereas in Machlup’s scheme patents would appear exclusively as the output of “inventive work”, in the conventional scheme patents would arise both from applied research work and - from experimental development. They would probably arise in larger numbers from experimental development work than from applied research. In any case the “input” categories to be related to patent “output” would be the combined inputs into applied research and experimental development. In some industries Mueller found strong correlations between input data for development work and lagged patent data but in others a stronger correlation for applied research and lagged patents. (52)

In either scheme of measurement, some “inventive work” would be done outside the professional R and D system altogether. Machlup’s scheme and the variants used by Ames(52) and Schmookler(54) are useful conceptually but all classification schemes do violence to the real complexity of the R and D process. Thus Schmookler, who also distinguished a special category of “inventive activity” whilst retaining ” applied research”, wrote:(54)

“Inventive activity is often so intertwined with research and development today that a scientist or engineer might have trouble deciding which function he was performing at a given moment . . . Thus a scientist, while studying a given phenomenon, may begin to think about a particular possible industrial application. At this point we might say inventive activity has begun. He may then attempt to create this application on a laboratory scale. If he makes the attempt and finds that his application does not work as expected, he may return to do more research (thereby temporarily

20

terminating his inventive activity) . . . While these shifts in r81e may be difficult to keep track of as a practical matter, the r8les . . . are different: understanding a phenomenon is one thing; creating an industrial process or product based on that understanding is another.”

In the conventional classification this scientist would probably be classified with “applied research”, but these difficulties would apply to all the various schemes of classification of R and D which have been put forward, and most statisticians would agree that the sub-division of R and D into “stages” or “categories” must be regarded as a very rough ap- proximation. The classification depends to some extent on subjective judgements by respondents on the nature of their work and on the degree of specialization within each organization. The conventional system of classification used in input surveys does have the merit of corresponding a little more closely to the actual division of labour within the present-day R and D system. Some laboratories are recognizable as being primarily “basic research” or “applied research” or “development laboratories’: but it would be difficult to distinguish the category of “inventive laboratories”. This reflects the in- creasingly science-based character of the inventive process.

LIMITATIONS OF PATENT STATISTICS

Whilst patents are undoubtedly one indicator of applied research and experimentaldevelopment activity (or of inventive work - whichever classification is preferred), they measure only a part of the output. Some inventions are not patented for a variety of reasons, including the following:

(i)

(ii)

(iii)

(iv)

I;!,

They are believed by the inventors to lack commercial applications; They are protected by secrecy and the inventors believe that security is greater without patent protection; The inventors believe that scientific and technical leadership is more important than patent protection; Peculiarities of patent law affecting certain classes of invention; The delay, expense or difficulty of patenting; Legal factors affecting patents, such as anti- trust legislation.

It may nevertheless be argued that in spite of these limitations patent statistics can be used to measure an important part of the output of applied research and experimental development, or of inventive work. The problem is to define the limits within which they may be used. Whereas most of the work on scientific papers has been done by sociologists, psychologists and historians of science, most of the research on patent numbers has been done by economists, with relatively minor contributions by sociologists. Whilst many economists have made

use of patent statistics for limited purposes for a long time, undoubtedly the major applications were made by Jacob Schmookler of the Universit of Minnesota before hisuntimelydeathin 1967. (55 3

As a result of his investigation into United States patents from 1937 to 1957 he concluded that non-patenting of inventions was much more common since 1940 and more common for large than small firms. This was attributed to changes in attitude of large corporations arising initially from (v) and (vi) above but later affecting (ii) and (iii). Where- as the absolute numbers of patents issuedannually have increased very little in the United States in the post-war period, they have continued to increase in most European countries, although more slowly than R and D expenditure. This suggests that these factors have had a much bigger impact in the United States than inEurope and Schmookler presents con- vincing empirical evidence on the reasons for this.

He nevertheless maintained that, used with caution, absolute numbers of patents are a better guide to inventive output than “important inventions’: which he also used in his major study. This is because information on “important inventions” is very spotty and subjectively biased by historians, and because their “importance” is as variable as that of patents. They are also usually a good deal less “important” than many lives of great men assume. On the other hand, even though the quality of individual patents does vary enormously, the proportion of United States patents which are used commercially has risen from about one-quarter in the nineteenth century to about half today. Few really important inventions are not patented, but more minor ones are discarded without patent applications. This means that patents represent an inventive output which has great economic significance and covers a very large number of contributions to technical progress. As with scientific papers individual quality variations do not exclude the possibility of useful aggregate statistics.

However, Schmookler is extremely careful not to claim too much for patent statistics. In addition to pointing out the change in patenting behaviour of large United States corporations since the Second World War, he emphasizes that they are of little value in relation to United States Government applied research and inventive activity and that there is a bias in the United States statistics towards small firms and private inventors, which over- estimates their contribution. Moreover, he dis- tinguishes very carefully between those contributions to technology which can be patented and those which cannot, emphasizing the importance of non- patented general lmowledge in applied science and engineering.

One important difference between American and some European patent statistics is that the existence of annual renewal fees permits analysis of some European statistics in terms of a “weight- ing” by years of renewal. Y. Fabian used this method inhis long-term study of innovations in the

21

iron and steel industry. (58) For obvious reasons this method can only be used in long-term historical studies. Schmookler’s own studies are also mainly in this category, but for different reasons.

Despite all these limitations, he and other economists have demonstrated that patent statistics do have valuable, if limited, applications as output indicators. As with statistics of scientific papers, it is a question of interpreting experimental statistical comparisons with great caution and sociological insight.

APPLICATIONS OF PATENT STATISTICS

The kind of economic studies in which patent statistics have been used include the long-run changes in the amount and direction of inventive output inpar- titular industries; the relationship between these changes and other long-run economic indicators; the relative efficiency of company-financed and government-financed industrial R and D; the contribution of individual firms to particular areas of innovative activity; the relative significance of foreign and home-generated technology; and the measurement of individual inventive output.

A United Nations study of “The r81e of patents in the transfer of technology to developing countries” showed that in most of these countries foreign applications accounted for more than 80% of total applications (Table 8)) whereas in the United States theyaccountedforlessthan 20%. (57) AnOECDstudy showed that patents granted to United States firms in Europe were far more numerous than patents granted to European firms in the United States.(58) Changes over time in the proportion of patents granted to local firms or to firms from various foreign countries could be significant measures of the comparative success of local and foreign R and D in each industry. They would be more reliable than comparisons of absolute numbers of patents taken out in each industry in each country, for reasons which have been mentioned. But even analysis of changing ratios would have to take account of such factors as the activity of foreign subsidiary enterprises in each country and the real possibilities of establishing local manufacture in various industries. Nevertheless, this type of analysis used with discrimination over long periods could provide useful indicators for industrial R and D policy. Par- ticipation by the socialist countries in the international patent system is still too recent to permit useful analysis of their experience in this context.

Several studies in the United States(59) and in Europe(80) have shown fairly strong correlation between R and D expenditures classified by industry and patent applications classified in the same way. Whilst this is interesting confirmatory evidence that the two types of measurement are in some way related, it unfortunately does not mean that statistics of patent numbers can be used directly as output measures for inter-industry comparisons of R and

D efficiency. This is because there are some differences in the “propensity to patent” between different industries. This propensity is highest in those industries in which a technical advance can very easily be copied by competitors, without much independent development work. An example is the drug industry. It is lowest in those industries in which technical advances can be copied only with great difficulty and with much independent design and development lasting, many years, or where government contracts play abig role. In such cases technical leadership may often be maintained without strong patent protection. An example is the aero-space industry. In other cases, differences in patentability may seriously affect the comparison. For example, in the computer industry advances in hardware are usually patentable, but those in soft- ware are not. Thus patent statistics can give only a very approximate indication of inventive output in each industry. Moreover. thev will reflect some

. ”

“inventive output” from outside the industrial R and D system as well as inside and therefore cannotbe directly related to R and D inputs without qualification.

Whilst absolute patent numbers are of limited value for inter-industry Rand D output comparisons, they may be more useful for comparisons between firms in the same industry. But, here too, caution is needed, because propensity to patent also varies between firms, depending upon the attitude of management and’ the particular circumstances in which the firm may be placed. Freeman used patent statistics to demonstrate that at critical periods the outstanding technicalleaders in certain industries were ahead both in numbers of “major” innovations and in terms of absolute numbers of patents taken out, as well as in volume of R and D inputs. (81) But he did not attempt to show that there was any general relationship between the efficiency of a firm’s R and D and its patent portfolio. Those firms whichuse patent numbers as indicators of individual output almost always take into account other factors affecting performance. A man who gets many hundreds of patents during his lifetime, as Edison (1, 100) and Lanchester (900)did, is almost certainly a highly creative engineer or scientist. But if a man gets no patents, this does not necessarily mean that he is not productive. More- over, whether used as an indicator of output by an individual or by a firm, patent numbers must be interpreted over a fairly long period to eliminate the “lumpiness ” in their incidence at the “micro” level. For this and other reasons, Mueller concluded that R and D input measures were a more satisfactory indicator for inter-industry comparison than patent numbers, strongly correlated. (82)

even though he found them

Differences in propensity to patent also com- plicate the interpreation of findings relating to patent numbers by size of firm. Some studies have suggested that smaller firms generate a greater number of patents per $ of R and D input than large firms. (88) But the conclusion that

22

smaller firms’ R and D is therefore likely to be more productive would not necessarily follow. Schmookler has pointed out that pre-application commercial testing is much more commoninlarge firms and that their established position, broader research base and technological strength will tend to reduce their propensity to patent as compared with the smaller firm or independent inventor:

“Cross sections of patents granted classified by industry will tend to be biased downward in the case of industries dominated by large firms or with a disproportionately large amount of corporate invention. “(84)

Patent numbers can be a useful guide to the type of output of private inventors compared with corporate inventors. Independents were responsible for only 3% of chemical inventions, compared with 9% of electrical inventions and 88% of mechanical inventions, in the United States. (85) This is valuable supporting evidence for the view that professional- ized R and D is dominant in the most advanced technologies.

Since “propensity to patent” and patentability are both normally much lower in relation to government R and D thaninrelation toindustrial R and D, comparisons between sectors have little value. But there may nevertheless be some merit in comparisons between the number of patents generated re- spectively by government-financed R and D and company-financed R and D where both types are performed in industry. Solo has noted the very striking differences between the number of patents arising from the two types of activity in United States corporations, (66) and draws a number of important policy conclusions relating to diffusion of military-space technology.

From the standpoint of economic analysis, by far the most important application of patent statistics has been Schmookler’s demonstration that inventive activity in several major United States capital goods industries was demand-induced. He compared the long-run waves in successful patent applications with indicators of output in the same industries and found a high degree of correlation:

“When time series of investment (or capital goods output) and the number of capital goods inventions are compared for a single industry, both the long-term trend and the long swings exhibit great similarities, with the notable difference that lower turning points in major cycles or long swings generally occur in capital goods sales before they do in capital goods patents.

He found that trends for the 900 “important inventions” which he also analysed exhibited the same features. He concludes:

“the fact that inventions are usually made because men want to solve economic problems or capitalize on economic opportunities is of over- whelming importance for economic theory . . .

The production of inventions and much other technological knowledge, whether scrutinized or not, when considered from the standpoint of both the objectives and the motives which impel men to produce them, is in most instances as much an economic activity as is the production of bread. “(87)

His findings, however, related only to three major industries.

“SCIENCE” AND “TECHNOLOGY” IN RELATION TO PAPERS AND PATENTS

Schmookler’s findings are ample justification for the careful use of patent statistics in economic analysis. But it is important to guard against over-crude interpretation of these findings. They do not mean, for example, that basic research is unimportant or that science does not influence the course of invention. On the contrary, Schmookler himself emphasizes that basic research provides the “multi-purpose knowledge base” on which inventors are able to draw for whichever practical purpose they may have in mind. The facility with which they are able to do so will depend on their own education, imagination and links with those who are engaged in enlarging this base. They thus point, as to Price’s findings, to the critical importance of the interaction between the “science” system and the “technology” system. Schmookleruses the metaphor of two blades of a pair of scissors, while Price makes use of Toynbee’s metaphor of two dancing partners.

But whereas for Price the output of “science” is published papers, whilst that of “technology” is artefacts, for Schmookler “technology” includes published papers as well as other types of information and is defined as a “social pool of knowledge of the industrial arts”. For Price “the one part has papers as an end product, the other part turns away from them”. (88) For Schmookler progress in “technology” is derived from four sources: discoveries in engineering, discoveries in the applied sciences (which generalize about classes of products and processes), inventions and sub-inventions (which embody knowledge about specific products and processes). In his system the first two are the output of applied research, the third of inventive activity and the fourth of experimental development. Together they constitute a pool of knowledge, part of which is published freely and part of which is protected. (89)

At first sight the Price definitions offer great advantages in simplicity in treatment of input and output but it should be noted that Price is able to maintain his definitions only by allocating electronics, industrial chemistry and computer science to “science”, instead of “technology”, since inhis view the publication pattern in these areas resembles that of “science”. These are, however, the three most important and fastest growing areas of all

23

modern industrial technology. It seems, therefore, that the “output” of new “technology” must be measured by an amalgam of several indicators, including both patents and technological publications of various kinds. Important work has been done on internal communication channels and patterns of communication between technologists in R and D projects. (See page 23 and references (91) to (96).) But little has so far been done on the published journal literature of technology. Price has made interesting observations on the lack of disclosure in technological journals which he attributes to pro- prietary “hoarding” of knowledge. From this point of view the socialist countries should enjoy major advantages in measurement, since internal technological reports of all kinds, as well as patents, and published papers are collected centrally for all R and D projects.

However, in considering applied research and

NOTES

(52) D.C. Mueller, “Patents, Research and De- velopment, and the Measurement of Inventive Activity”, Journal of Industrial Economics, No- vember 1966, Vol. 15, Part I, pages 26-37

(53) E. Ames, “Research,Invention, Development and Innovation”, American Economic Review, June 1961

(54) Jacob Schmookler, op. cit. (6), page 9

(55) Jacob Schmookler, op. cit. (6)

(56) Y. Fabian, “Sid&urgie et croissance dco- nomique en France et en Grande Bretagne (1735- 1913) - Les Brevets en G-B”, Cahiers de l’ISEA, February 1965

(57) United Nations, The R81e of Patents in the Transfer of Technology to Developing Countries, New York, 1964

(58) C. Freeman and A. Young, op. cit. , (12)

(59) National Science Foundation, Science and Engineering in American Industry, (NSF 56-16), Washington, 1956.

J. Schmookler, op. cit. (21)) page 46

D. C. Mueller, op. cit., (52)

(60) Y. Fabian, Measures of Output of R and D, OECD, 1963

experimental development in any country, there is one major advantage: the work is carried out in order to contribute to the achievement of specific policy goals. It is therefore possible to relate the whole of the applied research and experimental development in any particular area to the test of overall performance in relation to the policy goal. Inbasic research the ultimate applications of new knowledge are so diverse and the time lags so great that we are usually thrown back on publications as the only practical system of measurement, using if possible additional criteria of academic excellence, citations, or other quality adjustments. But in applied research and experimental development, although “spin-off” and “feed-back” certainly occur and are extremely important, the purpose of the R and Dis to contribute to specific goals, which may provide a yardstick for rating performance. In the final chapter of this paper we turn to these criteria.

(61) C. Freeman, op. cit. (60)

C. Freeman, R. C. Curnow, C. J. E. Harlow and J. K. Fuller, “Research and Development in Electronic Capital Goods”, National Institute Economic Review, No. 34, November 1965

(62) D. C. Mueller, op. cit., (52)

(63) F. M. Scherer, “Firm Size, Market Struc- ture, Opportunity and the Output of Patented In- ventions”, American Economic Review, Decem- ber 1965

(64) Jacob Schmookler, op. cit. (6), page 25

(65) R. Nelson, J. Peck and E. Kalachek, Tech- nology, Economic Growth and Public Policy, Brookings Institution, 1967, page 58

(66) R. Solo, “Patent Policy for Government- sponsored R and D”, Idea, Vol. 10, No. 2, 1966 Patent Practices of the Department of Defence, Committee on Judiciary, U. S. Senate, 72757, U. S. Government Printing Office, Washington, 1961

(67) Jacob Schmookler, op. cit., (21) page 208

(68) Derek J. de Solla Price, op. cit., (34)

(69) Jacob Schmookler, op. cit., (6)

C. Freeman, “The Plastics Industry: a Compara- tive Study of Research and Innovation”, National Institute Economic Review, No. 26, November 1963 page 35

24

Chapter V

INNOVATIONS AND THE R AND D SYSTEM

INVENTION AND INNOVATION

Schumpeter made the valuable distinction between “invention” and “innovation” , which has since been generally adopted by economists, althoughnot quite in the original form. It is avital distinction because

m=v “inventions” (and much other “output” of R and D) may never lead to an innovation. Since he regarded “inventions” as exogenous to the economic system, Schumpeter stressed the decision of the entrepreneur to commercialize an invention as the decisive step and defined the entrepreneur as the “innovator”. Today, when a great part of inventive work is professional, the expression “innovation” is often applied to the whole process from laboratory to industrial use. But it is also used in the more specialised sense of the first commercial introduction of a new product or process. Taking this second meaning, innovation may be regarded as the ultimate aim of most applied research and experimental development and the success of any R and D programme may in principle be measured by the economic benefits arising from the innovations. When the goal of the innovative process is not an economic one, a similar assessment mayneverthe- less usually be made in terms of “effectiveness”.

The distinction between inventions and innovations is extremely important for policy for science and technology, as it is theoretically quite possible for an R and D laboratory or system to have alarge flow of inventions, or publications, or other information output, but for few innovations to result from this flow. If the information output is regarded as a goal for its own sake, then this may be a satisfactory state of affairs. But this is rarely the case with applied rese,arch and experimental development. Consequently, it is only by measuring innovations directly or indirectly that the efficiency of the system in generating final output can be assessed.

This is not to say that the measurement of intermediate output is unimportant. Some very productive laboratories may be concerned entirely with “intermediate” output, just as many plants are in the chemical industry. But ultimately, measures

of intermediate output must be supplemented by measures of final output, for most policy purposes. For ex-post studies the measurement of intermediate output itself often involves reference to final output. For example, lists of “important” inventions or “key” patents can only be made ex-post using the criterion of successfulinnovation to estimate “importance”. Most histories of “inventions” are actually histories of “innovations”. It is the benefit to society, however measured, which is the ultimate criterion of R and D output.

COST-BENEFIT ANALYSIS OF INNOVATIONS

If a flow of successful innovations is considered as the final output of the R and D system, then these may be measured in terms of their contribution to specific goals, in relation to their cost. The measures used in this type of assessment may vary with the policy goal. For example, those reduc- tions in mortality attributable to specific innovations might be a criterion for effectiveness of medical R and D, whilst increased destructive power of new weapon systems mightbe the criterion for military R and D.

It is not possible within the scope of this brief paper to review methods of measuringthe effectiveness of military R and D. But it is important to note that analysis of the cost-effectiveness of new weapon systems, during and since the Second World War, played a major part in the development of techniques of operational research and systems analysis, which have since found much wider applications. Here the discussion concentrates primarily on economically-oriented R and D and on “welfare” R and D.

For economically-oriented research, the appropriate criteria might sometimes be cost re- ductions in the case of process R and D, or contributions to market growth and profitability in the case of new product R and D. The type of measurement which is relevant will vary with the level of decision-making. At the level of the firm a great

25

to measure the socialbenefits arisingfrom the diffusion of the innovation. In the case of the potato harvester, Grossfield was able to estimate the total savings arising from the difference between hand- picking and mechanical harvesting, whether these benefits accrued to the manufacturers, the farmers or the consumers. He was also able to demonstrate and measure approximately the cost of the contribution of the National Institute of Agricultural En- gineering and the National Research Development Council (NRDC) itself to the success of the innovation. But both he and Griliches encountered difficulties in calculating the cost of “failures”, i.e. parallel or abortive attempts to make similar innovations and research in related fields which made an indirect contribution. These difficulties of selecting, isolating and measuring the appropriate “costs” and “benefits” are multiplied many times over in the case of most industrial and agricultural innovations. Researchers usually find that, although occasional individual projects readily lend themselves to cost-benefit analysis, most do not.

This is probably one of the main reasons for the scarcity of practical examples of cost-benefit studies and rate of return studies at enterprise level in market economies. Other major factors are company security and a certain reluctance to subject past experiences to impartial scrutiny. Difficulties in quantifying costs and benefits are stressedvery strongly by Mansfield, who has contributed the most important econometric studies in the fieldof industrial innovation:(76) (77)

“The productivity of industrial research and development is an extremely important variable which is plagued by unusually difficult measurement problems. “(76)

One way of avoiding the problem of isolating Rand D expenditures attributable to a particular project is to relate the whole of the R and D budget of a firm to its technological progress. To calculate marginal rates of return on total R and D investments for ten major firms and for ten manufacturing industries, Mansfield attempted to use Cobb-Douglas production functions(79) on alternative assumptions as to whether technological change is capital- embodied or disembodied. On the assumption of capital-embodied technological change he found very high rates of return to R and D investment for major chemical and petroleum firms but not for the chemical industry.

The difficulties involved inusing Cobb-Douglas production functions to estimate the “contribution” of various inputs are notorious. Many economists reject altogether the validity of these techniques.(66) Mansfield himself stresses the severe limitations of his data and results(61) and states that “not much policy significance should be attached to them”. The principal finding of the inter-industry comparison of rates of return on Rand D investment is that they were highest in the apparel and furniture industries in the United States up to 1960.

deal of detailed information will be available, which can be used both in ex-ante and ex-post project evaluation, but cannot be used at a higher level of aggregation. In a market economy the main criterion may well be profitability of innovations. At the level of the industry, or the government, it may be possible to assess external and secondary benefits arising from an innovation, or external costs, which are disregarded at a lower level. The method of assessment may also vary with the type of economy: in a market economy the competitive position of the firm will be a relevant dimension, but this may not be so in a socialist economy. Just as with industrial productivity measurement, techniques which are appropriate in one firm or for one type of output may not be relevant for another.

Unfortunately, very few studies have been published relating to the project of programme evaluation of enterorise-level R and D. as it is actuallv

I

carried out. There are plenty of papers and books prescribing ways in which it ought to be carried out but there is some evidence that these techniques are not very frequently applied in practice, (76) except perhaps in the socialist countries, where in some industries they are systematically used. Work in progress at the Department of IndustrialEngineer- ing and Management Sciences at the Northwestern University will help to fill this gap for the USA.

In the Soviet Union “in all planning, design and scientific research institutes and organizations there are expert economic sub-divisions which, using a more or less uniform method, have to provide a detailed justification of the economic effectiveness of each project of an enterprise, of each unit or planned technological process”. (71) Eco- nomic effectiveness is assessed according to a code laid down by the State Committee for the Co- ordination of Scientific Research in 1963, but the detailed application of the code appears tovary con- siderably between different industries and different types of project. Whilst these techniques are mainly applied at the stage of R and D project selection and evaluation, that is ex-ante, there is also some ex- post analysis. Apparently development costs are often underestimated ex-ante, as in the market economies, but in spite of this, Soviet scientists calculate very high overall rates of return to Rand D investments. (72) It is not entirely clear whether in making such calculations they include the costs of generating intermediate output in the earlier stages of the process, or what type of simplifying assumptions are made about other contributions to technical progress. A thorough review of the Soviet methods has beenmade by R.W. Davies, M.J. Berry and R. Amann. (73)

Two much-quoted cost-benefit studies of important individual innovations are those of Griliches(74) and of Grossfield. (75) The frequency of their citation is a tribute both to their quality and to the ex- treme difficulty of assembling the relevant data for such analysis. One of the principal merits of both these studies was the care with which they attempted

26

INDIRECT MEASURES OF THE OUTPUT OF INNOVATIONS

Of greater interest are Mansfield’s studies of size and growth of a firm in relation to innovation, timing of innovations and rate of diffusion of innovations.(32) He points out that, because of data limitations in calculating rates of return on R and D investment, the effect of successful innovation on a firm’s growth rate is one useful alternative measure of R and D success. He found that the average effect of a successful innovation in the steel and petroleum industries was to raise a firm’s annualgrowth rate by 4 to 13 percentage points. Very striking differences were also found between the growth rates of innovating firms and those of other firms of comparable initial size. In the coal and petroleum industries he found that the larger firms made a disproportionately large share of the principal inno-

. vations in relation to their market share, but not in the steel industry. These estimates were made on the basis of direct identification and listing of important innovations from technical journals and correspon- dence with firms. In the course of his studies he was also able to develop useful models designed to explain the diffusion of process innovations.

Other economists have attempted to-relate industrial R and D expenditures and innovations to world market performance. It can be argued that, at least for capitalist countries, the world export market provides a fairly severe objective test of the innovative success of afirm’s (or anindustry’s) research and development activities for product innovations. If a firm is the first in the world to produce a new product, it may enjoy an export position based on monopoly supply up to the point of imitation abroad. Even then it may still be able to keep an export lead by technical improvements and the introduction of new models, through inten- sive and efficient R and D programmes. Suchsuc- cessful innovations would find their reflection in exceptionally high shares of the world market for particular new products or product groups.

Posner(33) had postulated such an explanation in theoretical terms and economic historians had explained the German lead in the nineteenth century chemical and optical instruments industries very much on these lines. Freeman(34) attempted to relate world export performance in the plastics and electronics industries to the flow of firm’s inventions and innovations and to R and D expenditures. His results indicated the importance of “lead-time” in innovative success, of the threshold levels of expenditure which this implied, and some of the cumulative advantages of large-scale R and D in particular industries.

Grossfield’s calculations on the potato harvester confirmed the sensitivity of the “benefits” from R and D to the length of lead-time in competitive markets. Development lead-times could thus be an important indicator of relative efficiency in R and D output. Such calculations are also widely used in the Soviet Union, (3$5) where it is accepted that measurement

oftime-lags is animportant indicator of efficiency in the whole R and D system.

Hufbauer(35) made a major contribution to the theory of international trade, elaborating Posner’s “technological gap” concept by relating innovations in synthetic materials to country export performance. Vernon(37) and Hirsch(33) made further important contributions, relating the technological gap theory to United States performance in world trade and the concept of the “product cycle”. Huf- bauer ‘s method was based on identification of major product innovations and measurement of “imitation lags” to first production for each country for each new product. These lags were weighted by the relative importance of each material in the world economy, to give an “aggregate imitation lag”. Thus all the principal countries could be ranked in order of their innovative and imitative success in this industry and this in turn could be related to their export performance.

Another indirect method of measuring innovation achievements is to use the assessment provided by the internationalmarket for licenses and technical know-how. It is reasonable to assume that such payments will flow mainly to the most successful innovating firms, industries and countries. The inclusion of “know-how” transactions in the statistics means that they do not refer only to that part of R and D output which is patented. Their main deficiency is that they do not reflect those transactions which are conducted on a barter basis or in secrecy. There are also difficulties in connexion with the transactions between parent and affiliated companies, which may take a great variety of different forms. Nevertheless, the regular statistics maintained by the Japanese Government since 1953 have demonstrated the value of these figures for policy purposes (Table 9). More recently most of the leading industrial countries have begun topub- lish them. A recent paper for United Nations Con- ference on Trade and Development (UNCTAD) (8g) attempted touse these published figures to estimate world flows and to assess the transfer of technology to the developing countries through this market.

This discussion of indirect measures of the final output of R and D emphasizes once more the difficulty of developing any standardized approach which can be applied across the whole spectrumof R and D activities. However, one ingenious and original attempt has been made to develop just such a method. Maestre(go) has suggested that an input-output matrix relating a country’s industry to its R and D system could be constructed by a novel method of measuring research productivity. This method would rank the “utility” of various types of research for each industry and would also take into account the upstream and downstream flows of information within the research systemby a separate matrix for the transactions within the R and D system. Its successful application would depend, however, not only on some measurement of these difficult intermediate and “feedback”flows, but also on a large number of peer judgements for

27

every industry about the “research productivity” relevant to that industry. Thus in the end the method comes back to dependence on the type of subjective evaluationof “quality”and “importance”of research output, which bedevils so many output measurement techniques.

Even by this method there is no real escape from the laborious and difficult work of case studies and systems analysis, industry by industry and even project by project. Such detailed analysis may ultimately yield the kind of data which could be applied in the way Maestre suggests. But as he recognizes, this is a long way off. Progress will probably depend on fundamental research at amicro- level on information channels and information flows between individuals, projects and organizations. Most of the pioneering work in this important field has been done at centres of management studies or operational research, notably at the Sloan School of Management (of the Massachusetts Institute of Tech- nology) by D. G. Marquis, T. J. Allen and colleagues(gl), at Northwestern University Depart- ment of Industrial Engineering and Management Sciences by A. H. Rubenstein et al. (g2), and at the Studiengruppe fiir Systemforschung (Heidelberg) by H. Krauch, H. Rittel, W. Kunz and others. (88) In the Socialist countries the technique of operations research has alsobeenused, notab1ybyG.A. Lakhtin and I. Male&i, to analyse problems of optimizing output in research organizations. G. A. Lakhtin has suggested that the information service withina research organization is of great importance(g4), but much of the American work points to the crucial significance of informal channels of communication. Of particular interest is Allen’s concept of “technological gatekeepers”(g5) and Diana Crane’s applications of the gatekeeper concept to science. (g6) It is not possible within the scope of this brief paper to review this extremely important literature systematically.

Whilst a great deal depends on the progress of this work in deepening our fundamental understanding of information systems, and creating new possibilities of measurement, meanwhile there is value in continuing attempts to generalize at the institute level, industry level or country level, crude though our output measures may remain. Such attempts may very well yield important guidelines for policy. For example, Ben-David’s secondary analysis of Jewkes’ data on inventions and some of the OECD and National Institute of Economic and Social Re- search (NIESR) studies(87) suggest the value of an approach which relates measures of inventive or research output and input to measures of innovative output. In the military field “Project Hindsight” studied a large number of “technological events” or sub-innovations and related them to the introduction of new weapons systems in terms of cost and time. (88) This type of comparison may throw up contrasts and unexplained discrepancies between firms or industries, for example, differences in lead-times or in “coupling” of social organizations,

28

which may in turn suggest better methods of measurement. By an altogether different route sociologists have come up with interestinghypoth- eses which could explain some inter-firm and inter- organization differences in research and innovative performance. (88) The problems are so complex that they necessitate variety in methods of approach and contributions from many disciplines. Solutions will not be found easily or quickly.

OUTPUT BUDGETING

In these circumstances a useful step towards a satisfactory system of measurement may often be simple “output budgeting”. This has been developed in response to the great difficulties of applying cost-benefit or input-output analysis in many parts of the public sector. Williams has defined its purpose as follows:(loo)

“Unfortunately, thorough-going cost-benefit analysis is not immediately feasible in many important fields of social policy, because of sheer lack of understanding as to what the relevant dimensions of the output are, because of difficulty in getting adequate data to measure them even where they are understood sufficiently, or because of the inadequacy of our present evaluation techniques in attaching con- vincing money-values to them. It is, however, possible to move part of the way towards comprehensive cost-benefit analysis by tackling a rather more restricted task, that of settingup an output budget for government activities. The basic idea of an output budget is to relate all cost items to broad functional objectives, by constructing a framework within which it is clear what resources are being devoted towards what end and with what results. ”

Such an approach may often involve complete re- arrangement of traditional forms of maintaining departmental accounts so that all the inputs relevant to a particular policy aim may be analysed together. An illustration of this is shown in Table 10 for such policy aims as protection of persons and treatment of offenders. It will be noted that research expenditures are grouped with many other items as one of the inputs designed to achieve specific policy goals. This has important implications for the reclassification of R and D input statistics by goals.

One of the merits of this type of analysis is that it may suggest areas of under-investment or over-investment in research and experimental development, which would not otherwise be readily apparent. For example, from Table 11 it is immediately evident that there is apparently little government research on treatment of offenders in the United Kingdom. This may be justifiable, but it is the kind of question which should be debated. Whereas the market mechanism may conceivably

_- ___.. --- -..- --.-. -_.. -----.-~. ~-.--.--- -..-. .- --.

allocate resources efficiently in some branches of industrial R and D, it will certainly not do so in this area. It is generally accepted that there is a wide area of social policy where similar considerations

apply. Krauch has pointed out forcibly(l61) that there is a permanent tendency tounder-investment in research in this social welfare area, whilst Nelson(l02) and other economists have suggested that the same tendency may operate in relation to fundamental research. This tendency to under- investment arises from the intangible and diffuse distribution of many of the social benefits arising from this research, the long time lags often involved, and the inability of some of the beneficiaries to finance the expenditures. For this kind of reason, both in market and in planned economies, these are areas of accepted social responsibility, usually at central government level. Political assessment of “benefits” is to some extent both inevitable andde- sirable.

As we have seen, it is difficult enough in the case of economically-oriented R and D to measure and relate costs to benefits. Even in cases where good statistics are freely available, there are often

judgements to be made about research output or innovative output, which are essentially non- economic in character. For example, in the case of a completely new product or service, its introduction may involve considerations of taste, quality of life and values which cannot be easily expressed in economic terms. In the last resort the assessment of “benefits” of all kinds depends on values. Man does not live by bread alone and economic growth is desirable not as an end in itself but as a means to other ends. Consequently, it is quite legitimate to assert the primacy of non-economic values in relation to the output of science and technology.

Bernal defined society’s investment in research as “all inquiries directed to changing the future state of civilization”. (163) The kind of civilization we want should determine the output of innovations and the way in which we rate theirbenefits. Consequently, the direction of investment in new science and technology must always be the subject of political and ethical debate and choice. The change in quality of civilization in the next generation is the final measure of the “output” of R and D.

NOTES

(70) W. R. Baker and W. H. Pound, “R and D Pro- ject Selection: Where we Stand”, I. E. E. E. Trans- actions on Engineering Management, Institute of Electrical and Electronics Engineers, 1964, and A. H. Rubenstein et al, projects listed in A direc- tory of Research on Research, Northwestern Uni- versity, Illinois, 1965

(71) Osnovnye Metodicheskiye Polozheniya po Opredeleniyu Ekondmicheskoi Effektivnosti Narechno-Issledovatel’skoi Raboty, MOSCOW, 1964

V. S. Sominskii, Ekonomika Novykh Proizvodstv, 1965, page 40

(72) V. S. Sominskii, Ekonomicheskaya Gazeta, 1967, No. 10, page 7

V. M. Petrov, Ekonomicheskie Problemy Sodvuz- hestva Nauki i Proizvodstra, Leningrad, 1967,

Page 9

(73) E. Zaleski, J. P. Kozlowski, H. Wienert, R. W. Davies, M. J. Berry and R. Amann, Science Policy in the USSR, Part 5, Section G, pages 457- 487, OECD, Paris, 1969

(74) Z. Griliches, “Research Costs and Social Returns: Hybrid Corn and Related Innovations”, Journal of Political Economy, October 1958

(75) K. Grossfield and J. B. Heath, “The Benefit and Cost of Government Support for R and D: a Case Study”, Economic Journal, September 1966, pages 537-549

(76) E. Mansfield, Industrial Research and Tech- nological Innovation, W. W. Norton, New York, 1968

(77) E. Mansfield, The Economics of Technologi- cal Change, W. W. Norton, New York, 1968

(78) E. Mansfield, op. cit. (76), page 198

(79) E. Mansfield, op. cit. (76), Ch. 4

(80) OECD, The Residual Factor in Economic Growth, Paris, 1964

(81) E. Mansfield, op. cit. (76), page 201 and page 80

(82) E. Mansfield, op. cit. (76), Chapters 5-9

(83) M. Posner, “International Trade and Techni- cal Change”, Oxford Economic Papers, October 1961

(84) C. Freeman, op. cit. (61)

(85) Puti Povysenija Effektivnosti Nauchnogo Truda, Novosibirsk, 1966, Vol. 2 and op. cit. (73)

(86) G. Hufbauer, Synthetic Materials in World Trade, Duckworth, 1966

(87) R. Vernon, “International Investment and International Trade in the Product Cycle”, Quarterly Journal of Economics, May 1966

(88) S. Hirsch. “The United States Electronics . I

Industry in International Trade”, National Insti- tute Economic Review, No. 34, November 1965

29

(89) C.H.G. Oldham, C. Freeman and E. Turkcan, The Transfer of Technology to Developing Coun- tries, with Special Reference to Licensing and Know-how Agreements, UNCTAD, November 1967

(90) Claude Maestre, “Vers une Mesure des Echanges Intersectoriels entre, la Recherche et l’Industrie”, Le Progres Scientifique, No. 102, November 1966

(91) D. G. Marquis, Research Programme on the Management of Science and Technology, Massuchesetts Institute of Technology, 1968

T. J. Allen “The Performance of Information Channels in the Transfer of Technology, Industrial Management Review, 8.1966

D. G. Marquis and T. J. Allen “Communication- Patterns in Applied Technology”, American Psycho- logist, 1966, pages 1052-1060

D. G. Marquis and W. H. Cruber, Factors in the Transfer of Technology, Massachusetts Institute of Technology, 1969

(92) A. H. Rubenstein, “Programme of Research on the Management of Research and Development”, Annual Reports 1966 et seq, Department of Indus- trial Engineering and Management Sciences, Northwestern University, Illinois. and op. cit. (70)

(93) H. Krauch, Studiengruppe fur systemsfor- schung, Jahresberichte 1966 et seq, Heidelberg.

(94) G. A. Lakhtin, “Operational Research in Research Management”, Minerva, Summer 1968

G. A. Lakhtin, “Nauchno-issledovatel’skaya Deyatel’nost i Materialovoe Proizvodstvo”, Ekonomischeskie Nauki, January 1966

K. Malecki, Popytka Otsenki Parametra Effektiv- nosti Nauchnyhh Issledovanie, (Polish-Soviet Symposium), Academy of Sciences, Moscow 1967

(95) T. J. Allen, Managing the Flow of Scientific and Technical Information, Ph. D. Thesis, Massa- chusetts Institute of Technology, 1966

T. J. Allen “Communications in the R and D La- boratory” Technology Review 70, November 1967

(96) Diana Crane, “The Gatekeepers of Science”, American Sociologist, November 1967

(97) J. Ben-David, Fundamental Research, OECD, Paris 1968

OECD, Gaps in Technology, Paris 1969

C. Freeman, op. cit. (61)

(98) C. W. Sherwin and R. S. Isenson, First In- terim Report on Project HINDSIGHT, Office ofthe Director of Defence Research, (ODDR) , Clearing- house for Scientific and Technological Information, AD 642400, 1967

(99) T. Burns and G. M. Stalker, Management of Innovation, Tavistock, 1961

G. Gordon, “The Problem of Assessing Scientific Accomplishment: a Potential Solution”, IEEE - Transactions, Institute of Electrical and Elec- tronics Engineers, December 1963

N. Kaplan, “Some organizational factors affecting creativity”, IRE Transactions, March 1960

(100) A. Williams, Output Budgeting and the Con- tribution of Micro-economics to Efficiency in Government, HMSO, London, 1967

(101) H. Krauch, Sqcial costs and benefits in R and D, paper submitted to first Frascati Con- ference, OECD, 1963

(102) R. Nelson, “The Simole Economics of Basic Scientific Research”, -Journal of Political Economv. 1959 M’ ~~~-

R. Nelson, “The Allocation of R and D Resources”, in Ed. R. Tybout, The Economics of R and D, Ohio, 1965

(103) J.D. Bernal, op. cit. (9)

30

Chapter VI

SUMMARY AND CONCLUSIONS

1. The rapid growth in the resources devoted to research and experimental development work has led to a growing demand for measures of the efficiency with which these resources areutilized. This involves in one form or another, direct or indirect measurement of outputs in relation to inputs.

2. I~JL& measures of scientific and engineering manpower and R and D expenditures have now become official statistical series in many countries, although there are still severe problems of definition and international comparability.

3. Output measures are still in the experimental stage and do not lend themselves readily to any standardization on a national or international scale. For example, there is no prospect of sub- stituting any “output” series for the “input” values now used in some national accounts systems.

4. The argument that output measurement is intrinsically unattainable is rejected and the literature is briefly reviewed to demonstrate some applications of output measures.

5. One major problem in attempting to measure output is the interdependence of the whole R and D system. If the final output of the system is considered as a flow of innovations, then the intermediate output of many parts of the system is a flow of new information. This information flows in many different directions and is used with varying time-lags. Only a part of it is published.

6. Limited success has been achieved inusing numbers of scientific papers and patent statistics as indicators of intermediate output for parts of the R and D system. Important contributions to economics and sociology have been based on such measures. Nevertheless, great caution is necessary in applying these measures, especially in international comparisons.

7. The use of patent statistics and counts of scientific papers, although justifiable for specific limited purposes, must usually be heavily qualified because of quality variations. These are generally less significant at higher levels of aggregation, but they may also be important in international comparisons, because of institutional factors.

8. Numbers of papers and patents have often been collected and analysed independently of the corresponding input measures, whilst input data have usually been assembled without any attempt to obtain any corresponding output indicators. It is often extremely difficult or impossible to relate one to the other. This is a grave deficiency of both series.

9. It would be valuable if Unesco and other interested organizations could promote experimental work in particular countries enabling a systematic comparison of input and output indicators in particular sectors and industries. For example, respondents to R and D input surveys could be asked also to enter information relating to publications, patents, licensing and know-how income and innovations. Abstracting journals and scientific journals could also be asked to collect and publish systematically data on the place and institution, where research leading to a paper was carried out and the citizenship of the authors.

10. The final output of the R and D system is a flow of innovations. These may be measured either by making lists of important innovations or in a more sophisticated way by measuring the ‘benefits” or “effectiveness” of each innovation. “Cost-benefit” analysis of innovations, although it has been successfully applied in particular cases, is however very time-consuming and sometimes impossible because of incomplete information on the costs and benefits. A particular difficulty is to make allowance for all the “intermediate” output of information which led up to the innovation and for all those experiments which failed but which nevertheless by their negative results contributed to ultimate success.

11. One way of getting round these difficulties is to relate innovations to total R and D budgets rather than to particular project budgets. In this way a “rate of return” may be calculated for investment in R and D in particular organizations. But this still leaves unresolved the problem of information flows derived from other parts of the R and D system. Measures of relative efficiency in

31

innovative performance maybe derived from “lead- times”, from operational research techniques and case study methods.

12. The “benefits” of innovations are often difficult to isolate by direct measurements, but may sometimes be measured indirectly by their contribution to the growth or export performance of a firm or an industry. In the case of non-economic benefits, alternative criteria must be applied which will vary with the policy objective. Research

which improves the quality of life, such as medical research or research on the urban environment must often be rated by essentially political judgements. This is quite legitimate, since economic growth too is not an end in itself but only a means. Both inputs and outputs of R and D should therefore be related to the principal policy objectives which they are designed to serve. Such “output budgeting” techniques may identify areas of relatively low inputs or outputs.

32

Table 1 The Flow of Ideas through the Stages of Research, Invention and Development to Application

Stage Intangible

INPUT

Tangible Measurable Intangible

OUTPUT

Measurable

I ” Basic

1. Scientific knowledge Scientists (old stock and output Technical aides

Research ” from I-A) Clerical aides 1

Men, man-hours Payrolls, current

A. New scientific knowl- Research papers edge : hypotheses and memoranda ;

and deflated and theories formulas

(Intended 2. Scientific problems Laboratories and hunches

Outlays, current output : Materials,

B. New scientific prob- and deflated lems and hunches

” Formulas ” ) (old stock and output fuel, power from I-B, II-B, and

Outlay per man C. New practical prob-

III-B) lems and ideas -

II ” Inventive

1.

Work ” (Including minor

2.

improvements but excluding further 3. development of inventions)

(Intended output : ” Sketches ’ )

Scientific knowledge (old stock and output from I-A)

Technology (old stock and output from II-A and III-A)

Practical problems and ideas (old stock and output from I-C, II-C, III-C and IV-A)

Scientists Non-scientist inventors Engineers Technical aides Clerical aides 1

Men, man-hours A. Raw inventions : Payrolls, current technological recipes and deflated a. Patented inventions a. Patent applica-

b. Patentable inventions and patents tions, not patented b. Technological but published papers and

Laboratories Outlays, current c. Patentable inven- memoranda

Materia’ls, and deflated tions, neither pat-

fuel, power Outlay per man ented nor published c.

d. Non-patentable, in- d. Papers and ventions, published memoranda

e. Non-patentable inventions, not published

f. Minor improvements F: - -

B. New scientific problems and hunches

C. New practical problems and ideas

-

III 1. Scientific knowledge Scientists ’ Development

Men, man-hours A. Developed inventions : Blueprints and

Work ” (old stock and output Engineers Payrolls, current from I-A) Technical aides

blueprints, specifi- specifications and deflated

(Intended 2. Technology Clerical aides cations, samples

output : ” Blueprints and Speci- fications “)

(old stock and output Laboratories Outlays, current B. New scientific prob-

lems and hunches from III-A) Materials, and deflated

fuel, power C. New practical prob- 3. Practical problems Outlay per man

lems and ideas and ideas Pilot plants Investment (old stock and output from I-C, II-C, III-C and IV-A)

4. Raw inventions and improvements (old stock and output from II-A)

IV 1. Developed inventions ‘New-type (output from III-A) Plant Construction”

2. Business acumen and market forecasts

(Intended output :

3. Financial resources

’ New-type 4. Enterprise (venturing) plant “)

Entrepreneurs Managers Financiers and bankers Builders and contractors t investment in Engineers new-type plant

Building materials Machines and tools

A. New practical problems and ideas

New-type plant producing a. novel products b. better products c. cheaper products

Source : F, Machlup. The Production and Distribution of Knowledge, Princeton, 1962.

34

Table 2 Schematic 7’able Showing Numbers of Authors of Various Degrees of Productivity (in papers per lifetime) and Numbers of Papers so Produced1

-- Papers/man Papers Remarks

1 100

2 25

3 11.1

4 6.2

5 4

6 2.8

7 2

8 1.5

9 1.2

10 1

10-11.1 1

11.1 - 12.5 1

12.5 - 14.2 1

14.2 - 16.7 I

16.7 -20 1

20-25 1

25 -33.3 1

33.3 - 50 1

50-100 1

Over 100 1

100

50

33.3

25 20

16.7

14.2

12.5

11.1

10 . .

lot

ll.l+

12.5 +

14.2 +

16.7 +

20 t

25 +

33.3 +

50 +

100+

. The 75 per cent of men who are low scorers produce one quarter of all papers.

. . Subtotal: 10 men produce more than 50 per cent of all papers.

The top two men produce one-quarter of all papers.

Total 165 586 t

Average papers/man = 586/165 = 3.54

1. Table constructed on basis of exactly 100 men with a single published paper. Other entries computed from Lotka’s law.

Source: Derek J. de Solla Price, Litlle Science, Big Science, Columbia Univ. Press, 1763, p” 45.

35

Table 3 Origin of Science Abstracts

Percentages of World Total

Share of GNP Share of Share of Share of 1964 rt6yls. Abstr. Chem. Abstr. Population

1965 1964

Country

(i) Larger participants :

USA

USSR

32.8 31.6 28.5

15.6 15.6 20.7

5.9

7.0

1.8

0.5

.W. Germany 5.2

E. Germany 0.8 I 6.2 6.3

2.2

UK 4.8 13.6’ 6.7 1.6

France 4.5 6.3 4.5 1.4

Japan 3.6 7.8 7.3 2.9

Italy 2.6 3.4 2.7 1.5

Canada 2.2 1.1 2.0 0.6

India 2.2 1.8 2.2 14.4

Poland 1.6 1.5 2.9 0.9

Australia 1.1 0.5 1.2 0.3

Romania 1.0 0.6 0.9 0.5

Spain 0.9 0.2 0.4 1.0

Sweden 0.9 0.7 0.9 0.2

Netherlands 0.9 5.2l 0.8 0.4

Belgium 0.8 0.3 0.6 0.3

Czechoslovakia 0.7 0.9 1.6 0.4

Switzerland 0.7 1.0 1 .o 0.2

Hungary 0.5 0.5 1.0 0.3

Austria 0.4 0.2 0.5 0.2

Bulgaria 0.4 0.2 0.5 0.2

(ii) Smaller participants and non-participants : -

All other countries 15.8 0.8 4.6 57.5

1. Data known to be swollen because of one or more large international journals published from this nation.

Source: Derek J. de Solla Price, ” Research on Research “, Journeys in Science: Small Steps - Great Strides, Report of the 12th Airforce Office of Scientific Research Science Seminar, University of New Mexico Press, 1967.

36

--- __--- --. _ ------_.I--.. .- _- -.-_-___

Table 4 Percentage of women on the prolessional staffs o/higher educational institutions in 1947 compared with percentage of articles in professional journals contributed by women in 1940, and 1960.

1947 1940% 1960+

Discipline Percentage Percentage Sample Percentage Sample of Women of Articles Size of Articles Size in Field by Women by Women

Physics-mathematics

Chemistry

Biology

Geology

Engineering

Agriculture

Veterinary medicine

Medicine

Physical culture

Social-political and philosophy

Economics

History

Geography

Philology

Arts

Law

Pedagogy

21.1 11 (247) 7

45.3 26 (206) 40

47.8 26 (384) 36

23.4 9 (186) 25

10.5 4 (868) 9

23.0 14 (156) 21

22.3 6 (274) 16

48.0 24 (203) 39

27.6 9 (127) 13

31.0 8 (131) 10

16.3 9 (168) 11

30.1 21 (154) 14

28.7 9 (105) 12

67.7 24 (134) 21

34.0 11 (216) 37

12.7 6 (121) 10

40.0 22 (294) 26

(1270)

(320)

(565)

(315)

(1877)

(409)

(331)

(549)

(115)

(180) (447)

(188)

(125)

(227) (207) (203)

(639)

Total 35.0 16 (3974) 19 (7967)

Source: Norton T. Dodge, Women in the Soviet Economy: their ro^le in economic. scientific and technical development, Johns Hopkins Press, Baltimore, 1966, page 228.

37

Table 5 Distribution of Research Publications and their Origin

Institution

Total Research Letters Technical Symposia Review Case Others no. of papers to editor reports and and reports publications and short conference informative

communi- papers articles cations

Universities1 3816 2492 613 6 40 353 287 25

CSIR* 766 537 115 22 23 69 -

Medical laboratories

and hospitals 1 023 543 108 - 203 157 12

Agricultural

laboratories 1187 764 291 4 1 123 4

Industrial and other

laboratories 3 1545 957 277 43 83 177 3

Other institutions,

individuals 1122 485 99 5 7 460 60 6

From abroad 345 170 20 6 34 102 13

Total 9804 5948 1 523 86 193 1 487 524 43

1. Including medical, engineeringand agricultural colleges.

2. Including research association laboratories supported by CSIR. 3. Including research organizations of all ministries not covered by the above groups, private maintained laboratories,

etc.

Source: B.V. Rangarao, ” Scientific Research in India : an Analysis of Publications “, journal o/ Scientific and

Industrial Research, 1967, Vol. 26, No. 4.

38

- . . - . --_l_---l . . - . - _ . . . . . . - . - - - - - -~ I .___

Table 6 Institutions from which more than 100 Papers were Published during the Year under Analysis

Total no. No. in No. of disciplines foreign in which contribution journals is

>lO 10-5 <5

University of Calcutta l

Agra University

Indian Agricultural Research Institute

Banaras Hindu

University

Delhi University

Indian Institute of Science

Rajasthan University

University of Madras 2

University of Bombay

Panjab University

Andhra University

Atomic Energy Establishment

Kerala University

University of Allahabad

University of Lucknow

Tata Institute of Fundamental Research

Vikram University

All Institutions (17)

279 50 a 3 1

260 10 5 1 4

227 12 3 1

226 22 6 3 2

206 58 6 1 4

194 26

177 32

174 16

162 7

160 la

154 20

133 26

128 4

110 49

110 15

107 63 2

107 2 3

4

5

2 914 430

1. Excluding Saha Institute of Nuclear Physics.

2. Excluding Agricultural College and Research Institute, Coimbatore.

Source: B.V. Rangarao, “Scientific Research in India: an Analysis of Publications “, Journal of

Scientijic and industrial Research. 1967, Vol. 26, No. 4.

39

Table 7 Number of Discoveries in the Medical Sciences by Nations, 1800-1926

Year U.S.A. England France Germany Other Unknown Total

1800-09 2 a 9 5

1810-19 3 14 19 6

1820-29 1 12 26 12

1830-39 4 20 18 25

1840-49 6 14 13 28

1850-59 7 12 11 32

1860-69 5 5 10 33

1870-79 5 7 7 37

1880-89 la 12 19 74

1890-99 26 13 18 44

1900-09 28 18 13 61

1910-19 40 13 a 20

1920-26 27 3 3 7

L

2

5

3

7

4

7

6

19

24

20

11

2

1

3

1

1

-

3

2

1

5

11

a

7

2

27

47

57

71

68

69

62

63

147

136

148

99

44

Source : J ~ Ben-David, ” Scientific Productivity and Academic Organitation in 19th Century Medicine ” American

Sociological Review, December, 1960.

Table 8 Patents granted to Foreigners as a Percentage of Total Patents, 1957-61

Country %

Belgium 85.55

Canada 94.65

Ceylon 83.39

Czechoslovakia 12.78

Federal Republic of Germany 37.14

Finland 78.07

France 59.36

India 89.38

Ireland 96.51

Italy 62.85

Pakistan 95.75

Poland 37.94

Sweden 69.30

Switzerland 64.80

Turkey 91.73

U.S.S.R. 00.72

U.S.A. 15.72

Yugoslavia 60.50

Source: United Nations, The Role of Patents in the Transjer o/

Technology to Developing Countries, (Dept. of Economic &

Social Affairs), New York, 1964, Annex E.

40

-_-..- _ _ ._...._ _^_..~. . .._. -_ .-_._ _._ _ -.. _.... ~- -- _ .- .___ . - . -- ---_.- _.

Table 9 Approved Enterprise-to-Enterprise Agreements for the Transfer of Technology in lapan

A-type contracts1 B-type contracts 2 Total payments for contracts

Year

Number $‘OOO

Payments

Million ye” Number

Jl’OOO

Payments

Million ye”

(A + B)

us pooo

1950 27 501 .l

1951 101 4 841.9

1952 133 8 156.1

1953 103 11 466.9

1954 82 13 011.1

1955 72 17 963.1

1956 144 28 416.9

1957 118 39 438.9

1958 90 44 370.0

1959 153 54 196.1

1960 327 83 466.1

1961 320 98 229.7

1962 328 104 491.1

1963 564 134 380.0

1964 500 139 000.0

1965 472

180.4

1 743.1

2 936.2

4 128.1

4 684.0

6 466.7

10 230.1

14 198.0

15 973.2

19 510.6

30 047.8

35 362.7

37 616.8

48 376.8

50.000.0

49 2 133.6

87 1 854.2

110 1 779.7

133 2 411.9

131 2 782.2

113 2 033.6

167 4 922.8

136 3 181.1

152 3 479.4

225 7 659.4

261 11 421.4

281 17 441.4

429 10 474.4

573 11 464.7

541 16 402.0

486

768.1

667.5

640.7

868.3

1 001.6

731 .l

1 772.2

1 145.2

1 252.7

2 757.4

4 111.7

6 278.8

3 770.8

4 127.3

5 900.0

2 634.7

6 696.1

9 935.8

13 878.8

15 793.3

20 222.3

33 339.7

42 620.0

47 849.4

61 815.5

94 887.5

115 671.1

114 965.5

145 844.7

155 402.0

160 000.0

Total 3 534 642 454.1 281 454.5 3 874 83 042.5 35 793.5 1 040 899.1

1. An A-type technological assistance contract is a” agreement whose duration ot period of royalty payment is one year

ot more and whose royalty is to be paid in foreign currency. These are governed by the Foreign Investment Law.

2. A B-type technological assistance contract is a” agreement whose duration or period of royalty payment is less than

one year ot whose royalty is to be paid in Japanese currency. These come under the Foreign Trade and Exchange

Control Law.

Source: Table reproduced from: C. H. G. Oldham, C. Freeman, E. Tutkcan, The Transfer of Technology to Developing

Countries with Special Reference to Licensing and Know-how Agreements, United Nations Conference on Trade

and Development (UNCTAD), November 1767, page 21”

UNESCO PUBLICATIONS: NATIONAL DISTRIBUTORS

Afghanistan Albania Alpe*ia

Argentina Australia

Austria Belgium

Chins Colombia

Congo (Dun. Rep. of) Costa Rica

Cuba CYPrt”

Czechoslovakia Denmark

Dominican Republic Ecuador

El Salvador EF’E;$

Fle”l2.s French West Indies

G-“Y (Fed$&

G*.XCe Guatemala

Haiti “=wfgr;

Iceland India

Indonesia Iran Iraq

Ireland Isteel

Ivory Coast Jamaica

JsPs” Jordan

:zz: Kuwait

Lebanon Liberia

Libya Liechtenstein Luxembourg Madagascar

Panusai, Press Department, Royal Afghan Ministry of Education, KABUL. N. Sh. Botirneve Naim Fasheri. T~ANA. Institut pddagogique national, II, rue Ali-Haddad (ex-rue 2aBtch.s). ALGER. Editorial Sudamericana S.A.. Humberto I n.” 54s. T.E. 30. 7518. BUENOS AIRES. Publications : Educational Supplies Pty. Ltd., Box 33, Post Office, Brookvale eroo, N.S.W. Perio- dicnls : Dominic Pty. Ltd., Box 33. Post Office, Brookvale aroo, N.S.W. Sub-agent: United Nations Association of Australia, Victorian Division, 4th Floor, Askew House, 364 Lonsdsle St., MELBOURNE

(Victoris) 3000. Verlag Georg Fromrne & Co., Spengergasse 39. Wrtt~ 5. All publications: Editions ‘Labor’. 34s. rue Royale, BRUXELLE~ 3 ; N.V. Standaard Wetenschsppelijke Uitgeverlj, Belgiglei, ANTWBRP~ I. For ‘The Courier’ and slides only: Jean De Lannoy,, I I 2. rue du TrBne, Bguxet.~ss 5, Comisi6n National Boliviana de la Uneaco, Mimsterio de Educaci6n y Cultura. casills de corre~ “0 4107. LA Paz. Libreria Universitaria. Universidad San Francisco Xavier, apartsdo sts, SVCRH. Funda$o Getulio Vargas, caixa postal 4081-ZC-05. Rro DH JANE~RO (GUANABARA). Raznoiznos, I Tsar Assert, SOFIJA. Librsirie Albert Portail, 14, avenue Boulloche. PHNOM-PGNH. Papeterie modeme, Mailer et Cie, B.P. 495, YAOUNDB. The Queen’s Printer, OTTAWA (Ont.). Lake House Bookshop, Sir Chittsnrpslam Gardiner Mawata, P.O. Box 244. COLOMBO z. Ah publicntiom: Editorial Universitsria S.A., casilla reuse, SANTIAGO. Fm ‘The Courier’: Cotnisidn National de la Unesce, Mat-Iver 764, dpt”. 63, S~~rt~oo. The World Book CO. Ltd., 99 Chungkiig South Road, section I, TAIPM (Taiwan/Formosa). Libreria Buchhols ,Galet+a, avenids Jimenez de Quesada 8-40. BOGOTA; EdicionesTercer Mundo, spsrtsdo aereo 4817, BOOOTA; Distrilibros Ltda., Pio Alvonso Garcia csrrers 4.s. n.0~: 36-119 y 36-rs5, CARTI\~ENA J. Germ&~ Rodriguez H., oficina 101, Edificio Bsnco de Bogota’, apartado “scions1 83, GIRARDOT. Cundinamarca; Lihreria Universitarla. Univenidad Pedag6gica de Colombia, TUNJA. La Librairie. Institut politique congolais, B.P. 2307, KINSHASA. Libreria Trejos, S.A., apartado 1313. SAN Jose. T&forms z&5 y 3200. For ‘The COURT’: CURIOS VslerinSa’en~& CO. Ltds., ‘ElPalacio de la Revistas’, apartsde 1924. SAN Josd. Instituto de1 Libm, Departamento Econ&nico, Err&a y San Pedro, Cerre, LA HAB~A. ‘MAM’. Archbishop Makarios 3rd Avenue. P.O. Box 1722. NICOSIA. SNTL, Spsle”s 51. PRAW r (Permanent dzsplay); Zahranicni literatura, I I Soukenicka, PRAHA 1. Ejnar Munksgaard Ltd., 6 Nerregade. t 165 KBBENHAVN K. Librerla Dominicana, Mercedes 49. apartado de correos 656, SANTO DOMINGO. Csss de 1s Culturs Ecusterisns, Nuclei de1 G~ayas, Pedro Moncayo y 9 de Octubre, casilla de corre~ 354s GUAYAQUIL. Librerla Culture1 Salvadorena, S.A., Editicio San Martin. 6.. calle Oriente n.0 t 18, SAN SALVADOR. National Commission for Unesco, P.O. Box 2996, A~ors ABABA. Akateeminen Kirjakauppa, s Keskuskatu. Hasmrtr. Librai& de l’U”es~~, place de Fontenoy, 75 P-7’. CCP rs598-48. Libnirie J. Becage. sue Lsroir. BP. ~8. FORT-D&FRAN(g (Msrtinique). R. Oldenbourg Verlsg. Unesco-Vertrieb f0r Deutschland, Rosenhehnerstrssse 145. MONCHEN 8. Methodist Book Depot Ltd., Atlantic House. Connncrcisl Street, P.O. Box roe. CAPE COAST. Librairie H. Ksulhnann, 28, rue du Stade, ATHENAI; Libraitie Elefthereudakis. Nikkis 4. ATHENAI. Comisi6n Nacionsl de la Unesco. 6.’ Calle 9.17, mns I, GUATEMALA. Librsirie ‘Ale Csrnvelle’, 36, rue Roux, B.P. I I I, PORT-AU-PRINCE. Swindon Book Co.. 13-15 Lock Road, Kow~oo~. Akademiai K6nyvesbolt V&i u a% BUDAP~ V. A.K.V. Konyvtavosok Boltja, Ndpk6zdvsss&g utja 16 BUDAPEST vt. Snaebjiim Jonsson & Co. H. F.. Hafnarstracti 9, R~~~~JAvIK. Orient Longmans Ltd.: Nicol Road, Ballard Estate BoMBAv I; 17 Chittsmnjan Avenue, CALCUTTA 13; 36s Mount Road, MADRAS ‘2; 3/5 Asaf Ali Road, NEW DELHI I. Sub-depots: Oxford Book& Stationery Co., r7Park Street, CALCITITA 16; and Scindia House, NEW DELHI. India” National Commission for Unesce. stt. : No. 214. Shastri Bhswan. NEW DELHI 1.

The Librarian. Ministry of Education, ‘C’ Wing, Reem

Indira P.T.. Dil. Dr. Sam Ratulanaic 37, DJAKARTA. Cornntissio” n&ion& irsnienne pour I’Unesco. svenue du Mu&e. TPH!&AN. McKenzie’s Bookshop, Al-Rsshid Street BAGHDAD ; University Bookstore, University of Bsghdsd. P.0 BOX 75. BAGHDAD. The National Press, e Wellington Road, Ballsbridge, DUBLIN 4. Emanuel Brown fornterly Blunrstein’s Bookstores: 35 Allenhy Road and 48 Nshlst Benjamin Street, TEL AVIV. Libreris Connnissionsria Sansoni S.p.A., via Lsmsnn~rs 45. cssells postale 55z,501st FIRENZE; Libreris Intemssionale Rizsoli, Gal&a Colonna, Lsrgo Chigi, ROMA; Libreria Zanichelli, Piazza Gslvsni t/h, BOLOGNA: Hoepli, via Ulrico Hoepli 5. MILANO; Librsirie frsrwise, piazza Castello 9, TORINO: Diffusione Edizioni Anglo-Americane, via Lima 28, 00198, ROMA. Centre d’edition et de difhts’on africaines, bolte postale 4541. ABIDJAN PLATEAU. Ssngster’s Book Stores Ltd.. P.O. Box 366, mr Water Lane, KINGSTON. Msruren Co. Ltd., P.O. Box 5050, Tokyo International, TOKYO. Joseph I. Bahous & Co., Dar-ul-Kutub, Salt Road, P.O. Box 66, AMMAN. ESA Bookshop, P.O. Box 30167. NNROSI. Korean National Commission for Unesco, P.O. Box Central 64. SEOUL. The Kuwait Bookshop Co. Ltd., P.O. Box 2942. KUWAIT. Librsiries Antoine, A. Naufal et Freres, B.P. 656. BEYROUTH. Cole& Ysncy Bookshops Ltd., P.O. Box 286. MONROVIA. Agency for Development of Publication and Distribution, P.O. Box 261, TRIPOLI. Eurecs” Trust Reg., P.O.B. 5, SC~AAN. Librairie Psul Bruck. sf GrsndeyRue. LLJXSMBOVRO. &&wbeb;gxtr: Comrmsston natmnale de !a Rdpublique malgache, Minis&e de I’Education natto”:.!c

Fm ‘The &tier’: Service des ocuvres post et p&i-scolaires, Mini&m de I’Education nation& TANANARIVE.

Malaysia Mali

Malta Mauritius

Mexico MO”8CO

MOIOCCO

Mozambique Netherlands

Netherlands Antilles New Caledonia

New Zealand

Nicaragua Norway

Pakistan

Paragg Philippines

Poland Pcrtugal

Puerto Rico Southern Rhodesia

Romania Senegal

Singapore South Africa

Spain

Suds” Sweden

Switzerland Syria

Tanzania Thailand

Turkey Uganda

U.S.S.R. United Arab Republic

United Kingdom

U.S.A. Um3”aY

Venezuela

Republic of V&-Nom Yuu0slav1a

Federal Publications Sdn. Bhd., Balai Ber~ta. 3r J&n Riong, KUALA LUMPUR. Librairie populalre du Mali, B.P. 28. BAMAKO. Sapienza’s Library 26 Kingsway. VALLETTA. Nalanda Co. Ltd., 30 Bourbon Street, PORT-Lows. Editorial Hermes, Ignacio Mariscal41. MEXICO D.F. British Libraw. 10. boulevard des Moulins. MONTE-CARLO. AlI public&k> “Lihrarie ‘AUX belles im&s’, 281 avenue Mohammed V, RABAT (CCP 68.74). For ‘The Courier’ (for teachers) : Commission nationale marocaine pour I’Unesco. 20. Zenkat Mourabitine. RABAT (CCP 307-63). Salema & Carvalho Ltda.. caixa postal 192. BEIRA. N.V. Marthus Nijhoff. Lange Voorhout, 9 ‘~GRAVENHAOB. G. C. T. Van Dorp & Co. (Ned. Ant.) N. V., WILLEMSTAD (Cura~o. N.A.). Reprex, avenue de la Victoire. lmmeuble Painbouc. NOUMCA. Government Printing Office, Govetnment Bookshops: Rutland Street, P.O. Box 5344, AUCKLAND; 130 Oxford Terrace. P.O. Box 1721. CHRISTCHURCH; Alma Street, P.O. Box 857, HAMILTON ; Ptinces Street, P.O. Box 1.04. DUNEDIN; Mulgrave Street, Private Bag, WELLINGTON.

Libreria Cultural Nicaragiiense, calle 15 de Septiembre y avenida Bolivar ,apartado “.o 807. MANAGUA. Allpub/icicotionr: A. S. BokhjGmet, Akersgt. 41, OSLO 1. For ‘The Courin’: A. S. Narvesens Litteratuqeneste. Box 6125, OSLO 6. The West-Pak Publishing Co. Ltd., Unesco Publications House. P.O. Box 374, G.P.O.. LAHORG. Showmoms: Urdu Bazaar, LAHORB, and 57-58 Mutee Highway, G/6-1. ISLAMABAD. Pakistan Publications Bookshop : Satwar Road, RAWAI.PlNol ; Parlbagh, DACCA. Melcbor Garcia, Eligio Ayala 1650, kWNCI6N. Distribuidora INCA S.A.. Emilio Althaus 470. Lime. apartado 3115. LIMA. The Modern Book Co., 928 Rizal Avenue, P.O. Box 632, MANILA. Osrodek Rozpowzechniania Wydawnictw Naukowych PAN, Palac Kultury t Nauki. Warszawa. Dins & Andrade Ltda., Libmria Portugal, rus do Carmo 70, LISBOA. Spanish English Publications, Eleanor Roosevelt I 15, apartado 19r2, HATO REY. Textbook Sales (PVT) Ltd. 67 Union Avenue, SALISBURY. Cartimex, P.O. Box 134-r35. 126 c&a Victonci, BUCURESTI. (Telex: 226.) La Maison du Livre. I 3. .wen”e Roume, B.P. 20-60. DAKAR. Federal Publications Sdn Bhd., Times House, River Valley Road, SINGAPORE 9. Van Schaik’s Bookstore (Pty.) Ltd., Libri Building, Church Street, P.O. Box 724, PRBTORIA. All publications: Libreria Cientifica Medinaceli, Duque de Medinaceli 4, MADRID 14. For ‘The Courier’: Ediciones Ibemameticanas S.A.. calle de Onate 15, MADRID; Ediciones Liber. apartado de correos 17, ONDARROA (Vizcaya). AI Bashir Bookshoo. P.O. Box I I 18. KHARTOUM. Al1publicotim.s: AjB. C. E. Fritzes i<“ngl. Hovbokhandel. Fredsgatan t, STOCKHOLM 16. For ‘The Courim’: The United Nations Association of Sweden, Vasagatan 15-17. STOCKHOLM C. Europa Verlag, Rtimistnase 5. ZL~ICH; Librairie Payot, 6. rue Grenus 121 I GEN~VE II. Libra& Sayegh, Immeuble Diab. rue du Parlement, B.P. 704. DAMAS. Dar es Salaam Bookshop, P.O. Box 9030. DAR FS SALAAM. Suksapan Panit, Mansion 9, Rajdamnem Avenue, BANGKOK. Librnitie Hachette, 469 Istiklal Caddesi, Beyoglu, ISTANBUL. Uganda Bookshop. P.O. Box 145, KAMPALA. Mezhdunarodnaja Kniga. MOSKVA G-zoo. Libra& Kasr El Nil, 38. rue Kasr El Nil, Ls CAERE, Sub-depot: Ls Renaissance d’dgypte, 9 SR. Adly Pasha, CAIRO (Egypt). H.M. Stationery Office. P.O. Box 569. LONDON S.E.r ; Government bookshops: London, Belfast. Birmingham, Cardiff. Edinburgh. Manchester. UnescoPublicationa Center. P.O. Box 433, New York. N.Y. 10016. Editorial Losads Uruguaya. S.A../Libreria Losada, Maldonado ro92/Colon1a 1340, MONTEVIDEO. Libreria Historia, Monjas a Padre Sierra. Edificio Oeste 2. n. o 6 (frente al Capitolio). apartado de correos 73.20, CARACAS. Libnirie-Papeterie Xuan-Thu, 185-193 rue Tu-Do. B.P. 283. SAIGON. Juaoslovensk;l KnJiga, Terpzije 27. BEOGRAD. Druvnn Zaluzbs Slorenijc Matni Trg. 26 LJUBLJANA.

yneasurement of output of research and experimental...

Documents