using social networks methodology to evaluate research & development programs
Post on 04-Jan-2016
32 Views
Preview:
DESCRIPTION
TRANSCRIPT
Using Social Networks Methodology Using Social Networks Methodology to Evaluateto Evaluate
Research & Development ProgramsResearch & Development Programs
Franco MalerbaFranco MalerbaCESPRICESPRI
Luigi Bocconi UniversityLuigi Bocconi University&&
Nicholas S. VonortasNicholas S. VonortasCenter for International Science and Technology PolicyCenter for International Science and Technology Policy
and Department of Economicsand Department of EconomicsThe George Washington UniversityThe George Washington University
Joint AEA/CES MeetingsJoint AEA/CES MeetingsToronto, CanadaToronto, CanadaOctober 27, 2005October 27, 2005
PRIOR RESULTS ON IST-RTD PRIOR RESULTS ON IST-RTD NETWORKSNETWORKS
Prior Results Prior Results European IST RTD NetworksEuropean IST RTD Networks
The network of research collaborations has:The network of research collaborations has:
A self-organizing structure, dominated by A self-organizing structure, dominated by “hubs”, which are also key nodes in National “hubs”, which are also key nodes in National research networksresearch networks
A scale-free architecture at the thematic levelsA scale-free architecture at the thematic levels
Prior ResultsPrior ResultsEuropean IST RTD NetworksEuropean IST RTD Networks
European research is characterized by “small world” European research is characterized by “small world” connectivityconnectivity
Strong tendency of scientists to cluster around national Strong tendency of scientists to cluster around national communitiescommunities
Strong tendency to cluster with research disciplines and Strong tendency to cluster with research disciplines and within industrial sectorswithin industrial sectors
The funding structure has a strong influence on research The funding structure has a strong influence on research co-operationsco-operations
Prior ResultsPrior ResultsEuropean IST RTD NetworksEuropean IST RTD Networks
As a result of the new As a result of the new Integrated ProjectsIntegrated Projects and and Networks Networks of Excellenceof Excellence::
The density of links is higherThe density of links is higher The share of participants in the principal component The share of participants in the principal component
is higheris higher The average path length is lowerThe average path length is lower Large firms and research institutes are more dominant Large firms and research institutes are more dominant
as gate-keepers of collaborationas gate-keepers of collaboration Small companies are “crowded out” relative to FP5Small companies are “crowded out” relative to FP5
Prior ResultsPrior ResultsEuropean IST RTD NetworksEuropean IST RTD Networks
The IST RTD network as a whole has “small world” The IST RTD network as a whole has “small world” characteristics - but this is not true for each and every characteristics - but this is not true for each and every one of its programmesone of its programmes
FP6 is more likely than other research collaboration FP6 is more likely than other research collaboration frameworks to:frameworks to:
Connect universities and industryConnect universities and industry Connect different research themesConnect different research themes Include new Member StatesInclude new Member States Include key patent-holdersInclude key patent-holders Include SMEsInclude SMEs
CURRENT STUDY ON IST-RTD CURRENT STUDY ON IST-RTD NETWORKSNETWORKS
Towards an ERA for IST:Towards an ERA for IST:Overall ObjectivesOverall Objectives
Develop and apply a quantitative analytical framework Develop and apply a quantitative analytical framework for the assessment of the characteristics and for the assessment of the characteristics and performance of networks supported by IST RTD in performance of networks supported by IST RTD in FP5 and FP6.FP5 and FP6.
Analyze knowledge and partnership networks in Analyze knowledge and partnership networks in selected IST RTD domains, concentrating on network selected IST RTD domains, concentrating on network nature, topology, time evolution and effectiveness.nature, topology, time evolution and effectiveness.
Supplement quantitative information with some Supplement quantitative information with some qualitative information, and inter-organizational qualitative information, and inter-organizational networks with inter-personal networksnetworks with inter-personal networks
Towards an ERA for IST:Towards an ERA for IST:Evaluation QuestionsEvaluation Questions
How do the characteristics of the How do the characteristics of the IST-RTD IST-RTD partnership and knowledge networkspartnership and knowledge networks compare with compare with the characteristics of the the characteristics of the global partnership and global partnership and knowledge networksknowledge networks of IST-RTD companies and of IST-RTD companies and with the characteristics of the related global with the characteristics of the related global networks?networks?
How well are the companies participating in IST How well are the companies participating in IST RTD programs positioned in the global partnership RTD programs positioned in the global partnership and knowledge networks?and knowledge networks?
Towards an ERA for IST:Towards an ERA for IST:Evaluation QuestionsEvaluation Questions
How effective are IST-RTD networks as How effective are IST-RTD networks as mechanisms for transmitting knowledge?mechanisms for transmitting knowledge?
Are the Integrated Projects (IPs) and the Are the Integrated Projects (IPs) and the Networks of Excellence (NoEs) creating Networks of Excellence (NoEs) creating leading “knowledge hubs”?leading “knowledge hubs”?
What makes these “knowledge hubs” What makes these “knowledge hubs” effective?effective?
Towards an ERA for IST:Towards an ERA for IST:Evaluation QuestionsEvaluation Questions
To what extent does the prominent network status of To what extent does the prominent network status of certain IST RTD companies of clusters match the EU certain IST RTD companies of clusters match the EU technological leadership in certain areas?technological leadership in certain areas?
Are the global networks of selected “hub” companies Are the global networks of selected “hub” companies with extensive ICT supply chains represented in the with extensive ICT supply chains represented in the FP6 IST RTD?FP6 IST RTD?
Are the perceived national IST “knowledge hubs” Are the perceived national IST “knowledge hubs” well integrated into the FP6 network?well integrated into the FP6 network?
Selection of IST
technology domains
IST-RTDFramework Programme 6
Matching of IPC codes with technological domains
Patent examiners
Field experts
Matching of SIC codes with technological domains
INNET alliances
EP-CESPRI patents/citations
KNOWLEDGE NETWORKS (Ib, IIb, IIIb)
PARTNERSHIP NETWORKS
(IIa, IIIa)
PARTNERSHIP NETWORK Ia
Towards an ERA for IST:Towards an ERA for IST:Network TypesNetwork Types
IST-RTD partnership networkIST-RTD partnership network
IST-RTD knowledge networkIST-RTD knowledge network
Global partnership network of IST-RTD project participantsGlobal partnership network of IST-RTD project participants
Global knowledge network of IST-RTD project participantsGlobal knowledge network of IST-RTD project participants
Global partnership network akin to the E technology unitsGlobal partnership network akin to the E technology units
Global knowledge network akin to the E technology unitsGlobal knowledge network akin to the E technology units
Towards an ERA for IST:Towards an ERA for IST:Examined ProgramsExamined Programs
FP6 Thematic Areas Strategic objectives
eSafety of road and air transports
eHealth
Technology-enhanced learning and access to cultural heritage
Towards a global dependability and security framework
Networked business and governments
eInclusion Applications and Services for the Mobile User and worker
Cross-media content for leisure and entertainment
GRID-based Systems and solving complex problems
1. Applied IST research addressing major societal and economic challenges
Improving Risk management
Broadband for all
Mobile and wireless systems beyond 3G
Networked audiovisual systems and home platforms
Open development platforms for software and services
2. Communication, computing and software technologies
Embedded systems
Pushing the limits of CMOS and preparing for post-CMOS
Micro and nano-systems
Advanced displays
3. Components and micro-systems
Optical, opto-electronic, photonic functional components
Towards an ERA for IST:Towards an ERA for IST:Examined ProgramsExamined Programs
FP 5
FP6
Key Actions Thematic Areas
1. System and services for the citizen
2. New method of wok and electronic commerce
1. Applied IST research addressing major societal and economic challenges
2. Communication, computing
and software technologies
4. Essential technologies and infrastructures
3. Components and micro-systems
TA 1-2-3 FP6TA 1-2-3 FP6
Not SelectedNot Selected 115115 27,327,3
SelectedSelected 307307 72,772,7
Projects
Not selected27%
Selected73%
TA 1-2-3 FP6TA 1-2-3 FP6
Participants
Not selectedNot selected 13401340 21,821,8
SelectedSelected 48144814 78,278,2
Not selected22%
Selected78%
ParticipantsParticipants: counted once for every project they have participated in
By instrument(projects)
CA: Coordination ActionIP: Integrated ProjectNoE: Network of Excellence
SSA: Specific Support ProjectSTREP: Specific Targeted Research Project
TA 1-2-3 FP6TA 1-2-3 FP6
CA IP NoE SSA STREP
0
0,1
0,2
0,3
0,4
0,5
Not Selected
Selected
Organization Type
HE: Higher EducationIND: industry
REC: ResearchOTH: Other
TA 1-2-3 FP6TA 1-2-3 FP6
HE INDOTH
REC
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
Not Selected
Selected
Not SelectedNot Selected Large CompanyLarge Company 10321032 21.1521.15
SMESME 260260 21.1721.17
SelectedSelected Large CompanyLarge Company 38463846 78.8578.85
SMESME 968968 78.8378.83
SMEs and Large Enterprises
TA 1-2-3 FP6TA 1-2-3 FP6
SMELarge
Company
00,1
0,20,3
0,4
0,5
0,6
0,7
0,8
Not Selected
Selected
Correspondence between TAs, IPC Correspondence between TAs, IPC and SIC Codes…and SIC Codes…
Technology Classes: PatentsTechnology Classes: Patents
ClassClass descriptiondescription ClassClass descriptiondescription
11 Electrical engineeringElectrical engineering 1616 Chemical EngineeringChemical Engineering
22 Audiovisual technologyAudiovisual technology 1717 Surface TechnologySurface Technology
33 TelecommunicationsTelecommunications 1818 Materials ProcessingMaterials Processing
44 Information TechnologyInformation Technology 1919 Thermal ProcessesThermal Processes
55 SemiconductorsSemiconductors 2020 Environmental TechnologyEnvironmental Technology
66 OpticsOptics 2121 Machine ToolsMachine Tools
77 Control TechnologyControl Technology 2222 EnginesEngines
88 Medical TechnologyMedical Technology 2323 Mechanical ElementsMechanical Elements
99 Organic ChemistryOrganic Chemistry 2424 HandlingHandling
1010 PolymersPolymers 2525 Food ProcessingFood Processing
1111 PharmaceuticsPharmaceutics 2626 TransportTransport
1212 BiotechnologyBiotechnology 2727 Nuclear EngineeringNuclear Engineering
1313 MaterialsMaterials 2828 Space TechnologySpace Technology
1414 Food ChemistryFood Chemistry 2929 Consumer GoodsConsumer Goods
1515 Basic Materials ChemistryBasic Materials Chemistry 3030 Civil EngineeringCivil Engineering
0
5
10
15
20
25
30
TA3
TA2
TA1
FP6 Patents/ Technological ClassesFP6 Patents/ Technological Classes
SIC Codes: AlliancesSIC Codes: AlliancesClassClass descriptiondescription ClassClass descriptiondescription
13111311 Crude Petroleum and Natural GasCrude Petroleum and Natural Gas 48414841 Cable and Other Pay Television ServicesCable and Other Pay Television Services
28212821 Plastics Materials, Synthetic Resins, and Plastics Materials, Synthetic Resins, and Nonvulcanizable ElastomersNonvulcanizable Elastomers 48994899 Communications Services, Not Elsewhere Communications Services, Not Elsewhere
ClassifiedClassified
28362836 Biological Products, Except Diagnostic Biological Products, Except Diagnostic SubstancesSubstances 49114911 Electric ServicesElectric Services
35773577 Computer Peripheral Equipment, Not Computer Peripheral Equipment, Not Elsewhere ClassifiedElsewhere Classified 50455045 Computers and Computer Peripheral Computers and Computer Peripheral
Equipment and SoftwareEquipment and Software
35713571 Electronic ComputersElectronic Computers 50655065 Electronic Parts and Equipment, Not Electronic Parts and Equipment, Not Elsewhere ClassifiedElsewhere Classified
36513651 Household Audio and Video EquipmentHousehold Audio and Video Equipment 67946794 Patent Owners and LessorsPatent Owners and Lessors
36613661 Telephone and Telegraph ApparatusTelephone and Telegraph Apparatus 73727372 Prepackaged SoftwarePrepackaged Software
36633663 Radio and Television Broadcasting and Radio and Television Broadcasting and Communications EquipmentCommunications Equipment 73757375 Information Retrieval ServicesInformation Retrieval Services
36743674 Semiconductors and Related DevicesSemiconductors and Related Devices 73737373 Computer Integrated Systems DesignComputer Integrated Systems Design
36793679 Electronic Components, Not Elsewhere Electronic Components, Not Elsewhere ClassifiedClassified 73717371 Computer Programming ServicesComputer Programming Services
37143714 Motor Vehicle Parts and AccessoriesMotor Vehicle Parts and Accessories 73797379 Computer Related Services, Not Elsewhere Computer Related Services, Not Elsewhere ClassifiedClassified
37113711 Motor Vehicles and Passenger Car BodiesMotor Vehicles and Passenger Car Bodies 87318731 Commercial Physical and Biological Commercial Physical and Biological ResearchResearch
45124512 Air Transportation, ScheduledAir Transportation, Scheduled 87328732 Commercial Economic, Sociological, and Commercial Economic, Sociological, and Educational ResearchEducational Research
48124812 Radiotelephone CommunicationsRadiotelephone Communications 87428742 Management Consulting ServicesManagement Consulting Services
48134813 Telephone Communications, Except Telephone Communications, Except RadiotelephoneRadiotelephone
1311
2821
2836
3577
3571
3651
3661
3663
3674
3679
3714
3711
4512
4812
4813
4841
4899
4911
5045
5065
6794
7372
7375
7373
7371
7379
8731
8732
8742
0%
2%
4%
6%
8%
10%
12% TA1TA2TA3
FP6 Alliances / Relevant SIC
Cliques Vs. StarsCliques Vs. Stars
Assumption about the structure of the Assumption about the structure of the affiliation network: clique vs. staraffiliation network: clique vs. star
Assumption about the role played by Assumption about the role played by prime contractor:prime contractor:
no specific role for participants - no specific role for participants - clique hp.clique hp.
vs.vs.
A participant serves as the coordinating agentA participant serves as the coordinating agent - - star hp.star hp.
PC
PC
Clique vs. StarClique vs. Star Both assumptions: rather strong and somewhat Both assumptions: rather strong and somewhat
arbitraryarbitrary
Equally reasonable,Equally reasonable, they represent the upper they represent the upper and lower limitsand lower limits
So we explore the main topological So we explore the main topological
characteristics of the network under both characteristics of the network under both assumptionsassumptions
Indicative Analysis: 3 subjectsIndicative Analysis: 3 subjects
Subject 1: Identifying HUBs andSubject 1: Identifying HUBs and their relative roles their relative roles
Hub definitionHub definition
• An organization is a hub in a specific network if it has many links and/or if it connects the otherwise unconnected parts of the network
The above translates into high degree centrality and/or high betweeness centrality
STYLIZED 3A PARTNERSHIP STYLIZED 3A PARTNERSHIP NETWORKNETWORK
The node labelled “HUB 3a” is the designated Hub for this network.
This is a stylized model of Network
3a (Alliances)
Give intuition behind the concept
of a Partnership Hub
A Hub is defined as a node exhibiting high value of betweenness
and degree
STYLIZED 3A PARTNERSHIP STYLIZED 3A PARTNERSHIP NETWORKNETWORK
Yellow nodes indicate
organizations participanting in
Framework Programme.
STYLIZED 1A PARTNERSHIP STYLIZED 1A PARTNERSHIP NETWORKNETWORK
The yellow node represents the relevant Hub in the stylized 1a partnership network
The blue node is the 3a network relevant Hub
This is a stylized model of Network
1a (FP Participants)
Links Between 1a Hubs and 3a HubsLinks Between 1a Hubs and 3a HubsBlue nodes are the 3a network Hubs
Yellow nodes represent the
1a network Hubs
1a Hubs are strongly inter-connected and
they are also connected with 3a Hubs
3a Hubs are NOT hubs in network 1a, BUT are gateways that connect FP organizations to the global network
1A FP6 (TA1) PARTNERSHIP 1A FP6 (TA1) PARTNERSHIP NETWORKNETWORK
Blue nodes are the 3a network Hubs
Yellow nodes represent 1a
network Hubs
Red nodes are other 3a network
participants within distance 1 from 3a
Hubs
1A FP6 (TA1) PARTNERSHIP 1A FP6 (TA1) PARTNERSHIP NETWORK (no IP)NETWORK (no IP)
This is the TA1 Network without the
links related to IP
The network is substantially different,
with many isolated nodes and diminished
complexity
Subject 2: Effectiveness ofSubject 2: Effectiveness of KNOWLEDGE HUBs KNOWLEDGE HUBs
Effectiveness of Knowledge HubsEffectiveness of Knowledge Hubs
Hubs as knowledge depositories
Hubs at the cross-road of information and ideas
• Number of Patents • Number of Citations Received• Number of Highly Cited Patents
• Degree Centrality • Betweeness Centrality
Effectiveness of Knowledge Hubs: Effectiveness of Knowledge Hubs: Hypothetical ExampleHypothetical Example
patent citation/patent
betweenness IIIb
degree IIIb
highly cited patent
0
0,02
0,04
0,06
0,08
0,1
0,12
KH IIIa
KH Ia
OTHER Ia
Effectiveness of Knowledge Hubs:Effectiveness of Knowledge Hubs:Hypothetical ExampleHypothetical Example
• closely matches that of global KHs in terms of three variables closely matches that of global KHs in terms of three variables (number of patents, network centralities);(number of patents, network centralities);
• lags seriously behind in terms of the remaining two variables lags seriously behind in terms of the remaining two variables that approximate the quality and the importance of their patent that approximate the quality and the importance of their patent portfolios;portfolios;
the FP KHs seem to perform better in diffusing knowledge the FP KHs seem to perform better in diffusing knowledge through their centrality roles in the networks than in creating through their centrality roles in the networks than in creating powerful and influential portfolios of new ideas.powerful and influential portfolios of new ideas.
Subject 3: LeadershipSubject 3: Leadership
Leadership
Two different definitions of Leadership:
• Technology Leadership:Technology Leadership: the role played by each the role played by each organisation in the innovative process organisation in the innovative process
• Market leadership:Market leadership: the share of revenues in ICT the share of revenues in ICT among EU25 among EU25
Technology Leadership
Technology leadership is defined in terms of two Technology leadership is defined in terms of two concepts:concepts:
• Niche overlapNiche overlap concerns the crowdedness of the concerns the crowdedness of the technological area explored by organisations. Its technological area explored by organisations. Its measure is based on similarity of technological measure is based on similarity of technological antecedents (i.e. co-citation).antecedents (i.e. co-citation).
• PrestigePrestige deriving from the direct technological ties deriving from the direct technological ties between actors (i.e. direct patent citations)between actors (i.e. direct patent citations)
Technology Leadership
Four different kinds of actors:Four different kinds of actors:
• Technology Leaders:Technology Leaders: a key source of knowledge a key source of knowledge spillovers for many other organizations in the industry. spillovers for many other organizations in the industry. Their research activity is focused on the exploitation of Their research activity is focused on the exploitation of opportunities in relatively mature and therefore highly opportunities in relatively mature and therefore highly crowded fieldscrowded fields
• Technology Brokers:Technology Brokers: sources of knowledge in sources of knowledge in relatively new and relatively new and unexploredunexplored fields fields
Technology Leadership
• Technology Followers:Technology Followers: they they do not contribute do not contribute significant spillovers to other organizations and engage significant spillovers to other organizations and engage into relatively mature and crowded technological into relatively mature and crowded technological subfieldssubfields
• Isolate Organisations:Isolate Organisations: they they do not receive direct do not receive direct citations from many other organizations and are citations from many other organizations and are exploring relatively untapped technological subfields.exploring relatively untapped technological subfields.
Technology Leadership:Hypothetical Example
-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
0 1 2 3 4 5 6 7 8 9 10
Crowding
Pre
stig
e
EU-Non FP
EU-FP KH
EU-FP Non KH
Global KH
Average prestige
Average alfa
Technology leaders
Technology isolates
Technology Leadership:Hypothetical Example
This analysis might suggest:
• The number of identified leaders and brokers that participate in the Framework Programme
• The number (and identity) of those who not only participate but they can also be characterized as Partnership HUBs in the Framework Programme.
top related