Measuring R&D Performance within Innovation Systems Perspective:

Illustrations from Ghana and Nigeria Agricultural Research Systems

Catherine Ragasa (IFPRI)

George Essegbey (STEPRI)

Aliyu Sabi Abdullahi (ARCN)

Motivation of the study• Investment in R&D (usually coupled with infrastructure)

is required for increasing agricultural productivity

• High ROR on investment on R&D (although with wide variability)

• Increased investment emphasized, but there is dearth in studies on the functioning of NARIs/NARS where the intended funding would go to• Are agricultural research organizations within Nigeria and

within Ghana performing differently and what are the factors that account for these differences?

• Are there patterns of differences between Ghana and Nigeria agricultural research performance measures that are explained by differences in their policies, emphasis and practices?

Measures of performance• Conventional indicators

• Number of publication (self-reported)• Number of technologies (self-reported)• Number of patents (self-reported)

• Organization theory and public sector motivation literature• Staff morale (staff perception; Likert scale)• Peer rating (perception of other organization; frequency)• Number of awards (self-reported)

• Innovation systems perspective• Interaction with various actors (frequency, perception of usefulness)• Involvement of actors in priority-setting; use of participatory approaches and

innovation platforms (dummy)• Level dissemination efforts of publications and technology (self-reported;

number)• Extent of use of publications (staff perception; Likert scale)• Extent of adoption of technologies (staff perception; Likert scale)

Potential explanatory factors• Organizational capacity (satisfaction on adequacy; actual ASTI data)

• Human, physical, financial resources

• Management systems (satisfaction on adequacy/implementation)• Planning processes • Information and coordination processes• Monitoring and evaluation system• Perception of transparency and other dimension of work environment• Training (technical and management)

• Organizational culture types – dilemma of control versus flexibility and people versus organization • Group, development, rational, hierarchical, balanced culture types

• Incentive systems & sources of accountability• Performance indicators reported being used (open-ended question)• Rewards/sanctions being used (open-ended question)• Salary (Salary costs/FTE; staff perception)• Openness of information about organization (staff perception, Likert scale)

Method and Data Sources

• NARIs under ARCN and CSIR, CRIG, and HEI

• Nigeria

• 47 organizations [all 15 RI, all 11 FCA, 21/48 faculties]

• 344 randomly-selected researchers (out of >3,000 total) [3-20 sample/organization]

• Ghana

• 16 organizations [all 10 RI; CRIG; and 6/15 faculties]

• 237 randomly-selected researchers (out of 706 total) [5-20 sample/organization]

• Face-to-face survey

• 2 types of questionnaire (for head and for sample researcher)

Majority of technologies produced are improved management practices

Nigeria (205) Ghana (109)0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

5838

72

59

56

10192

ChemicalMechanicalManagementBiological

* Preliminary

Limited awareness of adoption of technologies in both Ghana and Nigeria

Nigeria (87) Ghana (114)0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

37

763.44827586206897

17.2413793103448

528.735632183908

12.6436781609195

4

1.14942528735632

15

Wide adoption (>60%)Moderate adoption (41-60%)Some adoption (21-40%)Limited adoption (<20%)Zero adoptionNo knowledge

Observation (1)• Lack of farmer- or impact-orientation; mainly supply-driven • Heavy emphasis on publication and technology (N) and heavy

emphasis of internally-generated funds (G); little follow-up on what happens to the technologies

• Use of indicators such as “impact on/adoption of farmers”, “farmers’ problems solved”, “appreciation by farmers,” as performance indicators • 0% (G); 30% (N)

• Organization heads who mentioned these as motivations• 0% (G); 20% (N)

• Researchers who mentioned these as motivations • 15% (G); 5% (N)

• Organization heads who mentioned these or development outcomes (e.g., food security, poverty reduction) as central mission of organization according to org head• 20% (G,N)

What motivates researchers?

Nigeria

Prom

otio

n/hig

her s

alary

Skills

deve

lopm

ent/tr

ainin

g

More

timel

y re

lease

of f

unds

More

rese

arch

fund

s

Better l

abora

torie

s an

d facil

ities

-

5

10

15

20

25

30

35

40

45

50

Prom

otio

n/hig

her s

alary

Peer reco

gnitio

n

Apprecia

tion b

y fa

rmers

Skill d

evelo

pment/t

rain

ing

More

rese

arch

fund

s -

10

20

30

40

50

60

Ghana

What motivates organization heads?

Nigeria Ghana

Salary

& b

enefit

Staff

discip

line/m

orale

Recogn

ition/s

tatu

s

Job s

atisf

actio

n

Prom

otio

n

Career

deve

lopm

ent

Conduc

ive w

ork e

nviron

ment

0

10

20

30

40

50

60

0

10

20

30

40

50

60

70

Observation (2)• Different motivations across organizations and countries

• Most of the motivations are already being implemented (promotion, recognition, higher salary or bonus based on performance, training), but need more transparency in promotion and training participant selection; better implementation of M&E

• What are not being addressed are the basic needs (such as basic lab and research facilities, electricity, and internet). Conducive work environment has mainly to do with adequacy of resources.

• More timely research funds and research funds as motivating factors for both researchers and org leader

• Explore using other indicators not yet explored – appreciation by farmers

Average count of publications in last 3 years per researcher

Total With BS With MS With PhD0

2

4

6

8

NigeriaGhana

Observation (3) • Nigeria has more publications per researcher • Nigeria - stronger emphasis on publication (in both RI and HEI) in

explicit performance indicators used and implicit measures recognized by peers.

• Ghana – more focus on internationally-generated funds and associated more frequency of consultancy work rather than peer-reviewed publications

Limited linkages among various actors

Ghana

Farmers

Extensio

n agents

Private

secto

rNGO

Other r

esearc

hers

Intl o

rgMoA

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Nigeria

Farmers

Extensio

n agents

Private

NGO

Other r

esearc

hers

Intl o

rgMoFA

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Several times a month

About once a month

A few times a year

Never

Ideal frequency of interaction

GhanaNigeria

Farmers

Extensio

n agents

Private

NGO

Other r

esearc

hers

Intl o

rgMoFA

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Several times a month

About once a month

A few times a year

Never

Farmers

Extension agents

Private se

ctor

NGO

Other r

esearc

hers

Intl o

rgMoA

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Observation (4)• Linkages are important (staff, org head, correlation

analyses)• 2 strategies: (a) changing mindset; and (b) addressing the

binding constraints that hinder greater linkages, which include lack of funds, platforms, and transportation, time constraints, no incentive and no culture in the organization, or lack of interest or motivation from both parties

• Researchers in Ghana reported having better linkages with other innovation actors, but researchers in Nigeria reported having better linkages with other researchers. • Ghana - greater emphasis on non-state funds generated which

would very much require interaction and collaboration with possible sources of funds, such as private sector, NGOs, and international organizations.

• Nigeria – greater emphasis on publication, often requiring greater interaction with other researchers

Observation (5)• Variability in organizations within Ghana and Nigeria

• Good performing and bad performing organizations in the same country or institutional context; elements of good performance

• Factors (correlation analyses)• Greater satisfaction of facilities/physical resources more technology

produced per PhD, more awards received, peer rating (N); more technology and publication per PhD (N, G)

• International research collaboration more technologies, more publication, better perceived adoption, award, peer rating (G, N)

• Higher PhD/MS ratio higher publication per MS (G)• Higher operating funds per FTE more publication per MS (G)• More frequent linkages with farmers, private sector, MOA, other

researcher, Intl Org more technologies & publication, award, peer rating (G,N)

• M&E system, work environment, international research collaboration (N); control (G) more researchers perceiving more adoption

Observation (6)• Work environment almost all performance indicators• Majority of staff are satisfied with their job• Majority reported that their organizations are effective given its

budget and resources (Ghana >Nigeria)• Most important issue/dissatisfaction in both countries is

adequacy of physical resources and research funds• The second items that show much dissatisfaction among

respondents:• Not-so competitive salary and benefits seem to be a bigger

problem by staff in Ghana; while job security seems to be a much bigger problem in Nigeria.

• There is also reported dissatisfaction because of the level of corruption or misuse of funds in the organization and transparency in promotion and hiring processes for both countries (more in Nigeria)

Observation (7)• Output and productivity seems to respond to performance

indicators • If publication is mentioned as a performance indicator more publications

being produced by its researchers.• If technology is mentioned as a performance indicator more technologies

that researchers will contribute in producing.

• Presence of reward or sanction more technologies and publication per PhD and per MS

• Openness of information about performance of organization is correlated to publication, technology and peer rating

• Salary & benefits have been consistency mentioned, but variations of salary (variations in perception on competitive pay, adequacy of salary in relation to living expenses, and salary costs per FTE did not seem to be statistically correlated with variations in any of the performance indictors)

Observation (8)• Different performance ratings can be inconsistent or contradictory.

Puzzles:• More researchers in Ghana seem to have more linkages, more satisfied with

human and physical resources, organizational management practices, and work environment, but these do not seem to translate into greater productivity (publication, technology)

• More researchers in Ghana perceived that their organizations are effective given its budget and resources than in Nigeria

• Within Nigeria - more perceived adoption, award & peer rating all are positively correlated; but all are negatively correlated with productivity (publications and technology per PhD and per MS)

• Within Ghana – productivity and peer rating are positively correlated but all are negatively correlated with perceived adoption level

• From analysis perspective, avoid using composite index to capture all performance indicator, but to analyze them individually

• Trade-offs or balanced set of indicators? Trade-off between or combination of scholarly contribution versus impact of poor farmers and end-users

Concluding remarks• Complex and challenging process

• We tried a participatory approach, engaging researchers in several iterations of the questionnaire (technology, adoption, can mean differently to different people)

• Revisit definitions and measures especially when these metrics are to be scaled out to other countries

• Self-reporting (CV, not an evaluation, capacity strengthening)• Time lag of technologies and attribution to various factors• This study is a starting point; and more needs to be done• Qualitative approach (in-depth case studies with observational

approaches and historical perspective)• More research is needed to look at actual adoption and

impact of technologies (beyond perceptions of scientists)

Concluding remarks• Funding is required for basic institutional capacity

• Infrastructure seems to be a binding constraints • Linkages are important (but not automatic); linkage costs can be explicitly put

in the budget• Salary & benefit improvements are consistency emphasized by researchers

and leaders, but needs to be further studied• Human capacity strengthening (technical, management, value chain

approaches)• How can we make NARIs be more responsive and accountable to the needs

and demands farmers? • A lot of it is leadership/management commitment

• Performance indicators/targets, coupled with transparency• Openness of information about org performance is associated with greater

productivity and better peer rating • Hosting producer organizations within RI helps in the linkages

• External accountability• Greater international research collaboration • Producer organizations managing competitive grants