Mapping the Landscape of Operational Research in

Tuberculosis

Christian Lienhardt, Nita Bellare, Marcos EspinalTB Research Movement

Stop TB PartnershipWHO, Geneva

World Lung Conference Cancun, 3 November 2009

Background

• Promotion of research, including programme-based operational research (OR) is a full component of the WHO Stop-TB Strategy.

• Operational Research in TB is largely undefined and not properly evaluated

• Multitude of OR studies carried out worldwide, on a very large scope of TB control issues

• Little is known on the methods used, the results gathered and their impact on TB control programs.

Definitions of OR

• Operational Research in Public Health (Andersen, 1964) : OR as responding to "less industrialized societies" demand's for "systems composed of currently available techniques which give the optimal utilization of scarce economic resources". "Research that satisfies this demand can be called operations research, at some risk of criticism from other employers of the term”.

• Public Health Dictionary (Last, 2006): OR is “the systematic study of the way in which organizations function. This may be done by direct observation, a combination of observation and experiment, or statistical analysis of data from various aspects of the organization(s) under study. OR focuses on ways to improve performance of both individuals and groups, and of their work setting and equipment. OR includes health services research that aims at evaluating health services, outcomes and process by which services are provided. It involves epidemiology, economics and social and behavioral sciences”.

Definitions of OR

• WHO Global tuberculosis report (2009):

- OR defined as "research specifically aimed at developing interventions that result in improved policy-making, better design and implementation of health systems, and more efficient methods of service delivery"

- In 2007, 89 countries (including 20 high-burden countries) reported implementation of research activities related to TB control to WHO (compared with 49 countries in 2006); 400 research projects were reported.

Questions

• How to render OR more efficient and better formulated at helping policy decision-making ?– which type of OR projects are being conducted ? – what questions are being addressed ?– in which countries is research being conducted ?– by whom and with whom ? – with which methodology ?– which populations are being studied ?– what has been the impact of these studies ? Did the

research lead to changes in the programme, health systems organization or infrastructure ?

Objectives

Goal :This mapping exercise aims at exploring the scope of OR conducted in the area of TB worldwide and contribute to better defining OR in TB.

Objectives: 1. To map out the landscape of OR through a census of

institutions, NGOs and academia conducting OR studies on TB

2. To help identifying the gaps in research and developing an agenda on Operational Research in TB

3. To produce a database that enumerates projects conducted on OR worldwide

Methodology

• Broad definition of OR (2009 Global TB Report): “Research specifically aimed at developing interventions that result in improved policy-making, better design and implementation of health systems, and more efficient methods of service delivery”.

• Exploratory survey:– face-to-face interviews with Stop TB Partnership Working Group

(WGs) secretaries and WHO technical officers involved in OR on TB;

– contacts with TB Advisors in WHO Regional Offices; – internet search: academic institutions, research institutions,

NGOs, international organizations that mention OR in TB on their website, or published in peer reviewed journals.

Methodology

• Open ended questionnaire:– details from the OR conducted in the period 2006-

2009:• titles of project(s)• objectives of project(s)• region and country where OR was conducted• methodology and study population• status of project (i.e. data collection, analysis,

project beginning, not completed)– partners involved in projects – sources of funding– resulting publications in international journals– changes or enhancements made as a result of the

OR conducted

Results

Summary of responses:

• 74 contacts made (49 organizations, including WHO Regional offices as separate entities)

• 61 responders (82% response rate) - 32 organizations– government-based organizations (9), – non-profit organizations (10), – academia (10), – international organizations (3).

• 264 projects reviewed - 243 met the project’s broad definition of OR (92%)

• 134/243 (55%) : at least one main objective described

Classification of projects

• Projects classified according to Zachariah et al. criteria (LID, 2009):– improvement of programme outcomes in relation to

medical care and practice: 50 studies (21%) – feasibility of new strategies and interventions in

specific settings or populations: 35 (14%) – advocating for policy change : 158 (65%)

• These categories focus on how OR is relevant to health.

Geographical Spread

• Greatest number of projects were conducted in Sub-Saharan Africa (77)

• 61 projects conducted in the South East Asia region • 57 in the West Pacific region (primarily China)• 35 projects conducted in European region • 18 in American region (primarily Brazil)• 4 projects in East Mediterranean Region

• Many projects conducted in multiple countries

• 4 did not specify the country where the project was conducted.

Partners' Involvement

• Multiple partnership is the rule - technical or logistical capacity

• 243 partners associated with the 31 organizations • Average number of partners per organization: 9 (range:

1-64) – KNCV: 64 partners– CDC: 36– University of Leeds: 33– MSF Belgium: 19

• Most common key partners:– Ministry of Health (MOH) – National Tuberculosis Program (NTP) – NGO(s) and/or international organization

Funding

• 63 institutions provided funding to support the 243 projects listed:– 45% projects funded by local entities (MOH, National

TB programme or academic institutions) – 11% funded by USAID– 11% funded by the Global Fund for TB, AIDS and

Malaria– 5% funded by DFID (Department of Foreign and

International Development, UK) – 28%: multiple organizations.

Publications

• Overall, 157 publications were generated from 96 OR projects, with at least one publication

• 10 projects: publications in preparation, submitted or in press

• 48: projects not completed. • 89 projects (approximately 36%): no publication • According to the opinion of the project implementers,

108 projects (44%) led to change and/or enhancement of the programme performance

Discussion

• A major observation: definition of OR is still widely under discussion!!!

• For several responders, public health professionals "have their own understanding" of what OR entails.

• OR generally focuses on "how to improve the performance of a Programme"

• Various appreciations of what it entails:– implementation research– very wide scope as it relates to all aspects of health, so also

includes HSR, sociology, etc…

Discussion

• Laserson et al (2005): "OR is intended to provide managers, administrators, and policy makers with the information that they need to improve service delivery activities and plan future ones. It seeks practical solutions to problem situations and viable alternatives to unsatisfactory operating methods”.

• Zachariah et al (LID 2009): OR corresponds to “the search for knowledge on interventions, strategies, or tools that can enhance the quality, effectiveness, or coverage of programmes in which the research is being done”. This includes disease-control and programme-based research, health services research, feasibility, implementation research, or general public health research.

Limitations

• Greatest difficulty: receive complete responses from the respondents regarding their research !

• The limited information provided from respondents regarding the main objectives, methodology and study population, made it difficult to classify the majority of the projects, as only 109 out of the 243 projects (45%) provided either project titles or only publication titles, and few listed objectives clearly.

• The overwhelming majority of respondents did not provide the methodology employed or specific population studied on the questionnaire, nor could this information be gleaned from the project or publication title.

• As a result, a reliable analysis of which type of OR projects are being conducted, the main questions that are being asked, and the type of methodology being employed in the specific study population, could not be adequately assessed from the project data.

• It was sometimes not possible to determine if the primary authors originated from the countries where the OR was being conducted, or if publications were generated from international organizations outside the country where the project took place.

• Exploratory exercise - not intended to be a comprehensive list of all OR projects conducted in the period

Next Steps

• Further attention will be given to organizations that did not respond to the questionnaire in order to have a more complete map of the research landscape as possible:– objectives of the OR project(s) – methodology– study population– review articles arising from cited research project

• Efforts will also be made to contact those countries where information is lacking

• Link with the results from WHO World report 2008 to get information on projects conducted in high-burden countries

• Link with the Global Fund against AIDS, Tuberculosis and Malaria to get information on OR projects on TB they are funding

• The above will be incorporated into the mapping project database

Next Steps

• The project report and resulting Database will be published on the Stop TB Partnership/Research Movement website and shared with the DEWG

• Organizations/institutions who have responded will be encouraged to update their OR projects in the Database

• Initiators of OR projects will be invited to register on the site in order to obtain a concise map of OR projects carried out in the world with relevant information on their methods, place of conduct, partnership in charge of the study, and results

• All interested parties will be invited to access to the generated materials of these studies.

• The resulting outputs of this project will be used to develop guidelines to address the issue of OR in a concise and comprehensive way in collaboration with major stakeholders who conduct research in the field, in order to establish a well-supported agenda for OR in TB. This research agenda will then be used to provide guidance in the field on OR in TB by NTP managers, GFATM recipients and other prominent funding agencies

Acknowledgements

• All responders to the survey who made this exercise possible