Measuring corruption: ‘The basics’

By Marie LabergeUNDP Oslo Governance Centre

What challenges are involved in measuring corruption?

What challenges matter the most in the Tajik context?

Choosing appropriate assessment methods can go a long way towards mitigating these challenges.

Why measure corruption risks?

Corruption is usually performed ’in the dark’, ’in the shadows’, where it rarely can be observed directly

Therefore easier to focus on potential ’risk areas’ and collect evidence about vulnerability to corruption (’red flag indicators’)

In other words, it’s easier to measure the ’opposite’ of corruption, i.e. the effectiveness of anti-corruption mechanisms (and more useful to monitor the effectiveness of the ’remedy’ than to monitor the seriousness of the ’cancer’)

Complementarity of indicatorsInput indicators / ‘in law’

indicatorsImpact indicators/ ‘in practice’

indicatorsMeasures the existence and quality of formal rules found in documents, laws, regulations and the constitution

Measures what has been delivered to citizens

Answers the question: “What has been done?”

Answers the question: “Are citizens benefiting from specific institutions and policies?”

Says nothing about actual progress Measures actual improved governance

Example: In law, there is an anti-corruption agency mandated with the fight against corruption and protected from political interference.

Example: In practice, the public feels informed about the activities and services offered by the anti-corruption agency, and knows how to access these services.

Exercise: Combining input (’in law’) indicators with impact (’in practice’) indicators to assess integrity in the civil service

Two types of ‘tools’ for diagnosing and measuring corruption risks

1) Internal tools

Purpose: To identify institutional factors that enable corruption: leadership, institutional culture & incentives, internal procedures, regulations, enforcement mechanisms, transparency of operations

Sources of info: Water utility staff, experts, review of internal documents (procedural manuals, budgets, procurement documents)

Example: ’Vulnerability assessment’

2. External tools

Purpose: To validate the corruption risks identified by ’experts’ (by adding a ’water user perspective’)

Sources of info: water consumers, water utility, contractors, civil society, business community

Examples: Corruption surveys, participatory corruption apparaisals

Two types of ‘tools’ for diagnosing and measuring corruption risks

Research methods (for collecting information)

Qualitative approaches (document review, interviews, focus groups, observation) are better at describing and explaining situations

Quantitative approaches (questionnaires) are better at measuring

Used together, they provide a richer understanding of a situation

What kind of data collection methods can you think of?What are their strengths and weaknesses?

1. Surveys

2. Interviews

3. Focus groups

4. Observation

5. Field tests

6. Analysis of documents

Research methods (for collecting information)

Positive NegativeEfficient way of collecting info from a large area / population

Unavailability of sampling frames

Produces quantitative data (factual or opinion data), which is easily used by policymakers

High cost of administering a survey, especially face to face

Can identify trends through disaggregation

Questionnaire can hinder detailed exploration of a topic (‘response categories’ created by researcher)

Can make inferences about a population (random sampling)Cost of surveys brought down if attached to a planned census

1. Surveys

Positive NegativeQualitative, useful for preliminary and exploratory

More time consuming (data collection, transcription, analysis)

Helpful in explaining complex experiences, perceptions, identifying incentives, causes and effects

Small sample sizes can’t be generalized to entire population (not representative)

Reduces researcher bias, gives respondents more control over what to discuss, for how long

Interviewers must be well-trained

Often uncovers new issues Ideally best if interviews area carried out by the same person

2. Interviews

Positive NegativeQuick way of getting in-depth information from a group

Focus groups may be affected by intimidation and domination by elites

Can get a range of views on an issue

Potential of bias in sampling of groups

Cost and time-effective (no need for extensive preparation or training)

Need clear agenda and skilled facilitator

3. Focus groups

Positive NegativeObserving people or infrastructure in context gives greater insight / accuracy

Presence of researcher may cause people to act or respond differently

Cheap to carry out Bias in observation from researcher’s interpretation

Requires little training for analysis

Amount of time varies

4. Observation

Positive NegativeInnovative way of determining effectiveness of certain governance processes or reforms from a user perspective (e.g. use of trained teams compare ability to access information with groups of local volunteers)

Small sample sizes can’t be generalized to entire population (not representative)

Shows how different backgrounds and levels of know-how affect ability to access certain services

Timing of data collection varies

Cost effective

5. Field tests

Positive Negative

Necessary at early phase of research to determine available data sources, understand context and issues

Disaggregating administrative data is difficult

Event-based data: newspaper articles, individual records (e.g. testimonies collected by NGOs), information from independent researchers

Event-based data not generalisable, problems of under-reporting

Wide range of sources: academic, administrative, NGO, newspaperMany countries have high quality administrative data (e.g. through tax collection, registration for social security, census)

Bias of the media needs to be taken into account

6. Analysis of documents

Triangulation, a means to ‘verify’ information

The same topic is discussed with different groups of peopleAn issue is analyzed by the same group of people, but using

different methodsThe same group analyzes the issue at different points in timeResults from the analysis carried out by one group are shared

for discussion with another groupResults of the study are shared with the community at the end

to ’validate’ the findings