prevention of corruption, developing indicators and measuring achievements

Developing Indicators and Developing Indicators and Measuring AchievementsMeasuring Achievements

AdvancedTraining on AdvancedTraining on Prevention of CorruptionPrevention of Corruption

Techniques &MethodologiesTechniques &Methodologies

10-11 November 201110-11 November 2011Asmara AchcarAsmara Achcar

Applying the tools to the RBM life-cycle approach?IMPACT OUTPUT

PlanningEva

luat

ion

Monitoring

Participatory

process

Capacity

CapacityCapacity

Capacity

Capacity

Overview of presentation

1. Designing indicators

2. Measuring/Monitoring your indicators

3. Potential models/examples


1. Designing indicators Basic steps Actionable indicators Different indicator approaches



Establish a steering

committee

Identify national institution or civil

society organisation for hosting initiative

Raisefunds

Conduct multi-stakeholder dialogue on governance

priorities

Decide who will do the research

Select type of

assessment

Decide on assessment framework

Decide on indicators

Decide on sampling

Analyse results

Disseminate results

Conduct multi-

stakeholder consultation

Develop policy recommendations

Conduct policy reform

Institutionalize and repeat at regular intervals

Key steps in conducting an effective assessment

Identify key stakeholders

Decide how to

collect data

10 features of an effective assessment development

1. Strategy is aligned to national policy priorities and processes

2. Strategy is country contextualized3. Methodology is rigorous4. Selection of indicators is transparent and

participatory5. Results are stored in a public national database6. Indicators are pro-poor and gender-sensitive7. Capacity of national stakeholders is developed8. Assessment is cost-effective and timely9. Results are widely disseminated10. Assessment is repeated

Why participatory adaptation?

Technical benefits Customized Actionable

Political benefits Buy-in Credibility Legitimacy and public trust Consensus-building and political will

Efficiency benefits (usage) Adapted to needs Integrated in planning Sustainable

Increase

impact of

assessment

Participation at 2 levels

Through process: Participation in design Participatory monitoring

Through methodology: Inclusive data sources Gender-sensitive and pro-poor governance

indicators

Stakeholder Analysis

Identifying the key stakeholders and their interests in reform (positive or negative)

Assessing the influence and importance of each stakeholder

Identifying measuring needs of stakeholders (basis for engagement)

Significant influence

Some influence

Little influence

No influence

Significantly interested in reform

Some interest

Little interest

No interest

Significant influence

Some influence

Little influence No influence

Significantly interested in reform

Ministry of Local Government

state commission for the prevention of corruptioncitizens

NGO 2NGO 3The poorWomen

Some interest

President of municipal councils

ministry of transportNGO 1

state audit officebureau of public procurementMedia 2

NGO 4

Little interest Mayors Business communityMedia 1

ministry of environmental protection and urban planning

No interest Chief of administration

Local public servants

THE RBM LIFE-CYCLE APPROACH

Applying the tools to the RBM life-cycle approach?IMPACT OUTPUT

PlanningEva

luat

ion

Monitoring

Participatory

process

Capacity

CapacityCapacity

Capacity

Capacity

Which, if either, of these indicators are actionable?

1. How often do firms make extra payments to influence the content of new legislation?

2. How many judges and magistrates do you think are involved in corruption?

How would you make them more ‘actionable’?

Activity: two examples

Non-actionable

1. How often do firms make extra payments to influence the content of new legislation?

2. How many judges and magistrates do you think are involved in corruption?

Actionable

In practice, are the regulations restricting post-government private sector employment for national legislators effective?

In practice, are the requirements for the independent auditing of the asset disclosure forms of members of the national legislature fulfilled?

In practice, when necessary, does the judicial disciplinary agency initiates investigations?

In practice, when necessary, does the judicial disciplinary agency impose penalties on offenders?

Actionable or ‘action-worthy’?

Actionable does not mean action-worthy e.g. Measuring whether a country has an

independent anti-corruption commission when there is no guarantee it will reduce corruption

e.g. Measuring the speed of judicial proceedings when not clear that increasing the speed will ensure that justice is done

There is a risk of measuring things because they are easily measurable

Determining what is an ‘action-worthy indicator’

1. Is it measuring interventions that are truly ‘beneficial’?

Must be appropriate for country context Should be in line with reform priorities Must not further entrench corrupt systems

2. Can the performance measured be attributed to specific policy inputs? (‘actionable’)

Corruption Integrity

Perception

Public opinion

Experts

Public sector

Experience / victimisation

General population / vulnerable

groups

Public sector

Private sector

Diagnostic Assessments

Compliance monitoring

Institutions

Processes

Sectors

Local level

What indicators measure

Examples of corruption indicators

Number of reported cases of bribery within e.g. the police

Ranking of most corrupt institutions in public national perception surveys

Bribe payments as % of income among individuals in low-income neighborhoods

Proportion of sanctions to registered complaints

Corruption indicators can measure…

The incidence of corrupt transactions experiencial and perception-based data

The incidence of sanctions for corruption statistical comparison

The impact of corruption data sensitive to vulnerable groups & gender

Integrity indicators

Also called ‘transparency’ or ‘accountability indicators’

Measure the reverse of corruption: what is in place to prevent it

Focus on the effectiveness of anti-corruption mechanisms which are within the control of policy makers Therefore tend to be ‘actionable’

Corruption Transparency/Accountability/Integrity

Perception

Public opinion

Experts

Public sector



groups

Public sector

Private sector



Institutions

Processes

Sectors

Local level


Example of a fact-based / experience-based question

In some areas there is a problem of corruption among public officials. During 2011, has any government official, for instance a customs officer, police officer or inspector in your own country, asked you or expected you to pay a bribe for his service?

yes no don’t know

Example of a perception-based question

It is known that in some countries the problem of corruption among public officials is perceived to be high by citizens. Imagine a person who needs something that is entitled to him/her by law. Is it likely or not likely that this person would have to offer money, a present or a favour (e.g. more than the official charge) to get help from:

Member of Parliament? Elected municipal councillors? Customs officials? Tax/revenues officials?

Raise your hand every time you agree with a statement:

Perceptions are better indicators of progress than facts

Facts are better indicators of progress than perceptions

Both perceptions and fact-based indicators are useful

Neither is helpful, it is not possible to measure corruption

Drawbacks of perception data

Seen as biased, leading some to minimize its use, or rule it out entirely

Politically sensitive, especially in polarized political climates, where the media is seen to have undue influence in magnifying public perceptions of corruption

Lag effect: perceptions often lag behind reforms

But perception data matters!

High perceptions of corruption are correlated with low perceptions of state legitimacy

High perceptions can fuel corrupt practices by: encouraging people to believe they must pay

bribes reducing the likelihood of citizens reporting leading those with power to believe that there is

nothing wrong with accepting bribes. Perceptions and opinion reveal important

information about how corruption works in a specific context

Corruption Transparency/Accountability/Integrity

Perception

Public opinion

Experts

Public sector



groups

Public sector

Private sector



Institutions

Processes

Sectors

Local level


Inputs and outcomes

Inputs (de jure, laws)

vs.

Outcomes (de facto, practice)

de jure indicators vs. de facto indicators

De jure: In law, is there an agency with a legal mandate to address

corruption?

De facto: In practice, is the anti-corruption agency effective?

Does the ACA make regular public reports (e.g. to legislature)? When necessary, is the ACA able to independently initiate

investigations?

Can citizens access the anti-corruption agency? Does the ACA act on complaints within a reasonable time period? Can citizens complain to the ACA without fear of recrimination?

Using de jure and de factoindicators together

Combine de jure and de facto indicators to show discrepancies between: change in law & resources ($,

infrastructure, procedures, staff)

and change in practice (‘improved

governance’ in practice)

Discrepancies in law and practice are revealing

Are laws and organisational reforms translating into impact on the ground? If not, why not? Inappropriate reforms? Economically perverse incentives? Vested political interests? Lack of capacity (e.g. human resources,

information retrieval systems)? High tolerance of corrupt conduct by the public? Different understandings of what is corrupt

conduct?

Why generate poverty and gender sensitive indicators

Corruption exacts a higher price on women and the poor.

Rich/middle class men tend to be in positions of power, and thus are beneficiaries of corruption.

Need to be sensitive to effectiveness of anti-corruption mechanisms on these groups

Indicators too often tend to be gender and poverty blind

4 ways to make indicators sensitive to vulnerable groups

Disaggregating by poverty/gender Specific to the poor/women Implicitly poverty/gender sensitive Chosen by the poor/women

Methods of measurement

Objective analysis Statistics Surveys Quantifying the qualitative

Using benchmarks for Action Plans(vs. baselines)

It’s not just what you measure, it’s what you measure it against.

Benchmarks are imperative to judge progress, demonstrate change, measure success.

Use international treaties; standards from the World Bank, TI, etc.; or country-specific objectives

Developing a scale to quantify qualitative indicators

Example of qualitative indicator: In practice, are major public procurements effectively advertised?

Quantification: Score 10: There is a formal process of advertising public procurements.

This may include a government website, newspaper advertising, or other official announcements. All major procurements are advertised in this way. Sufficient time is allowed for bidders to respond to advertisements.

Score 5: There is a formal process of advertising but it is flawed. Some major procurements may not be advertised, or the advertising process may not be effective. The time between advertisements and bidding may be too short to allow full participation.

Score 0: There is no formal process of advertising major public procurements or the process is superficial and ineffective.

How are the tools above applicable to the RBM life-cycle approach?

Applying the tools to the RBM life-cycle approach?

PlanningEva

luat

ion

Monitoring

ActionableActionable

CorruptionCorruption

IntegrityIntegrity

Experience-based

Fact-basedFact-based

Experience-based

PerceptionsPerceptionsParticipatory

process

ImpactImpact

OutputOutput

SatisfactionSatisfaction

Exercise: Selecting Strategy and Action Plan indicators

Goal: To learn how to identify impact and output indicators for long-term and short-term results

Strategy Goal 1: A decrease in corruption in the judiciary

Step 1: Who are the stakeholders who could provide information on this goal?

• Citizens with contact with the judiciary• Lawyers• Civil society organizations that work on legal issues• Civil servants with contact with the judiciary

Step 2: What questions could you ask of citizens to help monitor this goal?

• Have you had contact with the legal system in the past 12 months? If so, were you asked to pay a bribe?

• How much have you paid in bribes to judges, lawyers, or other legal officers in the last 12 months?

Strategy Goal 2: A decrease in corruption in the judiciary (perceptions)

What questions could you ask of citizens to help monitor this goal?

• Do you think corruption in the judiciary has decreased over the past x years?

• In your opinion, do you think there is a higher, equal, or lower level of corruption in the judiciary as compared to [insert another key institution]?

• In your opinion, do you think there is a higher, equal, or lower level of corruption in the judiciary as compared to [insert neighboring country]?

Monitoring the Action Plan Objective: Increase in transparency of the legislature

What indicators would you propose to monitor this objective?

• What percentage of legislative proceedings are published on a publicly available website?

• What percentage of public requests for legislative information under the FOI law are granted?

• What is the average length of time it takes for an FOI request to be granted?

Benchmarking the Action Plan Indicator for Monitoring

No compliance: 0%-30% of legislative proceedings are published

Partial compliance: 30%-90% of legislative proceedings are published

Full compliance: 90%-100% of legislative proceedings are published

What percentage of legislative proceedings are published on a publicly available website?

Evaluating impact (longer term)

Corruption data and Integrity data Experience- based and Perception

based Actionable/not Baseline and Benchmarks

>> Goal to assess BUT ALSO to feeds into next planning stage

Don’t reinvent the wheel!

Many useful tools are already out there, they just require some adaptation.

Thank you!

Asmara Achcar

Regional Specialist, Governance Assessments

[email protected]

prevention of corruption, developing indicators and measuring achievements

Self Improvement