1

Measuring Progress in Democracy andHuman Rights: Why? How? To whom?

By Jan Robert Suesser and Raul Suarez de Miguel

Second OECD World Forum on Statistics, Knowledge and Policy:“Measuring and Fostering the Progress of Societies”

Istanbul, Turkey, 27-30 July 2007

2

1. INDICATORS OF PROGRESS

IN THE AGE OF GLOBAL HUMANISM

3

AN EVOLVING PARADIGM OF PROGRESS

Enlightenment: knowledge and freedom (18th cent.) Modern science and technology (19th–20th cent.) Economic growth / economic integration (40s-60s) Social integration / fair distribution of wealth (60s) Quality of life (70s and 80s) Human development (80s and 90s) Sustainable development (90s and 00s) Millennium Development Goals (late 90s and 00s) Now, happiness ?

4

A GLOBAL PARADIGM OF PROGRESS

Universally shared values

Universally shared goals

Universally shared tools

5

THREE KEY DIMENSIONS OF PROGRESS…

Human rights

Democratic participation

Governance and accountability

… THAT DESERVE PROPER MEASUREMENT

6

A NEED FOR EVIDENCE-BASED ASSESSMENT IS EMERGING WORLDWIDE

Huge demand for robust data and meaningful indicators (national policy actors and civil society, international institutions, development agencies).

Many measuring initiatives are being implemented in different regions of the world.

Partly identified through a global survey, these initiatives are being documented in an inventory, available on-line in: www.metagora.org.

7

8

ON-GOING MEASURING EFFORTS INCLUDE:

Global initiatives to build indicators of democracy, human rights and governance

Regional initiatives to set-up evidence-based monitoring mechanisms

National and local initiatives to measure human rights and democratic governance

9

THESE INITIATIVES PRESENT SEVERAL LIMITATIONS:

International indicators are of rather limited use in assessing the impact of national policies. Their sources and methods are often opaque and their comparative scope (ranking) is broadly questioned.

Regional initiatives are still incipient.

National and local initiatives are too often based on rather rudimentary quantitative methods. They are implemented with very limited human, technical and financial resources.

10

2. THE METAGORA PROJECT:

APPROACH, METHOD AND LESSONS

11

CAN HUMAN RIGHTS AND DEMOCRATIC GOVERNANCE REALLY BE MEASURED ?

By whom?

With which methods?

Under which conditions?

For which purposes?

12

METAGORA FORMULATES RESPONSES, RELYING ON ITS 2004-2006 EXPERIENCE

Several pilot national experiences were carried out in various regions of the world.

These pilot experiences, which willingly addressed sensitive issues, were selected by national stakeholders in different political, social and cultural contexts

13

A Decentralized Laboratory

Content

14

THESE PILOT EXPERIENCES WERE CONDUCTED BY SEVERAL ORGANIZATIONS

gathering together and sharing their specific expertise within a North/South community :

• Human Rights Institutions

• Research Organizations

• Civil Society Organizations

• National Statistical Offices

• Governmental bodies

15

ASSESSING STEERING

Independent Panel of Experts

Committee of Donors

IMPLEMENTING

AAAS

USA

PCBSPalestine

HSRC

South Africa

Fundar

Mexico

DIAL

France

SGCAAndean

Community

CHR

Philippines

Coordination Team

OECD / PARIS21

Group of NSIsBenin, Burkina Faso, Côte

d’Ivoire, Mali, Niger, Senegal, Togo & Madagascar

Group of NSIsBolivia, Colombia, Ecuador,

Peru & Venezuela

Partners’Group

16

ADRESSING SOME BASIC QUESTIONS

Can multidimensional human rights and democratic governance issues be measured through surveys?

Will people respond to sensitive questions?

Will the gathered information be statistically significant and politically relevant?

Can official statistical agencies be involved?

17

METAGORA METHOD OF WORK (1)

A bottom-up approach consisting of: identifying with stakeholders national key issues

for which evidence-based assessment could be policy relevant;

applying statistical methods adapted to each particular context;

assessing these methods for their capacity to provide policy-relevant results;

providing stakeholders with a shared knowledge on the policy issues at stake;

contributing to draw shared lessons from the pilot experiences.

18

METAGORA METHOD OF WORK (2)

The Metagora bottom-up approach complements the top-down global approaches to measure democracy, human rights and governance.

It aims at generating measurement tools which can contribute at designing, implementing and evaluating national and local policies.

Its primary goal is not to compare national performances or to make international ranking, but to address major national issues.

19

ENHANCING NATIONAL CAPACITIES

Metagora promotes and stimulates:

participatory processes based on large consultative mechanisms;

mutual learning among the national stakeholders and actors committed in national pilot experiences;

international transfer of skills;

fostering, through increased authoritative skills, the leading role of key national institutions and actors.

20

1. Measuring human rights and democratic governance is technically feasible and politically relevant. Sensitive data on human rights, democracy and governance can be collected and analysed using statistical tools.

2. On the basis of this information, indicators can be produced that are relevant and useful for political decision and action.

DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES

21

DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES

3. Quantitative data and qualitative information can and should interrelate to properly inform assessment of human rights and democratic governance.

4. Official Statistical Agencies can be efficiently involved in various forms in evidence-based assessment of human rights and democratic governance.

Nevertheless the decision on such an involvement should be taken along a national coherent strategy for the development of the national statistical system.

22

5. Quantitative indicators and statistical analysis bring a significant value-added to the work of national Human Rights Institutions.

6. Statistical methods can substantially enhance the research and advocacy of civil society organizations in the fields of human rights and democracy.

7. Available statistical data should be used to structure and inter-relate relevant indicators of social, economic and cultural rights.

DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES

23

DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES

8. Indicators gain relevance when they:

– Are related to specific public policies and programs;

– make evident the accountability of identifiable public authorities;

– allow to address specific human rights issues for specific populations;

– serve as basis for democratic dialogue, policy design and decision-making processes.

24

DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES

9. To become sustainable, statistics and indicators must:

• be nationally based and owned (appropriation);

• Rely on independent and robust professional basis;

• enjoy a broad social and political legitimacy;

• be authoritatively institutionalised;

• be financed through ad-hoc headings of public budgets.

25

3. TOPICAL EXAMPLES

OF WHAT CAN BE MEASURED

26

Example: measuring irregularities, abuse of power and ill-treatment in Mexico City (Federal District)

Persons withoutabuse47 %

Persons with contact

24 %

Personswith

abuse53 %

Persons with non-physicalabuse93 %

Persons with physical

ill-treatment7 %

Incidence of contact with public security and procurement of justice authorities

Incidence ofabuse

Type ofabuse

Target population:persons aged 15 or more living in the Federal District

(6,400,000 persons)Reference period:

events occurred between November 2003 and October 2004

Measuring method:random sample household survey,

conducted through face-to-face interviews.

Persons withoutcontact76 %

27

Example: Non-physical abuse in contacts with law enforcement authorities

(Survey results correspond to 2,300,000 contacts experienced by 1,520,000 persons)

34 000

72 000

114 000

120 000

163 000

248 000

667 000

Were you threatened with hurt to your family?

Were you compelled to confess?

Were you threatened to be hurt?

Were you threatened in order to obtain a confession or some information?

Were you threatened with accusation on false grounds?

Were you insulted or humiliated?

Were you asked for money?

(Number of contacts per type of non-physical abuse)

28

Example: Reasons for not reporting abuses

(spontaneous multi-answer)

0

5

10

15

20

25

30

35

40In

effic

ien

cy

La

ck o

f p

roo

f

No

t im

po

rta

nt /

lack o

f in

tere

st

Co

rru

ptio

n

To

avo

id tro

ub

le

La

ck o

f tr

ust

tow

ard

s a

uth

ori

tie

s

Ab

use

of p

ow

er

Oth

er

me

ntio

ns

%

29

Example: incidence of corruption within public agencies in Peru

0

5

10

15

20

25

30

35

40

Po

lice

Ju

dic

iary

Min

. o

f A

gri

cu

ltu

re

Dir

. o

f M

igra

tio

ns

Mu

nic

ipa

l a

uth

ori

ty

Offic

e o

f A

rbitra

tio

n

an

d C

on

cilia

tio

n

FO

NC

OD

ES

ON

PE

JN

E

RE

NIE

C

%

National

Urban

Rural

Poor

Non poor

30

Example: levels of corruption and civil servants’ wages in Antananarivo, Madagascar

0

5

10

15

20

25

30

35

40

45

1995 1996 1997 1998 1999 2000 2001 2002 2004

popula

tion v

ictim

of corr

uptio

n

90

100

110

120

130

140

150

160

170

180

base 100 = 1995%

Level of corruption Civil servants real w ages (right scale)

31

Example: in Peru, support for democratic regime weakens as corruption perception increases

32

Example: measuring dimensions of democracy: are they fundamental? Are they respected?

Lomé

020406080

100

Free, transparentelections

Political freedom (rightto choose party)

Freedom of expression,of the press

Equality before the law

Freedom to travel

Religious freedom

Fundamental

Respected

33

Example: comparing expectations for and respect of democratic principles in French speaking Africa

0

20

40

60

80

100

%

Cotonou Ouaga Abidjan Bamako Niamey Dakar Lomé Tana

The six principles of democracy are:

Fundamental Respected

34

Example: comparing support to democracy in Andean countries

35

Example: how important is democracy for people in Ecuador?

0

10

20

30

40

50

60

70

no education primary school secondaryschool

highereducation

%

very important

important

little important

not at all important

don't know

36

Example: linking quantitative and qualitative approaches to assess indigenous peoples’ rights in the Philippines

Quantitative approach:SURVEY FINDINGS

Qualitative approaches:FOCUS GROUP DISCUSSIONS

High perception and awareness of rights to ancestral domain and land (Bago, 68%), Bugkalot (70.8%), Kankana-ey (60.8%)Government is in second place as source of information of rights to ancestral domain and land: Bago 28.4%, Bugkalot 54.2%, Kankana-ey 22.4%.Experienced violations of rights consisting of encroachment (Bago-6%, Bugkalot 30.9% and Kankana-ey 13.2%); pollution (Bago, 4.7% Bugkalot, 17.7%, Kankana-ey, 8.8%); illegal entry (Bago, 5.3%, Bugkalot, 46.3% and Kankana-ey, 13.2%). Existence of violations on land grabbing by private individuals (Bago 50%; Bugkalot, 56.5%, Kankana-ey, 55.6%), council of elders and others Substantial awareness and availment of governmental programs and servicesEnjoyment of land ownership and acquisition of right to ancestral domainAverage satisfaction (68-78%) on delivery of government programs and services90% considered customary laws helpful in solving land issues; 52% of land issues are resolved by customary lawsTop five primary needs: adequate food, housing, water system, livelihood and education

Tribal leaders and women with higher awareness and perception of rights to ancestral domain and land. Youth has lowest awareness. Right of ownership, to develop lands and natural resources and to stay in territories well understood. Low or no awareness of other rights listed under IPRA. Apparent confusion of rights due to lack of knowledge about distinction or difference between rights to ancestral domain and landPositive effect of IPRA on their rights to ancestral domain and rightsExistence of violations and sources are tribesmates, other tribes, private mining companiesRecognition of government efforts in fulfilling rights to ancestral domain and landCustomary law as primary source of dispute resolution affecting rights to ancestral domain and land LOCAL CONSULTATIONSDemand for relevant and deeper human rights and IPRA educationNeed for livelihood and organizing especially from women sectorsDevelopment aggression of private sectors permitted/not controlled by government and co-opted by some tribal leadersLack of delivery of vital servicesDiscriminatory policies to access rights to education and other social servicesPollution of and inadequate water resourcesPeace and order to ensure personal security

37

COMPARING OFFICIAL STATISTICS WITH EXPERTS’ VIEWS

Appended Module to Regular Household Surveys conducted by National Statistical Offices in French Speaking Africa (35,600 persons interviewed; 4,500 for each capital city in average);*

Expert panel survey (“Mirror survey”) conducted by DIAL-France (246 persons surveyed; 30 experts for each country in average).

* In Madagascar, results are drawn from the 2003 survey.

38

How far can we trust experts’ opinion ?

0

10

20

30

40

50

60

70

Niamey

Coton

ou

Bamak

oLo

mé

Dakar

Ouaga

Antan

anar

ivo

Abidjan

Avera

ge

%

General population (% of victimsof corruption from householdsurveys)

Expert panel (what they believecould be the percentage ofvictims of corruption)

39

How far can we trust experts’ opinion ?

0

5

10

15

20

25

30

35

40

Dakar

Niamey

Lomé

Cotono

u

Bamak

o

Abidjan

Ouaga

Antan

anar

ivo

Avera

ge

%

General population (% whobelieves that making a bribe isacceptable)

Expert panel (what they thinkcould be the % of population whobelieves that giving a bribe isacceptable)

40

ASSESSING ACHIEVEMENTS, DOCUMENTING PILOT

EXPERIENCES AND PRODUCING TRAINING MATERIALS An independent panel of senior experts is monitoring

the process and assessing the outcomes of the Metagora pilot experiences.

Experiences, problems encountered and lessons learned were documented in the form of on-line training materials aimed at facilitating the replication and extension of the pilot experiences in other countries and other contexts.

41

UPCOMING CHALLENGES

Consolidating the working method

Replicating and extending the pilot experiences

Enlarging the Metagora community

Enhancing the policy impact

Enriching the methodology and tool box

Financing field operations and analytical work

42

Warm Thanks for Your Attention !