Decentralization Thematic Group/Public Sector Group Funding 2002-2003

“Assessment of the Impact of Decentralization:

The Case of Colombia (1991-2001)”

Presentation by:

Jonas Frank (LCSPS)March 10, 2004

The Puzzle:What is the impact of decentralization and can this impact be evaluated?

Better Services? Economic Efficiency? Democratic Government?

Evaluation is about finding causality

The Colombia Case: The Study:

Sample of 148 municipalities, out of a total of 1100 70 indicators 1000 pages It took ca. 2 years to complete the evaluation

Appropriate time-frame: process started in 1986, evaluation covers years following 1991

Integral reform: political, fiscal, administrative Typical sequence: political reform first, then fiscal and administrative

decentralization; municipal level

Three important questions: Why evaluate?What should be evaluated?How can one evaluate?

Objectives of Evaluation differ by actors of decentralization: Minister of Finance, Indigenous People, Legislators, Mayors,

Regional Governors, Donors,... There is no single objective, but evaluations can...

(i) improve decision-making (ii) create accountability among actors involved

I. The “Why” of Decentralization Evaluation

Implications: Evaluation is a continuous exercise Transformation of data into information A participatory exercise vs. purely

technocratic

I. The “Why” of Decentralization Evaluation

Objective of the Colombia Study: Crisis: fiscal pressures, weak budget constraints, bailouts Inefficient services Slow democratization

Lesson: Evaluate as early as possible Ownership Too many objectives of evaluation

I. The “Why” of Decentralization Evaluation

Some guiding questions: Where is decentralization supposed to have an impact? Were the initial goals of decentralization met and to what

extent?Dilemas: Decentralization goals remain vague They are not expressively formulated and agreed upon Decentralization is a “moving target”

I. The “What” of Decentralization Evaluation

Decentralization Goals in Colombia: In 1986: democratization In 1991: efficiency in services, redistribution,

services for the poor, improve popular participation In 2001: fiscal discipline, efficiency

I. The “What” of Decentralization Evaluation

1970-1975 1975-1980 1980-1985 1985-1990 1990-1995 1995-2000 2000-2002

Political Dec.

Fiscal Dec.

Administrative Dec.

SocialContext

VenezuelaEc

uado

rPe

ru

Boliv

ia

Colombia

ARG

Braz

il

RioGrande

Chile

LAC: Decentralization Objectives (1970 – 2002)

Colombia Study: “Comprehensive evaluation” (not only fiscal):1. Outcomes within eight sectors

Education Health Water and basic sanitation Fiscal performance Political decentralization Management capability Economic development

2. Municipal Progress: (i) sectoral outcome (ii) aggregate outcome,

II. The “What” of Decentralization Evaluation

1. Sectoral Evaluation: Results in Education

“Decentralization has improved and contributed to: Increase in coverage Improved teacher/student ratio Higher schooling levels Reduction in illiteracy rate”

II. The “What” of Decentralization Evaluation

Percentage of illiterate population aged 15 years and above

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0 22.0 24.0 26.0 28.0 30.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0 22.0 24.0 26.0 28.0 30.0 1985 (%)

2000 (%)

Source: ECLAC, Social development database.

Uruguay Ecuador

Brazil

Argentina Colombia Chile

Bolivia

Paraguay Mexico Venezuela

Peru

Results in Education

Sectoral Evaluation: Results in Education

1. Sectoral Evaluation: Results in Health

“Decentralization has improved and contributed to: Increased coverage Higher public spending Greater equity Lower infant mortality rates Greater ratio of physicians per inhabitant”.

II. The “What” of Decentralization Evaluation

Sectoral Evaluation: Results in Health Infant mortality by 5-year periods

0 10 20 30 40 50 60 70 80 90

100 110 120

0 10 20 30 40 50 60 70 80 90 100 110 120 1980-1985

1995-2000

Source: ECLAC , social development database.

Uruguay

Ecuador Brazil

Argentina

Colombia Chile

Bolivia

Paraguay Mexico Venezuela

Peru

Ratio of Physicians per inh.

0 300 600 900

1,200 1,500 1,800 2,100 2,400 2,700 3,000

0 300 600 900 1,200 1,500 1,800 2,100 2,400 2,700 3,000 1985

1998

Source: ECLAC , social development database.

Uruguay

Ecuador Brazil Colombia

Chile Bolivia

Paraguay

2. Performance Evaluation of Municipalities (i) Municipal Performance in Six Sectors (Health,

Education,...) (ii) Aggregate Performance: Most municipalities have achieved

an average performance; only 4% achieved and acceptable standard But:

What were the starting conditions? Where there several observations in the past and when?

II. The “What” of Decentralization Evaluation

Results in Economic Development

Lessons: Some important areas are excluded:

Allocative efficiency The regional level: departmental government

Conclusions about the causality of decentralization are primarily hypothetical

“Less is more”

II. The “What” of Decentralization Evaluation

1. Base-line of evaluation 2. Finding causality 3. Indicators 4. Selection of samples of local government 5. Periodicity of evaluation

III. The “How” of Decentralization Evaluation

1. Base-line Usually no such base-line has been established

before decentralization was implemented The use of proxies Select indicators that are available and mirror

the starting conditions

III. The “How” of Decentralization Evaluation

2. Finding CausalityTo date, impact studies of decentralization are either: (i) Evidence-based (ii) Subjective (iii) Hypothetical

All of these three types of studies provide a useful understanding and appraisal at various stages of the decentralization process.

III. The “How” of Decentralization Evaluation

2. Finding Causality The Colombia study is primarily hypothetical Statistical analysis is not sufficient to prove causality Lessons:

Discerning between decisions that are in hands of local government and which are not

Clarifying the value added of local government in carrying out new responsibilities?

Continuous monitoring

III. The “How” of Decentralization Evaluation

2. Finding Causality: Example of the Education sector Local government defines curriculum content:

”Instruction Indicator” Local government is responsible for construction and maintenance of school buildings

“Infrastructure Indicator” Local government develops education improvement plan

“Planning Indicator” Local government allocates budget

“Budget Indicator” Is this operationally possible?

III. The “How” of Decentralization Evaluation

3. Indicators 70 indicators were used in the Colombia study Lesson:

Selection of indicators depends on “what” will be used, but: use only a minimum set of indicators, and... indicators that can be monitored regularly and with low cost

III. The “How” of Decentralization Evaluation

4. Comparability of data and selection of local government Colombia: “representative sample” = most diverse

municipalities were selected Lesson:

Use only most similar cases because diversity increases during decentralization process

Split them into groups: eliminate some structural factors that lie outside control of municipalities (population size)

III. The “How” of Decentralization Evaluation

5. Periodicity of evaluation Colombia: evaluation after 10 years of time Lesson:

Time period sufficiently large, but... Regular monitoring is necessary It is important to separate between short-term, medium-term, and long-

term goals helps create accountability helps better decision-making

III. The “How” of Decentralization Evaluation

Conclusion: Risks and Opportunities

Evaluation as a purely bureaucratic exercise (i) Evidence and (ii) subjective evaluation are still important: the only

“early-warning system” that is operational Ready to propose and implement corrective measures: “exit” of

decentralization process? Giving “erroneous” answers to the “wrong” questions: “Decentralization

has not worked well enough because there was not enough of it”