Decentralization Thematic Group/Public Sector Group Funding 2002-2003
“Assessment of the Impact of Decentralization:
The Case of Colombia (1991-2001)”
Presentation by:
Jonas Frank (LCSPS)March 10, 2004
The Puzzle:What is the impact of decentralization and can this impact be evaluated?
Better Services? Economic Efficiency? Democratic Government?
Evaluation is about finding causality
The Colombia Case: The Study:
Sample of 148 municipalities, out of a total of 1100 70 indicators 1000 pages It took ca. 2 years to complete the evaluation
Appropriate time-frame: process started in 1986, evaluation covers years following 1991
Integral reform: political, fiscal, administrative Typical sequence: political reform first, then fiscal and administrative
decentralization; municipal level
Three important questions: Why evaluate?What should be evaluated?How can one evaluate?
Objectives of Evaluation differ by actors of decentralization: Minister of Finance, Indigenous People, Legislators, Mayors,
Regional Governors, Donors,... There is no single objective, but evaluations can...
(i) improve decision-making (ii) create accountability among actors involved
I. The “Why” of Decentralization Evaluation
Implications: Evaluation is a continuous exercise Transformation of data into information A participatory exercise vs. purely
technocratic
I. The “Why” of Decentralization Evaluation
Objective of the Colombia Study: Crisis: fiscal pressures, weak budget constraints, bailouts Inefficient services Slow democratization
Lesson: Evaluate as early as possible Ownership Too many objectives of evaluation
I. The “Why” of Decentralization Evaluation
Some guiding questions: Where is decentralization supposed to have an impact? Were the initial goals of decentralization met and to what
extent?Dilemas: Decentralization goals remain vague They are not expressively formulated and agreed upon Decentralization is a “moving target”
I. The “What” of Decentralization Evaluation
Decentralization Goals in Colombia: In 1986: democratization In 1991: efficiency in services, redistribution,
services for the poor, improve popular participation In 2001: fiscal discipline, efficiency
I. The “What” of Decentralization Evaluation
1970-1975 1975-1980 1980-1985 1985-1990 1990-1995 1995-2000 2000-2002
Political Dec.
Fiscal Dec.
Administrative Dec.
SocialContext
VenezuelaEc
uado
rPe
ru
Boliv
ia
Colombia
ARG
Braz
il
RioGrande
Chile
LAC: Decentralization Objectives (1970 – 2002)
Colombia Study: “Comprehensive evaluation” (not only fiscal):1. Outcomes within eight sectors
Education Health Water and basic sanitation Fiscal performance Political decentralization Management capability Economic development
2. Municipal Progress: (i) sectoral outcome (ii) aggregate outcome,
II. The “What” of Decentralization Evaluation
1. Sectoral Evaluation: Results in Education
“Decentralization has improved and contributed to: Increase in coverage Improved teacher/student ratio Higher schooling levels Reduction in illiteracy rate”
II. The “What” of Decentralization Evaluation
Percentage of illiterate population aged 15 years and above
0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0 22.0 24.0 26.0 28.0 30.0
0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0 22.0 24.0 26.0 28.0 30.0 1985 (%)
2000 (%)
Source: ECLAC, Social development database.
Uruguay Ecuador
Brazil
Argentina Colombia Chile
Bolivia
Paraguay Mexico Venezuela
Peru
Results in Education
Sectoral Evaluation: Results in Education
1. Sectoral Evaluation: Results in Health
“Decentralization has improved and contributed to: Increased coverage Higher public spending Greater equity Lower infant mortality rates Greater ratio of physicians per inhabitant”.
II. The “What” of Decentralization Evaluation
Sectoral Evaluation: Results in Health Infant mortality by 5-year periods
0 10 20 30 40 50 60 70 80 90
100 110 120
0 10 20 30 40 50 60 70 80 90 100 110 120 1980-1985
1995-2000
Source: ECLAC , social development database.
Uruguay
Ecuador Brazil
Argentina
Colombia Chile
Bolivia
Paraguay Mexico Venezuela
Peru
Ratio of Physicians per inh.
0 300 600 900
1,200 1,500 1,800 2,100 2,400 2,700 3,000
0 300 600 900 1,200 1,500 1,800 2,100 2,400 2,700 3,000 1985
1998
Source: ECLAC , social development database.
Uruguay
Ecuador Brazil Colombia
Chile Bolivia
Paraguay
2. Performance Evaluation of Municipalities (i) Municipal Performance in Six Sectors (Health,
Education,...) (ii) Aggregate Performance: Most municipalities have achieved
an average performance; only 4% achieved and acceptable standard But:
What were the starting conditions? Where there several observations in the past and when?
II. The “What” of Decentralization Evaluation
Results in Economic Development
Lessons: Some important areas are excluded:
Allocative efficiency The regional level: departmental government
Conclusions about the causality of decentralization are primarily hypothetical
“Less is more”
II. The “What” of Decentralization Evaluation
1. Base-line of evaluation 2. Finding causality 3. Indicators 4. Selection of samples of local government 5. Periodicity of evaluation
III. The “How” of Decentralization Evaluation
1. Base-line Usually no such base-line has been established
before decentralization was implemented The use of proxies Select indicators that are available and mirror
the starting conditions
III. The “How” of Decentralization Evaluation
2. Finding CausalityTo date, impact studies of decentralization are either: (i) Evidence-based (ii) Subjective (iii) Hypothetical
All of these three types of studies provide a useful understanding and appraisal at various stages of the decentralization process.
III. The “How” of Decentralization Evaluation
2. Finding Causality The Colombia study is primarily hypothetical Statistical analysis is not sufficient to prove causality Lessons:
Discerning between decisions that are in hands of local government and which are not
Clarifying the value added of local government in carrying out new responsibilities?
Continuous monitoring
III. The “How” of Decentralization Evaluation
2. Finding Causality: Example of the Education sector Local government defines curriculum content:
”Instruction Indicator” Local government is responsible for construction and maintenance of school buildings
“Infrastructure Indicator” Local government develops education improvement plan
“Planning Indicator” Local government allocates budget
“Budget Indicator” Is this operationally possible?
III. The “How” of Decentralization Evaluation
3. Indicators 70 indicators were used in the Colombia study Lesson:
Selection of indicators depends on “what” will be used, but: use only a minimum set of indicators, and... indicators that can be monitored regularly and with low cost
III. The “How” of Decentralization Evaluation
4. Comparability of data and selection of local government Colombia: “representative sample” = most diverse
municipalities were selected Lesson:
Use only most similar cases because diversity increases during decentralization process
Split them into groups: eliminate some structural factors that lie outside control of municipalities (population size)
III. The “How” of Decentralization Evaluation
5. Periodicity of evaluation Colombia: evaluation after 10 years of time Lesson:
Time period sufficiently large, but... Regular monitoring is necessary It is important to separate between short-term, medium-term, and long-
term goals helps create accountability helps better decision-making
III. The “How” of Decentralization Evaluation
Conclusion: Risks and Opportunities
Evaluation as a purely bureaucratic exercise (i) Evidence and (ii) subjective evaluation are still important: the only
“early-warning system” that is operational Ready to propose and implement corrective measures: “exit” of
decentralization process? Giving “erroneous” answers to the “wrong” questions: “Decentralization
has not worked well enough because there was not enough of it”