Evaluation of Results: An Analysis of the Outputs and Achievements of a Group of

Projects in the Areas of Civil Registry and Human Rights

March 2011

Contents1. Overview

2. Introduction

3. Evaluations

• Civil Registry: PUICA

• Human Rights: IACHR and I/A Court H.R.

4. Results

5. Institutional Implications

• Results-based management, a vehicle to achieve efficiency and effectiveness in development initiatives

• OAS and results-based management: present situation and perspectives

• Evaluation of results: a mechanism for accountability and the optimum use of resources; its significance for donors and within OAS

• A Monitoring and Evaluation Framework in the Organization, its institutionalization and sustainability

1. Overview

• Evaluation of achievements/ outcomes : its importance and advantages vis-à-vis the simple measurement of outputs

• Recognition of the achievements/results and their implications for the GS/OAS policies and projects

• Improves the focus and efficiency of the efforts undertaken by the Organization• Represents an effective tool for communicating with donors and other key

stakeholders (legitimization of the Organization’s initiatives and operations, and request for additional financing)

• Leads to the attainment of results• Assists in decision making• Increases knowledge and improves processes• Facilitates evaluating the effectiveness of a project and making optimum use of

resources • Efforts and results of the Organization to date

2. Introduction

• Evaluations made and their characteristics• Formative and summative evaluations• In compliance with the OECD Quality Standards for Development

Evaluation, in terms of:• Impartiality; • Independence; • Credibility; and • Usefulness.

• External and selected through competitive processes• Project groups evaluations by sector • Projects evaluated in the areas of Civil Registry and Human Rights

3. Civil Registry and Human Rights - Evaluations

Evaluations completed

Civil Registry• Identity, Registration and Citizen Participation

• Strengthening Systems of Civil Registry and Practical Tools Development

Human Rights• Strengthening the Jurisdictional Action of the I/A Court H.R., Phase I

• Strengthening the Jurisdictional Action of the I/A Court H.R., Phase II

• Itinerant Court, Phase I

• Systems of Information Management of IACHR Individual Cases

3. Civil Registry and Human Rights - Evaluations

Evaluations completed

• Civil Registry• Objective of the Evaluation: Assessing the performance of the projects

based on the following purposes:• “Identity, Registration and Citizen Participation” project

• Its purpose: - Strengthening civil registration and identification mechanisms in

Bolivia, Guatemala and Peru • “Strengthening Systems of Civil Registry and Practical Development

Tools” project• Its purpose: - Strengthening systems of civil registry and promoting civil identity in El

Salvador and Guatemala

4. Results

• Civil Registry• Main findings and results of both projects:Successes• Results at the output level:

• Registration and correction of errors in the registration of 3,762 persons in Bolivia (1,645 new registrations and 2,117 certificates rectified)

• Registration of 6,362 persons in Peru, both children and adults• Registration of 1,362 persons in Guatemala• Simplification of the procedures and requirements for obtaining a birth

certificate• Implementation of the civil registration service in two hospitals in El Salvador,

resulting in the registration of about 4,000 children and raising the awareness of some 500 mothers each month

• Successful local project management experience

4. Results

ChallengesThe following needs were identified:

• Focus efforts on the institutional strengthening of civil registry offices to supplement the achievements attained in terms of registrations and make them sustainable

• Redirect the resources to favor actions associated with the scope of the initiative and the fulfillment of its purpose

• Design and implement an appropriate system for monitoring and recording management information in order to show the efficiency and effectiveness of the projects

• Increase the core capacity of the program to ensure an effective transfer of knowledge as an input to the institutional strengthening of the civil registry offices in the countries

• Incorporate the forecasts necessary to sustain the benefits of the project once it is completed

4. Results (Cont.)

• Human Rights• Objective of the Evaluation: Assess the effectiveness of the actions

undertaken in a sample of projects with the aim of enhancing access to the Inter-American Human Rights System (IAHRS) and promoting the institutional strengthening of the IACHR and the I/A Court H.R.

• “Systems of Information Management of Individual Cases before the IACHR” project• Its purpose: - Strengthening the technological capabilities of the IACHR to improve efficiency and

access level of operators, victims and other users of the Inter-American Human Rights System

• Main findings and results:- 100% of the petitions received is handled pursuant to the standards of the new

system.

4. Results (Cont.)

• Human Rights

• Main findings and results (cont.):- The average time of internal processing and forwarding of

communications to the parties (regarding petitions and individual cases) was reduced by 83%.

- Benefits are likely to be sustained, since all the system maintenance costs are planned to be included in the regular budget.

4. Results (Cont.)

• Human Rights• “Strengthening the Jurisdictional Action of the I/A Court H.R.” project, Phases I

and II• Its purpose: - Strengthening the capacity of the I/A Court H.R. to protect human rights• Main findings and results:Results at the output level:- Number of orders on monitoring compliance with judgments increased by 59%,

exceeding the expected goal of 22 orders- 10 hearings held to monitor compliance with judgments, exceeding the original goal

by 66.66%- 8 studies on Comparative International Law conducted and included in the analysis

and resolution of cases- 84 cases reproduced in digital format and published on the I/A Court H.R. website

4. Results (cont.)

• Human Rights• Main findings and results (cont.):- Implementation of an intranet system for use by the judges and other judicial officers

of the CourtResults at the purpose level:- Orders on compliance with judgment increased by 198% from 2007 to 2010 - Overall rate of compliance with the reparations ordered by the Court increased by 7

percentage points from 2007 (24.25%) to 2010 (31.25%)- Average time devoted to a draft judgment reduced by 15.5% (from 20 months to 16.9

months)- As from phase II, the I/A Court H.R. attains sustainability of the benefits obtained

through the incorporation of technical measures and procedures. - However, the results reveal a high degree of dependence on external resources (at

present, more than 50% of the Court’s resources are from donations).

4. Results (cont.)

• Human Rights• “Itinerant Court” project

• Its purpose: - Improving the provision of justice • Main findings and results:Results at the output level:- 9 public hearings held- 3 special sessions held- 3 academic seminars held, attended by 1,700 personsResults at the purpose level:- The project has facilitated an increase in jurisdictional action as well as

the dissemination of the Inter-American System and, particularly, of the activities of the Court.

4. Results (cont.)

• Human Rights• Main findings and results (cont.):- The Itinerant Court has given rise to subsequent local initiatives in some

countries, such as the following: - Uruguay: A capacity-building program on the protection of human rights

for those who play a role in the justice system (justice operators) was launched.

- Colombia: The second Inter-American Congress on Human Rights was held with the participation of officers from the I/A Court H.R., and a training project for public defenders on the Inter-American System was implemented, with versions in Uruguay and Mexico.

- From a budgetary point of view, the initiative faces serious sustainability challenges.

4. Results (cont.)

• Specification of the organizational forms for promoting the participation of women. In project design, the widespread tendency to integrate a gender perspective into the definition of the beneficiaries to be favored is followed, but the actions and resources necessary for such intention to materialize are not specified.

• Strategic planning by sector. Due to the limited human and financial resources, and in view of the fact that the IACHR, the I/A Court H.R. and PUICA depend almost exclusively on specific funds to operate and expand or strengthen their action capacity, a more harmonious and planned operation is necessary (in order to avoid the multiplicity of initiatives, the false notion of isolated projects, and the dispersion of resources).

5. Institutional Implications

• Capacity building of technical teams: In the methodological aspect, weaknesses were identified in the planning and design of almost all projects (weak technical feasibility of the initiatives). Therefore, the understanding and command by the project teams of concepts as well as of Logical Framework and Results-based Management tools needs to be reinforced in their theoretical, practical and operational dimensions with the institutional support of the General Secretariat.

• Promoting a culture of evaluation in the Organization: Deficiencies in project design and the lack of an appropriate practice of a culture of evaluation encouraged dispersion in implementation and the dilution of efforts and results. If technical feasibility is not ensured, a model for controlling, recording and systematizing implementation performance, thus ensuring the sustainability of advances, can hardly be structured.

5. Institutional Implications (Cont.)

• Redefining the focus of civil registration projects: In the interest of an effective sustainability of the initiatives and to favor an appropriate technical logic, the main focus of civil registration and identity projects must not be attaining the goal of “zero under-registration,” but rather providing technical assistance to enhance the performance capacity of the States in order to turn the work of civil registries more efficient and effective, thus avoiding registration errors and the flow of persons whose civil identity is not registered.

• Focusing initiatives on their sphere of action: In designing and executing PUICA projects, the purpose is to implement actions in five different spheres/countries and contexts, without pointing out the specific situation of such contexts or establishing the way to articulate such actions. This strategy creates weaknesses in the coordination, monitoring and evaluation tasks, and generates deficient implementation reports, information gaps and lack of precision. It is advisable to prepare proposals suited to each country involved and capable of producing effects in line with each country’s particular problems.

5. Institutional Implications (Cont.)

• Sustainability of the IACHR and the I/A Court H.R. activities: The initiatives designed to date to support the activities of the IACHR and the I/A Court H.R. have primarily focused on financing recurring expenses, thereby not solving the issue of the sustainability of the benefits created. On the other hand, it is clear that these bodies depend on up to 50% of funding by donor countries in order to fulfill their mandates in the ordinary course of their activities.

5. Institutional Implications (Cont.)

• Civil Registry • The results of the evaluation led to the reformulation of similar projects in

execution in order to incorporate key recommendations. • Based on the recommendations, changes were also made with respect to

the coordination of PUICA aimed at strengthening the management of the program.

• The results were presented and validated successively by the Department of Effective Public Management, the Secretariat for Political Affairs, The General Secretariat, Permanent Observer Mission of Spain before the OAS, AECID and the Directors of the Civil Registry Offices in Member States involved in the projects.

• Other donors have expressed interest in the results of the evaluations.

6. Follow-up of the Evaluations

• Managers of other PUICA projects expressed an interest in having their programs evaluated.

• Human Rights • The results of the evaluation support the direction of the institutional

strengthening process of the Interamerican System of Human Rights • One of projects evaluated, received three awards, from various

independent organizations for the technical quality of the design and development of the information system for managing individual cases, in a US-wide competition.

• The lack of reliable conclusions in two of the projects evaluated resulted in modifications to certain aspects of the evaluation procedure.

6. Follow-up of the Evaluations (Cont.)