Assessing the Impacts of Citizen Security Projects in Latin America and the

Caribbean

Jennifer Peirce Institutional Capacity of the State Division (ICS) Institutions for Development Department (IFD)

Geneva Declaration Regional Review Conference Antigua, Guatemala, April 29, 2014

Overview: The IDB & citizen security

• Mandate: Crime and violence are obstacles to social and

economic development.

• Approach: Institutional strengthening – for prevention

• Trend: Scale and complexity of problems & demands

• Analysis & Strategy:

– 5 pillars: social prevention, situational prevention, police,

criminal justice system, and corrections

– Conceptual Framework: evidence-based models

– Emphasis on building data collection & analysis capacity

Protocol for Evidence-Based Crime Prevention

CRIME & VIOLENCE:

• Measured reliably and precisely by well-audited systems

• Classified in a way that supports crime prevention

• Analyzed in multiple units, or categories

• A power few units cause most harm

PREVENTION PRACTICES:

• Based on priorities set according to the degree of harm of the crime

• Theory: a logical sequence -outputs outcomes + how & why

• Ethics: Not dependent on success, but rather on respecting human rights and balancing harms of coercion vs. harms of crime

• Evidence from impact evaluations + local crime and context analyses

• Efficiency: concentration of resources based on predictions derived from past behavior.

• Effectiveness: determined by field tests that compare programs. Source: Lawrence Sherman, IDB 2012

Monitoring & Evaluation Systems

• Requirements for M&E systems for loan projects: – Shift from output information to outcome information – Shift from anecdotal to survey-based data – Shift toward more quantifiable and comparable data

• Increasing emphasis on experimental & quasi-experimental methods …with debates about cost, feasibility, adaptability, innovation …as complementary to non-experimental and process evaluations

• Investing resources in: – Dialogue on political & policy implications of M&E + data access – Administrative data systems – Baseline data collection; data collection & management tools

Examples of Indicators & Data Sources

INDIVIDUAL LEVEL • Risk factors (e.g. employment,

substance abuse) • Knowledge & Skills; Attitudes

& Behaviours • Access & Use of services

GROUP OR COMMUNITY LEVEL • Crime data • Victimization data • Perceived responsiveness of

institutions • Fear of specific types of crime • Attitudes & Behaviours • Re-offending rate (?)

Data Sources

• Pre/post-activity instruments

• Community-level survey (with match)

• National survey • Admin data (schools,

hospitals, police, courts, prisons, social services)

• Caseworker records • Qualitative: Focus

groups & interviews • Client mgmt system

Data and Information: Advances

• Mapping of details and trends in international funds – www.seguridadciudadana-centroamerica.org

• Mapping of types of interventions and types of evaluation and evidence

• More reliability in homicide data, including at city level

• More and better-quality victimization surveys

• Separate methods and modules on domestic violence

• More standardization, reconciliation & comparability of data – www.seguridadyregion.com

• More resources for data systems, data collection tools and methods, and inter-institutional coordination

IDB Study on Observatories

OBSERVATORIES

• 40%: no record of date, place, weapon for crimes

• 60%: cannot determine the time of a crime

• 50%: have data on drivers of crime, occupations and relations of victim and offender

• 25%: do not produce analysis reports

USE OF DATA

• 33% for policy-making

• 20% for information dissemination

• 11% for decision-making

• 7% for citizen or public oversight

M & E – Main Challenges

Harmonized indicators & quality data – incl for cities

Regular & systematic victimization surveys

Disaggregation: by community, age, sex, risk group

Individual & incident-specific characteristics

Access to admin data & methods, incl. non-traditional

Databases that can track individual beneficiaries

Common understanding of theory of change & eval implications

Lessons Learned

• Rigour vs. adaptability & practicality

• Overcoming political cycles Building Buy-in

• Balancing disbursement vs. quality

• Unintentional incentives in indicators Incentives

• Mainstream M&E skills & processes

• Oversight: data collection & application Capacity

• Process evaluation includes M&E itself

• Clarity on how data will affect decisions Implementation

KEY MESSAGES

• Crime and Violence in LAC is an over-diagnosed issue, yet we lack basic types of data, data systems, analysis capacity, and evidence-based models/cases.

• We need to focus on a more granular picture: individuals and communities, differentiating by gender, risk group, etc.

• More robust evaluation methods require investment not only in the evaluation itself, but also in building a common understanding, trust & buy-in, and follow-through with all stakeholders.

Technical Cooperation: Building & Applying Data

• IDB Citizen Security Initiative: data, evaluation, management – Mapping: International Cooperation, C&V Observatories – SES: Regional Standardized Indicators System on Citizen Security – Violent Crime in LAC cities (with UNODC) – Urban Citizen Security: data, mapping, evaluation, toolkit – Caribbean Crime & Victimization Survey + CentAm methodology – Standardization of Victimization Survey methodology – Support to SICA Security Strategy + Corrections Model & Baseline

• Research Products: – Methodology for a prison census – Violence against Women: costs; incarceration; effects of CCTs – Impact Evaluations of Citizen Security Policies (Bien Público Regional)

• Regional Policy Dialogue & Training Clinic

• SIN MIEDOS: blogs.iadb.org/sinmiedos & @BID_seguridad