learning and leading for large scale change scaling up: lessons for fundações dr. richard kohl,...
TRANSCRIPT
Learning and Leading for Large Scale Change
Scaling Up: Lessons for Fundações
Dr. Richard Kohl, Leading and Learning for Large Scale Change
Sao Paolo, Brazil 28 September 2012
Learning and Leading for Large Scale Change
Summary of the Presentation1. Successful large scale implementation as important
as new efficacious interventions2. Successful scaling up usually not spontaneous3. Chances for successful scaling up can be increased by
pro-active strategic management 4. Requires having
a clear scaling up strategy identifying large scale delivery system and funding
understanding intervention does not exist apart from the political, social context and delivery system
Explicitly managing political process (advocacy) and implementation (system strengthening and governance )2
Learning and Leading for Large Scale Change
Scaling up historically taken for granted: “Build it and they will come”
• Key is the intervention itself, emphasis on technical content vs. process and delivery
• Proof of effectiveness is sufficient• Scaling up assumed to be spontaneous• Activities limited to Dissemination, Research
Utilization, or RIP; peer-reviewed journals, conferences, dissemination workshops
slide 3
Learning and Leading for Large Scale Change
Spontaneous Scaling Up based on information alone is empirically false
• Ignores ALL of new neuroscience and behavioral economics research on how decision making happens
• Reinforced by foundation attitudes/incentives– Limited resources
– Institutional incentives to be innovative, leveraging, and catalytic (the reactant is?)
– Strong ties (staffing) from universities; research mentality – Academic culture of publishing, “theory of change”
• Result is neither no scaling, policy adoption only, or scaling (quantity) without quality and impact
slide 4
Learning and Leading for Large Scale Change
0.0
10.0
20.0
30.0
40.0
50.0
60.0
Median availability of selected generic medicines 2001-2008
(%) Public Sector
5
Learning and Leading for Large Scale Change
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0Median availability of selected generic
medicines (%) 2001-2008Private Sector
6
Learning and Leading for Large Scale Change
New Approach to Scaling Up• Recognizes the greater importance of politics and
implementation/delivery systems capabilities and capacities i.e. & org. culture (5/80%, vs. 10/10%)
• Develops pro-active strategies for each issue; strategic management of scaling up process
• New innovations in delivery mechanisms, systems, quality; Supply, Demand and Delivery
• Technical support for either – growth (expansion) to create new delivery mechanisms– System strengthening of existing delivery mechanisms – Monitoring, learning and feedback of delivery at scale
7
Learning and Leading for Large Scale Change
Two Key Concepts
• Capacity (Reach or Scale) – ability of an organization to delivery and implement the programme at the targeted or desired scale
• Capability: ability of an organization to delivery and implement the program according to the way it was designed (fidelity), with quality achieving the expected impact
• Delivery and Implementation Organizations have to have both Capacity and Capability to achieve the scaling up goals
Slide 8
Learning and Leading for Large Scale Change
Definitions of Scaling Up (2)
3. Scaling up Organizational Delivery and Implementation Capacity to achieve greater reach and deliver additional contents
a) Expansion: creating capacity from zero, pilot organization gets bigger
b) Replication: hand off from one organization to another, usually grafting onto an organization that already has large scale delivery capacity
c) Collaboration: partnership between one or more organizations to leverage either reach in specific places or capacity in specific content
slide 9
Learning and Leading for Large Scale Change
Defining Successful Scaling Up
• Policy Adoption: integrated into policy/strategy• Reach: the scale to be attained, number of people,
locations, demographic, % coverage• Cost and Efficiency: meets a budget/financial
constraints• Impact: impact on key outputs and outcomes• Sustainability: political, financial and institutional• Equity: reaching the hardest to reach;
marginalized, poor indigenous, and rural areasslide 10
Learning and Leading for Large Scale Change
Six Elements of Scaling Up1. Intervention: proven or promising model/program2. Evidence: demonstrating effectiveness, efficient, compatibility,
and robustness and relevance3. Advocacy: convincing institutions to adopt, fund/resource and
implement the model at large scale 4. Modifying and Strengthening Organizations:
– Creating the capacity and/or capabilities (HR, infrastructure, systems and supply) to implement at scale and
– Alignment of rules/procedures, incentives and org. culture
5. Implementation at Scale: – Coordination and cooperation among multiple stakeholders
– Monitoring and feedback for fidelity, quality and adaptation
6. Sustainability: political, financial and institutionalslide 11
Learning and Leading for Large Scale Change
The 8 Common Breakdowns in Scaling Up
1. Missing Key Invisible Elements of the Model2. Lack of Generalizability3. M&E not Designed for Scaling4. Lack of Large Scale Delivery Capacity and
Capabilities/Compatibility5. Limited, Narrow Advocacy6. Training ≠ Capability Building7. No Provision for Monitoring and Feedback8. Dropping the Baton: Resources and Responsibility for
Going to Scale12
Learning and Leading for Large Scale Change
Missing Elements of the Model Mistake: neglect key invisible elements
Focus only on technical contents; drugs, equipment, procedures
Ignore preparations, preconditions, processes and follow-up Favor technical elements over tacit and intangibles
Solution: Identify intangible, tacit elements at the pilot phase Ensure use of qualitative methods in pilot design, process
tracing Comprehensive documentation of the model
13
Learning and Leading for Large Scale Change
Pilot Design, OR/M&E not designed for Scaling
Mistake: measure impact/efficacy Ignores role of social environment and organizational context
in pilot phase For scaling, not what works, but what works where and why
(use Realistic Evaluation approach) Ignores that information for policy makers not the same as
information for research Assumes whole model will go to scale as is
Solution: Identify and test key social/org. assumptions at the pilot phase
through M&E for scaling up Design into pilot/OR anticipated variations in social
environment and organizational context Measure cost and impact and disaggregate by components M&E for advocayScaling Up March 2010 Chalkboard Project 14
Learning and Leading for Large Scale Change
Lack of large scale delivery system: capacity and capability
• Mistake: – Large scale delivery system not identified up front and there exists
no system that can deliver at scale (lack of CAPACITY)– no attempt to ensure compatibility with large scale systems (lack of
CAPABILITY)
• Solutions:– Clearly identify goals for scale (who/where) and who the large
scale implementer will be up front – Identify capabilities needed for the model– Design the model to be compatible with the implementer’s
capabilities and capacities– Anticipate needed capabilities and capacities building
15
Learning and Leading for Large Scale Change
Limited and Narrow Advocacy• Mistake:
– Believe impact results necessary and sufficient– Ignore political considerations of policy makers– Exclusive focus on policy champions (patrons)
• Solution: – Identify decision makers, stakeholders, their interests– Develop messaging & evidence to address interests– Build advocacy into pilot design and evaluation– Organize broad-based multi-stakeholder coalition
across party lines and beyond usual suspects– Extend advocacy beyond adoption and strategy to
programming, funding and implementation 16
Learning and Leading for Large Scale Change
No Matter What the Problem, Training is the Solution
• Mistake: Ignore complete set of capabilities needed for successful implementation– aligning incentives – Ensuring adequate financial and esp. human
resources, – infrastructure, supply, logistics and systems, – organizational culture; rules, regulations and
procedures– Adding programs to systems w/o adding capacity– Training is short-term, not institutionalized
• Solution: Comprehensive capability building for entire system with dedicated MT resources
17
Learning and Leading for Large Scale Change slide 18
Modifying and Strengthening Organizations
SYSTEMS
INCENTIVES (Internal Politics)
RESOURCES & INFRASTRUCTURE
POLICIES/REGULATIONS
ORG. CULTURE
Capacity to Build Capacity
Skills
Learning and Leading for Large Scale Change
No Monitoring of Performance and Fidelity at Scale
19
Mistake: model will be implemented as designed with quality and fidelityKey components are often dropped, especially the
more tacit, intangible or process orientedEmphasis on quantitative targets and
achievements, outputs not outcomesTensions between dilution or distortions of the
model versus necessary adaptations Solution
Principles vs. procedures, local adaptationBuild in a monitoring system to ensure fidelity and
quality, learning and adaptation
Learning and Leading for Large Scale Change
Passing or Dropping the Baton: The Missing Link?
1. Developing a Promising Innovation2. Generating Evidence of Efficacy (Internal Validity)3. Generating Evidence of Effectiveness and Generalizability
(External Validity)4. Dissemination5. Adoption into Strategies < Policies < Programs < Funding <
Buy-In in District & Front-Line6. Creating the Pre-Conditions for Large Scale Implementation:
Training, Capability Building, Incentives, Org. Culture, Coordination etc.
7. Implementation at Large Scale8. Ensuring Impact on Key Outcomes (monitoring)9. Sustainabiility
slide 20
Learning and Leading for Large Scale Change
Summary (1)
1. Build scaling up into discovery, design and pilot testing from the very beginning
– Identify processes and intangibles as part of model– Test for generalizability and robustness (social and
organizational)– Disaggregated evidence on cost and effectiveness by
component, able to simplify– Identify capabilities needed and cost– Identify large scale delivery system with capabilities and
cost constraints up front and ensure alignment– Generate evidence for advocacy in pilot design
21
Learning and Leading for Large Scale Change
Summary (2)2. Advocacy
– beyond dissemination and RIP, champions to broad-based advocacy (coalitions) for adoption , funding and implementation (sustainability)
– Anticipate political and institutional opposition
3. Capability building beyond training to aligning entire system (health, education, etc)
4. Coordination and monitoring for fidelity and adaptation to local circumstances
5. Responsibility and resources for going to scale
22