Prove It: How the Gates Foundation Ties Strategy to Results
Jodi Nelson, Interim Director
Emily Parker, Sr. Strategy Officer
Impact Planning and Improvement
September 30, 2010
Schwab Charitable Philanthropy Speaker SeriesCenter for Nonprofit and Public Leadership
Haas School of Business
What are your questions about
strategy and measurement
at the Bill & Melinda Gates
Foundation?
2© 2010 Bill & Melinda Gates Foundation |
Today we would like to:
Collect your questions
Share our aspiration: is it about “proving” it?
Share what we do in practice
Share our big questions
Ask for your ideas from other experiences and sectors
3© 2010 Bill & Melinda Gates Foundation |
Fast Facts about the Bill & Melinda Gates Foundation
4
(1) Number of countries in which the foundation has granteesSource: Bill & Melinda Gates Foundation Fact Sheet, September 2010, gatesfoundation.org
© 2010 Bill & Melinda Gates Foundation |
Year founded 1994
Employees 858
Offices 5
Number of strategies 25
Geographic reach (1) 100+ countries
Total endowment $33B
Grants since inception $23B
Grants in 2009 $3B
Changes over the past decade set the context for strategy and measurement
5
From To
Foundation growing rapidly
Pay out requirement hard to meet
A few strategies (i.e., high school education, US libraries)
No consistent approach to strategy development or measuremento little guidanceo let a thousand flowers bloom
Foundation at a stable size
Focus on use of limited resources
25 strategies across 3 broad sectors: global health, development, US education
Approach to strategy maturing
Actionable Measurement beginning to take shape o in learning mode—from inside and outside
© 2010 Bill & Melinda Gates Foundation |
Imagine…
You lead the Agricultural Development program at the Bill & Melinda Gates Foundation.
You have to decide how the foundation should invest in agriculture to help alleviate poverty in Sub-Saharan Africa and parts of Asia.
What will the foundation’s goals be? How will we achieve them? How will we know if they are achieved?
You have a meeting with Bill & Melinda in six months and need to tell them your plan.
What do you do?
© 2010 Bill & Melinda Gates Foundation | 6
Why is it so tough?
The world is complex — change is caused by many different things.
We are just one actor — governments, NGOs, other donors, private citizens are also trying to improve people’s lives
We are not on the “ground” — we are strategic donors, but still just donors.
Data can be hard to come by — we may know we need it, but collecting meaningful data is hard to do.
7© 2010 Bill & Melinda Gates Foundation |
© 2010 Bill & Melinda Gates Foundation |
So what do we do?
Use tools of strategy and measurement to plan our workdespite the complexity
8
Our aspiration is to create feedback loops so we can plan, execute, measure, learn, adjust, plan …
9
Execute
Develop/ refresh strategy
Track progress and evaluate what works
Review strategy
Learn, adapt, improve
data &
experience
data &
experience
data
&
expe
rienc
e
data
&
expe
rienc
e
© 2010 Bill & Melinda Gates Foundation |
Let’s break it down….
10© 2010 Bill & Melinda Gates Foundation |
What is a strategy at the foundation?
11
Causal pathway to impact. Outlines the investments and programmatic activities aligned with that pathway. Defines the results of these investments and activities over time.
© 2010 Bill & Melinda Gates Foundation |
Global Development Program Agricultural Development Financial Services for the Poor Global Libraries Water, Sanitation, and Hygiene Policy and Advocacy
Global Health Program Discovery Enteric Diseases and Diarrhea Family Planning HIV Malaria Maternal, Newborn, and Child
Health Neglected and Other Infectious
Diseases Nutrition Pneumonia Policy and Advocacy Tobacco Tuberculosis Vaccine Preventable
Diseases/Delivery
United States Program College Ready Early Learning Pacific Northwest Postsecondary Success U.S. Advocacy U.S. Libraries
Bill & Melinda Gates Foundation Approved Strategies
Policy and Government Affairs Charitable Sector Support
Our vision of success Strategies are constantly improving, based on measurement and learning
12
“No strategy survives first contact with the enemy.”
Measurement and learning—coupled with systematic review, reflection, and intellectual dialogue—drives improvement.
© 2010 Bill & Melinda Gates Foundation |
intended strategy
unrealized strategy
deliberate strategy
emergent strategy
realized strategy
Figure created by Henry Mintzberg, strategy theorist.
Our business process reflects this philosophy
13
Strategy development: creating a strategy “from scratch”
Strategy review: annual update for the CEO and co-chairs on progress against an approved strategy
Strategy refresh: re-examining an approved strategy with the expectation there will be a significant change in strategic direction
What When
Once: after a period of learning
Every strategy, every year
Every three to five years
© 2010 Bill & Melinda Gates Foundation |
Strategy development and refresh is an iterative, phased process with progressive analyses from broad to narrow
© 2010 Bill & Melinda Gates Foundation | 14
Three concepts underpin strategy development and refresh
15
segmentation(s) of the problem/ space and identification of opportunities for change
hypothesis about what needs to happen for the stated goal to be achieved
What must the world do?
what we will do (directly or through others) to help achieve the stated goal
What must we do? How will our actions
maximize leverage?
What it is
Key question
© 2010 Bill & Melinda Gates Foundation |
Opportunity map Theory of changeTheory of action solution leverage partner leverage
Where is the leverage?
16© 2010 Bill & Melinda Gates Foundation |
“Learning initiative”
established
Limited grantmaking
begins
Strategy developed & approved--
the sky’s the limit
Endless possibilities;
limited resources
Strategy review—
strategy is adjusted
???
What about measurement?
What is it anyway?
Why is it so difficult to do well?
What is our philosophy and approach?
17© 2010 Bill & Melinda Gates Foundation |
© 2010 Bill & Melinda Gates Foundation |
What is measurement?
By definition: Obtaining the magnitude of a quantity
Performance measurement o Assessing the achievement of pre-determined goals and objectives
o Produce objective, relevant information on program or organization performance
o Developing measurable indicators that can be tracked to assess progress
What is unfortunate about the word “measurement” o Not everything is about counting or “tracking”
o Under emphasizes evaluation, mixed methods, asking and answering questions about how things happen and why — not just what happens
18
What's our philosophy? Actionable Measurement
Direction from the co-chairs in 2008
Measurement should be strategic and actionable
We should not measure everything:
o Rely on grantee reporting as much as possible
o Align our data requests with those of other funders
o Learn from others
o Leverage technology for data collection and viewing results
19© 2010 Bill & Melinda Gates Foundation |
Actionable Measurement
20
Measure to inform decisions and actions. How you do it (the approach, methodology, evaluation design) is determined by the purpose. There is no one measurement approach that works all the time, for all purposes.
Why is it unique to be pragmatic?
Where do we weigh in on the philosophical debates?
The “gold standard” is use
© 2010 Bill & Melinda Gates Foundation |
© 2010 Bill & Melinda Gates Foundation | 21
Tracking results across time/space
o Tracking execution is good for accountability
o But tough to do well but can describe change over time
Evaluation designs
o Impact evaluation purpose: demonstrating effectiveness of a model/approach, good
for replication, informing the field, testing assumptions to check strategy
o Process evaluation purpose: to improve implementation/execution at key times
o Participatory evaluation purpose: to empower participants with information to act
o Developmental evaluation: to identify components of a model or approach in real
time
Getting useful data requires choosing the right methods or
approaches to measurement given what you need to do with it
Example of purpose driven measurement
It is an approach to measure results of our strategies
22
Inputs Activities Outputs Outcomes Impacts
Strategy
Initiative Sub-Initiative
Grant
Sub-Grant
How is this different from the measurement challenge facing implementingorganizations (grantees)?
© 2010 Bill & Melinda Gates Foundation |
© 2010 Bill & Melinda Gates Foundation |
What’s the essence of the approach?
Measure impact (change in people’s lives) to track collective progress toward targets, not to attribute impact to the foundation’s actions or investments.
Measure key output and outcome results to track progress of the foundation’s work; use evaluation to inform decision making by testing assumptions, learning what works, how and why.
Selectively track results of grants at critical points for accountability; evaluate to learn about implementation of key investments; use impact evaluation when it seeks to answer a strategic question for an initiative not as an accountability tool.
Strategy
Initiatives
Grants
23
Measuring investment types
24
Priority for evaluation resources
Research Product Development
Model Development
and Demonstration of
Effectiveness
Delivery at Scale
Systems Change
Policy and Advocacy
Measure attribution where it is
technically feasible and ethical
Track execution and
coverage/reach
Measure desired outcomes, track progress, and focus on shorter-term feedback
Measure success or failure and extent of fit with target
product profile
Monitor outputs and
track process
Types of Investment
Measurement Guidelines Specific to Investment Type
Use multiple measurement methods to draw conclusions
© 2010 Bill & Melinda Gates Foundation |
Our questions…
25
Execute
Develop/ refresh strategy
Track progress and evaluate what works
Review strategy
Learn, adapt, improve
data &
experience
data &
experience
data
&
expe
rienc
e
data
&
expe
rienc
e
© 2010 Bill & Melinda Gates Foundation |
How do we identify, articulate, and evaluate
alternative paths to impact?
What results do we aggregate? What are the
most promising interventions to evaluate?
How do we adjust our strategies while maintaining
continuity in our partnerships?
How do we keep the bar high and hold people accountable
for results, while leaving room for failure and learning?
How do we develop theories of change where evidence
of what works is slim?
Thank You
© 2010 Bill & Melinda Gates Foundation. All Rights Reserved. Bill & Melinda Gates Foundation is a registered trademark in the United States and other countries.