Chlorine Dispensers: Bringing Safe Water to Scale

Final Performance Report: July 2012 – December 2015 Prepared by: Evidence Action

Cooperative Agreement No.

AID-OAA-A-12-00018

Dispensers for Safe Water is Implemented by Evidence Action via

Sub-Award from Innovations for Poverty Action

I. Executive Summary ......................................................................................................................... 1

1. What were the key objectives? How did the project aim to achieve these key objectives? .......... 1

2. What are the key results and lessons? ............................................................................................ 1

3. What are the next steps? ................................................................................................................. 2

II. Overview .......................................................................................................................................... 2

1. Overview of Grant ............................................................................................................................ 2

2. Organization Description ................................................................................................................. 3

III. Program Design and Implementation .............................................................................................. 3

1. Project Goals .................................................................................................................................... 3

2. Project Implementation ................................................................................................................... 4

3. Stakeholder Engagement ............................................................................................................... 11

4. Beneficiaries ................................................................................................................................... 14

5. Challenges ...................................................................................................................................... 15

IV. Evaluation Design ........................................................................................................................... 16

1. Verifying Goals and Objectives ...................................................................................................... 16

2. Impact Evaluation .......................................................................................................................... 17

3. Final Results ................................................................................................................................... 17

4. Data Quality ................................................................................................................................... 20

5. Analysis Limitations ....................................................................................................................... 20

6. Additional Evaluation Plans ........................................................................................................... 21

V. Findings .......................................................................................................................................... 21

1. Lessons Learned ............................................................................................................................. 21

2. What do the lessons learned from this project imply for future funders and/or policy makers? 24

3. Context Extrapolation .................................................................................................................... 24

VI. Cost-effectiveness & Competitive Landscape ................................................................................ 25

1. Competitive Advantage ................................................................................................................. 25

2. Extent of Impact ............................................................................................................................. 25

3. Cost-Effectiveness .......................................................................................................................... 28

VII. Scaling Plan .................................................................................................................................... 30

1. Paths to Scale ................................................................................................................................. 30

2. If Ongoing Public Investments: Sustained Operations without DIV Support ................................ 30

3. If Not Ongoing Public investments: Who Will Benefit? ................................................................. 31

4. Scale Plan ....................................................................................................................................... 32

VIII. Appendix: ....................................................................................................................................... 33

1

As set out in our original proposal in 2012, our key objectives were to reduce childhood diarrhea by at least 20

percent for 5 million people and to transition Dispensers for Safe Water to scale, establishing them as a standard

approach to water treatment over the course of 3 years. For safe water access, very rural and very poor communities

are still underserved in Sub-Saharan Africa. With water really becoming an issue of equity, we believe we have a

different approach. Through our dispensers, we aimed to: Provide six million people across three countries with

access to chlorine via cost-effective, sustainable dispenser programs (we revised our goal from five to six million

after Year 2); Seed Dispensers for Safe Water in multiple countries; Disseminate project results to achieve support

for evidence-based Dispensers for Safe Water replication; Consolidate development of a reliable, low-cost supply of

commercially manufactured dispenser hardware, spare parts, and chlorine that are readily available to meet

dispenser demand globally; and Establish replicable, cost-effective, scalable, and sustainable operational models

that are available for adoption by dispenser programs in multiple and diverse contexts.

In December 2015, we measured dispensers in Kenya, Uganda, and Malawi to be serving 4,497,382 people, 25%

below the revised target of 6 million, which includes the 333,108 people served from the baseline in Kenya and

Uganda. In retrospect, we were too confident in our initial projections about scale. Decreases in adoption rates at

times forced us to curtail the number of dispenser installations in Uganda and Malawi, therefore reducing the

number of people with access to safe water. Additionally, we updated the way we calculate the number of people

served in 2015. We realized that a major consequence of moving from high population density to low population

regions is that the number of people served per dispenser will decrease as the program expands. This means that

while the way we calculate the number of people served is more accurate now, our original projections were too

high.

With a meta-analysis summarizing 15 studies from 9 countries in Africa, Asia, and Latin America showing that treating

water at the household level with chlorine reduced childhood diarrhea by an average of 29%1, the basis on which

we established our goal of decreasing childhood diarrhea by 20%, we knew dispenser up-take would be key. Our

baseline adoption rates were 40% in Kenya, and the program-wide adoption average across all countries has grown

to 51% in December 2015 over a much larger territory, driving home how effective the program is. We also surpassed

our set targets on the number of published articles mentioning the program, number of reputable press mentions,

number of conference presentations, and the total value of additional resources supporting scale up. Indeed, the

value of third party resources raised was 50% above set targets at over $9 million cumulatively in both individual

donations and additional grants. The program has also consistently kept the manufacturing costs per dispenser per

year low, resulting in a $5.09 reduction in targeted costs. On average, the program has achieved above expectations

on metrics measuring the quality of the program’s operations. We were able to expand our operations to Uganda

and Malawi, and we have planted a few promising seeds in India and Ethiopia. Over this period, with USAID’s support,

we have managed to join a select club of social enterprises that have rapidly scaled their operations to cover millions

of beneficiaries.

1 Arnold, B. F., J. M. Colford Jr. 2007. Treating water with chlorine at point-of-use to improve water quality and reduce child

diarrhea in developing countries: A systematic review and meta-analysis. The American Journal of Tropical Medicine and

Hygiene 76, (2): 354.

2

We have learned a lot over the course of these three years. First, we learned what can happen to revenue projections

when the price of carbon credits collapses. And after scaling our pilots, we learned that efficiencies at scale take

time. There are inherent challenges with rapidly scaling a pilot due to the inherent bandwidth restriction that

accompanies small staff sizes during a fast growth period. We learned that specific governmental partnerships are

important for Dispensers for Safe Water to operate at scale. For example, our partnership with the Ministry of Health

is the linchpin for our far-reaching operational model in Malawi. That said, these types of quality partnerships are

not common; we tried to cultivate the same kind of relationship in Kenya and Uganda, but due to a change in the

administrative structure in Kenya and lower government enthusiasm in Uganda, we decided to expand the program

on our own while keeping government informed but not actively engaged.

We also learned that maintaining high adoption rates, our highest priority over the last two years and key to the

success of the program, is difficult to maintain at scale. Consistent provision of chlorine at the village-level, functional

hardware, and quality meetings prior to installations were key to maintain high adoption rates. Additionally, we

learned that while full saturation coverage is desirable from at-scale programs, it is very difficult to actually achieve

that level of scale. It is important to acknowledge the fact that in most cases, behavior change is very hard, and 100%

population coverage is not a realistic expectation. Nevertheless, this expectation persists, which has made gaining

traction with some donors and peer organizations a challenge.

Our main priority going forward is to fundraise to cover our net deficit in total program costs and strengthen the

sustainability of our program. Since there has been a downturn in the carbon market, we can no longer rely on

carbon finance as our sole stream of revenue. We are therefore looking to test new sources of revenue such as

charging fees and commercializing program assets. On the commercialization side, we can offer branding

opportunities for corporations or NGOs through our 18,000 dispenser network in Kenya alone, as well as

opportunities in market research, last-mile product distribution, and messaging platforms.

Additionally, we intend to engage more heavily with governments going forward. During the award period, we

managed to leverage the existing government health surveillance network in Malawi with a commitment to procure

chlorine for our dispensers. We expect this partnership to continue in the future, and we will continue to engage the

government of Malawi to increase this support. As for Kenya and Uganda, a stronger engagement with the

government is on the agenda for 2016 and beyond as we seek to commit budgetary allocation to augment our carbon

revenues. This intervention is perhaps the cheapest way to make water clean at the source and clean at home in

rural areas.

Finally, as we look at scale in an international context, we plan to develop our partnership strategy. This would

consider all options, from a franchise model to a consulting model, whereby we would transfer our knowledge and

expertise to implementing partners such as large international NGOs working in water, while maintaining the rigor

of the approach through licensing or some other means.

Please give an overview of project including timeframe, funding level (DIV and leverage/cost share) and brief

contextual information about the location of implementation and testing.

Upon initiation of the project in 2012, we started with $4,610,336 of Stage 3 funding for 3 years, from June 2012-

June 2015. While IPA originally signed the award in 2012, Evidence Action officially spun off as its own independent

3

organization on October 1, 2014. We then became an official sub-awardee on October 1, 2015. After a no-cost

extension, the award period was extended until January 15, 2016.

A modification increased our total funding to $7,416,557 in 2014 to increase the number of people served from 5 to

6 million. Initial leveraged funding included $3.8 million, including $1.8 million toward a long-term health impact

study from The Bill & Melinda Gates Foundation, as well as $0.3 million from other donors including the World Bank,

Japanese International Cooperation Agency, and the International Initiative for Impact Evaluation for operational

research. While we originally pressure tested the program in other locations such as Zimbabwe and Haiti, in the end

we implemented Dispensers for Safe Water in Kenya and Uganda, and officially introduced a third country, Malawi,

in 2014.

Please give a one paragraph description of organization/company

Evidence Action scales proven development solutions to benefit millions of people around the world. We fill the gap

between knowing “what works” and having impact at scale. We lead and manages two programs incubated by

Innovations for Poverty Action: Dispensers for Safe Water and the Deworm the World Initiative. We also run

Evidence Action Beta where we are currently testing a number of other rigorously-evaluated interventions for scale-

up.

What were the goals/targets of the project?

The project’s overarching goal is to reduce under-five mortality from diarrhea by seeking:

1. To reduce childhood diarrhea by at least 20 percent for five million people that will be reached by this

project during its three-year implementation phase; and

2. To transition Dispensers for Safe Water to scale and establish them as a standard approach to water

treatment that will reach at least 75 million people.

In order to achieve the two project goals, we aimed to deliver the following results:

1. Provide six million people across three countries with access to chlorine via cost-effective, sustainable

Dispenser programs;

2. Seed Dispensers for Safe Water in multiple countries;

3. Disseminate project results to achieve support for evidence-based Dispenser for Safe Water replication;

4. Consolidate development of a reliable, low cost supply of commercially manufactured dispenser hardware,

spare parts and chlorine that are readily available to meet dispenser demand globally; and

5. Establish replicable, cost-effective, scalable and sustainable operational models that are available for

adoption by Dispenser programs in multiple and diverse contexts.

4

Describe the implementation of this project.

We broke project implementation down into our five output categories: Provide six million people with access to

safe water; Increase adoption in successful pilots; Disseminate project results; Supply of commercially manufactured

dispensers; and Optimize quality of operations.

Our implementation took place over the course of three years: Year 1, June 2012-July 2013; Year 2, June 2013-July

2014; and Year 3, July 2014-December 2015. Note that this section will focus more on the sequencing of events; we

will discuss our target figures in Section IV.3. Final Results and highlight important insights given these experiences

in Sections III.5. Challenges and V.1. Lessons Learned.

2 The method used to calculate the Number of people served changed starting October 1, 2015 to reflect a more sensitive

method of detecting changes in population density across counties within each country; the figures shown in the graph

retroactively reflect this new methodology since the program’s inception.

5

Kenya

In our first year, Dispensers for Safe Water installed a total of 1,119 dispensers in Western Kenya, bringing the total

number of dispensers to 3,208 reaching a total of 489,624 people served. We reduced field activities from January

to March 2013 to prepare for potential heightened tensions during the presidential and county elections. This period

gave the program an opportunity to evaluate learning from the latest round of installations, update the best

practices and plan for the expansion activities to be conducted in over June 2013-14. We also finalized the

comprehensive “toolkit,” fully systematizing best practices for implementation and providing a strong foundation

for field operations for successful, rapid scale up.

In the second year, we installed a total of 5,554 dispensers across 6 counties to serve an additional 682,472 people

for a total of 1,172,096 people served in Kenya. For each installation, we hosted a local leadership meeting, a

waterpoint verification, a village-level sensitization meeting, and a community education meeting. We also learned

that as we scale up, fewer water points pass our verification requirements, which ensures that we are expanding to

regions with high population and water source density, leading us to hit 85% of our target of 9,411 by June 2014.

While this is challenging in terms of reaching program targets, it is an important lesson learned, emphasizing the

nuances of waterpoint selection for maximum program impact and long term cost-effectiveness.

By the end of December 2015, we installed a total of 17,784 dispensers serving 2,215,704 people in Kenya. We

expanded into existing and new areas in Western, Nyanza, Uasin Gishu, and Tranzoia parts of Kenya by opening up

nine new field offices to facilitate expansion activities. The reporting year also recorded the highest number of

dispenser installations in a single year since the program began, a result of a stable chlorine supply and dispenser

maintenance throughout the year. This ensured that the communities received an uninterrupted supply of safe

water through the circuit rider distribution model managed through the field offices.

Uganda

By the first year of the program, we selected Uganda as our second country for expansion after an initial pilot in

early 2012 in Eastern Uganda yielded high adoption rates. Significant scale-up activities began in January 2013, with

6

stakeholder meetings and district meetings across the 23 sub-counties and 90 parishes in the 2 districts. In total,

12,376 village members and stakeholders participated in these meetings. We installed an additional 499 dispensers

in Uganda by June 2013, providing 204,619 people access to safe water.

During the second year, we installed 1,339 dispensers in Kibuku, Budaka, and Manafwa districts. This brought the

Uganda program to 1,861 total dispensers serving 601,531 people. We also began our expansion into Sironko and

Mbale districts, completing 1,244 water point verifications in Siroko and finding 790 suitable water points. In Mbale,

we verified 1,573 water points and found 913 suitable water points. We also hired 16 additional full-time staff

members, bringing our team to 46 total in Uganda; this was essential to building a robust team to implement the

program.

By December 2015, we installed a total of 5,585 dispensers, providing a total of 1,622,347 Ugandans with safe water.

We also made the strategic decision to scale down on our expansion rate in Uganda from April-June 2015. We did

this to direct most of our field teams’ efforts towards tackling the low adoption rates the Uganda program was

experiencing at the time. After improving our supply chain, consistent chlorine delivery, quality program

management, and restructuring the team in Uganda, along with working with a pro-bono Human Centered Design

consultant to understand and guide dispenser consumer behavior economics, our team implemented various

adoption-boosting activities that helped turn our declining adoption rates around. We will discuss this more in

Output 2: Increase Adoption in Successful Pilots and Section III.5. Challenges.

Malawi

From June 2013-14 we tried to pilot the program in Malawi and Tanzania. After trying for a year, we were unable to

gain clarity on the Clean Development Mechanism project approval process in Tanzania and decided to focus our

attention on Malawi, where we got enthusiastic support from the government due to annual cholera outbreaks in

Southern Malawi. We installed 87 dispensers in Zomba district for the pilot. These dispensers provided

approximately 27,743 people with access to safe water; 430 people per dispenser was the highest number of people

per dispenser than we have seen in any of our countries to date. In line with our expansion strategy, we worked

closely with the government of Malawi and health service assistants, full-time government employees to implement

the pilot program, with health service assistants completing the last mile of chlorine distribution, an integral step in

ensuring a cost-effective and far-reaching distribution network.

Based on evaluations carried out three months and six months into the pilot, we eventually decided to focus on

Malawi as Scale-Up Country #3, registering Evidence Action in Malawi in 2014. Since then, we installed 3,724

functioning dispensers by the end of December 2015, providing 659,331 people with access to safe water. We

achieved such a high rate of expansion because of our operational model, working closely with health surveillance

assistants, grass root health care providers, and fast tracking the verification exercise following the purchase of 40

additional cell phones. With rapid program growth in Malawi, we grew to 17 employees by June 2015. With more

staff in operations, we were able to provide better support to the 415 health service assistants, ensuring that we

completed all dispenser installation preparations quicker than in the past while maintaining our quality standards.

Other Countries

We initially considered Rwanda for Country #3 during the first year. However, the Minister of Health requested to

work with another safe water project, which we will discuss more in Section III.5. Challenges. We eventually

concluded it would be more prudent to invest resources for expansion in another country given these facts.

As mentioned above, we later identified Tanzania and Malawi as countries with strong potential for expansion based

on a newly developed scoping framework that included factors like population density, national chlorination rates,

7

under-five population proportion, chlorine availability, national under-five diarrheal incidence rates, and logistical

support. In the end, we placed the Tanzania program on an indefinite pause due to issues with the Clean

Development Mechanism approval process, which we will also discuss more in Section V.1. Lessons Learned. As a

result, we decided to hold off on the Tanzania dispenser pilot.

Kenya

At the project’s commencement, our M&E systems were too immature to systematically collect adoption data. At

the end of the 3-month adoption evaluation exercise for the 1,010 dispensers installed in Ugunja in the first year,

we achieved an overall adoption rate of 41% in that region. Separately, we began to work with researchers from

Harvard University and the University of Maryland to evaluate how varying the selection mechanism of promoters,

as well as the level of compensation that promoters receive for their volunteer work, affects chlorine adoption rates.

We would roll out and scale a promoter incentive scheme if it proved to boost adoption rates for the long-term over

the evaluation period in a cost-effective manner.

Over the second year, we achieved an encouraging 10% increase in adoption, from 33% overall in June 2013 to 42%

in June 2014. We credit this achievement to the introduction of the circuit rider model, resulting in high refill and

dispenser maintenance rates, a more systematic rolling installation model, and increased program scale.

Additionally, analysis showed that in locations where a promoter’s water tested positive for chlorine, the entire

community was 16.8% more likely to test positive for chlorine. We also attributed the higher adoption rates to a

number of interventions targeted at incentivizing promoters, from chlorine delivery scripts to biweekly phone calls.

By the end of the third year, we recorded an average adoption rate of 57%. We achieved this in spite of a dip in our

adoption rate from February-April after we implemented a number of strategies to increase the chlorine usage and

curb false rumors started by other safe water programs. These included re-zoning dispensers for effective service

delivery and management, awareness raising at local government administrative meetings to address the negative

rumors, as well as organizing consultative meetings with competing water programs. Our strategies paid off in May

3 Because Zomba 1 & 2 were only in the pilot stage in 2014 and adoption data was not collected continuously in Malawi until

2015, we did not include 2014 Malawi adoption data in Figure 3.

8

2015, with the overall adoption rate of 47%, up from 33% in April, and we maintained these rates, ending the year

with our highest quarterly adoption rate to date for Kenya at 57%.

Uganda

In the first year, the 40-dispenser pilot program yielded a 49% adoption rate compared to the baseline rate of 5%.4

These figures came from 40 spot checks, 40 promoter surveys, and 436 household surveys. We then started

monitoring and evaluation activities for the 448 expansion dispensers during the January-March 2013 quarter and

then randomly selected 24 water points and conducted 204 baseline community surveys. We also conducted a 12-

month adoption evaluation for the 10-dispenser pilot program initially rolled out in February 2012, which found a

sustained 73% adoption rate. Because this adoption rate was significantly higher than other dispenser programs, we

developed pilot programs to further examine the factors leading to such a high rate.

Over the second year, we assessed the data from the pilots. The first pilot included a series of activities modeled

after the activities carried out in the Kibuku pilot, which had the highest adoption rate in the program. Activities

included additional education for promoters, education for local leaders, weekly calls to the promoters, and

waterpoint visits. In addition, we hosted a competition among field staff where they worked in pairs and were given

a budget of roughly $10 per waterpoint to realize their own ideas. To qualify any one of the activities described

above as a success, we were looking for an at least 5% increase in adoption. Unfortunately, none of the first pilot’s

activities showed any significant increase in adoption.

In the third year, we saw low adoption rates from January-March 2015 consistently falling below 30%. We primarily

addressed this decline through improvements in our supply chain, consistent chlorine delivery, quality program

management, and restructuring the team in Uganda. Additionally, we commissioned a Human Centered Design

study through a pro-bono consultant to further understand the causal factors of this situation, which we will discuss

more in depth in Section III.5. Challenges of this report. Using the insights from the study, found in Appendix 1:

Human Centered Design Report, we implemented activities such as community re-engagement and promoter

engagement meetings. In the end, our research showed that our impovements to our delivery mechanisms, as well

as the re-trainings, generated positive results. Before these updates, adoption rates were below 30%. Afterwards,

adoption rates broke through the 40-percentile margin, achieving 46% in June 2015. We also improved the chlorine

supply chain processes to and from Nairobi and the respective dispensers in response to chlorine shortages in the

field. We developed accurate and timely chlorine forecasts, sorted out customs clearance challenges, increased our

storage space for chlorine, and streamlined field-based chlorine delivery processes. Consequently, the percentage

of dispensers found empty during random visits by our internal auditing team reduced from 43% during the first

quarter of the year to 25% during the second.

Malawi

In the second year, we evaluated our two rounds of dispenser installations in Malawi: the first from the December

2013 installations and the second from early March 2014. The results were encouraging; we found adoption rates

of 78% with dispensers functioning properly 98% of the time.

By the third year, we recorded a 49% adoption rate in January 2015 in Malawi, the lowest recorded rate in Malawi

to date. Similar to what deterred consumers from using chlorine in Kenya and Uganda, the reasons for this drop

ranged from the refusal to use chlorine due to the smell of the water to the belief that chlorine was a family planning

method aimed at making those who use it sterile. As we will discuss in Section III.5. Challenges, we needed to

4 Demographic and Health Survey 2011 Data for rural Eastern Uganda.

9

support our working model with a system that gave us up-to-date information from the field-level. Our “super-

engagement model,” which involved more interaction among our field staff, promoters, and traditional leaders,

resulted in adoption rates moving from 49% in January to 90% in February 2015. Since then, this active community

engagement enabled us to maintain a 6-month average chlorine adoption-rate of 84.5% and enabled targeted

communities to view chlorinated water as a way of life, maintaining high adoption rates.

We worked on sharing our dispenser model with other institutions and organization throughout the project period.

We have been featured on many news publications, from Wired Magazine to The Huffington Post to The Stanford

Social Innovation Review. We have also had the opportunity to present Dispensers for Safe Water to others through

a variety of mediums, including a White House Google Hangout to the MDG Countdown, a side event to the UN

General Assembly hosted by USAID’s DIV. Our work has frequently been highlighted by USAID leadership in both

internal and external communications materials and when briefing Congressional staffers about USAID’s

investments. For the full list of publications and presentations, see Appendix 2: Disseminated Project Results.

The physical dispenser models have undergone an evolution over these last three years. In the first year, we focused

on improving the Generation 3 dispenser models to meet the challenges faced in field implementation. Generation

3.5 dispensers included stronger, marine grade padlocks, more durable dispenser lids, stainless steel bolts allowing

for a stronger bond, screen printed instructions as opposed to stickers that would peel off, stronger, anodized QR

asset tags, and a newer plastic mold design to decrease cracking and leaks.

By the second year, the engineering team made strides in driving down the costs of dispensers through a number of

manufacturing and supply chain improvements, such as working with new suppliers to decrease longer lead times

for imported parts. As a result, we were able to drop the cost of a dispenser from $57.80 to $51.84, surpassing the

final 2015 cost target. Additionally, with a better understanding of the manufacturing supply chain in our second

year, the export to Uganda in January took one month from order to delivery with the total cost to export at 6% of

the value of the consignment. This was an improvement to the first year when shipments took 2 months and the

exports cost 27% of the shipment’s value. With the program’s growth, a higher volume of purchase orders also

enabled us to switch to larger, more cost-effective suppliers and gain enough leverage to re-tender past purchase

agreements. Lastly, our low hardware failure rates from the Generation 3.5 dispensers allowed us to mainly focus

on systematizing field level maintenance and improving our assembly work. The only design improvements were a

stronger valve design, lid injection mold alterations, and testing a solid chlorine model prototype.

In the third year, we hired an engineer to further improve the maintenance and quality of our dispensers and trained

more than 50 staff members in the field to carry out dispenser maintenance and installations. They also started

working on other hardware improvements, such as improving the safety of the Barcode ID tags, an important

component for claiming carbon revenue. Supply chain improvements this year included identifying a local

manufacturer in Kenya that could alter the molds that create the plastic dispenser components. This gave us an

opportunity to address design weaknesses in Dispenser Generation 3.5 and reduce part breakages in the field. As a

result, we altered the basket by introducing ribs and webs on the padlock lip to reduce lip breakage, the lid by

introducing ribs and webs on the lid hinge to reinforce and strengthen the hinge, and the valve by modifying it to

reduce instances of stiff valves. The local sourcing also helped us reduce the cost of hardware procurement by 45%

and therefore improved the potential production capacity, which increased from 2,482 dispensers produced in

January-March 2015 to 5,165 dispensers in April-June 2015.

10

Kenya

We significantly improved the quality of our information systems by June 2013. We integrated electronic data

collection tools and protocols into every component of our “Best Practices” operations guidelines, ensuring that high

quality data collection became an integral part of every activity. Additionally, as previously mentioned, we built tools

to enable field teams in different geographical regions to have access to the central database and to efficiently make

use of the information to further optimize our operations. We launched an online data portal, an internal

organizational resource that houses many data sets compiled from operations and evaluations surveys, as well as

several real-time and static reports that aggregated data in order to help field teams keep track of key indicators by

regional office. It is also important to note that as we finished initial rounds of meetings culminating in sub-county

dispenser installations, we transitioned from having a field office to a satellite office. A satellite office is much smaller

in size and cheaper to maintain. A field office during intial program expansion needs to accomodate about 20 staff

but in steady state, after we complete our initial installations, we only require 5-6 staff to work from the satellite

office to serve about 1,200-1,500 dispensers.

During the second year, we made significant gains towards achieving our cost-effectiveness goals. We were able to

reduce dispenser hardware costs through new dispenser designs, reduce installation costs by purchasing our own

vehicles and managing transportation in house, and reduce chlorine delivery costs through the circuit rider model.

Over the third year, to increase the quality and effectiveness of the field chlorine supply chain and services to the

communities, we created chlorine storage hubs in order to reduce time taken and distance covered during deliveries

to our promoters and waterpoints. We also re-distributed an equitable number of waterpoints to the community

service assistants who deliver chlorine and maintain our dispensers. As mentioned in Output 4: Supply of

Commercially Manufactured Dispensers, our engineering department hired more personnel to assist with dispenser

repairs at field office levels. Lastly, in January 2015, Evidence Action evaluated and analyzed our staffing needs

compared to our annual goals, which resulted in reducing our personnel by 21% for cost-effective and efficient

operations without compromising quality. Out of the 168 staff in Kenya at the beginning of the year, 21 were

released in March and an additional 14 between April and June.

Uganda

In the first year, we were able to leverage the products that the Kenya program developed to support our operations

in Uganda. We adopted the electronic survey program that we developed in Kenya, with each team member

receiving a smart phone used for data collection. We also modified and adopted the “Issue Tracker” and “Chlorine

Delivery Tracker” that the Kenya program adapted. Unfortunately, liquid chlorine was not available in Uganda

through June 2013, and we needed to source chlorine and dispensers from Kenya, which added to supply chain costs

and time.

In order to improve our operations by June 2014, we continued to use the Kenya program to model several elements

of the Uganda program off of. During this time, we hired 5 community service assistants to carry out chlorine

deliveries to our 1,837 dispensers. We also hired two field officers to manage the community service assistant team.

We have implemented a direct delivery circuit rider model that allowed our community service assistants to interact

with the promoters when delivering chlorine, as well as eliminate the need for our volunteer promoters to have to

walk long distances in order to collect refills of chlorine. The circuit rider model also allowed our community service

assistants to perform routine maintenance on the dispensers as needed.

We continued to improve the Dispensers for Safe Water business model to make it more robust and cost-efficient

in the third year. This included recruiting a program manager to provide strategic leadership and management over

11

program operations. We reduced the number of the field team by 27 field officers; this came after reducing dispenser

installation targets for the second quarter, as mentioned above, to channel more efforts towards adoption boosting

activities. This resulted in a team that is leaner but more effective. We also established satellite offices in

Namutumba, reduced the travel time and cost required to service each dispenser in the area while also improving

chlorine storage space. Lastly, we rolled out a new management information system over January-March 2015. This

cloud-based system is central in providing key information to management on the planning, implementation, and

reporting processes of Dispensers for Safe Water, giving us more direct line of sight into the reasons behind the

various adoption rates we achieve and allow us to respond accordingly.

Malawi

By June 2014, the Malawi team prioritized developing a cost-effective method for delivering chlorine to rural villages.

The first part of the supply chain is to identify the best manufacturer to produce the chlorine, so we began by making

contacts with two local suppliers and the Malawi team made site visits to their factories. In addition, we conducted

research on identifying possible logistical and operational options for the last mile of distribution. As a result, the

Malawi team has started a partnership with the Ministry of Health in Malawi to use their health service assistants to

serve as the last mile of distribution, essential to making rural chlorine availability the norm.

The next year, we increased the number of staff from 3 in October 2014 to 17 in June 2015, enabling shared roles

and responsibilities and ultimately resulting in smooth activity expansion. This contributed towards an increase in

the number of installed dispensers from 86 to 1,951 and average adoption rate from 76% to 84.5%. We also re-

energized 1,500 promoters through refresher trainings via our super engagement model, which helped them stay

motivated and constantly monitor dispenser functionality. Additionally, we strived to create a conducive work

environment for all staff, which has kept them motivated and trusting of one another and thus operate as a unified

team. Sufficient induction and regular refreshers on our best practices have also helped staff master their work and

thus provide sufficient support to health surveillance assistants and promoters, resulting in a better working

relationship in their respective communities.

What organizations or stakeholders did you engage to implement the project? Please describe those relationships

Ministry of Public Health and Sanitation

After the pilot study, we sought approval from the Ministry of Public Health and Sanitation to scale Dispensers for

Safe Water. The ministry issued an approval letter in February 2011, essential for new expansions and partnerships

like the municipal councils of Busia and Mumias, as well as the Lake Victoria North and South Water Services Boards

in Kakamega and Kisumu respectively.

Local Governments in Western Kenya and Lake Victoria North Water Services Board

We partnered with the municipal councils of Busia and Mumias and with the Lake Victoria North Water Services

Board. The municipal councils included a budget allocation for dispensers and chlorine in their Local Authority

Service Delivery Action Plan. Field officers facilitated the training for the councilors and other council staff about the

program. The Lake Victoria North Water Services Board included chlorine dispensers in their community projects of

drilling water boreholes in rural communities in Western Kenya. The same was done to Tana Athi River Water

Services Board in Eastern Kenya. With the partnership through two boards, we installed a total of 35 pilot dispensers.

12

Kiva

Dispensers for Safe Water entered into a partnership with Kiva for up to $900,000 in loans to cover the up-front

costs of dispenser installations in 2013. This opportunity required external assessments of the program’s financing

strategy and reinforced the potential of our carbon credit business model at the time. We installed a total of 2,111

dispensers through this partnership.

One Acre Fund

In 2011 Dispensers for Safe Water entered into a partnership with One Acre Fund to scale dispensers. This

partnership allowed Dispensers for Safe Water to leverage the existing network and resources of One Acre Fund to

achieve significant cost-savings in expansion activities while sustainably serving large numbers of new farmers. As a

result, we installed total of 1,327 additional dispensers.

Mercy Corps

We worked closely with Mercy Corps in Uganda, who provided oversight during the implementation of the USAID-

Mercy Corps and Skoll Foundation sub-award to Evidence Action to scale chlorine dispensers in Eastern and Western

Uganda. We hosted Chris Walker, Social Innovations Director, twice who guided us on how we could partner with

local government and with peer organizations in Uganda to further our program objectives. We also participated in

the Mercy Corps all staff learning forum in Uganda, discussing sustainable business models and effective monitoring

and evaluation systems. During this forum we shared our innovative business model using carbon revenues to

sustain chlorine dispensers and our rigorous monitoring and evaluation systems.

The Skoll Foundation.

We partnered with The Skoll Foundation to scale the Dispensers Program in Uganda. The Skoll Foundation gave us

a $1,000,000 grant which, in addition to funds from USAID/Mercy Corps, enabled us to undertake our dispenser

activities in Uganda.

Local Governments in Eastern Uganda

We worked very closely with the local governments to implement this program. We held various meetings with local

government staff in order to educate them about the objectives and modalities of the program and to seek their

consent to implement chlorine dispenser in their locations. Additionally, we held an annual review meeting with

peer organizations in the water sector including the government to share our progress and lessons as a means to

continue cementing a relationship of mutual collaboration and accountability. As a result, the government played a

large role in creating program awareness among the masses, especially when rumors were spreading about the

dangers of using chlorine to treat water. This support in creating awareness contributed towards the rise in adoption

rates from 14% in January 2015 to 62% in December 2015.

USAID mission in Uganda

The USAID mission in Uganda is one of the key partners to the program. The USAID mission supported Dispensers

for Safe Water to acquire tax exemptions for all of the program inputs in 2016, including dispenser hardware and

chlorine re-fills. Consequently, we saved over $10,000 in taxes, which has enabled us to translate those savings to

serving more people with chlorine dispensers.

BRAC

We have held several meetings with BRAC over 2015. We are optimistic about partnering with BRAC to service

dispensers in various locations in Eastern Uganda, thereby reducing the cost of maintaining chlorine dispensers in

13

those respective areas. BRAC is supporting community health promoters in Uganda who work on the prevention and

treatment of malaria, pregnancy-related care, basic curative care, family planning, immunization, and health

nutrition education, including safe water.This partnership model therefore proposes using BRAC’s community health

promoters to carry chlorine to the community. We have reached an agreement with BRAC to pilot this partnership

model, which is on schedule to be carried out starting April 2016.

Abt Associates

Based on recommendations from the USAID mission in Malawi, Dispenser for Safe Water and Abt Associates held

several meetings to explore a potential working relationship between the two organizations. Subsequently, we

signed an agreement in December 2013 in which Abt Associates provided funding for the pilot program while the

team in Malawi managed all operations of the pilot, including site selection, community sensitization, and on-going

chlorine supply. Abt Associates also supported the solid chlorine dispenser pilot after noticing the supply chain

challenges we were facing at the time. Summary and outcomes of the solid chlorine pilot can be found in Section

V.1. Lessons Learned.

Vice President of the Republic of Malawi

In April 2015, the Vice President of the Republic of Malawi, Saulos Chilima, met with Evidence Action’s then Executive

Director, Alix Zwane; Director for Africa, Laliteswar Kumar; and Malawi Program Director, Express Moyo. During this

meeting the Vice President assured our team of the Malawi government’s commitment to providing a conducive

operating environment. This was essential to formalize the government’s commitment to the program, essential for

growing the program in Malawi.

Director of Preventative Health Services in the Ministry of Health

The Director of Preventive Health Services served as the main point of contact between Dispenser for Safe Water

and the Ministry of Health. Some of the key responsibilities fulfilled by the Director of Preventive Health Services

included providing the letter of approval in 2013 that allowed us to carry out a pilot program in Zomba, approving

the program scale-up after a successful pilot phase, committing to make health surveillance assistants available to

support the implementation of the program in May 2014, and committing to purchase chlorine for the dispensers

through the Zomba District Health office in May 2014.

Designated National Authority for Clean Development Mechanism in the Ministry of Environmental Affairs

In March 2015, the Designated National Authority for Clean Development Mechanism in the Ministry of

Environmental Affairs, Shamiso Najira, issued the Letter of Approval that enabled us to register Dispenser for Safe

Water for the carbon project in Malawi. This was an essential step in enabling us to start generating carbon credits

in Malawi and strengthening the sustainability of the program.

USAID mission

We worked with the USAID mission in early March 2013 to introduce the program and learn about USAID’s WASH

strategy in the country and their supported programs. Subsequently, they recommended that we explore a working

relationship with the Abt Associates’ SHOPS project that had experience in implementing chlorine related

interventions. These initiatives resulted in Abt Associates and the Dispenser for Safe Water signing an agreement in

Malawi. Once we started accessing funding from USAID through Abt Associates, several officials from the mission

made monitoring visits to the project impact area which kept the USAID mission up to date on the progress that we

were making towards the implementation of the Dispenser for Safe Water program in Malawi. The advice that we

received during each field visit contributed to improvements to our program implementation model. They also

improved our process for monitoring the disposal of our 5-liter empty chlorine jars so that they do not become an

14

environmental hazard. The mission’s desire to improve the chlorine delivery system resulted in the solid chlorine

dispenser pilot aimed at reducing the frequency of refilling dispensers with chlorine and if successful, do away with

the 5-litre liquid chlorine jars, which are bulky and take up a lot of storage space both at our offices and health

centers. Summary and outcomes of the solid chlorine pilot can be found in Section V.1. Lessons Learned.

What were the demographics of the beneficiaries (or customers, if this is a private sector project)? How did you

determine who participated / benefited?

At a glance, Kenya’s 2015 total population was 46.05 million with an average life expectancy of 62 years;5 in 2014,

the under-five mortality rate was 52 per 1,000 live births (decreasing by more than half since 2003), with virtually

the same rate in urban and rural areas.6 Uganda’s 2015 population was 39.03 million with an average life expectancy

of 59 years;7 2011 under-five mortality rate was 90 per 1,000 live births, with rates lower in urban areas than rural

ones.8 Malawi’s population in 2015 was 17.22 million with an average life expectancy of 64 years;9 the 2010 under-

five mortality rate was 112 per 1,000 live births, with the rate higher in rural areas compared to urban areas.10

When considering countries for program expansion, our framework has two stages. The first stage includes a desk

review considering factors such as population density, under-5 population proportion, percentage of rural

population, liquid chlorine availability, national chlorination rates, national under-5 diarrheal incidence rates, and

logistical support to estimate potential for impact and cost-effectiveness.

The second stage of the evaluation involves a two-week in-person scouting trip to short-listed countries, during

which meetings were held with various ministry officials, sector stakeholders, and other start-up organizations to

better understand the general WASH and operating environments. Further research and analysis can be found in

Appendix 3: Country Research for New Program Expansion.

Within each country, we determine the beneficiaries by first expanding to the most cost-effective districts before

moving into the less cost-effective districts after that. Essentially, we prioritize targeting regions with high population

and water source density over regions with a low population and water source density. This also means that we

5 2015 World Bank data Kenya. 6 Demographic and Health Survey 2014 Data Kenya. 7 2015 World Bank data Uganda. 8 Demographic and Health Survey 2011 Data Uganda. 9 2015 World Bank data Malawi. 10 Demographic and Health Survey 2010 Data Malawi.

15

operate in rural areas that are likely poorer than urban areas, that may already have access to clean water. Indeed,

when randomly surveyed in their homes, 82% of people in current dispenser regions report that they collect their

water exclusively from a water source with a dispenser. This is a distinguishing factor for Dispensers for Safe Water.

Through sustained last-mile service delivery, we reach the hardest-to-reach and most rural communities in Kenya,

Uganda, and Malawi. As we secure more support from governments and as unit costs start to reduce as we scale,

we will likely be able to enter new areas that previously were not viable for us to serve.

What challenges arose during program implementation, and how were they addressed?

Maintaining high levels of adoption is critical to our cost-effectiveness strategy (maximizing carbon credits, and

eventual revenue for a given fixed investment in infrastructure) and the impact of the program (reducing diarrhea

rates, which is a major cause of death for children under five). Yet high adoption rates are also challenging to achieve

and maintain. For example, towards the end of 2014, Uganda began to see very low adoption rates, with the

substantial part of January-March 2015 experiencing rates below 30%. We engaged with a pro-bono human-

centered designer to investigate what prevented people from using dispensers and what our customer thought was

needed for them to routinely chlorinate their household drinking water.

We focused on our key constituents: people who influence decision making about safe water in their households,

including mothers of young children, our community promoters who are a critical factor in increasing user adoption,

and other community influencers. We asked questions about a typical day, routines around water use, and

perceptions about chlorination. We talked to lots of promoters to understand better the pivotal role they hold as

community experts on water. We also reinforced our existing knowledge that a superbly functioning supply chain

and quick responses to maintenance issues are key to keeping our customers confident that dispensers are filled

and working.

This exercise highlighted the importance of understanding the user perspective and having the courage to halt

dispenser installations in order to adequately address the core issue, uncovering key negative associations with

chlorine, such as the bad smell, that it makes women infertile or men impotent, or turns men homosexual. The study

also reinforced our understanding of who the key influencers were to address these issues, allowing us to pinpoint

our solutions: community re-engagement meetings and promoter engagement meetings, which ultimately reversed

the decline in adoption rate. Before these updates, adoption rates were as low as 14% in January 2015; afterwards,

adoption rates broke through the 40-percentile margin in May and have been sustained through December 2015.

We took a similar approach in Malawi: more attention to help the user understand why chlorine usage was

important. There were a couple of districts where adoption rates were declining, albeit from a much higher starting

point; at 49%, the adoption rate for January 2015 was the lowest recorded in Malawi at the time. We called our

approach to address the low rates “super engagement”: two field offers would ride every day for weeks to the

communities that needed attention to meet with the community service officers, whose job is to maintain the

dispensers and deliver the chlorine. The field officers would go over weekly goals with the community service

officers, talk about any problems or issues with the dissemination of chlorine to the villages, and talk to the local

promoters as well to ensure that they had what they needed from the community service officers. This approach

enabled us to understand and resolve any challenges relating to chlorine availability, community beliefs and

misconceptions, and hardware malfunctions. This strategy proved successful, resulting in adoption rates moving

from 53% in January 2015 to 91% in February, 100% in March, and eventually achieving a 2015 annual average of

85%.

16

We will talk about this more in Section VII.1. Paths to Scale, but in late 2015 and early 2016, we had to rethink a

sustainable revenue plan in light of declining global carbon markets and lower-than-expected carbon revenues. Our

initially aggressive growth plans have been throttled and now we need to assess how we can sustainably finance

Dispensers for Safe Water from a variety of revenue sources, not just carbon. Also, we are focusing on how we can

further decrease unit costs and how we can more sustainably grow consumer access to our dispensers.

We have learned that while full saturation coverage is desirable from at-scale programs, it is very difficult to actually

achieve that level of scale. By full saturation coverage, we mean providing chlorine dispensers at every water point

in a district. We have been able to cover about 60% of the areas where we operate and are piloting other

interventions to cover the remaining areas, but not all water points satisfy our inclusion criteria, such as more than

9-month availability of water supply, low levels of turbidity, and being used by more than 10 households. It is

important to acknowledge the fact that in most cases, 100% population coverage is an unrealistic expectation.

Nevertheless, this expectation persists, which has made gaining traction with some donors and peer organizations

a challenge.

There is also an unrealistic expectation of some funders to integrate additional WASH interventions into the

dispenser infrastructure. This includes handwashing interventions, borehole maintenance services, among others.

Although it has been in fashion the last few years, we have not seen much evidence attesting to the success of such

interventions. Furthermore, where additional interventions have a cost implication, we would need to see an

additional revenue increment of some kind or new revenue streams to cover the cost of such interventions.

However, where integrating other programs reduces our cost or brings in an incremental benefit, then we would

able to take this on. This is the case in Kenya, where we are exploring a partnership with a clean cook stove supplier

that could incentivize and motivate our village-level volunteer promoters. Therefore, such interventions need to be

assessed on a case-by-case basis.

Finding ways to augment our carbon revenue stream has been a challenge. It is forcing us to rethink our business

model and is encouraging us to take more risks in prototyping different income generating activities. For example,

some have argued that charging for services is essential from a sustainability perspective. Although a range of

literature demonstrates that charging a fee for a service can cause a drop in adoption for people below a certain

income threshold, we could explore the option of segmenting payment to those who can afford to pay to use

dispensers; this payment group would then help cross-subsidize the program for the rest of the users.

How did you verify whether or not you met project goals and objectives? What indicators and instruments were used

to measure them?

We derived Appendix 4: Final Indicator Report based off of the original workplan; this includes indicators measuring

self-reported diarrheal disease reduction and DALYs averted calculated from our model, which will act as proxies to

the health benefits of the program. This, along with program cost records, will provide information on cost-

effectiveness and estimates of cost per DALY saved.

17

The cumulative self-reported cases of diarrhea averted totaled 563,716 in Kenya, 221,549 in Uganda, and 51,941 in

Malawi11. Similarly, cumulative program-wide DALYs averted were 22,324, with 15,054 DALYs averted in Kenya,

5,882 in Uganda, and 1,388 in Malawi12. Calculations for cases of diarrhea averted and DALYs averted can be found

in Appendix 5: Impact Calculator.

For impact evaluations: What was the evaluation design? Please include if applicable: key study questions,

randomization design, sampling strategy, power calculations, qualitative methods used, etc.

For this Stage 3 DIV award, we focused on scaling operations, not testing new evaluations. Dispensers for Safe Water

is based on a series of randomized, controlled trials by Harvard and Berkeley researchers who tested chlorine

dispensers in Kenya against a variety of other water treatment options. They found two things: that chlorine

dispensers had a much higher usage rate than any other treatment, and that use stayed high over time. We input

our measured outputs into a mathematical model to determine three proxy indicators that we use to track the likely

impact of the program: Cases of Diarrhea Averted, Disability Adjusted Life Years (DALYs) averted, and Deaths

Averted. We estimate the number of cases of diarrhea in order to recognize the number of households and children

that have individually benefitted from a reduction in childhood sickness. We also project disability adjusted life years,

which are a measure of how many years of life would have been lost due to poor health, disability, or early death in

the absence of the chlorine dispenser. One DALY can be thought of as one lost year of "healthy" life, because

sicknesses such as diarrhea detract from the quality of life even if they are not fatal. Finally, we project the number

of deaths averted due to the program and the presence of the Dispensers. Our calculations for our impact calculator

model can be found in Appendix 5: Impact Calculator, and we describe the methodology used to calculate these

indicators in Appendix 6: Impact Calculator Narrative.

Summarize preliminary (or final) quantitative and/or qualitative results of the intervention.

Output Indicator Country Target Achieved

Output 1. Provide six million

people13 across several

countries with access to

chlorine via cost effective,

sustainable dispenser programs

Number of chlorine

dispensers installed

(broken out by country)

Kenya 16,250 17,784

Uganda 6,250 5,585

Malawi 2,500 3,333

Number of people with

access to chlorine

dispensers

Kenya 2,654,374 2,215,704

Uganda 2,364,092 1,622,347

Malawi 983,119 659,331

11 Cumulative since inception of the program in each country through December 31, 2015. 12 Cumulative since inception of the program in each country through December 31, 2015. 13 The original goal in the proposal was originally to provide safe water to five million people. However, by the end of September 2014, we had reached 50% of our Kenyan installation targets for Year 3 with only 25% of the year complete. Similarly, we reached 37% of our Ugandan installation targets for Year 3 with only 25% of the year complete. Additionally, by revising our focus to shift more energy to Malawi, where we reach over 2.5 times more people per dispenser, we believed we would be on track to reach our new target number of clients (six million) by the end of Year 3.

18

The basis for achieving Output 1 in the indicator report - provide six million people across several countries with

access to chlorine via cost effective, sustainable dispenser programs - is the growth of the program via new

installations. Growth in installations begets an increase in the number of people with access to chlorine dispensers.

In December 2015, we measured the dispensers in Uganda, Malawi, and Kenya to be serving 4,497,382 people, 25%

below the targeted six million in Output 1.

Output Indicator Country Target Achieved14

Increase adoption, knowledge

and proper dispenser use across

dispenser programs

% of households in target area

with total chlorine residual in

household drinking water

during unannounced visits,

broken out by country

(objective adoption rate)

Kenya 55% 57%

Uganda 50% 59%

Malawi 50% 81%

Output 2 specifies the goal of increasing adoption, knowledge and proper dispenser use across dispenser programs.

On average, the Kenya and Uganda programs performed below adoption targets. However, quarterly reports for the

second half of 2015 attest to the rise in adoption for these regions to an average above the targeted 55% for Kenya

and 50% for Uganda. Malawi consistently registered adoption above the June 2015 target of 50% (see Appendix 4:

Final Indicator Report for December 2015 figures). The program-wide adoption average across all countries was 51%

in December of 2015. The final evaluation of adoption rates indicates the success of Output 2.

Output Indicator Country Target Achieved

Disseminate project results to

achieve support for evidence-

based dispenser program

replication

Number of published articles

about the project and/or its

findings

Total 4 6

Number of reputable press

mentions per year about the

project and/or its findings

Total 5 9

Number of conferences/

workshops where dispenser

programs are presented

Total 5 5

Total value of additional

resources supporting dispnser

scale-up

Total $ 6,090,284.49 $ 9,125,659.49

14 These figures are quarterly averages for the October-December 2015 period. Please contact us for cumulative figures over

the life of the grant.

19

An additional metric by which the program can be evaluated is the dissemination of project results to achieve

support for evidence-based dispenser program replication (Output 3). We achieved above set targets on the number

of published articles mentioning the program, number of reputable press mentions, number of conference

presentations, and the total value of additional resources supporting scale up. Indeed, the value of additional

resources was 50% above set targets at over $9 million cumulatively in both individual donations and additional

grants.

Output Indicator Country Target Achieved

Consolidate development of a

reliable, low cost supply of

commercially manufactured dispenser

hardware, spare parts, and chlorine

that are readily available to meet

dispenser demand globally

Manufacturing

costs per dispenser

per year

Total $ 52.50 $ 47.41

The program has also consistently kept the manufacturing costs per dispenser per year low at $47.41, resulting in a

$5.09 decrease on targeted costs (Output 4).

Output Indicator Country Target Achieved

Optimize quality of operations

for dispenser programs in

multiple and diverse contexts

Average number of working

days between when a

dispenser hardware issue is

reported and when it is

resolved

Kenya 5 5

Uganda 6 7

Malawi 6 6

% of dispensers that are

operational at unannounced

visits

Kenya 95% 99%

Uganda 95% 99%

Malawi 95% 91%

% of dispensers that are full at

unannounced visits

Kenya 95% 97%

Uganda 90% 91%

Malawi 90% 89%

On average, the program has achieved above expectations on metrics measuring the quality of operations (Output

5). We found dispensers to be operational in more than 91% of unannounced visits program-wide, with very few

hardware problems registered. Hardware problems are resolved within 5-7 days. In addition, on average between

89-97% of dispensers in Kenya, Uganda, and Malawi are filled with chlorine during unannounced visits. For additional

details regarding the performance of the program, see Appendix 4: Final Indicator Report.

20

Did anything occur during implementation or data collection that could have threatened data quality?

On several occasions, the mobile data collection platform used to assess chlorine dispenser functionality and conduct

household surveys failed to submit collected data to the cloud-based server. This would result in “losing” data for a

set of dispensers in the same geographical region (approximately 100 data points) for a given month. However, this

occurred infrequently. For example, we estimate that less than 1% of the dataset for household surveys is lost out

of more than 27,300 data points collected per year. Therefore, this phenomenon would not have affected the validity

of the data when aggregated across offices, over the course of the year as reported in the indicator report.

In addition, we noted one incident where a field officer evaluating the presence of chlorine in household water

samples was found to be color blind (the testing process involves using a color wheel). However, the data collected

by this person for the previous three months was discarded and households were retested. We also changed the

hiring process to test for color blindness.

Are there any methodological limitations to the analysis?

We updated our original calculation of number of peopled served, “People Served = # of Dispensers × # of

Households per Dispenser × # of People per Household,” in October 2015. While the calculation itself has not

changed, the way that the inputs, "# of People per Household" and “# Households per Dispenser” are calculated

have. As Dispensers for Safe Water expands into lower density areas, countrywide averages for average number of

people per dispenser and average household size would change to reflect these new demographics.

Calculating a national average will therefore keep changing as we expand in to new geography and therefore

changing the figures in previous reports. Using the methodology, these inputs are calculated as geography specific

and then incorporated into the dispenser database for dispensers from that specific geography. This reduces the

variance created by having to weigh the figure to get a national average. Additionally, you can find the official write-

up for our Number of People Served methodology update in Appendix 7: Updating Number of People Served.

In addition, several limitations have been noted in the literature regarding the assumptions that underlie the

calculation of cases of diarrhea averted and subsequently, DALYs, such as the percentage of children with diarrhea

in a 48-hr period. This metric is self-reported during household surveys. Self-reported data is often unreliable as

respondents may not be able to recall incidents accurately, may have a different threshold for defining what

constitutes diarrhea or may be subject to demand characteristics in that they tell field officers what they think they

want to hear. In addition, there are several global assumptions drawn from the literature that may have been

superseded by more up-to-date estimates. These assumptions are listed below:

Average diarrhea case duration: This value is drawn from a paper by Lamberti (2012) who reviews the

average case duration of mild, moderate, and severe cases of diarrhea across low and medium income

countries.

Number of deaths per 1,000 cases of diarrhea: This value comes from a Review of Childhood Pneumonia

and Diarrhea for the Africa region by Walker et al., (2013) in the Lancet.

Disability Weight: The disability weight for diarrhea is drawn from a WHO “Global Burden of Disease”

document that references a table of disability weights published by Murray & Lopez (1996).

21

Overall reduction in diarrhea given that an individual adopts chlorine use: The impact that using chlorine

has on reducing diarrhea comes from an average included in review Fewtrell et al., (2005).

Standard life expectancy at age of death: The standard age of life expectancy at age of death is drawn from

WHO published life expectancy tables for each country and was last updated in June 2013.

If you have significant new or different evaluation plans for your future work, please describe.

We plan to conduct an updated secondary review of the literature used to support global assumptions in our DALY

impact calculator and the subsequent calculation of key impact metrics; we are currently hiring a cost-benefit

modeler to assist in analysis like this. In addition, we aim to supplement our quantitative data collection with a small

sample of qualitative evaluations. Qualitative interviews with community members will greatly add to our ability to

explain any major and sustained changes in program performance and can be used to inform innovations in regular

implementation activities. The proposed purpose and methodology for qualitative data collection activities can be

found in Appendix 8: Dispensers for Safe Water Biannual Qualitative Research Protocol.

What lessons were learned during implementation? What are these lessons based on? If applicable, please provide

details about lessons in the following areas, especially those that could inform future scale up or replication:

After scaling our pilots, we learned that efficiencies at scale take time. There are inherent challenges with rapidly

scaling a pilot due to the inherent bandwidth restrictions that accompany small staff sizes during a fast growth

period. That was one of the main reasons we focused on hiring in the beginning, growing our staff to 164. Without

the proper personnel to absorb all of the operational and programmatic growth, scaling a pilot will bottleneck and

it will be difficult to succeed. Ultimately, we were able to achieve a degree of efficiency at later stages of the scaling

process—not because efficiency was not a focus from the beginning, but because personnel growth was an

important step to take first.

On the other hand, after the initial hiring phase, we also learned the importance of optimizing the work force. For

example, in January 2015, Evidence Action evaluated and analyzed our staffing needs compared to our annual goals,

which resulted in reducing our personnel by 21% for cost-effective and efficient operations without compromising

quality. Out of the 168 staff in Kenya at the beginning of the year, we released 21 in March and an additional 14

between April and June. This resulted in a lighter team that still met all of our staffing and operational needs.

Specific governmental partnerships are important for Dispensers for Safe Water to operate at scale. For example,

our partnership with the Ministry of Health is the linchpin for our far-reaching operational model in Malawi. The

health surveillance assistants under the Ministry of Health provide the last-mile delivery of chlorine, allowing us to

cost-effectively make chlorine a rural water supply norm.

That said, these types of quality partnerships are not common. For example, we tried to cultivate the same kind of

relationship in Kenya and Uganda, but due to a change in the adminstrative structure in Kenya and lower government

enthusiasm in Uganda, we decided to expand the program on our own while keeping government informed but not

actively engaged. The intention is to attempt to re-engage at regular time intervals, with the assumption that officials

22

come and go, and therefore a new ‘champion’ to support our work may appear within government. Equally, initial

government opinions may change when we transition from a pilot to an at-scale program.

We have also been open to partnerships with other organizations. Specifically, through one of our first partnerships,

with One Acre Fund, we learned how to better identify strategic relationships. Most importantly, that we should

only make partnership decisions based on the feasibility, practicality, and analysis of context, rather than

connections or other agendas. We initiated the original partnership on the basis of friendly connections between

our office and One Acre Fund management back in 2010-2011. Additionally, expectations of roles, commitments,

and communications for each partner should be well defined beforehand so that no one is caught off guard during

the partnership. Until August 2013, our partnership suffered from a lack of field level commitment on both sides,

both in principle and in time. This often led to a reactive rather than proactive relationship.

We have learned from these lessons to inform our engagement go/ no-go decisions to ensure partnerships are

feasible, strategic, and beneficial to both parties involved.

As we mentioned in Section III.5. Challenges, knowledge of the cultural and social lenses through which dispenser

users were viewing the dispensers and chlorine usage played a role in turning around Uganda’s low adoption rates.

By understanding their perspectives and the key influencers in the communities, we were able to adjust our

messaging and implementation strategy to help close the gap between the negative perceptions surrounding

chlorine usage and dispenser benefits.

Additionally, we also learned that as we scale up, fewer water points pass our verification requirements as expected.

For example, in Kenya during the second year of implementation we only hit 85% of our installation target. While

this is challenging in terms of reaching program targets, it is an important lesson learned, emphasizing the nuances

of waterpoint selection for maximum program impact and long term cost-effectiveness.

At first, we identified Tanzania as a strong potential for expansion. In the end, however, we placed the Tanzania

program on an indefinite pause. Because additional revenue streams are integral to the sustainability of Dispensers

for Safe Water, being able to generate carbon credits was essential when deciding whether or not to move forward

with a pilot. Unfortunately, the government body responsible for approving Clean Development Mechanism carbon

projects was non-decisive and caused a prolonged halt in the approval process. As a result, we decided to hold off

on the Tanzania dispenser pilot. We learned that a country’s political palpability towards carbon credit policy was

important to our programs and if necessary, we would not expand into new countries where our carbon credit

program could not operate.

We learned that we should continue to pilot innovative solutions within the program. One such idea came in the

form of using solid chlorine instead of liquid chlorine. We carried out a pilot study to investigate the use of solid

chlorine in dispensers in Malawi, which presented the possibility of longer storage time, easier deliveries, and lower

costs. We designed the pilot study to understand user experience, identify physical challenges when the dispensers

were installed, and record the frequency of use, obstacles, concerns, and preferences. We manufactured 10 solid

chlorine dispenser prototypes for this pilot. The results of this study are summarized below:

There was a high (40%) hardware failure rate related to inability or hardness in turning the valve early in

the pilot, making it difficult to consider the result as representative.

23

The test for dose accuracy showed that a higher than the recommended dose was being dispersed, and this

resulted in a decision to immediately replace these with liquid chlorine dispensers.

In terms of community perception, 84% indicated that if given a choice they would opt for solid chlorine to

liquid chlorine. However many people were concerned with the smell of chlorine, which was not surprising

since the dispensers were giving out higher doses than was expected.

Despite these challenges, we still believe this is an avenue worth investigating considering the significant potential

cost savings. One of the key advantages of a solid chlorine dispenser would be the three-month time interval

between refilling the dispenser, compared to every two weeks for liquid dispensers. Based on the first pilot, our

MLIS team recommended an additional pilot study with an improved solid chlorine dispenser design. We will pick

this back up once we secure additional funds.

Product modifications

The dispenser unit itself underwent an evolution over the course of the program. The original metal dispensers had

their own challenges: they were prone to breakage and rusting, and higher temperatures lead to faster chlorine

disintegration. Since then, we incrementally improved the dispensers leading us to Generation 3.5. These included

marine grade padlocks, more durable dispenser lids, stainless steel bolts allowing for a stronger bond, screen printed

instructions as opposed to stickers that would peel off, stronger, anodized QR asset tags, and a newer plastic mold

design to decrease cracking and leaks. In this version we continued to improve the design; we updated the basket

by introducing ribs and webs on the padlock lip to reduce lip breakage, the lid by introducing ribs and webs on the

lid hinge to reinforce and strengthen the hinge, and the valve by modifying it to reduce instances of stiff valves.

We also now have a sophisticated sourcing mechanism in place where we source the bulk of the dispenser

components in the region and chlorine in respective countries. For example, in the third year of the program, we

identified a local manufacturer in Kenya that could alter the molds that create the plastic dispenser components,

helping us reduce the cost of hardware procurement by 45%, thereby improving the potential production capacity,

which increased from 2,482 dispensers produced in July-September 2015 to 5,165 dispensers in October-December

2015. On the other hand, while we have been trying to localize our product sourcing as much as possible, we

acknowledge quality and cost considerations of the high-precision products from abroad. The quality of the

dispensers parts affect the frequency of hardware issues; keeping costs low is important, but not if it interferes with

the dispenser from functioning correctly. As a result, although more expensive, we still source padlocks from China

and high-precision valves from the US.

Service Delivery modifications

We made an early improvement in service delivery by developing our standard five-step implementation process.

These five steps describe the way we move into a new area, from first engagement with government officials, right

through to ongoing chlorine delivery. The five steps transformed our program into a replicable model ready for

geographic expansion, which gave us a clear understanding of what was required, when, and provided for efficient

planning of resources. This has been continually optimized over time, helping us to continually lower costs.

We quickly learned the importance of strong service delivery at the very outset. For example, after our first year of

scaled operations in Kenya, we learned the importance of a healthy supply chain of chlorine. Based on data collected

during this period, households were 20% less likely to have chlorine in their drinking water where a promoter reports

the dispenser has ever been empty. Forty-percent of cases where dispensers were empty were due to promoters

not having chlorine to refill it. We realized the need to adapt our supply model so that each dispenser was routinely

visited by a circuit rider, mapping the dispenser locations to determine optimal circuit routes. This model has proven

24

very effective, leading to high chlorine adoption rates observed in pilot counties, likely a result of the increased

consistency of both chlorine refills and dispenser functionality. It also allowed us to increase the per staff dispenser

coverage for chlorine delivery from about 150 in the beginning of the program to 400.

There are several challenges that any program will need to overcome in order to achieve implementation at scale.

The first challenges are operational constraints. Going from a pilot to scale requires enough staff of sufficient quality

to allow the organization’s capacity to meet the growing demands of a scaling program. Funders need to be sure to

invest not only in the product but also in the team that brings the product to scale, which requires a tremendous

amount of operational coordination.

Continuous and diversified revenue streams are also important. We will discuss this more in Section VII.1. Paths to

Scale, but diversifying revenue sources is an important part in program sustainability. These streams need to be able

to absorb market shocks, as we saw the challenge in decreasing prices in the carbon market, the original pillar of our

sustainability plan. Allowing a program to test and iterate new business model strategies over a longer period, and

not be restricted by the duration of a typical USAID program, will provide a stronger long-term strategy for the

implementing organization.

Although full saturation coverage is desirable from at-scale programs, it is very difficult to actually achieve that level

of coverage. That said, it is possible to achieve a high degree of penetration in the areas we do cover, and although

it is possible to do this without governmental distribution partnerships as we have demonstrated in Kenya and

Uganda, our partnership in Malawi through the Ministry of Health demonstrated how with the right kind of

partnership, last-mile chlorine delivery is possible and that funders and policy makers can make rural chlorination

the norm, just like it is in urban water supplies.

To what degree and in what contexts can these results be generalized?

After starting with Kenya, we piloted the dispensers first in Uganda and then in Malawi. We believe that our results

as mentioned in Section IV.1. Verifying Goals and Objectives have been positive enough to warrant continuing to

pilot dispensers in other countries and contexts in hopes of finding similar results. We are currently exploring

partnerships in Ethiopia and India with other implementation organizations, following a light technical assistance

approach.

Of course, we say this cautiously, knowing that each country we consider offers its own cultural norms and

operational constraints that will influence the take-up of dispensers and the program’s cost-effectiveness.

Additionally, we are careful to identify contexts that align with our scoping framework, considering factors like

population density, national chlorination rates, under-five population proportion, chlorine availability, national

under-five diarrheal incidence rates, and logistical support. Key to the roll out of the Dispensers for Safe Water

program is our community outreach prior to installation. We would be hesitant to extrapolate our results to contexts

in which the environment does not present a need or support for our dispensers.

In addition, the results of our regular monitoring have allowed us to identify three key determinants of program

performance (i.e., adoption or “take-up” by the community), which are always functioning hardware, regular

promoter engagement, and continuous chlorine availability. By focusing on these elements of the program in Kenya,

Uganda, and Malawi, we have been able to promote consistent adoption of chlorine at our high target levels. Based

on the results of our evaluation and consistent improvement over time in all three countries, it is reasonable to

25

project that a similar focus by an implementing agency with similar capabilities would result in the same performance

in a different context.

What is your solution’s competitive advantage over existing solutions (including cost considerations)?

Dispensers for Safe Water aims to address the challenge of providing safe water to millions. We know that with

unsafe water and poor sanitation as the second leading cause of childhood mortality, access to water is not enough.

It matters if there is clean water at the source and clean water at home. Unlike other solutions, dispensers not only

provide clean water at the source but provide clean water at home to millions.

When we talk about existing solutions, we are talking about chlorine products, not household water filters. That

said, looking at other chlorine solutions, Dispensers for Safe Water differentiates itself in terms of pricing and take-

up.

Results from a randomized controlled trial documented that the dispensers approach increases chlorine use more

than six-fold compared to the standard model of selling small bottles of chlorine through retail outlets. In rural

Kenya, 55-60% of households had sustained detectable chlorine levels in household drinking water during

unannounced visits 3-6, 18, and 30 months after dispenser installation, compared to less than 10% with access only

to the retail model (Kremer, Miguel, Mullainathan, Null, and Zwane 2009).

Studies also found that while some people may be willing to pay for water, most of those with free access to water

from a communal source will not pay for water treatment. In a series of randomized evaluations, chlorine take-up

was highly sensitive to price, with more than half of households using chlorine when it is provided for free, even

though less than five percent used it under the standard retail model.

In practice, we have found that dispensers work; maintaining high adoption rates over 40% at scale in all three

programs since May 2015.

We do not have a current market analysis to verify the cost-effectiveness of dispensers versus other existing chlorine

solutions, but we’re intending to conduct one in 2016.

Please assess the extent to which your solution yields (or has the potential to yield) greater impact per dollar than

alternate ways of achieving the same development impacts. Provide estimates of impact per dollar for your solution

if possible, taking into account the full cost of implementation.

Dispensers for Safe Water looks at four costing indicators: installation costs per year, on-going costs per year, Cost

per beneficiary per year, and Cost per anticipated DALY averted per year. We calculated these costing indicators

using the costing information summarized in Appendix 9: Costing Model for 2015. We have met all of our costing

goals, and in Section VI.3. Cost-Effectiveness, will show how we calculated each of the costing indicator values and

how the costing metrics decrease as we scale.

We have summarized our actuals versus targets for each below:

26

Kenya Uganda Malawi

Target Weighted Average

Installation Costs per Dispenser $ 280.70 $ 269.05 $ 258.43 $ 260 $ 272.07

On-going Costs per Dispenser $ 175.20 $ 170.50 $ 181.83 $ 180 $ 174.48

Installation costs refers to the cost of installing a dispenser as well as the ongoing costs of maintaining this new

dispenser for the rest of the 12-month period. These activities include local stakeholder management, conducting

water-point site verification for dispenser installations, holding village community sensitization meeting to secure

buy-in, installing dispenser hardware at the water point, and conducting community education meetings to educate

dispenser users. It also includes monitoring costs for the dispenser through the rest of the year.

On-going costs refers to the costs associated with monitoring activities to ensure proper functioning, decent usage,

and regular chlorine refills. These activities includes monitoring the amount of chlorine used, conducting random

spot checks on dispenser functionality, and monitoring adoption levels of chlorine through randomly selected

“confirmed use” surveys, where we ask households to prepare a glass of water as they would for a child to drink on

un-announced visits.

Looking at Table 6, with the program-wide average installation and average on-going costs weighted by the share of

new and existing dispensers in each country respectively, we can see that although we did not achieved our target

of $260 for installation costs per dispenser, we did achieve our $180 target for on-going costs per dispenser.

Although we did not hit our installation target, our on-going costs are competitive as a result of increasing economies

of scale as we expand the dispensers program. This is also a result of learning from our experience of the years at

improving our distribution and supply chain.

Note that the we break costs down between the first year of a dispenser (installation costs) and the second year of

a dispenser (on-going costs) to better frame our business model, which incorporates carbon revenue that only kick

in at some point in the third year a dispenser is active. This way we are better able to understand the pre-finance

amount required to sustain the dispensers up until the point when we begin to receive carbon revenues.

Additionally, note that the costing indicators for Uganda are lower than Kenya, despite being a new program, for

two reasons. First, Uganda had only 1 field office per 4,000 dispensers for the majority of 2015. When we re-looked

at operations in early 2015 to determine reasons for low adoption rates, we realized that a main culprit was because

all dispensers operated out of one field office in Mbale. Consequently, we set up two more field offices:

Namatumbwa was set up in August 2015; Palisa was set up in November 2015. As a result, the costs per dispenser

(and consequently costs per DALYs averted and per beneficiary) are low in Uganda for the first year. That said, we

expect Uganda figures to rise and plateau starting in 2016. In comparison, we had 1 field office per 1,200 dispensers

in Kenya. And second, on top of the higher field office to dispenser ratio, the cost to maintain the Kisumu office in

Kenya was drastically more expensive than the cost to maintain the Kampala office in Uganda, adding to the higher

Kenya costing indicators.

Kenya Uganda Malawi Total

Target Weighted Average

Number of Beneficiaries 2,215,704 1,622,347 659,331 4,497,382

Cost per Beneficiary per Year $ 1.63 $ 0.70 $ 1.22 $ 1.20 $ 1.24

27

Our beneficiaries are the number of people who have access to our dispensers, given the number of dispensers

installed and the corresponding average number of households per dispenser. Knowing our overall program costs

and total number of beneficiaries, we can then calculate the program-wide average cost per beneficiary, weighted

by the percent of beneficiaries in each country. Looking at Table 7, we did not beat our cost target of $1.20 per

beneficiary; through economies of scale, we expect we will be able to break this target by 2018.

In the beginning, new country programs like Uganda and Malawi worked with their single field office and small core

staff teams while servicing a growing number of dispensers and beneficiaries. This generally drives down costing

indicators for a time before the country program reaches a tipping point, when they need to invest in additional field

offices and/or core staff members in order to continue to provide quality service to dispensers and maintain quality

adoption rates. This change will temporarily drive up country costs before continued program expansion is able to

drive down the country’s costing indicators over time.

Specifically, Kenya has 15 field offices servicing over 2 million beneficiaries. But while Kenya is operating at a greater

scale than Uganda, Uganda is still servicing 73% as many beneficiaries as Kenya (1,622,347 vs. 2,215,704) with one-

fifth of the field offices. So considering Uganda’s relatively high beneficiary figure and significantly lighter

programmatic costs, Uganda has the lowest Cost per Beneficiary figure out of the three countries.

Malawi follows a similar pattern. The Lilongwe office only has one core staff member and their sole office is less

expensive to maintain than Kenya’s Kisumu office. Although Malawi serves 30% of the beneficiaries as Kenya

(659,331 vs. 2,215,704 beneficiaries), they operate at 22% of the costs as the Kenya office ($805,812.24 vs.

$3,619,651.76), ultimately making Malawi slightly more cost effective.

Kenya Uganda Malawi Total

Target Weighted Average

Total Anticipated DALYs Averted 15,054 5,882 1,388 22,324

Cost per Anticipated DALYs Averted $ 240.44 $ 193.40 $ 580.56

$ 207.55 $ 249.20

While we cannot measure actual diarrhea reductions because that is very complex and costly, our detailed models

suggest that we have averted more than 830,000 cases of diarrhea and more than 22,000 DALYs over the project

lifetime at a very low cost. Although Table 8 shows that we did not achieve our Cost per Anticipated DALYs Averted

goal of $207.55, dispensers are still cost-effective compared to other interventions. Clasen (2008) et al’s review of

cost-effectiveness of interventions to improve water quality to prevent diarrhea concluded that home chlorination

usually costs between $242 to $1,371 per DALY averted (in current USD).15 You can see that our DALYs averted

figures in Year 3 are competitive, with the program-wide weighted average of $249.20 per DALY averted in Year 3

slightly above the $242 per DALY averted lower limit cited by Clausen. After we look at the figure broken down by

country, we can see that we met the target in Kenya and Uganda, but did not in Malawi. This is because out of all

15 Clausen, F. T., and L. Haller. 2008. Water Quality Interventions to Prevent Diarrhoea: Cost and Cost-Effectiveness. World

Health Organization: 19.

28

dispensers in Malawi in 2015, the majority were newly installed dispensers (79%). Dispensers cost the most the first

year, which includes both installation and monitoring costs, while older dispensers are less costly, only requiring

maintenance and monitoring costs. This is in comparison to Kenya’s 17,784 dispensers installed, with only 24% of

dispensers being new dispensers.

While we have refined our models for tracking diarrhea and DALY reductions, it is important to note that our DALYs

Averted model is just a model and only indicative of health outcomes. Please refer to Section IV.5. Analysis

Limitations for more information regarding the limitations to calculating these figures.

Can DSW rigorously show costs associated with clean water delivery and model those into the future. We would see

this particularly important to a future franchise program.

We break down our costs into five categories: 1. Field operations costs, 2. Field office costs, 3. Program office costs,

4. Coporate costs, and 5, Financing costs. Costs 1 and 2 are direct costs, while we consider costs 3, 4, and 5 indirect

costs. Full table of costs for Dispensers for Safe Water can be found in Appendix 9: Costing Model for 2015. The

costing model and costing indicators are based what the 2015 costs should be based on the current program’s size

and operations; they were not based on actuals due to limitations in our accounting and financial systems.

Field operation costs take into account the labor and supply costs it takes to install and maintain dispensers in the

field, ranging from promoter trainings to fuel consumption costs for last mile chlorine delivery to the installation

costs for dispenser hardware and installation materials like cement and ballast.

Field office costs are the costs associated with running our field offices. These include the salaries for the field office

supervisors, such as our area coordinator, associate area coordinator, and field officers. These costs also include

contractual costs to keep the field offices running, like field office rent and internet.

Program office costs are the costs include office costs for regional offices in Kisumu for Kenya, Kampala for Uganda,

and Lilongwe for Malawi; this also includes staff time for our regional area leads and other staff responsible for

running the regional office.

Corporate costs can be broken down into two subcategories: Regional and U.S. Costs. These are the highest-level

indirect costs associated with running the program. Regional costs include supporting arms like engineering and

MLIS staff in Nairobi, while U.S. Costs are a share of the costs associated with our DC office staff. While in general

these corporate costs are the central costs that are shared among our other programs, like the Deworm the World

Initiative and Evidence Action Beta, the $1,289,566 in Table 9 below are Dispenser for Safe Water’s share of

corporate costs. These costs are primarily salary costs, but also travel costs associated with these staff members.

Regional staff members includes our M&E team, information system, and engineering. US-based costs include

management, strategy, and fundraising team members. We distributed these U.S. costs across each country,

weighted in proportion to the percent of dispensers installed in each country. For example, in 2015, 20% of all

dispensers were installed in Uganda; we then allocated 20% of U.S. Costs to Uganda.

Lastly, there are financing costs associated with our carbon financing. This includes costs associated with MLIS and

tracking carbon crediting procedures over 2015.

29

Costs Categories Kenya Uganda Malawi Total

1. Field Operation Costs $ 1,186,367.88 $ 349,054.44 $ 288,241.86 $ 1,823,664.18

2. Field Office Costs $ 812,555.60 $ 175,277.45 $ 49,287.20 $ 1,037,120.25

3. Program Office Costs $ 317,618.49 $ 204,039.20 $ 224,060.00 $ 745,717.69

4. Corporate Costs $ 858,873.56 $ 269,726.09 $ 160,966.35 $ 1,289,566.00

5. Financing Costs $ 444,236.24 $ 139,510.76 $ 83,256.83 $ 667,003.83

Total Costs $ 3,619,651.76 $ 1,137,607.95 $ 805,812.24 $ 5,563,071.94

You can see that Kenya assumes the majority of the program costs, spending $3,619,651.76 over 2015. Uganda is

next, with $1,137,607.95, and the youngest program, Malawi, assumes the least at $805,812.24. It is from these

total costs that we have derived our costing indicators for 2015 mentioned in Section VI.2. Extent of Impact: Total

installation costs per year, Total on-going costs per year, Cost per beneficiary per year, and Cost per anticipated DALY

averted per year.

Overall, our cost effectiveness stems from over three years of experience in achieving high household chlorination

rates, streamlined logistics, supply chain, and maintenance procedures. You can see how costs per dispenser

decreases as we install more dispensers, and therefore serve more people in Table 5 below.

16 Per person per year refers to those with access to the dispenser within a given year.

30

In 2016, our focus will be on strengthening our business model which will mean slowing down expansion and

consolidating our costs further; we have also developed a much more robust budgeting process and financial

tracking system to employ this year. This means that our projected costs, though higher than 2015 total costs per

person, will be more rigorous and accurate.

For the purposes of demonstrating lower cost per person per year as we scale, because our fixed costs spread over

more dispensers, we have modelled receiving another futures contact by the year 2017. Thus, 2017 costs are

projected to be higher than 2016 since they mainly represent the first year cost of installations. After that, costs will

continue to decline as we scale the program from 2018 and onwards.

What should be the scaling path of this intervention? How does this differ from the scaling plan proposed at the

beginning of implementation?

In the beginning, we prioritized the carbon market in supporting our scale-up plan. While a consistent revenue

stream through payment mechanisms like the carbon credit market is important, we found that we cannot rely on

a single source of revenue. Therefore going forward, we are a building a diversification of revenue streams, such as

government budgetary allocations for chlorine procurement, leveraging government networks like community

health promoters, and commercializing our program assets for use by corporates and other NGOs.

If we look at scale in an international context, we are weighing a franchise model; Evidence Action would become

the network partner to other organizations, transferring our knowledge and expertise to the implementing partners,

while maintaining the rigor of the approach through licensing or some other means.

We will be refreshing our scaling strategy over the course of 2016. We are therefore unable to provide DIV with a

definitive scaling path at this time.

If your project results justify ongoing public investments: How will your solution continue to be operationalized in the

absence of DIV support? Please address the following points in your response: What, if any, commitments (funding

the solution, incorporating learnings from the evaluation, adopting the solution in their own programming etc...)

have you secured from the stakeholders necessary to scale your solution? Who are the stakeholders that are still

needed? What are the roles each stakeholder could play? What is the engagement strategy and timeline? What role

would the evaluation findings play in influencing each stakeholder’s decision making?

We intend to more heavily engage with governments going forward. During the award period, we leveraged the

existing government health surveillance network in Malawi with a commitment to procure chlorine for our

dispensers. Documents pertaining to this commitment can be found in Appendix 10: Malawi commitments from the

Government. Government involvement is certain to continue there and we will continue to engage with them to

increase this support. As for Kenya and Uganda, a stronger engagement with the government is on the agenda for

2016 and beyond as we seek to commit budgetary allocation to augment carbon revenues. These stakeholders are

31

the Ministry of Health and devolved country health departments (Kenya only). The role they could play is to provide

a subsidy, as low as $0.50 per person per year, to augment our revenues from carbon.

Originally, we thought we could be self-sustaining. However, after carbon prices fell by 2/3 and the way credits are

measured led to 50% fewer credits per dispenser, we will continue to need partners and funders, and perhaps even

DIV, to support our operations; but this still is worth doing. Governments, donors and foundations are currently

paying significant sums for the provision of water in rural areas. And we have one of the cheapest ways to get clean

water at the source and at the home in rural areas, meaning our approach could help those parties obtain more

bang for the buck. Thus it makes sense for them to wish to make their investments go further through future grants

or partnerships with us.

An initial strategy to firm up their commitment will involve combining our program data with government data on

the prevalence of cholera to demonstrate the cost-effectiveness of dispensers as a preventative measure. We are

currently seeking funding to initiate our government engagement strategy, which will fund the investment of a small

policy team in each country over a three-year period, by which time the additional revenue from government will

likely cover their cost.

Additionally, we will ramp up our engagement with big international NGOs who already implement water programs,

and who seek out cost-effective solutions to the issue of waterborne disease. In 2016, we have already initiated

work to gain a deeper understanding of our costs and benefits versus competing interventions and we will use this

to disseminate the dispensers for safe water solution globally, with the aim of making a significant contribution to

the new Sustainable Development Goals (SDG6) for water.

If your project results do not justify further testing or public investment: What stakeholders would benefit from

learning about the results of this project? Please describe plans to disseminate lessons learned from the project.

What role would the research findings play in influencing each stakeholder’s decision making? What is the

engagement strategy and timeline?

Our partnership strategy in the short-term will be on securing additional revenue streams and lowering our costs to

sustain existing dispensers over time. As mentioned above, this would involve augmenting carbon revenues with

governments, corporations, NGOs, and others. We are currently discussing partnerships with WaterAid in Malawi,

BRAC in Uganda, and UNICEF in all three countries (Kenya, Uganda, and Malawi).

Our longer-term strategy would be for Evidence Action to play the role not just of implementer but of knowledge

center for other organizations to replicate dispenser programs in their countries. Our first step is to more rigorously

understand our costs and benefits versus competing interventions, which is currently underway in the first quarter

of 2016. Then we will double down on our dissemination efforts with the aim of making dispensers more widely

accepted as the norm for rural safe water globally. Finally, we will be targeting big NGOs who have a focus on water

and under-five child mortality. This will likely be via direct meetings and training seminars with the aim of reducing

the knowledge gap about how to make the program a reality in other geographies.

USAID missions could help disseminate the benefits of the program and provide much needed introductions to

implementing organizations in other geographies who show interest. The missions could also help cover costs

needed to undertake feasibility studies with those organizations and Evidence Action. Part of the feasibility studies

would be gauging the interest and willingness of governments to be involved, and USAID could help facilitate this.

Finally, USAID could examine its current water investments and see if dispensers could be bundled within it, as well

32

as provide much needed funding to actually implement pilots to determine the feasibility and likely success of a

program at scale.

Carefully detailing thoughts on scaling, what partners will be needed, etc. Discussion of Carbon Credits program.

These are our first draft thoughts on a scaling plan, which will be derived from our strategy refresh process that we

are undertaking over the course of 2016.

We realize that with the downturn of the carbon markets, scaling the program with carbon as our sole revenue

source is no longer viable. At steady state, we would likely recover only 43% of total program costs from carbon

revenues. Consequently, we will be making small investments over the course of 2016 to ascertain the size and

viability of three main additional revenue streams: revenue from users, revenue from governments, and revenue

from corporations or NGOs.

Users

The goal of the program when it was housed at IPA was to sustain high levels of chlorine adoption at the household-

level. To do this, the original RCT and research from JPAL17 concluded that asking very low-income households to

pay for preventative health measures was a barrier to uptake. We have scaled the program following this approach.

Within Evidence Action, our goal is to grow Dispensers for Safe Water sustainably. We understand that there are

studies indicating this may not work, but one option we will investigate will be for richer users to cross-subsidize

poorer users over time.

Governments

Through our contacts at the UN, we have learned that Kenya is struggling to curtail cholera, with more than 12,500

cases currently reported and over 28 counties affected. We suspect that governments invest more in cholera

response than in cholera prevention. For example, the Kenyan government allocated 500 million shillings to fight

cholera in 201518. One of our strategies in 2016 is to share the evidence of chlorine dispensers keeping cholera at

bay in our areas of operation. One tactic we are currently employing is to combine our program data with

government data on the prevalence of cholera in order to show the cost effectiveness of chlorine dispensers as a

preventative measure compared to existing cholera fighting measures. However, our immediate task is to raise funds

for a three-year program to invest in a policy team in order to better engage the government in an effective manner

in all three countries of operation.

Corporations

We have built a very strong program with a number of assets that we could commercialize to generate revenue into

the program. For instance, we visit over 3,000 households on a monthly basis who trust us and invite us into their

homes as we take water quality samples and a few simple survey questions on our mobile phones. We have the

ability to update the survey remotely on a daily basis, so adding a few extra market research questions for a

corporation would be easy. This will be attractive to market research firms and their corporate clients who target

the hard to reach, ‘deep rural’ consumers. Other such assets that could be commercialized include our extensive

chlorine rural distribution network via our many motorbike circuit riders who visit our promoters approximately

every 45 days, our regular community meetings, plus our thousands of dispensers, villager promoter t-shirts,

17 JPAL, “The Price Is Wrong,” JPAL Bulletin (April 2011);

https://www.povertyactionlab.org/sites/default/files/publications/The%20Price%20is%20Wrong.pdf. 18 “Kenya: Sh500 Million Budget to Fight Cholera” http://allafrica.com/stories/201505220574.html (accessed March 2016)

33

motorbikes, and chlorine jerry cans which could be branded for advertising purposes. We are testing out this strategy

throughout 2016 to investigate a stronger corporate engagement team in the future.

Our ultimate goal is to make chlorine dispensers the norm for rural water sources, where formal piped water supplies

do not exist. To succeed in this will mean that every other water sector organization and government needs to be

aware of chlorine dispensers and the operational model required to achieve a sustained use by households. This will

require continuous evaluation and presenting our results in easily understood and appropriate formats,

disseminating them through appropriate channels and modalities. Additionally, this will require developing

strategies to effectively engage with other organizations through appropriate partnership structures, such as

licensing, franchising, consulting, or open source type arrangements. This will require an ongoing investment in

product innovation to ensure we achieve the desired impacts of chlorine usage generated through a more expansive

and varied program.

Appendix 1: Human Centered Design Report

Appendix 2: Disseminated Project Results

Appendix 3: Country Research for New Program Expansion

Appendix 4: Final Indicator Report

Appendix 5: Impact Calculator

Appendix 6: Impact Calculator Narrative

Appendix 7: Updating Number of People Served

Appendix 8: Dispensers for Safe Water Biannual Qualitative Research Protocol

Appendix 9: Costing Model for 2015

Appendix 10: Malawi Commitments from the Government

Appendix 11: Dispenser Maps