OFFICE OF INSPECTOR GENERAL

AUDIT OF USAID’S OFFICE OF FOREIGN DISASTER ASSISTANCE ACTIVITIES IN ZIMBABWE AUDIT REPORT NO. 4-613-12-008-P MAY 10, 2012

PRETORIA, SOUTH AFRICA

Office of Inspector General

May 10, 2012 MEMORANDUM TO: Bureau for Democracy, Conflict and Humanitarian Assistance, Office of Foreign

Disaster Assistance Director, Mark Bartolini FROM: Regional Inspector General/Pretoria, Christine M. Byrne /s/ SUBJECT: Audit of USAID’s Office of Foreign Disaster Assistance Activities in Zimbabwe

(Report No. 4-613-12-008-P) This memorandum transmits our final report on the subject audit. We have considered management’s comments on the draft report and have incorporated them into the final report, as appropriate. They have been included in their entirety in Appendix II. Management’s comments indicate that corrective actions are planned for both recommendations. Therefore, we consider that management decisions have been reached. Please provide the Office of Audit Performance and Compliance Division with the necessary documentation to achieve final action on the recommendations.

I want to express my sincere appreciation for the cooperation and courtesy extended to my staff during the audit.

U.S. Agency for International Development

100 Totius Street

Groenkloof X5, 0181

Pretoria, South Africa

www.usaid.gov/oig

CONTENTS

Summary of Results ................................................................................................................. 1 Audit Finding ............................................................................................................................. 5

The Office of Foreign Disaster Assistance Does Not Have a Systematic Way to Determine Whether Its Activities Are Working ......................................... 5

Evaluation of Management Comments .................................................................................... 8 Appendix I—Scope and Methodology ..................................................................................... 9 Appendix II—Management Comments .................................................................................. 11

1

SUMMARY OF RESULTS

The U.S. Government declared a disaster due to a complex emergency1 in Zimbabwe every year between fiscal years 2002 and 2011. On October 18, 2001, the U.S. Ambassador to Zimbabwe declared a disaster in response to the country’s food security crisis. The Fiscal Year 2002 Annual Report for USAID’s Office of Foreign Disaster Assistance (OFDA) in the Bureau for Democracy, Conflict and Humanitarian Assistance (DCHA) notes that the complex food security emergency was “caused by a combination of factors, including drought, political violence, and the Government of Zimbabwe’s chaotic seizures of once productive commercial farms.” After the drought subsided, the other problems persisted. For example, the 2006 OFDA annual report notes that although Zimbabwe received favorable rains in 2005 and 2006, many Zimbabweans faced emergency conditions due to an 80 percent unemployment rate, hyperinflation, and limited access to basic social services. This complex emergency was further exacerbated by political violence during the 2008 presidential and legislative elections, which displaced some 30,000 people according to the United Nations Office for the Coordination of Humanitarian Affairs. From August 2008 to July 2009, Zimbabwe’s poorly maintained water and sanitation infrastructure and fragile health system led to 98,600 cases of cholera, resulting in nearly 4,300 deaths (concentrated approximately as shown below).

Areas of Zimbabwe Affected by Cholera, August-December 2008

Source: United Nations Office for the Coordination of Humanitarian Affairs.

1 According to Chapter 251 of USAID’s Automated Directives System (ADS), a disaster declaration is a

written determination by a U.S. Ambassador or designee of the Secretary of State that “a disaster exists in the host country which meets three criteria: it is of a magnitude with which the affected community cannot cope; recognized representatives of the affected population desire the assistance; and it is in the USG’s [U.S. Government’s] interests to respond.” ADS Chapter 251 defines a complex emergency as “[a] disaster, usually of long term duration, that includes a combination of humanitarian, political and military dimensions which hinders the provision of external relief.”

2

Conditions began to improve in 2010, and the U.S. Ambassador to Zimbabwe decided not to reissue a disaster declaration in October 2011. To address these emergencies, OFDA spent over $100 million in Zimbabwe between 2002 and 2011. In fiscal year 2011, USAID provided $13 million through OFDA to support agriculture and food security; protect urban livelihoods and vulnerable populations; and support water, sanitation, and hygiene activities. The scope of the audit included all of the projects that were not implemented by public international organizations (such as United Nations organizations) and were active as of February 2012. The audited projects totaled $6.5 million and are described below:

Rural Water Supply and Treatment Infrastructure and Hygiene Promotion in Gokwe North District. On February 9, 2011, USAID/DCHA/OFDA awarded $1,206,694 to Medair International to address the risk of waterborne diseases through the provision of protected hand-dug wells, rehabilitation of protected and unprotected wells, and provision of rainwater harvesting tanks to schools in ten wards of Gokwe North District (pictured on page 3).

Water and Hygiene Promotion Project—Phase II. On August 9, 2011, USAID/DCHA/OFDA awarded $569,000 to Adventist Development and Relief Agency International to establish community and school health clubs, build reinforced rainwater harvesting tanks for schools and communities, and promote improved water treatment and storage.

Scaling Up Point-of-Use Water Treatment and Hygiene. On July 28, 2011, USAID/DCHA/OFDA awarded $1,098,415 to Population Services International to expand behavior change communication and sustain nationwide delivery of WaterGuard, which was launched under a prior USAID/DCHA/OFDA grant.2

Mitigation of Risks of Waterborne Disease Outbreaks Focusing on Community Adoption of Household Water Treatment Practices in Manicaland Province. On July 7, 2011, USAID/DCHA/OFDA awarded $500,000 to International Rescue Committee to employ social marketing techniques to promote the use of WaterGuard for treating drinking water in four districts of Manicaland Province.

Promoting Improved Hygiene and Sanitation. On February 22, 2011, USAID/DCHA/OFDA awarded $643,188 to International Medical Corps to build the capacity of communities to implement their own interventions, mobilizing locally available resources and contributing labor to construct latrines and hand-washing facilities.

Food Security Support Project. On August 2, 2011, USAID/DCHA/OFDA added $1,420,714 to its grant to GOAL (a relief and development organization based in Ireland), which started on June 29, 2010. The additional funding in fiscal year 2011 made it possible to increase the number of farmers receiving support from 5,000 to 12,000. Support included training on conservation farming, 12,000 specially designed hoes, and 7,000 farming supply vouchers each worth $80.

2 WaterGuard is a brand name of sodium hypochlorite solution (NaOCI) manufactured and distributed by

Population Services International with 1.25 percent available chlorine and pH raised above 11 to ensure adequate shelf life.

3

Non-Governmental Organization Joint Initiative for Urban Zimbabwe: Community Based Support to Vulnerable Urban Populations. On July 19, 2011, USAID/DCHA/OFDA awarded $1,095,732 to Mercy Corps to promote, improve, and protect sustainable livelihoods for urban and peri-urban communities in Zimbabwe through effective response and information coordination.

A school rainwater harvesting tank (left) and a hand-dug well (right), both built by Medair International, help reduce waterborne diseases. (Photos by RIG/Pretoria, Gokwe North District, January 26, 2012)

In fiscal year 2012, OFDA operations in Zimbabwe shifted from activities under a state of disaster response to risk reduction programming. The current risk reduction strategy covers a 3-year period, fiscal years 2012 to 2014. The stated goal of this strategy is to “reduce risk through enhanced institutional and community capacities to respond to and mitigate the effects of disasters, strengthen the resiliency of vulnerable communities, and reduce exposure to hazards.” For example, the strategy states that water and hygiene activities will include skills training for purifying water for household use and the construction of rainwater harvesting tanks. The Regional Inspector General/Pretoria conducted this audit to determine whether USAID/DCHA/OFDA’s activities in Zimbabwe were achieving their goal of mitigating the effects of the country’s complex emergency. On the basis of limited site visits and interviews conducted, the audit determined that the project was mitigating the ongoing effects of the complex emergency in Zimbabwe. Specifically, the digging of wells and, to a lesser extent, the construction of rainwater harvesting tanks increased the availability of clean drinking water; the promotion of sanitation and hygiene increased awareness of the risks of disease; the introduction of WaterGuard began to provide a commercially available product for households to use to treat their water; and the distribution of inputs and promotion of conservation farming resulted in increased food availability. However, the audit noted that OFDA does not have a systematic way to determine whether its activities are working (page 5). Because it provides disaster relief, OFDA has been exempted from having performance monitoring plans. Yet for long-term risk reduction programming, it needs a monitoring and evaluation system to guide strategic decisions about activities in Zimbabwe and measure their effectiveness.

4

To address this issue, this report recommends that USAID/DCHA/OFDA: 1. Conduct an independent evaluation of key programmatic decisions regarding the complex

emergency in Zimbabwe to determine whether OFDA is allocating resources to projects that are demonstrating meaningful results (page 7).

2. Develop a timeline for the implementation of a monitoring and evaluation system for its disaster risk reduction activities in Zimbabwe (page 7).

We are also making a recommendation in a supplement to this report with limited distribution, as publically disclosing sensitive information contained in the supplement may be harmful to grantee operations or personnel. The detailed finding appears in the following section, and a description of the audit scope and methodology appears in Appendix I. Management comments are in Appendix II, and our evaluation of management comments is included on page 8 of the report.

5

AUDIT FINDING

The Office of Foreign Disaster Assistance Does Not Have a Systematic Way to Determine Whether Its Activities Are Working

On September 22, 2010, the White House issued a fact sheet on U.S. Global Development Policy, which states that the United States will “set in place rigorous procedures to evaluate the impact of policies and programs, report on results and reallocate resources accordingly.” In response to this policy, USAID notes on its Web site that it is “strengthening performance monitoring and impact evaluation capabilities so [it] can easily identify what works, what doesn't and why.” Further, USAID notes that it will emphasize effective resource management in order to allocate “resources toward programs that are demonstrating meaningful results.” Effective monitoring and evaluation strategies are key to achieving these objectives. Nevertheless, OFDA does not have a comprehensive monitoring and evaluation system to determine the success of its activities. While OFDA officials conducted periodic site visits and received information from grantee monitoring and evaluation systems on the status of activities in Zimbabwe, OFDA had no comprehensive system for assessing whether these activities were appropriate or successful. As a result, OFDA may have based programming decisions on untested information and been unaware of deficiencies of implemented activities. For example, OFDA’s strategy for disaster risk reduction emphasizes rainwater harvesting tanks rather than wells. An OFDA official noted that this decision was made because well pumps tend to break if they are not maintained regularly. Therefore, the underlying assumption to this decision is that well pumps break more frequently than rainwater harvesting tanks. However, during site visits, auditors found that 5 of 14 recently constructed USAID rainwater harvesting tanks were no longer functional or were not being maintained to allow maximum clean water collection and retention. Two other tanks (pictured on the following page) had not been constructed properly to prevent leaks, although the grantee was confident that the concrete would cure properly.3 Notably, only 3 of these 14 tanks were constructed before fiscal year 2011. Meanwhile, all five OFDA-constructed hand-dug wells observed during site visits were functional. OFDA officials were unaware of the problems with the rainwater harvesting tanks before this audit. Another reason for emphasizing rainwater harvesting tanks was to mitigate cholera outbreaks. Although clean water sources (both tanks and wells) help prevent cholera, rainwater harvesting tanks would likely not mitigate a large cholera outbreak, such as the one in 2008-2009. In general, rainwater harvesting tanks collect water only during the rainy season. Water collected by a tank may then be consumed in a matter of days or weeks, depending on the size of the tank and the number of beneficiaries, while wells provide water throughout the year. However, the cholera outbreak of 2008-2009 (one of the worst in a century) started in August, one of the

3 Curing newly placed concrete requires maintaining a favorable moisture content and temperature.

Neglecting to cure concrete properly can irreparably damage the structure’s ability to retain water.

6

driest months of the year, when the rainwater harvesting tanks would likely have been empty or at low levels.

The rainwater harvesting tank at left, built by Adventist Development and Relief Agency International, did not cure properly and shows water seeping through the side. The school rainwater harvesting tank at right, built by Medair International, has a broken tap. (Photos by RIG/Pretoria, Gokwe North District, January 26-27, 2012)

Related to other project site visits, a private chemistry lab contracted by the audit found that 2 of 14 water samples collected from USAID-sponsored tanks and wells tested positive for coliform bacteria, indicating that some beneficiaries were drinking potentially unsafe water. Additionally, beneficiaries trained by OFDA grantees did not build some latrines at the recommended distance from their water source. To allow OFDA to respond to emergencies, USAID exempts the office from some management requirements, like performance monitoring plans and data quality assessments. OFDA lacks a systematic monitoring system because it is designed to respond quickly to disasters. However, when OFDA engages in long-term, disaster risk-reduction programming designed for sustainability, this rationale is no longer valid. By not monitoring or evaluating the strategic decisions made for activities in Zimbabwe, OFDA may be investing in strategies that do not work. This use of resources without effective monitoring and evaluation undermines the President’s global development emphasis on determining “what works, what doesn’t and why.” The new 3-year risk reduction strategy gives OFDA an opportunity to implement a systematic monitoring plan to evaluate performance. Such a plan would make OFDA more aware of issues like the deficiencies discussed above. The resulting monitoring and evaluation system would allow lessons learned to be incorporated into future activities. Accordingly, the audit makes the following recommendations.

7

Recommendation 1. We recommend that the Office of Foreign Disaster Assistance conduct an independent evaluation of key programmatic decisions regarding the complex emergency in Zimbabwe to determine whether the office is allocating resources to projects that are demonstrating meaningful results. Recommendation 2. We recommend that the Office of Foreign Disaster Assistance develop a timeline for the implementation of a monitoring and evaluation system for its disaster risk reduction activities in Zimbabwe.

8

EVALUATION OF MANAGEMENT COMMENTS In its comments on the draft report, USAID/DCHA/OFDA agreed with both recommendations, and management decisions have been reached on them. Our evaluation of management comments follows. Recommendation 1. USAID/DHCA/OFDA agreed to conduct an independent evaluation of the activities in Zimbabwe by April 16, 2013. In addition, management responded to our analysis of the water, sanitation, and hygiene programs in the report. These are all factors that should be considered by the evaluation team, but we have not modified the body of the report. Recommendation 2. USAID/DCHA/OFDA agreed to prepare a timeline for the implementation of a monitoring and evaluation system for its disaster risk reduction activities in Zimbabwe by December 16, 2012. In addition, management elaborated on the grantees’ monitoring and evaluation systems, which the report briefly notes on page 5. Therefore, no change has been made to the audit report in response to these comments.

Appendix I

9

SCOPE AND METHODOLOGY

Scope The Regional Inspector General/Pretoria conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions in accordance with our audit objective. We believe that the evidence obtained provides that reasonable basis. The objective of this audit was to determine whether USAID/DCHA/OFDA’s activities in Zimbabwe were achieving their goal of mitigating the effects of the country’s complex emergency.

Audit fieldwork took place from January 17 to February 14, 2012, and included all OFDA activities funded under the fiscal year 2011 budget that were not implemented by public international organizations and were ongoing as of our audit fieldwork. OFDA’s fiscal year 2011 budget for Zimbabwe was $13 million. Of this amount, we included $6.5 million, or 50 percent of the funding for Zimbabwe, in the audit scope.

In addition to answering the audit objectives, the audit team assessed internal controls related to data quality, ongoing monitoring, reporting, and evaluations. In planning and performing the audit, we reviewed the following:

The fiscal year 2011 Federal Managers’ Financial Integrity Act of 19824 certification for

OFDA.

OFDA’s exemptions from U.S. Government and USAID regulations, including sanctions on

public figures in Zimbabwe; restrictions on working with the government; and standard

USAID rules about planning, monitoring, and acquisition.

The 2011 edition of the Humanitarian Charter and Minimum Standards in Humanitarian

Response handbook, published by the Sphere Project.

OFDA Guidelines for Unsolicited Proposals and Reporting, October 2008.

Grantee and quarterly progress reports for fiscal year 2011.

Grantee proposals, USAID technical comments on those proposals, and the final grant.

Budgets for the funding provided to grantees.

We also interviewed key USAID/Southern Africa, USAID/DCHA/OFDA, and USAID/Zimbabwe personnel, grantee personnel, local Government of Zimbabwe officials, U.S. Embassy officials in Harare, and beneficiaries. Sampling was done on a judgmental basis because grantee

4 Public Law 97-255, codified in 31 U.S.C. 3512.

Appendix I

10

activities were either not numerous enough to justify statistical sampling or covered too vast an area. We reviewed the significant partner activities in 22 of 119 district wards where OFDA activities were ongoing. We visited seven grantee offices in Harare and conducted meetings to learn about their operations. We conducted the audit at USAID/Southern Africa, USAID/Zimbabwe, grantee offices in Harare, and partner activity locations in the Manicaland, Mashonaland Central, Mashonaland West, and Midlands Regions.

Methodology To answer the audit objectives, the Regional Inspector General/Pretoria identified the current USAID/DCHA/OFDA activities in Zimbabwe intended to address the complex emergency. We identified the process used by OFDA to issue grants to address the complex emergency. Our review of this process included reviewing the unsolicited proposals prepared by the grantees, the technical comments sent back by OFDA, and the final proposals prepared by the grantees. We reviewed the budgets prepared by the grantees and determined whether the allocation of funding was reasonable given the goal of their activity and the allocation used by other partners for similar activities. If there were differences, we talked with partners to learn the reason for these differences. We met with grantees at their offices in Harare to discuss their experience with OFDA and determine how their activities were addressing the complex emergency. We also reviewed quarterly reports and corresponded with grantees to identify their major deliverables. We conducted site visits to verify that these deliverables were provided, accepted by the community, and of high quality (according to applicable disaster assistance guidelines). To determine whether the communities accepted the deliverables, we talked with members of the communities. Our review of the quality of the deliverables included contracting a private lab in Harare to test water samples from wells and rainwater catchment devices, and verifying that latrines would not contaminate the wells or other means by which the communities collected water. Sites visited were judgmentally selected for all of the activities except for Population Services International, which operates primarily in Harare; and Mercy Corps, which directly operates only one office in Mutare (other program activities are operated through subgrantees). The audit team’s primary concern when choosing sites to visit was maximizing the number and types of activities funded by USAID/DCHA/OFDA that we could observe. In addition, the audit team prioritized activities with higher levels of funding, easier access (largely based on proximity to major roads), proximity to other activities, and overlap with other programs, given the potential for duplication of efforts. In total, we visited activities in 22 of 119 district wards where activities operate. After understanding the quantity, quality, and geographic coverage of the deliverables, as well as the efficiency and effectiveness of activities, the audit determined that the activities were mitigating the ongoing effects of the complex emergency in Zimbabwe. The audit did not rely on performance indicators because the performance indicators did not address the audit objective. Therefore, the audit did not establish any materiality thresholds.

Appendix II

11

MANAGEMENT COMMENTS April 16, 2012 MEMORANDUM TO: Regional Inspector General/Pretoria, Christine M. Byrne FROM: Director, Bureau of Democracy, Conflict, and Humanitarian Assistance (DCHA), Office of

U.S. Foreign Disaster Assistance (OFDA), Mark Bartolini SUBJECT: Audit of USAID’s Complex Emergency Project in Zimbabwe Managed by the Office of

U.S. Foreign Disaster Assistance This memorandum transmits the USAID/DCHA/OFDA response to the draft report on the subject audit. USAID/OFDA concurs with the two audit recommendations, and details on the management decisions and target completion dates are provided below. In addition, USAID/OFDA is providing the following further information to clarify or update on the audit findings related to the two recommendations. Recommendation 1: We recommend that USAID/DCHA/OFDA conduct an independent evaluation of key programmatic decisions regarding the complex emergency in Zimbabwe to determine whether OFDA is allocating resources towards projects that are demonstrating meaningful results. Management Decision: USAID/OFDA’s management concurs with the recommendation to conduct an independent evaluation of the key WASH programmatic decision in Zimbabwe noted in the text. The target completion date for this evaluation is April 16th, 2013. In regards to the findings related to water, sanitation, and hygiene (WASH) programming, USAID/OFDA has followed up with the two non-governmental organization (NGO) partners that constructed WASH infrastructure assessed by the audit team. Both NGOs indicated that repairs are either underway or already completed on the tanks with issues identified in the audit. Prior to the audit, Medair International had reported that minor repairs were needed on several tanks, which would be completed in the final quarter of the grant. The Adventist Development and Relief Agency (ADRA) noted that the certain ferro-cement tanks constructed earlier in the project did not cure properly due to the high clay content of local sand used in the plastering mixture. The tanks in question have since been repaired by removing the initial plastering where the tank wall met the base slab, and replastering with the proper ferro-cement mixture. ADRA also added training for community builders on the importance of sieving the sand to remove clay particles. The audit findings noted that two of 14 water samples from WASH infrastructure tested positive for coliform bacteria. Typically, a positive test for total coliforms is followed by a subsequent test to isolate and enumerate thermotolerant, fecal coliforms in order to fully assess the public health risks. This is particularly necessary in the tropics where many bacteria of no sanitary significance occur in almost all untreated community water supplies. Given that, in the case of these two positive tests, it is difficult to infer more.

Appendix II

12

In addition to source water quality, USAID/OFDA recognizes that post-collection water handling behaviors are a prime source of contamination of drinking water supplies. As such, USAID/OFDA places special emphasis on the quality of household water supplies requiring all grantees constructing or rehabilitating water sources to report against the standard indicator “Number and percent of household water supplies with 0 coliform bacteria per 100ml.” Both grantees reported on water testing during the program period, using DelAgua kits to conduct tests for coliform bacteria. In addition, all partners include hygiene education for all project beneficiaries. USAID/OFDA also encourages all partners to stress the importance of point-of-use water treatment for all household drinking water supplies, using low-cost water treatment products available on the local market, such as PSI’s Waterguard. The findings also note that rainwater harvesting tanks could not mitigate against waterborne disease if, as in the case of the 2008/2009 cholera outbreak, the disease began to spread in the dry season when tanks would be either empty or at their lowest levels. The 2008/2009 cholera outbreak was exceptional in part because the first cases were reported in August 2008, but according to the U.N. World Health Organization approximately 11,700 cases occurred from August to November 30, with the remaining more than 86,000 cases occurring after the end of November. November is generally the beginning of the rainy season in Zimbabwe, so the period of highest risk for most Zimbabweans coincided with the rainy season in the case of this major outbreak as well. Therefore, while rainwater harvesting tanks may not be able to fully mitigate against disease risk during the dry season, this intervention is meant to address the highest risk period, during the rainy season, when the tanks will have a renewable source of water even with heavy use. Recommendation 2: We recommend that USAID/DCHA/OFDA develop a timeline for the implementation of a monitoring and evaluation system for its disaster risk reduction activities in Zimbabwe. Management Decision: USAID/OFDA’s management concurs with the recommendation to develop a timeline for the implementation of a monitoring and evaluation system for USAID/OFDA’s Zimbabwe programs. The target completion date is December 16th, 2012. In regards to program monitoring-related findings, USAID/OFDA has the following monitoring and evaluation systems in place for grants in Zimbabwe. As stated in the USAID/OFDA Guidelines for Unsolicited Proposals and Reporting, all potential grantees are required to provide information on planned monitoring and evaluation as part of their grant proposal. All seven NGO grantees included in the audit report provided information on their internal program monitoring and evaluation. In addition, USAID/OFDA encourages independent evaluations of programs. Five of the seven grantees audited included external program evaluations in their grants, while two grantees also included additional research studies related to the program. Project sites and grantees have also been visited by USAID/OFDA field staff and headquarters staff, including technical specialists.

U.S. Agency for International Development Office of Inspector General

1300 Pennsylvania Avenue, NW Washington, DC 20523

Tel: 202-712-1150 Fax: 202-216-3047 www.usaid.gov/oig