603.1 WB96

An Evaluationof the UNDP—World Bank

Water and Sanitation Program

Team

603.1—13532

An Evaluationof the UNDP-World BankWater and Sanitation Program

~ ~ ,.~.

A Forward-Looking Assessment

February 1996qcooE 1~q~ :.

~H ~ ~jgq~

© The World Bank1818 H Street, NW

Washington, DC 20433USA

All rights reserved

Manufacturedin the UnitedStatesof America

This documenthasbeenpreparedby an independent,externalEvaluationTeamappointedby theUnited NationsDevelopmentProgrammeand theWorld Bank. Copiesof thedocumentmaybeobtainedfrom theUNDP—World BankWaterandSanitationProgrammethroughthe World Bank.Material maybe quotedwith properattribution.The findings,interpretationsandconclusionsexpressedin thispaperareentirely thoseof theEvaluationTeamandshouldnot be attributedin anymannerto theUNDP—World Bank WaterandSanitationProgramme,the UnitedNationsDevelop-mentProgramme,theWorld Bankor anyaffiliated organizations.

Abbreviations

ADBCAMCCWSSCESIDGIP

ESAFAOGWPHABITATHRDIBRD1DBIDWSSDILOINUWS

IPFITN

MISNGOPAHOPPERPPUPREPROWWESS

RWSGRWSNRWSSSARAR

Asian DevelopmentBankcountryassistancemanagementCollaborativeCouncil on WaterSupplyandSanitationcountryexternalsupportinformationDivision for Global andInterregionalProgrammes,UnitedNationsDevelopmentProgrammeexternalsupportagencyFoodandAgriculturalOrganisationof theUnitedNationsGlobalWaterPartnershipUnitedNationsCentrefor HumanSettlementshumanresourcesdevelopmentInternationalBankfor ReconstructionandDevelopmentInter-AmericanDevelopmentBankInternationalDrinking WaterSupplyandSanitationDecadeInternationalLabourOrganisationInfrastructureandUrbanDevelopmentDepartment,WaterandSanitationDivisionindicativeplanningfigureInternationalTrainingNetworkfor WaterandWasteManagementmanagementinformationsystemnongovernmentalorganisationPanAmericanHealthOrganisationprojectperformanceevaluationreportProjectPreparationUnitPolic~ResearchandExternalRelationsPromotionof the Role of Womenin WaterandEnvironmentalSanitationServicesRegionalWaterandSanitationGroupRegionalWaterandSanitationNetworkruralwatersupplyandsanitationself-esteem,associativestrengths,resourcefulness,actionplanningandresponsibility

An Evaluation of the UNDP-World Bank WaterandSanitation Program U’

SDT sector developmentteam~TAG TechnicalAdvisory GroupTWUWS Transportation,WaterandUrbanDevelopmentDepartment,

WaterandSanitationDivisionUEST UrbanEnvironmentalServicesTeamUNIDTCD UnitedNationsDepartmentof TechnicalCooperationfor

DevelopmentUNDP UnitedNationsDevelopmentProgrammeUNICEF UnitedNationsChildren’sFundVLOM village-leveloperationandmaintenanceVPA Vice-Presidentfor PersonnelandAdministrationWASH WaterandSanitationfor HealthProjectWHO World HealthOrganisationWSS watersupplyandsanitation

iv An Evaluationofthe UNDP-WorldBankWaterandSanitationProgram

Contents

Preface viiExecutiveSummary 11. ProgramHistory andBackground 72. Building NationalandLocal Capacity 143. The Effectivenessof NationalPolicies,Strategies

andPlans 214. Impacton DomesticandExternalInvestment 285. SystematicLearningandExchangeof Experience

andInformation 346. SectorcoordinationandCollaboration 397. ProgramManagement 438. ProgramFinancing S89. ConclusionsandRecommendations 64

Annexes 731. Statusof Recommendationsfrom 1991 Evaluation 752. UNDP—WorldBankWaterandSanitationProgram:

Logical FrameworkAnalysis 843. List of PersonsMet 874. UNDP—World BankWaterandSanitationProgram:

Criteria for SelectingProjectsof Opportunity 975. Organizationof IJNDP—World Bank Waterand

SanitationProgramandCountriesof Concentration . . . .996. Summaryof Recommendations 101

Figures1. Capacity-BuildingModel 162. Respondents’Evaluationof theProgram’sAchievement

AgainstSelectedObjectives 243. Donors’ Contributions 60

Tables1. ProgramDisbursementsby Donor, 1978—1995 60

An Evaluationof thetJNDP—WorldBankWaterandSanitationProgram V

Preface

The UnitedNationsDevelopmentProgramme(UNDP)—World BankWaterandSanitationProgramcontinuesaseriesof joint undertak-ings by the World Bankandotherpartnersto increaseaccessto safewatersuppliesandsanitationthroughoutthe world. Theseeffortsbeganmorethantwo decadesagoin joint programswith the WorldHealth Organization(WHO). Today,manyinternationalandbilat-eraldonororganizationswork with recipientgovernmentsthroughtheWaterandSanitationProgramtowardthis sameobjective.Thisdocumentreportsthe resultsof adelayedmid-termevaluationof atrancheof theProgramfinancedunderUNDP InterregionalProjectINT/92/OO1. Two regionalprojectscoveringAfrica andAsia,,respectively,supportthe interregionalproject(RAF/92/007and

RAS/921001). Therewas no regionalprojectdocumentfor LatinAmerica. Programoperationstherewerecovered by country specific

documents. Althoughthe three project documents were notprepared at the sametime andthe needs of the regions arenot thesame,thereareonly minor differencesin the projects’ objectives.The evaluation,therefore,is basedprincipallyon an examinationof

how well the InterregionalProjectis meetingits goals.An independentteamcarriedoutaprior evaluationof thePro-

gramin 1990_91.1Its “Summaryof PrincipalFindingsandRecom-mendations”contains54 recommendationsfor Programmanagers2andsponsors.Many of thesehavebeenimplemented.Othersarestill in theprocessof beingimplemented.Somehavenot beenactedon becauseProgrammanagersdo not considerthemappropriate.Annex 1 commentson recommendationsfrom theprior evaluationthatarestill beingimplementedor havenot beenactedon. Someofthesearealsorevisitedin this report.

An Evaluationof the UNDP-World BankWaterand SanitationProgram vu

Methodology

The Evaluation Team looked atcapacity-building, promotionand support of sustainableinvestments, the use ofalternative approaches and thelearning process

TheEvaluationTeam3hadtwo main objectives:(i) to assessimple-

mentationof the Program’sstrategy,identify the main lessonsemergingfrom theuseof the strategyas away to improveaccessofthepoor to servicesandadvisethe UNDP and the World Bank onchangesthatwould improvethe effectivenessof the strategyandtheProgram;and (ii) to determinehow the key lessonsemergingfrom the evaluationcouldbestbe usedin theproposedGlobalWaterPartnership.

The EvaluationTeamanalyzedwhathasbeenlearnedthusfarusingthe 1992 Strategy4to addressthe Program’soverall goalofprovidingsustainableaccessfor thepoor to waterandsanitation.In

particular,theTeamlookedat: capacity-building(asdefinedin the1992document)with respectto sectorinstitutions,communitiesandsupportorganizations,especiallyInternationalTrainingNet-work (ITN) centers;the useof alternativeapproaches(includingappropriatetechnology)to expandserviceaccessfor thepoor; thepromotionandsupportof sustainableinvestments;as well as thelearningprocessandthedevelopmentof mechanismsfor analyzinganddisseminatingthe lessonsgainedfrom experience.The evalua-tion alsoreviewedtheextentto which the Program’sapproachhasbeenan effectivemeansof technicalcooperation.In this areatheTeamfocusedon the Program’sdecentralizedorganizationandfieldstructure,andits emphasison usingpartnershipsto work withdevelopingcountriesandexternalsupportagencies.

The evaluationexercisewas to havebeenprecededby tripartite

reviewmeetingsin Africa andAsia in July andAugust1995,heldinaccordancewith UNDPprocedures.In preparationfor thesemeet-ings, ProjectPerformanceEvaluationReports(PPERs)were pre-paredby the RegionalWaterandSanitationGroups(RWSG).ThePPERsdocumentedoutputsandactivities,as well as theextenttowhich the objectivesarebeingachievedin participatingcountriesinthe Program.

The evaluationteamundertookits work in thecontextof theproposedto createaGlobalWaterPartnership.This Partnershipisexpectedto consolidateandbuild on UNDPandWorld Bankcollab-orativeefforts,of which theWaterandSanitationProgramis thelargestandlongestrunning.ThePartnershipwill influenceandimprovepolicies,build capacityandgeneratesustainableinvestmentprojectsto supportintegratedwaterresourcemanagement.With itsfield structure,staffanddecentralizedoperations,the WaterandSanitationProgramprovidesan excellentmodelfor planningand

v,ii AnEvaluationof the UNDP—’World BankWaterandSanitationProgram

The Global Water Partnershipwill influence and improvepolicies, build capacity andgenerate sustainableinvestment projects to supportintegrated water resourcemanagement

implementingtheglobalpartnership.Consequently,the EvaluationTeamsoughtto determinewhich of the lessonsthatemergefrom itsexaminationof the Programaremostpertinentto theoperationofthe GlobalWaterPartnershipand,wherepossible,recommendhowPartnershipplannersshouldtakeaccountof theselessonsin plan-ning its ownstructure,organization,operationandfinancing.

It is alsoimportantto notethatthe evaluationtookplaceduring

aperiodof adecliningbudget.In effect,theProgramwasbeingaskedto achieveits objectiveswith lessthanthecoreresourceswhich hadbeenplannedatthe outset.A majorpreoccupationofthe Programmanagersat the time of the evaluationwas finding thefunds to supportthework programto which theyhadcommitmentsbeforeresourceswere cut.

To beginthe evaluationtheTeamreviewedthe projectdocumentsto identify the objectivesof thecurrentphaseandidentify indicatorsthatcould be usedto assessthe successof the Program.It wasdiffi-cult to find theselinks in projectdocuments.The beststatementofexpectedimpactsappearsin a sectionthatdescribesthe expectedend-of-projectsituationfor eachof the Programobjectives.Thus,theTeamdevelopedindicatorsto judgeeachof the end-of-projectsitua-tions. Annex 2 containsatablesummarizingthesewithin the formatof a LogicalFrameworkAnalysis,alongwith theWork ProgramSum-marypreparedby theTeam (thesereflectminor modificationsmadefollowing apresentationto representativesof World Bank andUNDP

managementin July 1995).In effect, thesetwo documentsconstitutethe revisedtermsof referencefor the EvaluationTeam.TheTeambelievesthat thelack of clarity in comparingindicatorsof achieve-mentwith projectobjectivesis theresultof trying to fit theProgram,largelya learningor actionresearchprogram,into themold of astandardIJNDPprojectdocument.The Teamrecommendsthat, inthe nextphaseof theProgram,managementpaygreaterattentiontodefining indicatorsof Programimpactthatcanbe usedto demon-stratethatobjectivesarebeingachievedwhile still keepingin mindthe learningnatureof theProgram.

The Teamgatheredthe informationrequiredto evaluateprogressagainsttheindicatorsfrom anumberof sources:• A reviewof Programpublicationsandinternaldocuments,

includingwork programs,annualreports,andPPERSprepared

by the RegionalWaterandSanitationGroup(RWSGs)for thetripartitereviews.

• Interviewsof managersandstaffof the Program,UNDP andtheWorld Bank,as well asof representativesof otherdonors,gov-ernmentagencies,departmentsandbeneficiariesof projects.

An Evaluationofthe UNDP—WorldBankWaterand SanitationProgram ix

The lack of clarity in comparingindicators is the resuh of tryingto fit the Program into the moldof a standard UNDP projeddocument

• Participation in tripartite reviews (governments-UNDP-World

Bank) of the threeUNDPprojectsheldin New Delhi, Jakarta,Nairobi andAbidjan. Thesehadslippedin their timingandwerethenrescheduledto makeparticipationby theTeampossible.

• Missionsto nine countries,5whereamemberof the teamcon-ducteda focus-groupmeeting,visited siteswherethe Programmight havehadan impactandinterviewedgovernmentofficials,nongovernmentalorganizations(NGOs)staff, donorrepresenta-tives,communityleadersandbeneficiaries.The namesof thosewho attendedthe meetingsor wereinterviewedareprovidedinAnnex 3.

• A questionnairecompletedby anumberof thosewho attendedthe tripartitereviewmeetings,as well as by awider groupofpartnersfamiliarwith the Program.ThroughouttheevaluationtheThamusedaparticipative

approachto the evaluation.In this way the Teamhopedto modelthe behaviorwhich theProgramseeksto helpothersto adopt.Atthetripartitereviewsthe attendeestogetherconducteda self-evalua-tion of the Program’sperformanceagainstthe desiredend-of-projectsituationstatementswith the assistanceof EvaluationTeammem-bers.Participantsfound this a useful approachto the tripartitereviews.It alsogaveevaluatorsveryrich information.Teammem-bersfollowed thesameapproachwhentheyconductedfocusgroupmeetingsin the ninecountriestheyvisited, andtheyhadsimilar,positiveresults.

The EvaluationTeamalsodevelopeda questionnaire,which wasdistributedto thosewho attendedtheregional andcountrymeetingsin Asia andAfrica andto otherswith knowledgeof the Program.Aseriesof close-endedquestionsrequestedthat the progressof theProgrambe evaluatedagainstend-of-projectdescriptors.Answerstothesequestionsprovidedaquantifiableassessmentof accomplish-ments(seefigure 2). A secondseriesof open-endedquestionspro-videdvoluntary commentsandmoredetailedinformationusefultothe evaluators.Onehundredresponseswereanalyzedto determinetrends.Interpretationof thesedatais given in the relevantsectionsof the report.

Early feedbackfrom the evaluationwasneededfor anumberofreasons.First, this mid-termevaluationwaslate in starting.TheTeam’sreportwas neededas soonas possiblein orderto haveanimpacton thefinal yearin thecurrent trancheof activities.Second,UNDP is planningits nextthree-yearfunding cycle.The evaluation

x An Evaluationof the UNDP—WrldBankWaterandSanitationProgram

results are an input to decision-making on future supportfor theProgram.Third, the evaluationwasproceedingat the sametime asdiscussionswerebeingheldon includingthe Programas apartofthe proposedGlobalWaterPartnership.TheTeamthereforeheldabriefing on the preliminaryfindings andrecommendationsin Wash-ington on November30.To reachabroaderaudience,additionalbriefingswereheldin NewYork on December1 andin Stockholmon December4. The exchangeof viewsat thesebriefings helpedtheTeamorient its reportto respondto the concernsof the stakehold-ers.The organizingmeetingfor the GlobalWaterPartnershipwasheldin Stockholmin earlyDecember.A discussionpaperpresentedthereon governanceof the Partnershipincludedrecommendationsbasedon the findings of this evaluation.

Organization of the report

This report is astand-alonedocumentthatputsthecurrentphaseofactivitiesin the historiccontextof the sectorandof the Programitself. Chapter1 providesthis setting.Chapters2, 3, 4, S and6examinethe findings,future issues, challenges and recommendations

for eachof the main objectivesof the Program.Theseare: Chapter2: Building National andLocal Capacity, Chapter 3: Effectiveness of

NationalPolicies,StrategiesandPlans,Chapter4: Impacton

Domestic and ExternalInvestment, Chapter 5: SystematicLearningand Exchange of Experience and Information and Chapter 6: Sector

CoordinationandCollaboration.Underits termsof reference,theTeamalsoneededto examine

the efficiencyandeffectivenesswith which the Programis managedandthemannerin which it is financed.Reportson thesetwo aspectsarepresentedin Chapter7: ProgramManagementandChapter8:Financing.Finally, Chapter9, ConclusionsandRecommendations,summarizesthe principal findingsof the EvaluationTeam.Detailedrecommendationsappearin eachchapter.Annex 6 presentsacom-pletelisting of them.

While theEvaluationTeamwaspreparingthis report, thePro-gramwaspreparingits 1995Annual Report.The annualreportpre-sentsa comprehensivepictureof the recentactivitiesandcurrentstrengthsof the Program.Readersof this reportwill find furtherinformationaboutthe Programthere.

An Evaluationofthe TJNDP-WorldBank WaterandSanitationProgram xi

Acknowledgments

As notedearlier, theTeamundertookthis evaluationas a participa-tory exercise.It could not havebeensuccessfulwithout thecoopera-tion of manypeople.Of particularimportancewas the willingnessof governments,UNDP and Program representatives in the regions

to agreeto modify the usualformat of tripartitereviewmeetings,turningthem into exercisesin participatoryself-evaluation.Arrange-mentsmadeby governments,communitiesandProgramrepresenta-tivesdeterminedthe effectivenessof thecountryvisits.Their successwasa demonstrationof this group’scommitmentto maketheevalu-ation ausefulexercise.The Teamcameto understandthe strengthsandweaknessesof the Programthroughfrankexchangesduringinterviewswith staffandmanagersof UNDP, the World Bank andthe Program;theother donorsandNGOs; officials andstaff of gov-

ernmentagencies;andthe beneficiaries.We hopethat all aremen-tionedin thelist of participantsin Annex 3. It would be animpossibletaskto singleout individuals for specialmentionwhenso

manyhavebeenhelpful.

Notes

1. “An Assessment of the UNDP—WorldBankWaterandSanita-tion Program,”Reportof an IndependentTeam,May 1991

2. The term “Programmanagers”whereusedin this text refersto the ProgramManager,his Deputyandmanagersof the Regional

WaterandSanitationGroups(RWSGs).3. Membersof the EvaluationTeamwere:William J. Cosgrove,

Eng.;President,EcoconsultInc.,Montreal,Canada(TeamLeader);JerryDelli Priscohi,Arlington, Virginia, USA; (Mrs.) MamonahengRamonaheng,ActingChief HousingOfficer, GovernmentofLesotho,Maseru,Lesotho;RolandSchertenleib,Eng. HeadSANDEC;EAWAG;Dubendorf/Switze’rland;JackTitsworth,Bethesda,MD, USA; (Mrs.) Budhy Tpajati Soegijoko,AssistantMin-ister,NationalDevelopmentPlanningAgency,Indonesia;Jakarta,Indonesia

4. July 1992,ImprovingServicesfor thePoor: A ProgramStrategyfor the 1990s,UNDP—World BankWaterandSanitationProgram.

5. Benin, Bolivia, Ethiopia,Ghana,India, Indonesia,Pakistan,PhilippinesandUganda

x An Evaluationofthe UNDP—WorldBankWaterandSanitationProgram

Executive Summary

The UNDP—WorldBank WaterandSanitationProgramis acontinu-ation of adecades-longeffort to improvepoor people’saccesstosafewaterandsanitation.This documentreportstheresultsof anexaminationof the Programcarriedout in 1995 by a teamof inde-

pendentevaluatorswho haveanalyzedits achievementsandshort-comings,andthenmaderecommendationsfor changes.The teamhasalso attemptedto drawlessonsfrom the Program’sactivitiesthatwill benefitthe now-formingGlobalWaterPartnershipinto whichthe Programwill be incorporated

The origins of theWaterandSanitationProgramgo backto 1978whenthe UNDP andthe Bank beganto work togetherto promotelow-costsanitationandwatersupplysolutionsfor poorpeopleinrural andmarginalizedurbanareas.In subsequentyears,througha

seriesof programsand international meetings, emphasis shifted from

technicalresearchon equipmentsuchas handpumpsto broadercon-cernabouthowto getthejob donewith communityparticipation.This led to arealizationthatthe taskcould not be accomplishedwithoutan effectivepolicy framework,planningandproject prepara-

tion andthatthe Programhadarole to play in assistingcommunitiesandnationalgovernmentsin theseareas.Along with the obviousneedto consolidateexpertiseandexperiencescatteredamongvari-ousprograms,thesetrendsled to a1987mergerof anumberof sep-arateprojectsinto asingleWaterandSanitationProgram.

Sinceits formation, the Program’smissionhasevolvedandcannow bestatedas “creatingcapacityso theycando it themselves.” Itsfocusremainsthepoor in ruralandmarginalizedurbanareas.How-ever,it hasincreasinglystructuredprojectsto supportinvestmentsinthewaterandsanitationsectorby helpingcommunitiesandgovern-mentsdeveloptheir capacityto solveproblems,to treatwateras aneconomicandasocialgoodandto involve all of thestakeholdersinthe selection, operation andmaintenanceof systems.The Programhasattractedwidespreadsupportandhasmademajorcontributions

AnEvaluationoftheUNDP-WorldBank WaterandSanitationProgram 1

in manycountries.The EvaluationTeamrecommendswithoutquali-fication thatthe ProgrambecontinuedandserveasamajorpartoftheGlobalWaterPartnership.But, thereareseveralareasof concernandseveralchangesthat shouldbeconsidered.

At the outsetof its work theteamdiscoveredtwo significantproblems.First, the objectivesthathavebeenset for the Programareexceptionallybroadandambitious.With a smallstaff anddwindlingfinancial resources,the Programis beingaskedto do too muchwith

too little. Justas important,thereis not acleardefinition of how theProgramis to bemeasured,how its managementor sponsorscandemonstratethatanyof its objectivesare,or arenot, beingreached.As the EvaluationTeam’swork progressedthroughmeetings,visitsto ninecountries,anda questionnairecompletedby alargenumberof peopleinvolved in thewaterandsanitationsector,otherthemesemerged.Forexample,althoughthereareimportantexceptions,themanagementandstaffof the Programarenot doing an adequatejobof transferringlessonsfrom onecommunityto another.At the sametime the Programstaff is perceivedto be isolated;it hasseldomlookedoutsideto learnfrom the approachesandtechniquesof oth-ersinvolved in similaractivities.

In an effort to measurea programthatdoesnot haveits ownwaysof measuringthe successof its mission,the EvaluationTeamusedavariety of Programdocumentsto establishfive statementswhich describethedesiredsituationat theendof the project,andthenmadean effort to measureprogresstowardeachof these.TheProgram’sgoalsandthe sumof theTeam’sconclusionsare:

1. National and local capacityfrom communitylevel to thegov-ernmentministerial levelwill havebeenstrengthened.Measurestobuild capacityincludeextensionof participatorytrainingandskills,including greaterattentionto genderissues;creationandsupportofnationaltrainingnetworkslinked into regionalandinterregionalnetworks;andrecruitmentof nationalstaff for in-countryandregional posts.In manycountriesthe Programhasstrengthenednationalandlocal capacitiesin thewaterandsanitationsector.But,thereoften remainsagapbetweenintentandimplementation.Deci-sionsaboutprojectsaremadeby donorsandgovernmentofficialswho do not knowwhatthe commurLitywantsor needs.Responsibil-ities of ministries andagenciesarefrequentlyconfficting andover-lapping,asituationthatprojectmanagerssometimesattempttoremedyby proposinga parallelstructureof nongovernmentalorga-nizations(NGOs).The Programmustreviewits efforts to buildcapacityatall levels,to developanddisseminatemodelsthatdemonstratehowto increasethe ability of usersto takecareof sys-

2 AnEvaluationoftheUNDP—WrldBankWaterand SanitationProgram

temswith aminimumof externalhelp. It alsoshouldincreaseitsemphasison involving the entirecommunity,especiallywomen,onwhomtheheaviestburdenof inadequatefacilities often falls.

2. Participatingcountrieswill havemadesignificantprogressinshiftingsectorpoliciesandstrategiesaway fromgovernment-driven,top-downapproachestowardsdecentralized~demand-driven,bottom-up approachesthatlegitimatizea variety ofoptionsfor provisionofservicesto thepoor. The Program’sworkplansfor 1991—9S includeprojectsto helpdeveloppolicy in 16 countries,strategyin 22 coun-tries,andactualplansin 24. The Programhassuccessfullyaccom-plishedmuchof this. It hasprovidedpolicy developmentandplanningassistanceon national, regionalandmunicipal levels,andhashelpedplanmissionstargetedatspecific issues,suchas invest-mentor the involvementof women.But, adoptionandimplementa-

tion of theseplanshasvariedwidely. In somecases,includingGhana

andBenin,policieshavebeenimplemented.In Pakistanimplementa-tion is slow. TheProgrammustidentify the reasonsfor the gapsorlagsbetweenplansandaction.It shouldalsodevelopagroupof casestudiesthatillustratehow acountry or communityhasaccomplishedits goal.And it shouldsponsortours so thatpoliticiansandsectoradministratorsfrom a countrythatdoesnot havea workingpolicyframeworkcanvisit andlearnfrom acountrythatdoes.

3. Qualitative improvementswill havebeenmadein thedesignandimplementationof large-scale,sustainable,investmentprojectstargetingtherural andurbanpoor, fundedby theWorld Bank, theregionaldevelopmentbanksandother donors,asa result ofPrograminputs.Althoughtherehavebeenmissedopportunitiesandafairlygeneralfailure to attractinvestmentfrom private enterprises,theProgramhashada positiveeffecton investmentsin the sector.It has

beeninvaluablein assistingin donorcoordinationandcollaborationandhasinfluencedthe approachtakento project designby theWorld Bank andothers.It hasdevelopednew approachesto com-munity financingschemesanddevelopedstrategiesthathavegener-atednew investment.But, the Programcanplay amorepositiverole in coordinatingandstreamliningplansfor projects.Somebilat-eraldonorsbelievethatthe RegionalWaterandSanitationGroups(RWSGs) throughwhich the Programcarriesout much of its work,arenot sufficiently linked to their programsor the programsofotherdonors.TheEvaluationTeambelievesthat the Programmustplay amorefocusedandstrategicrole, that this might increasetherateof investment.It shouldgive greaterpriority to micro-levelpreparationof investmentprograms:proposals,financingschemesandpilot anddemonstrationprojects.And it should advocate

AnEvaluationoftheVNDP-WorldBankWaterandSanitationProgram 3

financingof demonstrationprogramsby the Bankandthe regional

developmentbankssincesuchprogramsareessentialprecursorstolarger,moreexpensiveprojects.

4. A systematiclearningprocessfor testingnewapproachesto sec-tor development,monitoringtheres4tsandfeedingthembackintothesectordevelopmentprocesswill havebeendevelopedandinstitu-tionalizedin mostoftheparticipatingcountries.This is the weakestelementof theProgram.Thereis little focuson systematiclearning,identifying topicsandquestionsto be studiedduringan on-goingprojectto learnandimproveon the project.Thereis alsoresistanceto studyinganddrawing lessonsfrom the work of others.At thesametime the Programhasnot usedits informationstaffproperly,andbudgetcutshavefurtherhamperedthe publicationanddistribu-tion of information.Lessonslearnedfrom projectshavenot beencommunicated broadly. Even when publications on the transfer of

replicablemodelshavebeenproduced,therehasbeenno systematiceffort to learnwhetherthematerialis usefulor usedeffectively.

5. Sectorcoordinationwill havebeensignificantlyimproved.TheProgramhashada positiveeffect on sectorcoordinationwithincountriesby involving andworkingwith agenciesandorganizationson specificprojects.Generallyviewedas an independentorganiza-tion with a specialrelationshipwith the World Bank,it canserveas

achannelfor informal andnon-threateningcommunicationbetweengovernmentagencieson onesideandcommunityorganizationsandprivate enterpriseson the other. But, it hasnot exploitedthis advan-tage as muchas it could. It hasnot often initiatedcoordinationactivities beyondthecountrylevel. And severalgovernmentagencies

andnon-governmentgroupscomplainthat it doesnot involve themor collaboratewith them sufficiently.Among othersteps,thePro-gramshouldtakeastrongerlead in initiating anddevelopingcoordi-nationandcommunicationsamongall the playersin thesector,especiallyin connectionwith theplannedestablishmentof regionalorganizationswithin the GlobalWai:erPartnership.This initiativecould includeclearlydefining its own role in the sector,thusencour-agingotheragencies,includingUNICEF, WHO andthe regionalbanksto definetheir respectiveroles.

Management staffing and style

Partof thejob of the EvaluationTeamwasto look at theway inwhich theprogramis managed,staffedandstructured.The teamfoundashortageof staff atheadquartersandin someregions,signs

4 AnEvaluationofthe UNDP—~XbrldBankWaterandSanitationProgram

of low morale,high employeeturnoverandaneedfor a full reviewof managementpolicies,proceduresandstyle. If the Programis tobethe flagshipof the newGlobal Partnership,it needsstrengtheningas quickly as possible.The teamhasmadeanumberof recommen-dationsin this area.A few are:

• The Programhadno current missionstatementexceptfor onedevelopedby theEvaluationTeamitself. Onemustbe developedand agreed upon.

• Therehavenot beenreviews,evenat the endof the fiscalyear,to evaluateprogressagainstwork plans.Reviewsof perfor-manceagainstgoalsshouldbe heldat leastsemi-annually,if not

quarterly.• Means of measuring whether Program activities are making a

differencein the lives of the disadvantagedmustbe developedandpromulgated.

• Personnel policies, now set according to Bankrules,aswell as

work methodsand proceduresshouldbe designedto be moresuitableto thework andstyleof the Program.

• Opportunities for career development of staff members must be

improved.• Regional groups should be strengthened, perhaps made centers

for specialized sector expertise, and given some responsibility forfund raisingandliaison with donorsin their region.

• Staffing, structure and systems should flow from strategyandpriorities.

Money: declining support

Fundingfor theProgrampeakedin 1991 andthendeclinedasdis-bursementsfrom UNDPdroppedby nearly50 percent.Althoughtherehasbeensomeincreasein the flow of fundsfrom bilateraldonors,the losshasnot beenoffset.In 199S,out of funding ofnearly$11.1 million, bilateral donors,now the Program’schiefsupport,contributed$6.4million. The UNDP provided$3.7mil-lion. Aside from reimbursementfor servicesprovided,the WorldBank contributed$355,000,only 3.2 percentof the total. Thefinancial situation has beenfurthercomplicatedby ties betweenUNDP funding andindividual regionalor interregionalprojectsand,especially, by the tendency of bilateral donors to support spe-cific projectsrather thancoreor headquartersactivities,whicharenow largelyunderwrittenby the Bank’s relativelysmallandUNDP’s diminishing contributions.

An Evaluationofthe UNDP—WorldBank WaterandSanitationProgram S

Programmanagerstried to keepexpendituresin line, but they

did not cut back activities fast enough or far enough to bridge afundinggap. By 1995,the Programlacedasubstantialshortfall in its

budget for the current year. Mainly as a result of new commitments

of supportfrom afew donors,the programwasableto closethe gapso that it will beable to continue its activities attheir presentleveluntil the end of 1996.

In addition to the obviousrestrictionson activities,financingproblemshavecontributedto low staffmorale,high turnoverandawidespread perception among the staff that the Program lacks Bankcommitment. Among otherdifficulties, individualsfrequentlyhaveto be supportedwith relatively short-termprojectfunds. Somestaffmembers are never certain whether they will continue to be

employed. One other serious consequence is the burden on manage-ment time. As muchas half of thetime of key managersis spentonactivities related to fund raising.

TheProgram’sfinancingproblemswill alsobe facedby theGlobalWaterPartnership.Partnershipplannershavesuggestedthatit funds its activities through a water fund thatwould be at arm’slengthfrom anyspecific donorandable to attractfundingfrom dif-

ferentsources.The Partnership,however,will not be establisheduntil August1996.In the meantimethe Programmustfind funding.The Teamproposesthatit consideran adaptationof thewaterfundconcept, with the size based on a strategy agreed on by all donors

and a share set aside for the core costs associated with the global

learningprogram.Giventhe impactof the Programon the capacityof countries to carry out and sustainlargeprojectsof the typeusu-

ally fundedby the Bank,the teamsuggeststhatthe World Bank playa majorrole in raisingthe financial supportfor the Program.

A unique function

In sum,theEvaluationTeamconcludedthatthe Programperformsaunique function. It is properly focused on providing the world’s

poor in rural and marginalized urban areas with waterandsanita-tion services.It is trustedby nearlyall of its partnersas an indepen-dententity that is still closeenoughto the Bankto influenceitsdecisions.It is accessibleandcredible. Improvementsarepossible.Insome areas they are critical. But the Programshouldbe continued.

6 An Evaluationofthe VNDP—WorldBankWaterandSanitationProgram

Chapter 1

Program History and Background

In 1980 theWorld Bankcalculatedthatabouttwo billion peoplein

developingcountries,about45 percentof theworld’s population,did not have access to safe drinkingwaterandadequatesanitationfacilities. It was in this contextin 1978 thatthe UNDPandtheWorldBank launched the first global projectdesignedto apply moreappro—priatewater andsanitationtechnologiesafterdecadesof uncoorcli-

nateddonoractivity in the waterandsanitationsector.At the timeboth organizations were readying to meetthechallengesof the Inter-nationalDrinking WaterSupplyandSanitationDecadeof the 1980s.Seventeen years later, an estimated 1.1 billion people (75 percentliving in ruralareas)still do not havesafedrinking water.Abouthalfof the world’s population,2.9 billion people(80 percentof whomarerural), still lackdecentsanitationfacilities. And between3 and4

million people,including 2.5 million children,die annuallyfromdiseasesrelatedto inadequate,unsafesanitationfacilities.

At first glance it wouldappearthatlittle progresshasbeenmade.Butbetween1980and1995, theworld populationgrewfrom 4.5 to5.7 billion, a 28 percentincrease.During the WaterSupplyandSan-itation Decadealone,onebillion peopleobtainedsafedrinkingwaterfor the first time andmorethan750 million gainedaccesstoimproved sanitation facilities. Since then, progress has continued atasimilar pace.

An emerging consensus

The UNDP—World BankWaterandSanitationProgramis rootedinwhat hasbeena growingawarenessof waterandsanitation-relatedproblemsandan evolving globalconsensuson howto dealwith theseissues.Sincethelate 1970stherehavebeenaseriesof internationalmeetingsthatstimulatedthisawarenessandhelpedshapethe

An Evaluationofthe UNDP—WorldBank Waterand SanitationProgram 7

During the Water Supply andSanitation Decade alone, onebillion people obtained safedrinking water for the first timeand more than 750 milliongained access to improvedsanitation facilities

IJNDP—WorldBankapproach.The first of thesewastheUnitedNationsWaterConferenceatMar del Platain March, 1977.Thismeetingwasfollowed by: The GlobalConsultationof SafeWaterand

Sanitationfor the1990sheldin New DeLhi, India in September1990;the symposium“A Strategyfor WaterSectorCapacity-Building,”heldin DeIft, The Netherlands in June1991; International Conference onWaterandthe Environment,held in I)ublin, Ireland in January1992;UnitedNationsConferenceon EnvironmentandDevelopmentheldin Rio deJaneiro,Brazil in June1992~RoundTable on WaterandHealth,SophiaAntipolis heldin Francein February,1994;andtheMinisterial Conferenceon Drinking Water andEnvironmentalSanita-tion heldin Noordwijk, The Netherlandsin March 1994.

The originsof theProgramcanbe tracedto 1978.At thattimethe UNDP andthe World Bankbeganto collaboratein thepromo-tion of low-costsanitationandwatersupplytechnologiesfor poorpeoplein rural andmarginalizedurbanareasas an alternativetocostly conventional sewage and piped water systems. The main

effort was to demonstrateandpromotelow-cost,on-sitesanitationusing ventilated pit latrines and pour-flush toilets. Work was also

doneon the developmentof computersoftwareto facilitate thedesignof low-costpipedwatersystems.

Over the next 15 years the UNDP—WorldBank Program evolved,

both in organizationalstructureand in thethematicissuesthat it

addresses,througha seriesof projectsthatwere,in fact, successivephasesof its core components. In 1981, following an in-depth eval-uationof its earlierwork, the Program wasexpandedthroughtwoUNDP projects: Development and Implementation of Low-Cost

Sanitation Investment Projects (INT/8 1/047) and Laboratory andField Testing andTechnologicalDevelopmentof Rural WaterSupplyHandpumps(INT/81/026). By 1986 some2,700handpumps,repre-sentingabout70 differenttypesofequipment,hadbeenfield-testedin 17 African, AsianandLatin Americancountries.This work wascontinuedby two follow-on projectsin 1987:WaterSupply forLow-incomeCommunities(INT/87/013)andSanitationfor Low-income Communities (INT/87/014).

In 1991—92the UNDPapprovedthreemoreprojects:the Coun-try, RegionalandInterregionalIJND P—WorldBankWaterandSani-tationPrograms(INT/92/001), WaterandSanitationfor thePoorinAfrica (RAF/92/007)andWaterandSanitationfor thePoorin AsiaandthePacific (RAS/92/001).

In Central America, a Regional Witer andSanitationNetwork(RWSN)was funded, initially by UNI)P regional funds andsubse-quentlyby SwissandSwedishfunds. En Latin AmericaProgramactiv-

8 AnEvaluationof the UNDP—WorldBankWaterand SanitationProgram

itiesweresupportedin theearly 1990sby UNDP preparatoryfundsfor theAndeanNetworkand, later, by fundsprovidedby SwedenandtheNetherlands.Elsewhere,the UNDPand theBankagreedtocooperatein establishingseveralInvestmentProjectPreparationUnits(PPU) in 1982—twoin Africa, locatedin Abidjan andNairobiand

fundedby UNDP’s RegionalBureaufor Africa, andone in Asia, ini-tially locatedin Colombo,Sri Lanka(later movedto Bangkok)andfundedby the UNDP’s RegionalBureaufor AsiaandthePacific. Thetheory behindthe PPUs,which wasnot completelyborneout in sub-sequentevents,was thatthe main obstacleto increasedexternalinvestmentin the sectorwasthe lackof awell-preparedpipelineofinvestmentprojects.This lack, in turn,stemmedfrom countries’meagerexperiencewith projectpreparation.The PPUswereintendedto remedythissituationby providingexpertassistancein

identifying andpreparingprojectsfor considerationby internationaldonorand lendingagencies,includingtheWorld Bank.

An InternationalTrainingNetwork (1TN) for WaterandWasteManagementwasanothereffort of the early 1980s.The ITN wasestablishedto helpcountriesbuild their capacityto deliverwater

supplyandsanitationservices.The developmentandoperation of anetwork of institutions, althoughnot in itself an objective,waspositedas an effectivemeansto help achievethe overall objective.The creationof the1TN wasendorsedandfundedby theinterna-tional donorcommunityandtheProgramwasgiven responsibility

for its promotionandsupport.The UNDPandthe Bankalsocollaboratedfrom 1980to the

early 1990son ResearchandDevelopmentin IntegratedResourceRecoveryandWasteRecycling—asubjectthattranscendsthewaterandsanitationsectorin thenarrowsense,but clearlyhasabearingon the sector, as well as on emergingenvironmentalconcerns.Throughout,andsince,the InternationalDrinking WaterSupplyandSanitationDecade(IDWSSO),all of the regionalandinterregionalumbrella projects werecomplementedby manyrelatedcountrypro-jects fundedeither from UNDP countryindicativeplanningfigures(IPF) or from bilateralassistanceagencies.

By 1987 the UNDP andthe Bank,with growingsupportfromother donors,werecooperatingin agroup of interregionalandregional projects related to theWaterSupplyandSanitationDecade.Eachwasseparatelyfundedandhadits particularobjectivesspeci-fied in aprojectdocument.Initially, the projectsweremanagedbythesenioradvisor,water andwastes.Later,managementwastakenover by adivision chiefwho wasresponsiblefor research,policydevelopment,cooperativeprojectsandotherfunctionsin thewater

An Evaluationofthe UNDP—WorldBankWaterandSanitationProgram 9

supplyandwastessector.The projecl:sweregroupedby technical

contentinto sub-units,eachheadedby managerswho heldthe rankof division chiefor seniortechnicalofficer. Onemanagerwasresponsible for urban andruralwatersupplyandurbanandruralsanitation,anotherfor handpumpdevelopmentandresourcerecov-ery anda third for theITN. A Bank-~taffedsub-unitwasassignedresponsibilityfor the World Bank functions.

Evolution in orientationThe goal was to use what hasbeen learned about alternahveapproaches and appropriatetechnology in theimplementation of large scalewater and sanitation programsfor the world’s poor

In the meantimetwo importanttrendshademergedin theorienta-tion of severalprojectsas a result of periodic evaluations. The firstwasa shift away from the heavy emphasis on technology research

andtowardinstitutional issuesandthedevelopmentof servicedeliv-ery mechanismsbasedon community participation.The secondwastowardmore unified managementof thecollectionof projectsin aneffort to achieve better coordination and impact. The Technical

Advisory Group(TAG) Low-Cost SanitationstaffmeetingheldinNairobi in 1984,the FourthMeetingof theHandpumpsAdvisoryPanelheldin Chinain August1984,andthe evaluationof theAfrican PPUsin 1985,for example,all acceleratedtheshift awayfrom technology research and development and toward greater

emphasison softwareissues.The African PPUevaluationalsorec-ommendedthatthe PPUsbe replacedby interdisciplinarySectorDevelopmentTeams(SDTs)which could bring togethertheexper-tise thatwasscatteredamongindividual projectsunderasingleman-agement.The SDT conceptwasaccepted.Beginningin 1988,it wasdevelopedinto a networkof broader-basedRWSGswhich,withoffices in Abidjan, Jakarta, Nairobi, NewDelhi and, as of 1995,LaPaz,now constitute the principal avenuethroughwhich Program

activitiesareimplementedin thefield.A third trendthatemergedas the Decadeprogressedwasagrow-

ing realizationthat, to be really effeciive,projectpreparationandothermicro-levelactivitieshad to be carriedout within asupportivenationalsectorpolicy andplanningframework.Severalevaluationsof the PPUscarriedout in Africa andAsia confirmedthis andled toincreasedemphasison broadersectorwork, includingsectorstudiesandpolicy advice,as amajorcomponentof Programactivities.

Thesetrendsculminatedin 1987in the Bank’s decisionto mergethe separateprojectsinto a singleProgramwith anintegratedstrat-egy. To helpaccomplishthis,the Programwasplacedunderthemanagementof the WaterandSanitationDivision (INUWS) of the

10 An Evaluationof the UNDP—WorldBankWaterand SanitationProgram

This approach is aimed at givingthe Program a more rigorousstructure for disseminatinglessons and establishing astrategic learning process

PROWWESS was integrated intothe Program to promote waysto include women more fullyin water and sanitation projects

Bank’s centralPolicy, ResearchandExternalProjectsRelations

(PRE)complex.In 1988,following the consolidation,the ProgramissuedToward

EquitableandSustainableDevelopment:A Strategyfor theRemain-der oftheDecadeandBeyond.The strategythusoutlinedusedthelessonslearnedduringthe final yearsof the 1980sas the challengesfor the 1990s.The goalwas,andis, to usewhat hasbeenlearnedaboutalternativeapproachesandappropriatetechnologyin theimplementationof largescalewater andsanitationprogramsfor theworld’s poor.

The reorganizationof the Programinto asingleentity wasintendedto improveits overall cohesionandsenseof direction.But,it alsodisruptedresearchandpublicationactivitiesandled to thedisbandingof theformer TAG for low-costsanitation,agroupthathad initiated successful activities in some20 countries.Furthermore,it weakenedheadquarterspromotionandguidanceof the ITN.

Since 1988 the Program’smissionhasevolved.The initial mis-

sion was to increase the capacityof countriesto deliver watersupplyand sanitation services to low-income groups, primarily with low-

costandcommunity-basedapproaches.This missionshiftedto assistdevelopingcountriesin improvingpoor people’saccessto sustain-ableservices.Mostrecently,the missionhasbeenmodifiedto place

moreemphasison communitymanagementthancountryresponsi-bility and,thus,createcapacityfor communitiesto provideservicesfor themselves.

In 1992Promotionof the Roleof Womenin WaterandEnviron-mentalSanitationServices(PROWWESS)was integratedinto theProgram.PROWWESSbeganin 1983 asan interregionalUNDPprojectwith the overall objectiveof promotingwaysto includewomenmorefully in watersupplyandsanitationprojects.Overtimeit becameactive in morethan20 countriesandexpandeditsapproachto includefour areas:genderanalysis,communityinvolve-ment,participatorytechniquesandcapacity-building.A 1991 evalu-ation recommendedthat PROWWESSexpandits alreadystronglinks with theProgramin orderto improvethe useof participatoryapproacheswithin large-scaleWorld Bank projectsand to helpitnetworkwith thelargenumberof agencieswith which the Programworks.Since1992 it hasbeenfully integratedandhasfocusedongenderissues(andlesson womenper se), training andtoolsfor useatthe communitylevel, credit andmicro-enterpriseandhealthandhygieneeducation.

In the early 1990sa consensuswasreachedamongsectorprofes-sionalsthat,althoughlow-costtechnologieshadbeenrefinedand

An EvaluationoftheUNDP-WorldBankWaterand SanitationProgram 11

The Program has increasinglystructured projects to supportinvestments by buildingcapacit~çtrealing water as aneconomic and as a social goodand employing demand-basedapproaches to involvestakeholders in the selection,operation and maintenanceof systems

madeavailable,waterandsanitationsystemsfell idle andinto disre-pairbecauseinsufficient attentionwasbeingpaidto the ability ofinstitutionsandcommunitiesto manageand fund them. In 1992 theProgramissuedImprovingServicesfor thePoor: A ProgramStrategyfor the1990s,which reinforcedits triad strategy:workingwith part-ners to supportsustainableinvestments,building the capacityofgovernmentsandpeopleto developandmaintainsystemsandexchangingthe knowledgecultivatedin sodoing.Sincethen,theProgramhasincreasinglystructuredprojectsto supportinvestmentsby building capacity,treatingwateras an economicandas asocialgoodandemployingdemand-basedapproachesto involve stake-holdersin the selection,operationandmaintenanceof systems.

In 1994 the Program,still operatingwithin thecontextof its1992 Strategy,beganto put greateremphasison learningandstrate-gic supervision.It fosteredstructuredlearning,anapproachthatrequiresthehypothesesuponwhich the designof anyprojectisbasedto be identified whenthe project is beingdesigned,andthenreportedagainst,so thatsubsequentsuccessor failure canbe moresystematicallydocumentedandanalyzed.At thesametime strategicsupervisioncomplementsregularprojectmonitoring. It is orientedtowardmonitoringtheprinciplesuponwhich aprojectis basedas itis beingimplementedin orderto providefor critical reviewby stake-holdersanda changeof coursewhensomethingis not working. TheProgramthusintendsto establishasetof designpremisesthatwillallow it to monitor,compareandanalyzethe performanceof similarprojectsacrossdifferent regionsor countries.This approachis aimedat giving the Programamorerigorousstructurefor learning,docu-menting,disseminatingandcommunicatinglessonsandestablishingacontinuous,global, strategiclearningprocess.

As part of theincreasedemphasison learningthreefiltersl havebeenput in placeagainstwhich anyproposedprojectmustbeassessedbeforeimplementation:• Project characteristics—Theprojectmustprovide a vehiclefor

developinglessonsthatwill likely haveanationalandinterna-tional impacton future projects.

• Project environment—Theprojectshouldhaveahigh potentialtopositively affect servicedelivery andsustainabiityin thecountry~Both donorandrecipientgovernmentmanagementshouldsup-portthe learningapproach,as well as the principlesembodiedinprojectdesign,andshouldbe preparedto apply themto futureprojectsor policies in thesector.

• Locationin a countryofconcentration—Theprojectshouldbelocatedin oneof the Program’scountriesof concentration.In

12 AnEvaluationofthe UNDP—WorldBankWaterand SanttationProgram

this way sufficient resources canbe broughtto bearto providethe high level of strategicsupervisionrequiredto ensuresuccessandahighstandardof documentation.

Program management

Between1988,whenthe UNDP—WorldBankWaterandSanitationProgramassumedits presentform, and1994 the Programwasheadedby amanagerwho reportedinitially to theDirectorof theWaterandSanitationDivision (INUWS) of the Bank’scentralPol-icy, ResearchandExternal Relations(PRE) complex.Followingchangesto the Bank’s structure,he reportedto theDirectorof theTransportation,WaterandUrbanDevelopmentDepartment(TWIJ), oneof threedepartmentsin theEnvironmentallySustain-able DevelopmentVice Presidency.Sinceearly1995,reportingrela-tionshipshavebeenrevised.The Headof the Programnowreportsto the Division Chief responsiblefor the WaterandSanitationDivi-

sion within TWU.The 1995changeaddedto the professionalexpertiseavailableto

the Programandwas undertakento bringmoreintellectualrigor,substantivethinking andfocusto the Program’srangeof activity. Ithasalso changedthe statusof the Programwithin the World Bank.Taskmanagersandotherswithin the geographicvice presidencies

identify morecloselywith theattemptto bring moreintellectualrigor to the Programandto combineit with the field strengthandgrass-rootsexperienceof the RWSGS.But at the sametime blendingthe Programinto the WaterandSanitationDivision atheadquartershasrenderedthe Programlessdistinct from the World Bank.Thisrisksweakeningits comparativeadvantagein that it diminishestheperceptionof governmentsanddonorsof the Programas beinginthe Bank,but not of the Bank,a trait thatfacilitatedclosercollabo-ration betweenthe Programand its principal clients.

Note

1. SeeAnnex 4 for moredetail.

An Evaluationofthe UNDP—WorldBankWaterandSanitationProgram 13

Chapti~r2

Building National and Local Capacity

Nationalandlocal capacityfrom communitylevelto thegovern-mentministerial levelwill havebeenstrengthened.Measurestobuildcapacityincludeextensionofparticipatory trainingandskills, includinggreaterattentionto genderissues;creationand

supportofnational trainingnetworkslinked into regional andinterregionalnetworks;andrecruit,nentofnationalstafffor in-countryandregionalposts.1

As emphasizedin Programstrategydocuments,capacity-buildingis along-term,continual,dynamicprocessinvolving policies,institu-tionsandpeople,which mustpermeateall activitiesin the sector.The needsandrequirementsare changingcontinuouslyandhavetobereviewedregularly.At thepolicy level capacity-buildingmeansimprovingthe rulesgoverningthe sector,aswell as the regulationsandpracticesthatdefine theenablingenvironmentwithin whichsectordevelopmentoccurs.Capacity-buildingalsomeansenhancingthe performance,variety andnumberof organizationsactivein thesectorandstrengtheninghumanresourcesthroughoutthe sector.The cornerstoneis the involvementof nationalandlocal institutions

in theprocessof learningandadaptation.Althoughthe Programwasintendedto helpstrengthenexisting

managementstructures,traditional managementsystemsarefailingin mostdevelopingcountries.Moderndevelopmentplacesheavydemandson the traditionalsystems,andit hasbecomemoredifficultto agree on commonpriorities within any onecommunity.To helpcountriescopewith thissituationthe Programhasoftensupportedthe developmentof parallelNGO structures.Butsuch supportmight renderthe situationmorecorriplex in the long term.It canremoveresponsibilityfrom governmentandultimately weakenthecapacityof governmentdepartmentsor agencies.

14 An EvaluationoftheUNDP—WorldBankWaterand SanitationProgram

Capacity-building isa long-term, continual, dynamicprocess which must permeateall activities in the sector

Involvingcommunitiesin the Program’swork is a two-wayprocessthat requiresthe provision of information, training and edu-cation.The coordinationof functionsfrom the grassroots to thecentrallevel is an importantelementin achievingbetterplanning,implementationandmanagementof water andsanitationprojects.Governmentsmustchangetheir role from that of centralprovidersof servicesto thatof facilitatorsandcoordinators.

The educationalprogramsin mostdevelopingcountriesstillemphasizeknowledgeratherthanskill development.Educationandtraining takeplace throughformal channels,like schools.In mostplaceson-the-jobtraining is still not usedeffectively.Moreover,manypeoplewho aretrainedin waterandsanitationtendto moveout of thesectorover time.

Waterandsanitationissuesaffect the lives of men,womenandchildrenand,therefore,all needto beinvolved in sectoractivities.Therole of womenis especiallyimportantsincetheymostoften

mustfetchthe waterandtheir quality of life in thehomeandin thefield is mostaffected.Capacity-buildingat the local level cannotsucceedwithoutthe full participationof women.Their leadershipisespeciallyimportant.The EvaluationTeamnoteda tendencytoassumethat theimportanceof women’srole is well understoodandaccepted.But thisis not alwaysthe case.For example,theyobservedthatwomenarelittle involved in the processof planningandpro-viding watersupplyandsanitationin IndiaandPakistan.

Capacity-building model

To build the capacity to implementlarge-scaleprojectsbasedon ademand-driven, bottom-up approach,activitiesandknow-howarerequired on three different levels (figure 1). The base, or pilot level

(level A) includes the development andevaluationof alternativeapproaches(including technologies),methodsfor communicatingwith beneficiary communities, different methods and approachesfor delivering water supply and sanitation services and models for

financing water supply and sanitation services. These technologies,

methodsandapproachesthataredevelopedandevaluatedat levelA mustbeappliedandtestedor demonstratedin individual com-munities(level B) beforetheycan be appliedin large-scaleinvest-mentprojects(level C). The demonstrationprojectsin level B areespeciallyimportantfor policy dialogueandfor demonstratingthepracticalityof newpoliciesandstrategies.

An EvaluationoftheUNDP—WorldBankWaterand SanitationProgram is

The cornerstone is theinvolvement of national andlocal institutions in the pro(:essof learning and adaptation

Capadty-build~g model

Po~kyiindstmtegy

In the 1980sthe Programwasveryactiveatthe pilot level indevelopingandevaluatinglow-costtechnologiesin watersupplyandsanitationfor predominantlyrural areas.At the samelevel, theWaterandSanitationDivision of the World Bank,thenthe managerof theProgram,conductedresearchon delivery andfinancingmechanismsfor water supply and sanitationservices,andPROWESSfocusedonthe developmentandevaluationof mechanismsandtechniquesforcommunityinvolvementandparticipatorytraining processes.

More recently,realizingthatgreaterresultscould be obtainedathigherlevels, the Programhasbeenconcentratinginsteadon demon-

strationprojectsandon assistingin the developmentof large-scaleinvestments.For instance,the Programhasbeengraduallyturningover its responsibilityin the areaof handpumptechnologydevelop-

ment, in which it hadbeenvery activefor manyyears,to SKAT, aSwissNGO, thoughit remainsoneof thepartnersin theHandpumpTrainingNetwork. TheProgramhasalsoconcludedits development

work on participatorytraining andis nowencouragingotherstoadaptthetools thathavebeendevelopedto their local needs.In fact,becauseof its ability to influencetheBank, whichis themajorfinancierof large-scaleinvestmentsin thewaterandsanitationsector,theProgramcanprovidea link betweenpilot-level andlarge-scaleinvestmentprojectsby implementingdemonstrationprojectsand

participatingin the preparationandevaluationof large-scaleprojects.

Program results

The Programhasstrengthenedbothnationalandlocal capacitiesinmany countries. This strengthening hasbeenmorepronouncedwheregovernmentalandnon-governmentalsectoragenciesand

C App&uhon of~nxffive~pmothes(nchithng tochrmk~es)inIore pro~eds(wml/por~on)

I. A~pbwoIogyontheaIe(courtiyw~commuriiy~pecific)

Al. A2. A3. A4.D~e1opmentaid Commwucation Aitecnmvo Finonangevalualionof meitEdsfac delivery modesaltenintive oppioaches commuiaty mecloinisas(mckdmgtediidognis) involvement

16 An Evaluationofthe UNDP—WorldBankWaterandSanitationProgram

In several projects there isevidence of a lack of skills foroperation and maintenance.A number of facilities are indisrepaii or soon will be

organizationshavebeenactivelyinvolved in the implementationofpilot anddemonstrationprojects.At thenationallevel therearemanyexamplesin which the Programhassuccessfullyinfluencednationalsectorpoliciesandstrategies(seechapter3). But, in most

countriesthereis still a gapbetweenofficial declarationsandactualimplementationof policy andstrategy.In mostcasesthereis noappropriateinstitutional frameworkin placefor implementingthenewpolicy, thereis a generallack of nationalcapacity-buildingstrategiesandthe participatory-approachmethodologyhasrarelybeeninstitutionalized.Decisionson local water andsanitationinter-ventionsarestill madetoooftenat theheadquarterslevel by donorsandhigh-level governmentofficials, neitherof whomareknowledgeableaboutthe wishesandneedsof the local population.

In all countriesseveralministriesandagencieshavepartialresponsibilitiesfor thewaterandsanitationsectorwhich areoftenconflicting or overlapping.The responsehasoftenbeento createaparallelstructureof NGOs,but this approachmaynot be sustain-able. Still, it is importantto usethe NGOsas muchaspossiblebecausetheyareusuallycloseto communities.So far, the Programhascontributedrelatively little to developingnewmodelsof respon-

sive institutionalframeworks.Similarly, little work hasbeendonetodevelopsuccessfulmodelsthatencouragetheparticipationof pri-vate sectorentrepreneursin thewaterandsanitationprojectsthatbenefitpoor urbanandrural areas.

Thereareseveralexamplesin whichlocal ability to adoptnew

approachesto deliveringandfinancingwatersupplyandsanitationserviceshasbeenincreasedasa resultof Programactivities.But ben-eficiarieshaveacquiredthe skills requiredby theseparticipatoryapproachesto only alimited extent.In severalprojectsthatthe teamvisited thereis evidenceof alack of skills for operationandmainte-nance.A numberof facilitiesarein disrepair,or soonwill be,becauseof alack of knowledge,organizationandthe financialresourcesrequiredto repair,operateandmaintain the system.

PROWWESS and alternative technologies

Establishedas aspecialinterregionalproject,PROWWESSwaspro-viding technicalsupportandguidanceto community-basedwaterandsanitationprogramsevenbeforeit wasofficially integratedintotheProgramin 1992.The cornerstoneof PROWWE.SS’sactivitieshasbeenteachingtrainersandfield personnelto designandusepar-ticipatory methodsandmaterials.This trainingprocesscalled

An EvaluationoftheUNDP—WorldBankWaterand SanitationProgram 17

In the last few years theProgram has been evaluatingand judging theappropriateness of differenttechnologies under differentcircumstances. This oftempt hasbeen only partly successful

SARAR (self-esteem,associativestrengths,resourcefulness,actionplanningandresponsibility)involvescommunities,particularlywomen,in planning, implementingandevaluatingwaterandsanita-tion projects.The Programhassuccessfullytakenadvantageof

PROWESSactivitiesin somecountries.The RWSG for EastandSouthernAfrica hasbeenworking

with WHO andUNICEF on regionalinitiativesaimedatdevelop-ing aparticipatoryapproachto hygieneandsanitation.SARARmethodsandmaterialshavebeenadaptedandtestedthroughaseriesof country-levelprojectinterventions.In Zimbabwe

PROWWESStraining in theparticipatoryapproachhasbeencar-ried out mainly throughthe Institute for WaterandSanitation.ParticipatoryIWSD Developmentmethodshavebeentheprinci-pal subjectof somecoursesandhavebeenincludedin others.Thistraining hasmainly enabledtraineesto implementtheir programs

in amoreparticipatoryway but hasnot led to the applicationofthe principlesof the SARAR methodandcommunityempower-ment.Many of thoseinterviewedby the EvaluationTeam feel thatthe Programhasnot adequatelybuilt on thework completed

underPROW’WESS.The development,evaluationandapplicationof alternativetech-

nologiesis anotherimportantaspectof capacity-building.In the last

few yearsthe Programhasbeentrying to move awayfrom attempt-ing to select“winners” anddeveloptechnologies.Instead,it is

keepingawatchingbrief on applicationsin the field andevaluatingand judgingthe appropriatenessof different technologiesunderdifferentcircumstances.This attempthasbeenonly partly success-

ful. The rangeof alternativeapproachesbeingconsideredfor andevaluatedin Programactivitiesis still very limited andthereis abias towardlow-costtechnologiesbr predominantlyrural areas.With the exceptionof activitiesin connectionwith thedevelopmentof amethodologyfor StrategicSanitationPlanning,field evaluationof alternativesanitationtechnologiesfor pen-urbanareashasbeenlargely neglected.Also, while the participatoryapproachis beingtried at thecommunitylevel, expeaienceis still lacking in how touseit in thesectorat a nationalscale.

ITNs: Need for a new strategy

The InternationalTrainingNetwork for WaterandWasteManage-mentwasoriginally createdto assibtcountriesin building their

18 An Evaluationofthe UNDP—WorldBankWaterand SanitationProgram

The range of alternativeapproaches being consideredfor and evaluated in Programactivities is still very limited

capacityto deliver watersupplyandsanitationsystemsthroughtheestablishmentof anetworkof ITN centers.The Programoriginallycreated,promotedandsupportedtheseinstitutionsandis continu-ing to do so in thecurrentphaseof its activities.

The ITN, however,hasevolvedinto acomplexcollectionofcountrycenters(Ghana,Zimbabwe),nationalnetworks(India,

Philippines)andsub-regionalnetworks(WestAfrica, EastAfrica)thatareonly looselylinked togetherin aglobalnetwork.Eachcen-ter hasgrown alongdifferent lines, developeddifferentspecialties

andestablisheddifferent ties with host andothernationalinstitu-tions, donorsandassociatedinstitutionsin developedcountries.This divergenceis mainly dueto the fact thatmostof the ITN cen-tersarefundeddirectly by differentdonorswith differentviewsandexpectationsandthat the centershavebeenencouragedto bemore independentfinancially and, therefore,havebecomemore

marketoriented.The Program’sinputsinto the ITN centershavevariedover time

andby regionandcountry.Especiallybecauseof thedepartureandfailure to replacethe 1TN Coordinatorin Washington,supportfrom the Programhasdeclinedsubstantially,exceptin Kenya,Zimbabwe,Ghanaandthe Philippines,wherethereis still consider-able collaboration.

A 1994 reviewof the statusof the 1TN centersanalyzedpossiblefuture directionsandprioritiesandconcludedthatanew ITN pro-gramstrategyshouldbe developed.The EvaluationTeamagrees.This strategyshouldbe basedon theconceptthat eachITN center

shoulddevelopits ownobjectives,guidingprinciplesandbusinessplansaccordingto theidentified needsanddemandsin their respec-tive regionandcountry.The Team doesnot seetheneedfor theProgramor anyotherorganizationto coordinatethe different1TNcenters.Coordinationshouldbe left to the initiative of thecenter

managersif andwhentheywant to exchangeinformationandexperience.

The ITN centersmostly needadviceandassistancein specificareas.BecausetheProgramcan beinvolvedat thecountrylevel andcan collaboratewith andbe consideredindependentby nearlyallpartners,it is well-positionedto ensurethatthe ITN centersareincludedin the developmentof nationalcapacity-buildingstrategies.Sincethe Programis working interregionallyandshouldbe familiarwith the activitiesandstrengthsof the variousreference,trainingand researchinstitutions,it shouldalso be ableto advisethe centersas to wheretheycanget assistanceandadvicein specificareas.

An Evaluationofthe fJNDP—WorldBankWaterandSanitat:onProgram 19

Recommendations

Models should be developedthat demonstrate how toincrease the capacity ofbeneficiaries to operate, repairand maintain their systems withminimal external assistance

It is recommended that:

• The Programadvocateandassistin thedevelopmentof the

capacity-buildingstrategiesin eachcountry;which shouldinclude: aregularreviewof capacity-buildingneedsat all levels,identificationof institutionsandorganizationsthatcanbestfulfilltheseneedsandcoordinationof their activities.

• TheProgramadopta newstrategytowardthe ITN program,basedon the conceptthat each[TN centerdevelopsits ownobjectives,guidingprinciplesandbusinessplansandthat theProgramensurethatthe ITN centersareincludedin thedevelop-ment of nationalcapacity-buildingstrategies.

• The ProgramassisttheITN centersin identifying suitableorgani-zationsfor externalsupport,assistanceandadvice.

• Modelsbe developedthatdemonstratehowto increasethecapacityof beneficiariesto operate,repairandmaintaintheirsystemswith minimal externalassistance.

• Modelsalso bedevelopedon howto makeoptimal useof theNGOsystemwithout creatingparallelstructures.

• The Programfocuson making thelink betweenpilot projectsandlarge-scaleinvestmentprojectsby implementingdemonstra-tion projectsandparticipatingin thepreparationandevaluationof large-scaleprojects.

• The Programensurethatnew developmentsandlearningat alllevels,aroundtheworld, arecapturedandfed into the sector.

• The Programcontinueto emphasizetheneedto involve thediverseelementsof the community,especiallywomen,in theplanningandoperationof waterandsanitationsystems.

• The Programseekways to apply the participatoryapproachinthe sectorat anationalscale.

Note

1. Descriptionsof theexpectedend-of-projectsituationin thisandchapters3—6 havebeenexcerptedfrom UNDP InterregionalProjectINT/92, SectionB, ProjectJustification.

20 An Evaluationof the UNDP-WorldBankWaterandSanitationProgram

Chapter 3

The Effectiveness of National Policies,Strategies and Plans

Participatingcountrieswill havemadesignificantprogressin shift-ing sectorpoliciesandstrategiesawayfromgovernment-driven,top-downapproachestowardsdecentraii~ed,demand-driven,bot-tom-upapproachesthat legitimatizea varietyofoptionsfor provi-sion ofservicesto thepoor.

During the InternationalDrinking Water Supply andSanitation Decade it wasgradually realized that effectiveproject preparation activitieshave to be carried out within asupportive national sector polkyand planning framework

During theInternationalDrinking WaterSupplyandSanitationDecadeit wasgraduallyrealizedthateffectiveprojectpreparationactivitieshaveto be carriedout within a supportivenationalsector

policy andplanningframework.This realizationledto placingincreasingemphasison broadersectorwork, suchas sectorstudiesandpolicy advice,as a majorcomponentof the Program’sactivities.

The Program’sapproachto sectorpolicy is not prescriptive.Whatworksin onecommunitymaynotwork in a neighboringcommunity,muchlessin acommunityon theothersideof theworld. Policy advicehasthusneverbeencodified or setout in asingledocumentor handbook.Rather,it hasevolvedthrougheval-uationsby the ProjectPreparationUnitscarriedout in Africa andAsia duringthe 198Os andin the courseof deliberationsamongsectorprofessionalsat internationalmeetings.As aresult,policyadviceis embodiedin variousresolutions,statementsof principleandworking papers.

The Dublin Statement,which emergedfrom the InternationalConferenceon Waterandthe Environmentin January1992,whenthe currentphaseof the Programbegan,embodiesan emergingglobal consensuson the directionsthatarecritical to makingmajorimprovementsin the sector.It setsout aseriesof recommendationsbasedon four guidingprinciples.Two of thesehaveemergedas thebasisuponwhich to establishpoliciesfor managingwaterresources

An Evaluationofthe tJNDP—WorldBankWaterandSanitationProgram 21

andfor deliveringwater supplyandsanitationservicesin efficient,

The Program’s approach tosector policy is not prescriptive.What works in one communitymay not work in a neighboringcommuni1~much less in acommunity on the other side ofthe world

equitableand sustainable ways.~ These are:

• Waterhasan economicvaluein all of its competingusesandshouldbe recognizedas an economicandas asocialgood.

• Waterdevelopmentandmanagementshouldbe basedon apar-ticipatory approach,involving users,plannersandpolicy-makersat all levels, with decisionsmadeat the lowestappropriatelevel.The challengefor the Programhasbeento work with clientgov-

ernments,externalsupportagenciesandotherpartnersin the sectorto transformthesegeneralprinciplesinto policy andinto practice.

Achievements in policies, strategies and plans

In the 1991—95Programworkplanstherearespecificplansfor 36countries(thoughall 36 do nor appearin everyannualplan).Withintheseplans,therearestatedintentionsfor the Programto helpgov-ernmentsdeveloppoliciesin 16 countries,developstrategiesin 22countriesanddevelopplansin 24 countries.Only four countries

thathaveworkplansareexcludedfrom this categoryof activity. Theevaluationteamdocumentedmanyaccomplishmentsrelatingtotheseitems.

Severalof thepolicies,strategiesandplansfor whichthePro-gram providedassistancewerenationalin scope.Othersaddressedthe regionalor municipal level Andsomeweretargetedto specificprogramaspects,suchas investmentor theinvolvementof women.The Program:• Hashelpedto establishnationalsectorpolicies in Bangladesh,

Benin, Bolivia, Eritrea,Ethiopia,Ghana,India, Malawi, Pakistan,the Philippines,Sri Lanka,Tanzania,UgandaandZimbabwe

• Providedadviceandassistancein thedevelopmentof sectorstrategiesin Benin, Bolivia, Ghana,India, Indonesia,Laos,Malawi, Mali, Nepal,Sri Lanka,Tanzania,UgandaandVietnam

• Helpedpreparesectorplansin Benin, Botswana,Burkina Faso,Ghana,Indonesia,Sri Lanka,Tanzania,ThailandandUganda.Whathavebeenthe effectsof thesepolicies,strategiesandplans?

The ultimatedevelopmentobjectiveof the Programis to increasethe deliveryof safewaterandsanitationto low-incomecommuni-ties, bothruralandurban,primarily throughtheapplicationof low-costtechnologiesandcommunity-basedapproachesto projectorganizationandmaintenance.A primary objectiveis to shift thesectorpoliciesof participatinggoveinmentsandexternalsupportagenciestowardassigningahigherpriority to poverty-focusedpro-

22 AnEvaluationof the UNDP—W,rldBankWaterandSanitationProgram

The ultimate developmentobjective is to increase thedelivery of safe water andsanitation to low-incomecommunities, both rural andurban, primarily through theapplication of low-costtechnologies and community-based approaches

grams,while concurrentlybuilding up governments’capacityto for-mulateandimplementsuchprograms.

The ninecountriesselectedby the EvaluationTeamfor amore

detailedreviewof the Program’sachievementsvary enormouslyinsize,population,form of governmentandeconomiccomplexity.Intermsof size andcomplexityBenin andBolivia representoneendofthe spectrum,India andIndonesiatheother.Amongthesecountriesadoptionand implementationof the policiespromotedby thePro-gramvary widely.

The focusGroupin Pakistannotedthe resistanceof thegovern-

mentand traditionalsectorprofessionalsto acommunity-based,demand-drivenapproachinsteadof thecurrent,centralized,top-down,supply-drivenapproach.AlthoughProgramarticulationof

the government’suniform policy for thewatersupplysectorwasachieved,therewerecertaindifficulties in making it operational.Forexample,the governmentgenerallyexpressedits policy asdirectives,but procedures,rulesandregulationsfor actingon thesedirectiveswerenot well developed.The newpolicy~oncostrecoveryandcom-munity participationhad to be supportedby aframeworkof rulesand regulationsfor effective operation.This wasnot easilyachieved.It wasgenerallyperceivedthat the uniform policy hadnot beenthatwell publicized,andthatawarenessof it shouldbeenhancedsothat

all partiesconcernedcould work in unison.The participationof thecommunityandthe costrecoveryissueswere not advancedby thesystem,thusencouragingover-design,inefficiencyandlackof trans-parencyin the bidding,contractingandaccountingsystem.

The Pakistanexampleillustratesboth theProgram’saccomplish-mentsand theproblemsconfrontingpolicy reformandimplementa-tion in mostcountries.However,the EvaluationTeamfoundinstancesin which countriesassistedby the Programprogressedinconceivingandimplementingpoverty-orientedpolicies.A notewor-thy exampleis Benin, which hasachange-orientedgovernmentandseniorcivil servantsin thewaterandsanitationsector.SupportedbytheProgram,Benin officials havebeenableto makesignificantprogresstowardimplementingnationalpoliciesbasedupon theDublin principles.

Gainshavealsobeenachievedin Ghana,wherethe government

hasadoptedadecentralizationpolicy thathasled to thecreationof110districtassemblieswhich havethe legal andadministrativeauthoritythattheyneedto function. Newpoliciesandimplementa-tion strategieshavebeendevelopedin collaborationwith the RWSGfor both the ruralwaterandthe environmentalsanitationsub-sectors.Thesestrategiestakeinto accountthe demand-driven

AnEvaluationof the VNDP-WorldBankWaterandSanitationProgram 23

Supported by the Program,Benin officials have been ableto make significant progresstoward implementing nationalpolicies based upon the Dublinprinciples

approach,costrecoveryandinstitutionalreformsneededfor effec-tive servicedelivery. TheRWSGhasalsosupportedstudiesof thelegal statusof waterandsanitationcommittees,includingcommu-nity ownershipof watersupplyandsanitationsystemswithin thecontextof the decentralizationpolicy, to ensuretheimprovedandeffective provision of services.

A notablefeatureof thework in Ghanahasbeengovernmentsupportof privatesectorparticipationin thenationaldevelopmenteffort. Reformsin the waterandsanitationsectortakeinto accountthe activeparticipationof the privatesectoras aproviderofservices.As aresultNGOs andprivateconsultantsandcontractors

are involved in providingimprovedwaterandsanitationservices.One questionin thesurveythatthe EvaluationTeamconducted

was“How would you ratethe Program’sachievements(on ascaleof

1- 5) with respectto helping to generatebeneficialimpactson sectorpolicies in the countriesin which you work?” The Programwasratedat 4 or higherby 63.9percentof the respondents(figure 2).

The surveycirculatedin Asia posedan extraquestion:“howwouldyou ratethe Program’scontributionsto the overall goal ofexpandingaccessto waterandsanitationfor thepoor?”The resultsindicatedthat77.3percentof the respondentssawapositivecorre-

lation betweenthepolicy-relatedactivitiesof the ProgramandtheProgram’scontributiontowardexpandingaccessfor thepoor towaterandsanitationfacilities. AnotherquestionregardingthePro-gram’scontributiontowardgeneratingbeneficialeffectswascross-tabulatedwith addingvalueto Bank or otherdonorinvestmentprojectswith which participantshadbeeninvolved. In this case7S.4

b~wokigWSS accasa for ~ poor(AiIaor~y)

Conibjilig to toreased Iannai andl~htoflorO~

Mug vth~o to Bo* hmstmentPto$ects

Contttivfing to tiueased cokàon~n

~ oreasehd to ~ work

He~to geneorto benefoidm~octsan sect potdas

— —ato-

F-2Reiposdeuts’evàatlo.of theProgram’sadilevemeutagainstselectedoblectives

20 40 60 80100

24 AnEvaluationof the (TNDP-WorldBankWaterandSanitationProgram

percentof the correspondentssawa positiveinfluencebetweenthepolicy-relatedactivitiesof the Programandvalueaddedto theBank’s or otherdonors’investmentprojects.

Comparative advantages of the program

The Program is ideally suitedto bringing together the labor-intensive capacity of the NGOsand communities with theintellectual and financingcapacity of the World Bank

A numberof peoplethat the EvaluationTeaminterviewedexpressedviewson the comparativeadvantagesor strengthsof the Programorits RWSGsrelative to otherWaterPrograms.Someof theseare:• The Programcan be seenas acatalystor neutral facilitatorat the

countrylevel. It is not seenas an instrumentof the Bank.Manyof thewater andsanitationsolutionsaretoo small for the WorldBankandtoo largefor smallorganizations.Solutionsrequiretheintellectualandfinancialcapacityof the Bankandthe labor-intensivecapacityof theNGOsandcommunities.The Programisideally suited to bringingthe two together.

• The RWSG’shavea uniqueability to concentrateglobal informa-tion and thendisseminateit locally. NGOs aretoo fragmentedtohavean impacton countryandsectorpolicy strategiesandplans.The Bank cannotdealwith micro-economicproblems.The Pro-gramcan.

• TheProgramconcentratesglobalknowledgeof watersupplyintoasmall,easy-to-accessinstitution.

• The Programis focusedon WaterandSanitationfor thepoor in

rural andmarginalizedurbanareas.• The Programis ableto facilitateexchangesof informationand

documentationrelatingto internationalexperience.• The Programhascredibility acquiredover time andwith the

establishmentof trustingrelationshipswith keysectorpeople.• The Programhasthe ability to carryout relevantresearchlocally

and internationally.• The RWSGstaffarecolleagues,partners.Theyplay a supporting

role, theydo not hideanything.The RWSG’sapproachis vision-ary and inculcatesself-reliance.The RWSGbringsauniqueglobal perspectiveto bearon local waterandsanitationissues.Itpreparesanddisseminates(sometimesinadequately)high-quality,leading-edgeinformation.The RWSGhasbeneficiallinks to theWorld Bankandother donors,but unlike the Bank,is moreableto influencethe governmentconstructivelyin waysthat do notrankle or offend.

• If the Programhadnot existed,sectoraccomplishmentswouldhavetakenconsiderablylonger,or theresultswould not have

An EvaluationoftheVNDP-.WorldBankWaterandSanitationProgram 25

The Program and itsdecentralized RWSGs havemany of the qualities necessaryfor working with governmentson policies, strategies and plans

beenas satisfactory.No other organizationcould haveplayedtheroleof the Program.As theseviews indicate,the Programandits decentralized

RWSGshavemanyof the qualitiesnecessaryfor workingwith gov-ernmentson policies,strategiesandplans.It hasthe right size andmodeof operation.It hasaccessto andhasdemonstratedthat it can

work with the full spectrumof organizations,governments,NGOs,the United Nations group, the World Bank and the regional banks

andbilateraldonors.And it canbring a globalperspectiveto thelocal level.

Future issues

The people with whommembersof the EvaluationTeammetofferedgeneralsuggestions.Theseinclude:• TheProgramshouldretain its focuson poverty alleviation.• The Programshouldstrengthenthe links betweenwaterandsan-

itation andmovebeyondthemto includedisease,hygieneandhealth.

• The water andsanitationsector:thouldbe developedpayinggreaterattentionto demographictrends,especiallyin relationtourbanandpen-urbangrowth.

• A higherpriority shouldbe assignedto sanitationandsolid wastemanagement.

• Waterresourcesin their entiretyshouldbe managedon acom-prehensive,integratedbasis.

Basedon all of theaboveobservations,the EvaluationTeamrecom-mendsthat:• The Programidentify on a country-specificbasisthe reasonswhy

new policiesare not alwaysbeingimplementedon acountry-specificbasisandwhat role the rrogramcanmostappropriatelyplay in eachcase.

• The Programdevelopas soonas possibleagroupof casestudiesbasedon countriesthat haveimprovedthe well-beingof theircitizensin general,especiallythepoorest,as a resultof havingdevelopedandappliedpolicies,strategiesandplansbasedon theDublin principles.

26 An Evaluationof theUNDP—WorldBank Waterand SanitationProgram

• TheProgramcontinueto help countriesdevelopandapplypoli-cies,strategiesandplansbasedon the Dublin principles.

• The Programsponsorwell-thought-outstudytoursfor politiciansandseniorsectoradministratorsfrom countrieswishingto adoptnewpolicy frameworksto countriesthathaveone.Thesecouldbefollowed by acontinuingmentoningprocesswhereeffectiverelationshavebeenestablished.

• Giventhe enormousdisparitybetweenthenumberof peoplewhohaveaccessto potablewaterand the fewerwho haveknowledgeof appropriatehygienepracticesandaccessto ade-quatesanitationfacilities,the Programassignahigherpriority tohygieneandsanitationthanit hasin the past.

• The Programrecognizethat individual capacity-buildingthemesor solutionssuchas Womenin Development(WID), technologytraining,policies,strategiesandthe Programitself aretools thatarenot universallyapplicable.

An EvaluationoftheUNDP-WorldBankWaterandSanitationProgram 27

Chapter 4

Impact on Domesticand External Investment

Qualitative improvementswill havebeenmadein thedesignandimplementationof large-scale,sustainable,investmentprojectstar-geting therural andurbanpoor, fundedby theWorldBank,theRegionalDevelopmentBanksandotherdonors,asa resultofPro-gram inputs.

The nature of investmentdecisions is generallythe result of the conjunctureof many forces

The goalof promotingdomesticandexternalinvestmentis closelylinkedto the Program’sobjectivesof buildingnationalandlocal

capacities,disseminatingexperienceandinformationandfacilitatingsectorcoordinationandcollaboration.It is alsorelatedto the Pro-gram’spolicy andmanagementandeachcountry’sconditionanddevelopmentpolicy or strategy.

Investmentsin the sectorcanbe groupedinto variouscategories:physicalinfrastructure,humanresourcesdevelopment,institutionbuilding andtechnologydevelopment.Investmentscanalsobeviewedaccordingto thevariousstagesof projectdevelopment—preparationandformulationof the project,implementation,or con-struction—aswell as supportingactivitiessuchas preparationof thecommunity,training of personnel,scalingup of pilot or demonstra-tion projectsandsoon. Investmentscan alsobe definedin relationto otherstakeholders:collaborationwith the World Bankor otherbilateraldonorsandcoordinationwith localandnationalgovern-ments,theprivate sectorandcommunities.

Becauseof thiscomplexity,andbecausethereis no effectivebase-line againstwhich to measureoverallimpact, it is not easyto assessthe impactof the Programon domesticandexternalinvestments.Itis alsodifficult to discernwhetherchangesin acountry’swaterandsanitationsectorpoliciesor investmentsaredirectlyrelatedto the

28 An Evaluationofthe UNDP—WorldBankWaterandSanitationProgram

The Program has hadconsiderable positive impact oninvestments in the participatingcountries, but there have beenmissed opportunities

Program.The natureof investmentdecisionsis generallythe result

of the conjunctureof manyforces.In preparingfor large-scaleinvestments,the Programfollows the

World Bank’s project processandprinciples:capacity-buildingis

treatedas an integralpart of projectsratherthanasan add-on;tech-nical assistanceandborrowerandbeneficiaryinvolvementare

soughtat everystageof the projectcycle; andexplicit risk manage-ment links theextentof commitmentto a structuredprocessof

learning.As a consequencethe Programhasbeenamongthosepioneering

a new projectcycle,a four-stageprocessthat consistsof:• Listening—Developingthe frameworkfor preparationby listen-

ing to the needsandaspirationsof thebeneficiariesandthegovernment.

• Piloting—Testinglocal leadershipand identifying developmenttasksthatareexpectedto be replicatedat a laterstage.

• Demonstrating—Bringingpilot projectsto a larger-scaleandpro-viding opportunitiesfor the fine tuningandadaptationof project

conceptsbeforelarge-scalereplication.• Mainstreaming—Transformingdemonstrationprojectsinto a

sustainableandcontinuingnationalprogram.

Program results

TheEvaluationTeamfound thatthe influenceof the Programoninvestmentsshouldbe differentiatedaccordingto whetherthe influ-enceis direct or indirect.A direct impactis definedas direct involve-menteither in supportiveactivitiesor in actualimplementation.Indirectimpactincludesinfluencingdonorsandfinanciersto makelargerinvestmentsand influencingtheir approachto startingandimplementingnewprojects.

The Programundoubtedlymissedanumberof opportunitiestoencourageinvestmentby not, for example,pursuingmorepersis-tently the implementationof sectorpolicies in Pakistan.In manyinstances,however,theProgramhasbeeninfluential or ableto fol-low throughon opportunities,andthe impactof the Programoninvestment,whetherdirect or indirect,hasbeenpositive.In just theninecountries’visited by the EvaluationTeam,theProgramhascontributedthroughits involvementin pilot projects,projectdesign,monitoringor strategicsupervisionto somethirty-five large-scale projectswith a total costin excessof $2.6 billion, with donor

An EvaluationoftheUNDP—WorldBankWaterand SanitationProgram 29

In the rural water supply sector,the Program has chieflyinfluenced World Bank projects.Results from the survey showedthat, overall, the Program hada medium-to-high level ofachievement in adding valueto investment projects

contributionsof morethan$2.1 billion, andatotal beneficiarypop-ulation of morethan53 million people.

Other examplesof the positiveimpactsof the Programon invest-

mentinclude:• Coordinationandcollaboration with donors—TheProgramhas

beeninvaluablein assistingdonorcoordinationandcollabora-

tion. For example,it hassecuredthe confidenceof thedonors(aswell as the Government)in Benin andas aresulthashelpedgen-eratenewexternalinvestmentin thatcountry.The Programhas

alsohadan impacton the approachesthatdonorsandtaskman-agersat the World Bankhavetakento projectdesign,largelythroughthe Bankstaff’s contactswith thedecentralizedRWSGs.

• Introducingnewapproaches—Newapproachesthathavebeen

developedby the Programincludecommunityparticipationin allstagesof projectdevelopment,communityfinancingschemesandspecific instancesof low-costsanitationbeingincludedin water-supplyprograms.Someof thesehavebeenadoptedby donorswhohave,as a result,takensimiLar approachesto projectsinwhichthey investin thewaterandsanitationsector.

• Supportto generatenewinvestnunts—Inaddition to developingstrategiesthatgeneratednew investmentin Benin ($14 millionfrom theWorld Bank,DANTDA andotherdonors),theProgramhassupportedthepreparationandformulationof large-scalepro-jects in Bolivia, Ethiopia,Nepalarid Pakistan;helpedin theplan-ning, supervisionandappraisalof ongoinglarge-scaleprojectsinEthiopiaandthe Philippines;assistedin a$125million SectorInvestmentPlanin Ghana;andaidedin thepreparationof aSocial

Action ProgramandCommunityInfrastructureProjectin Pakistan.• Direct involvementin projects—TheProgramhasalsobeendirectly

involvedin anumberof projects.Theseincludea $2.4million ruralcommunitydevelopmentprojectin Bolivia, projectsin Ethiopia,Ghana,Indonesia andUgandaanddemonstrationprojectsin Indiaandthe Philippines.It hasassistedin scalingup projectsinBangladesh,Ghana,India, PakistanandSri Lanka.In addition,theProgramhassupportedthelocalmanufactureof handpumps,thedistributionof sparepartsand,in Benin andUganda,helpedtoincreasethecapacityof privatesectorenterprises.

Donor interviews and survey results

The responsesof donorsandothersectorpartnersto aquestion-naireindicatethatthe impactof the Programon investmentshas

30 AnEvaluationoftheUNDP—WbrldBank WaterandSanitationProgram

It will be more strategic for theProgram to focus primarily onbuilding capacity for sectorinvestment

beenmostly indirect. In the ruralwatersupplysectorthe Programhaschiefly influencedWorld Bankprojects.But respondentsacknowledgedthat, throughits involvementin the field, thePro-gramcould directly or indirectly influencethe strategyandapproachesadoptedin projectformulationandpreparation,aswell

as the implementationof large-scaleprojects.In SouthAfrica, forexample,a respondentto the evaluationquestionnairestatesthat,

“...the Mvula Trust, afinancingfacility managingapproximatelyR260million ($72 million) hasbeenfundamentallyinfluencedbyapproachespromotedby the Program.We in turn havea significantinfluenceon the SouthAfrican Government’sbudgetof R730mil-lion ($203million) for CommunityWaterandSanitation.In one

country alone,therefore—acountrynot evenmentionedin its cur-rent CountryAction Plans—youhaveaPrograminfluenceoveraverysignificantportionof public finance.A [complete] calculation

of the scaleof Programinfluencedirectly andindirectly on institu-tions andfinanceflows wouldrevealaremarkablepicture...” Otherdonorsexpressedthe view thatthereis scopefor the Programtoplay amorepositiverole in coordinatingandstreamliningpoliciesandprojectapproaches.

Somebilateraldonorsalsobelievethat theProgramhashadlit-tle effect on their programs.For them,the RWSGswere not suffi-ciently linked to their work or to that of theWorld Bankandother

donors,possiblydueto a lack of communicationandcoordination.This hassometimesresultedin minor conflict or disagreementbetweenProjectOfficers/TaskManagersandthe RWSGstaff. Sucha lack wasalsofelt in somecountriesbetweenthe Programandlocalgovernment.

Otherdonorsfelt that theRWSGsservedas their in-house“cen-tersof excellence.”Theyhad,theserespondentsbelieved,helpedgreatlywith studiesandprojectpreparation.Resultsfrom the surveyshowedthat, overall, the Programhadamedium-to-highlevelofachievementin addingvalueto investmentprojects.Of the 100responseson thequestionof whethertheProgramwasaddingvalue

to investmentprojects,70 percentsaidthat it hada positiveimpact(seefigure 2).

Future issues

A clear conclusion from the evaluationof the Programis that theUNDP—World BankWaterandSanitationProgram,notwithstandingsomeproblemsandshortcomings,hashadapositiveimpacton

An Evaluationofthe UNDP—WorldBankWaterand SanitationProgram 31

In South Africa the Programhas an influence over a verysignificant portion of publicfinance for community waterand sanitation

investmentandthe approachto developmentof the sector.But, theProgramis engagedin awide rangeof activities.A morefocused

andstrategicrole within the frameworkof the upcomingGlobalWaterPartnershipmight helpincreasethe rateof investment.ShouldtheProgramlimit its assistanceto suchsoftwareas sectorpolicy development,humanresourcedevelopmentandcapacity-building?Or shouldit also beinvolvedat theprojectlevel: in theformulation,implementation,monii:oringandevaluationof pro-jects; developmentof pilot anddemonstrationprojects;andthe up-scalingof activities?Buriedin thosequestionsis afundamentalissue:Whois theProgram’sclient? Whoshouldit assist:thecoun-

try, the World Bank, or otherdonors?It will be morestrategicfor theProgram,given its currentstruc-

ture,to focusprimarily on building capacityfor sectorinvestment.Itcan playamajor role by advocatingstate-of-the-artapproachestoinfluencegovernmentpoliciesanddonors’lendingor assistancepro-cedures;by informal coordination;andby fosteringpolitical willamongdonorsandcountriesfor demand-driven,community-basedinvestmentprograms.

Recommendations

Thereis astrongneedfor demonstrationprojectsto testcommunity-wide applicationof waterandsanitationsectorpoliciesandapproaches,Theseareexpensiveto implementandoftenexceedlocal financingcapacity.But theyareessentialprecursorsto the large

investmentprojectsthat lenderssuch asthe World Bankareorga-nizedto support.The Programshouldadvocateto the World Bankandtheregionaldevelopmentbanksthatwaterandsanitationdemonstrationprojectsbe financedandmonitoredunderproject

preparationfunds.The Programshouldgive greaterpriority to preparationof

investmentprograms,which includespreparingprojectproposals,financingschemes,developingpilot anddemonstrationprojectsandupscalingpilot or demonstrationprojects.This can beaccomplishedby providingtraining or capacitydevelopmentin the variouscoun-triesregardingtheseaspects,rather thanrelying on the Program’sregionalofficesto undertakethem.

The regionalstaff shouldnot be in-houseconsultantsfor thecountriesin which theyarelocated.Theymustcontinueto focusonthe objectivesof the Programwithin eachcountryandleaveonce

32 An Evaluationofthe VNDP—WDrldBankWaterand SanitationProgram

they haveachievedtheseobjectives.Otherwise,the Programwill bespreadtoo thinly andwill loseits uniquepositionandcompetence.

Note

1. Benin, Bolivia, Ethiopia, Ghana,india,Indonesia,Pakistan,thePhilippinesandUganda.

AnEvaluationof theUNDP—WorldBank Waterand SanitationProgram 33

Chapter 5

Systematic Learning and Exchangeof Experience and Information

A systematiclearningprocessfor te.~tingnewapproachesto sectordevelopment,monitoringthe resultsandfeedingthembackinto thesectordevelopmentprocesswill havebeendevelopedand institu-tionalizedin mostof theparticipatiiig countries.

Systematic learning forcapacily-building must takeplace mainly at the pilot anddemonstration levels, and thenapplied in large-scaleinvestments

The EvaluationTeamfoundthat systematiclearningandexchangeof

informationis the weakestelementof the program.No clearconceptor strategyto addressthis importantelementof capacity-buildinghas

beenestablishedwithin or outsidethe Program.Referringto thecapacity-buildingmodel(seefigure 1) introducedin chapter2, sys-tematiclearningmusttakeplacemainly at thepilot anddemonstra-tion levels,and thenappliedin largescaleinvestments.

The Programhasyet to define “systematiclearning”adequately

in eitherconceptualor operationalterms.Referenceto it canbefound in differentsituationsanddocamentsin suchforms as “actionlearning,” “learningagenda”and“learningapproach.”In a few

casestheconceptof systematiclearningwasaddressedas“structuredlearning,”wherebyprojectdesignandformulationisbasedon hypotheses,assumptionsandindicators.Thereis muchmoreto learnfrom ongoing projectsin whichtopicsandquestionsare identifiedin advancefor projectlearning.Thismight leadtomodificationof the projectsthemselves.The goal in thecurrentstructuredlearningagendais systematicallyto learnparticipativeapproachesto projectsthatinvolve stakeholdersin decisionsandsubsequentimplementation,monitoringandevaluation.

In 1994,the RegionalWaterandSanitationGroupfor EastAsiaandthePacific (RWSG-EAP)helda workshopto helpparticipantsidentify andclarify issuesfor systematiclearning.Thisworkshopledto productionof a draft technicalpaperon strategicmonitoring

34 AnEvaluationofthe [JNDP—WorldBankWaterandSanitationProgram

The extent of the influence ofdissemination has never beenassessed either in terms of itsoutreach or of its contributionto improving the well-beingof the poorest

conceptsandtools. Learningactivities in the regionnow focuspri-marily on strategicmonitoringof watersupplyandsanitationpro-jectsandon the role of informal institutionsin the sector.In thePhilippinesthe currentUNDPprojectmanagedby the Programisplanningsystematicallyto documentlearning.It is hopedthat theRWSGwill providea consultantfor thiswork. Work hasstartedonpreparinga learningagendasothatstructuredlearningcanbe used

effectively. In IndonesiatheWaterSupplyandSanitationfor Low-IncomeCommunities(WSSLIC)projectis an exampleof aprojectthatprovidesan ideal substratefor structuredlearning.A numberofhypotheseshavebeendevelopedandarebeingtestedwith the aimof identifying, and thenreplicatingsuccessfulpracticesin otherpartsof thecountry~In Nepal systematicreviewswereundertakenin thePeoplesWaterandSanitationProject(JAKPAS) thatwereusedtofield testservicedelivery optionsfor rural watersupplyandsanita-

tion. This projectis alsoonein which beneficiariesor thecommuni-ties participatedin all the phasesof the project.

Interviewswith anumberof peoplein thewaterandsanitationsectorin differentcountriesindicatethat the Programis learninglittle from projectsoutsidethoseof the UNDP andthe WorldBank.The belief is that thereis muchmorethat could be learnedfrom others,especiallywherepilot projectshavebeenimplementedandcompleted.It is felt that theProgramshouldbuild on what hasbeenlearnedfrom suchpilot phases,ratherthan

managenew pilot programs.The Program’s1995 work plan for Indiafocuseson understand-

ing the institutionaldelivery optionsfor providingwatersupplyandsanitationservices.A numberof casesin which efficient andsustain-ableserviceshavebeenprovidedbothby informal andformal gov-ernmentagencies,NGOs,the private sectorandcommunityself-helporganizationsarebeinganalyzedanddocumented.The keyelementsthatcontributedto their successarebeingdocumentedas

“caselets”so thatlessonslearnedcanbe usedandreplicatedelse-where.Thesanitationworkshopin Harare,ZimbabwesupportedbytheEastAfrica RWSG,is anotherexampleof the transferof suchlearningor’ aregionallevel.

Information dissemination

During the 1992—95evaluationperiod,theProgramhascarriedout agooddealof analysisandresearchwhich haveled to the publicationin headquartersof 47 differentstudiesandpublications.The RWSGs

AnEvaluationof theUNDP—WorldBankWaterandSanitationProgram 35

Information dissemination hasnot been integrated into thestructured learning program

collectively haveproducedanother46 publications,34 iii English,8in Frenchand4 in Spanish.

The Programwas to developsev~ralcommunicationvehiclesfordisseminationandto allocateadditionalresourcesto ensurethat thehigh-qualityinformationproducedby the Program,andcollectedsystematicallyfrom othersourcesreachestheintendedaudiences.But, the staffof the InformationDisseminationUnit at headquartershasdeclinedfrom five peopleto two, althoughathird personisaboutto be added.Someservicesthat theunit usedto provide,suchas the UNESCO-compatibleMicro isis programwhich wasset upandusedspecificallyto facilitate accessto technical,scientific andothersectorinformationrecordedin books,journals,studies,reports,etc., havebeendiscontinued.Nevertheless,theProgramdoeshaveamasterpublicationsdistributionlist in placewhich hasgrown from someS,000namesin 1987 to about8,000namesatpresent.In 1994,the DisseminationUnit distributedselectively17,770publicationsto peopleandInstitutionson its distributionlist.In addition it sentout publicationsin responseto another9,150generalrequests,mostof which emanatedfrom individualsandorganizationsin the field.

The approachof the DisseminationUnit hasevolvedover theyears,but the Programmanagementdid not agreeuntil recentlyto

implementanydisseminationstrategyproposedby its staff.As aresult information dissemination hasnot beenintegratedinto thestructuredlearningprogram.Therehavebeendifferencesof opinionbetween Program management and the DisseminationUnit regard-ing who the targetaudienceshouldhe andwhatmight be thebestmeans to reach it. Currentdisseminationtoolshavenot beenselectedanddesignedfrom the groundup on thebasis of a strategi-cally selectedtargetaudience.Thee~tentof the influenceof dissemi-

nationhasneverbeenassessedeither in termsof its outreachor ofits contributionto improvingthe well-beingof thepoorest.

Somestaff expressedthe viewthat the Programhasnot usedstafftalentswell, andthatthe generalaudiencefor waterandsanitationinformation hasnot beenwell servedby currentpublications.Bud-get cutswhich led amongstotherthingsto the eliminationof thepositionof the personresponsiblefor the Micro IsisProgram,haveadverselyaffectedthe publicationanddistributionof information.On theotherhand,over64 percentof the respondentsto theevalu-ation questionnaire(seefigure 2) ratedpositively the usefulnessofthe Program’spublicationsto their work.

Headquartersandthe RWSGsdo not coordinatefully with

respectto the publishinganddisseminationof information.Head-

36 AnEvaluationof the TJNDP—WorldBank Waterand SanitationProgram

quartersdoesnot knowwho is on the RWSG’sdistributionlists andvice versa.But this is aboutto be corrected.

Therehasneverbeenasystematicassessmentof how recipientsusethe informationthatthe Programpublishesanddisseminates.Therearenumerousunsolicitedthankyounotesandreferencestothe free informationthat theProgramhason hand,muchof whichis not alwaysavailableeitherat theWorld Bank bookstoreor intechnicallibraries.

It is important that the program Documentation of successful projectsnot only disseminates, but thatit communicates

The Programhasproducedseveralpublicationson the transferof

replicablemodels,but therehasnot beenanyactive follow up tolearnwhetherthe materialsproducedwereused.A wealthof lessons

andexperiencehasbeengathered,but it is not adequatelydissemi-nated,communicatedor evaluated.The teamfound thatEnglishisthemostwidely usedlanguagefor documentationof lessonslearned.Consequently,non—Englishspeakingcountriesmusttrans-lateandadaptthe disseminatedmaterials.It is importantthat theprogramnot only disseminates,but thatit communicates.

Both theEvaluationTeamanda numberof peopleinterviewedobservedthatuseof booksor literaturehasbeentheProgram’smaintool for dissemination,it was alsoobservedthat in manycasesthese

publicationsdo not reachthe intendedbeneficiaries.It is importantto focuson identifying targetaudiencesandhowtheymaybereachedusingavarietyof widelyavailablemediatechnologies.TheProgramcould explorethegreateruseof video andthe Internetfor

disseminatingandcommunicating.In today’sworld speed-of-transferof knowledgeis alsocritical to effectiveuseof resources.

In many countries country staff haveexamplesof their bestprac-tices,which could be documentedanddisseminatedto othercom-munitiesor countries.In severalothers,staff arereluctanttodocumenttheir bestpracticesfor fear thatthey wouldbe perceivedas not meeting a Bank standard and would then be rejected. In othercasesprojectstaff timesimply hasnot beenbudgetedor reservedtorecordprogress,difficulties andlessonslearned.

Transfer and exchange of experience

Programstaffmove in asmallorbit. In the countriesthat the Evalu-ation Teamvisited, manymembersof the staff felt theycouldlearn

AnEvaluationofthe UNDP—WorldBankWaterand SanitationProgram 37

Communities often do not knowwhat their neighbors are doingto solve problems similar totheirs

more andcould communicatewhattheyaredoingin their homecountriesto othercountriesif theyweregiven the opportunity.Such communication would not: only benefit the countries to which

staff are transferred, but would also be an additional incentivetothe staff themselves.

Theteamalsonotedthat cornmurtitiesoftendo not know whattheir neighborsaredoingto solveproblemssimilar to theirs.Know-

ing that a newmethodhadbeentried’ andacceptedin a neighboringcommunitywould influencethosewho havebeenresistanttochange.Studytourswould be oneway to accomplishthis. In thePhilippines,communityvisits by individualsfrom aneighboringareahavebeenoneof themostsuccessfulpractices.

In somecountriesdisseminationof informationis carriedout byindividuals and personal contacts. In several instances a few willing

andperseveringindividualswho havedevelopedinterestin sectorissueshavebeenrelied uponalmostetcclusively.Whentheyaretransferred or leave the sector or the Program, there is a breakdown

in the transfer of information andknowledge.Thereis a needfor asystematic process involving a~many stakeholders as possible to

ensure continuity.

III~~~vuullI~uIuu I IVIIJ

TheProgram should develop a strategy for systematic learning fromexperiencewithin andoutsidethe Program.This shouldtakeaccountof thedifferentrequirementsat thelevelsof pilot, demonstrationandlarge-scale investment projects. It should include at least:

• A definition of the Program’s approach to structured learning as

an operational concept and process• A conceptof howto ensurethat learningexperiencesfrom around

theworld arecapturedandcommunicatedas rapidly as possible.The Programcould makemoreinnovativeandcreativeuseof

available information-sharing methods and technologies, such as an

interactive electronic mail (e-mail) dialogue among regions or

portable audio-visual demonstrations using video cassettes. It couldbe usefulfor the Programto maintainan electronicbulletinboard,the equivalent of the telephone yellow pages, with the names and

coordinates of who knows what.The Program should develop a system for measuring the number

of users and usefulness of the information that is being

disseminated.

38 An Evaluationofthe UNDP—WorldBankWaterandSanitationProgram

Chapter 6

Sector Coordination and Collaboration

Sectorcoordinationwill havebeensignificantlyimproved.

Cooperation among countriesof a region and amongdifferent regions is neededto accelerate the learningprocess throughout

Sectorcoordinationis oneof the mainobjectivesof the Program.

Coordinationis requiredat different levels.First, thereis aneedforcollaborationandcoordinationamongdifferentgovernmentagen-cies.This is verydifficult to achievesincethedifferentagenciesactive in thesectorareusuallylocatedin differentministries:health,public works, local government,environmentandso on. Second,thereis a needfor coordinationamongmultilateralandbilateralexternalsupportagenciesandNGOs,which oftenhavedifferentagendasandapproaches.Third, coordinationis urgentlyrequired

betweenthe nationalsectoragencies,on the onehand,andthe ESAson theother. Finally, cooperationamongcountriesof aregion andamongdifferentregionsis neededto acceleratethe learningprocessthroughout.

The UNDP Interregionalprojectdocument,which formsthe basisfor the Program’sactivitiesduringthepresentprojectcycle, statesthat the promotionof closercollaborationat thecountrylevel amongsectoragenciesandamongESASandNGOsactive in thesectoris animmediateobjective.However,in contrastto theotherimmediateobjectivesenumeratedin the document,the Programhasnot beengiven specificdirectionson howto accomplishthis. Only “possibleapproaches”havebeenmentioned.In aspecialnoteit is pointedoutthat, “further consultationswith governmentsandamongUNDP, theBankandthe Program’sbilateralpartnersareneededto defineeffec-tive collaborativemechanisms.”Thustherewereno firm guidelinessetout for theProgramto follow in the areaof sectorcoordinationandcollaboration atthe nationalandregionallevel.

Governmentsconsiderformal sectorcoordinationat thenationallevel to be their own responsibility.Although this is not

An Evaluationofthe UNDP—WorldBankWaterandSanitationProgram 39

disputed,mostgovernmentsareuinditig it difficult to carryout thisrole. Most needexternalsupport,especiallywhenthereis no agree-mentwithin the governmentregardingwhich agencyshould takethe lead.

To acertainextent,sectorcoordinationis alsoneededat theregional level but few government agencies areableto assumethisresponsibility.Regionalsectorcoordinationmustbe the responsibil-

ity of a leadcountryor an internationalorganizationactivein thesectorandin the specific region.Givenits independentstatus,theProgramis well suitedto play or to supportthis role.

The Programhashadapositiveimpacton sectorcoordinationin allof the countriesin which it hasbeenactive. This effecthasbeenmorepronouncedwherethe Programhasbeenpresentcontinually.

The Program’simpacthasbeenlargelyindirect,eitherthroughpar-ticipatingactivelyin the sectordialogueor through involving andworkingwith differentsectoragenciesandorganizationson specific

projects.Lookedupon as an independentorganizationwith aspecialrelationshipto the Bank,amajorsourceof funding,the Programcan be a channel for informal andnon-threateningcommunicationwithin andamongESAs,NGOs(includingthe privatesector)andgovernmentagencies.But, it hasbeenusingthis advantageto only a

limited degreeandhasbeeninvolved in the formationof joint sectorworkinggroupsin only a few countries.

At theregionallevel, wherethe Programcanhaveamoredirectimpact,it hasinitiated very few sectorcoordinationactivities. WestAfrica, wheretheRWSGhasbegunsponsoringannualmeetingsofthoseactive in the sector,is oneexception.Anotherpositiveandfrequentlypraisedexampleis theRWSG’sactive involvementin aregionalworkshopon preparingandimplementinglargerural watersupplyandsanitationprojectsin Asia,which washeldin September1994 in Colombo,Sri Lanka. Theworkshopbroughttogetherpro-ject managersof IDA-assistedprojects,whosharedexperiences,ideas,problemsandsolutions.

An exampleof wherethe Programhasnotactedsuccessfullyisthe RegionalNetwork for WaterandSanitationin CentralAmerica.This groupwas launchedin 1992to enhancecollaborationandcoordinationamonggovernmentinstitutions,externalsupportagen-cies andNGOs.With the Programtaking the leadagencyrole, the

Regional sector coordinationmust be the responsibility of U Results of sector coordinationlead country or an internationalorganization active in the sectorand in the specific region. TheProgram is well suited to playor to support this role

40 An Evaluationofthe LJWDP—WorldBankWaterand SanitationProgram

Network wasintendedto be supportedby sector specialists from theWorld Bank,UNDP,UNICEF, USAID-WASH, PAI-lO, WHO and1DB. The partneragencies,however,contendthat the Programwasnot able to developan effectiveframeworkof coordination,that itdid not selectanappropriatemanagerandthatit failed to makeuseof existingorganizationsin the region.

Collaboration with other partners

The Program can be a channelfor informal and non-threatening communicationwithin and among ESAs, NGOsand government agencies. But,it has been using thisadvantage to only a limiteddegree

The Program’srecord of direct collaborationwith otheragencies,and especially with other external supportagencies,is mixed. Sinceoneof the main reasonsfor its placementwithin theWorld Bankwas to influencethis majorlender,the ProgramandespeciallyitsRWSGs,havetendedto focuson Bankinvestmentprojects.In somecasesthedangerarosethatthe RWSGshadbecome“servicecenters”

for World Bankstaff taskmanagers,as seemsto havehappenedinPakistan.Consequently,severalgovernmentagenciesandotherESAscomplainthatthe Programdoesnot involve or collaboratewiththemenough.

Thereis alsoa commonfeelingamongthe partnersthatthePro-gramis not learningenoughfrom the experiencesof others.Althoughthe Programhasincreasedits participationin theworkinggroupsofthe Collaborative Councilon WaterandSanitationduringrecent

years,severalESAsnotedduring interviewsthatthe Programappearsreluctantto participateactivelyin multi-agencyworkinggroups.Some intervieweesgaveas an examplethe fact thatthe Programdidnot participatein the developmentof asanitationhandbook.

Recommendations

The EvaluationTeam recommendsthat the Programshouldmoresystematicallyandconsistentlymakeuseof its independentimageby

beingmoreactive in sectorcoordination.This could include:• Assistinggovernmentsin thedevelopmentandnormalizationof

coordinationmechanismsat thenationallevel, particularlyfacili-tating andenhancingcoordinationamonggovernmentagenciesactive in the sector.

• Providingforumsfor informal andnon-threateningcommunica-tion mechanismsamongESAs,NGOsandthe privatesector,andbetween these groups andgovernmentagencies.

An EvaluationoftheUNDP—WorldBankWaterand SanitationProgram 41

There is also a common feelingamong the partners that theProgram is not learning enoughfrom the experience of others

• Taking aleadrole in initiating anddevelopingformalandinfor-mal coordinationandcommunicationmechanismsat the regional

level, especiallyin connectionwith the establiskmentof theregional organizations of the GlobalWaterPartnership.This

position might include clearly defining and disseminating a state-ment of its own role in the sector,thusencouragingthedefini-

tion of the respectiverolesof theotheragenciesin thesector,including UNICEF, WHOandthe regional banks.TheProgramshouldenhanceits collaborationwith otheragen-

ciesandorganizationsactivein the sectorby regularlyinvolving allpartnersin definingProgramprioritiesandactivitiesandworkingmoreactivelywith otheragenciesandorganizationsininternationalworkinggroups,including thoseof theCollaborative

Council, thataredealingwith specificsectorissuesanddevelopingtools for the field.

42 An Evaluationofthe UNDP—WorldBankWaterandSanitationF~’ogiam

Chapter 7

Program Management

Partof the taskfor the EvaluationTeamwasto reviewthe efficiencyandeffectivenessof the Programat headquartersandat the fieldlevel.This chapterdescribesthe resultsof this part of the review,conductedusinga traditionalapproachto organizationanalysis.

Mission

During the first of the regionaltripartite reviewsheldin New Delhi,oneof the participantsaskedwhatwas the missionor purposeof theProgram.Althoughthe staffweresurethattherewasone,theywereunableto find it in a searchthroughthe documentscurrentlyin use.The evaluationteampreparedonebasedon their readingof thepro-ject documentation.Participantsat subsequenttripartitereviewsdis-cussedandrefinedthis statementto producethefollowing statement:“The UNDP—World BankWaterandSanitationProgramseekstohelpcountriesfind betterwaysof providingpoorpeopleaccesstowatersupplyandsanitationserviceson a sustainablebasisandtohelp thembuild the capacityto do this with progressivelylessexter-nal support.To thisend,the Programworkswith partnersat thecountr~ regional andinternationallevelsto build local capacity,introducedemandmanagement,supportsustainableinvestmentsandlearnaboutwhatworksandwhat doesnot,andwhy, anddisseminatethe lessonsof experiencewithin countriesandinternationally.”

As notedin chapter1, the missionstatementdevelopedduring

the evaluationreflectsthe Program’schangein emphasis—fromhelpingparticipatingcountriesto expandandextendservicecover-agein collaborationwith otherexternalassistanceagencies,tohelpingcountriesfind betterways to provideserviceto poorpeopleandto build capacityto do this without externalsupport.Thenewmissionstatementalsocontainsa strategicstatementas to how this

AnEvaluationoftheUNDP-WorldBankWaterandSanitationProgram 43

There is a feeling that there is anew sense of direction undercurrent Program management.Yet, some feel that the Programmay be losing touch withrealities on the ground

goal is to be accomplished.It alsois consistentwith the latestformulationof internationallyagreedobjectivesfor the sector.

As describedin chapter1, the Programadoptedastrategyduringits evolution of usingatriad of linked approaches:building the

capacityof governmentsandpeopleto developandmaintainsys-tems;supportingsustainableinvestments;andexchangingtheknowledgedevelopedin the process.The immediateobjectivesofthe Programas set forth in theInterregionalProjectDocumentanddiscussedin the precedingchaptersareobviouslyvery broadandambitious.The resourcesavailableto the Programcould not permitsignificantprogressto be madeagainstall of themwithin theprojecttime frame.

TheProjectDocumentnotesthatthe emphasisplacedon theseobjectivesvariesfrom countryto countryandregion to region,dependingon acountry’sstageof seccordevelopment,specificneedsandgovernmentpriorities.As notedearlier,the effect is to makeitdifficult to monitorandevaluateachievementagainstthe objectives.For this reason,andbecauseof the learningnatureof the Program,

the ProjectDocumentdoesnot containcriteriafor monitoringprogressmadeagainstthe objectiveson aglobalbasis.As aresulttheEvaluationTeam foundit necessaryto developcriteria againstthedescription of end-of-project situations.With moreclearlydefinedobjectivesandpriorities,Programmanagerswould havefound iteasierto select the systems,staffandstructurerequiredto achieveobjectives,allocateresourcesand, probably, raise funds.

Theteamnotedatensionbetweenthedesireof Programpartnersandstaff to haveobjectives,strategyandprioritiessetcentrallyand,at the sametime,wantingto participatein decisionsthat influencethedirection of theProgram.Thereis afeeling on thepart of manythatthereis a new senseof directionundercurrentProgramman-agement.Yet, somewho welcomethisalsofeel that thenewdirec-tion is not supportedby asenseof ownership—thatthe Programmaybe losingtouchwith realitieson theground.

To managethistension,Programmanagerscoulddevelopstrat-egythroughan iterativeandparticipativeplanningprocess.Thisprocesscouldbeginwith a centrallygeneratedstraw-manstrategybasedon informationgatheredby the tripartitereviewsandthisevaluation.The straw-manstrategycould thenbe examinedinregionalconsultationsto establishspecific regionalneedsand,ifnecessary,helpto reorientit. The consultationscould be run in away similar to theworkshopsheldwith the EvaluationTeam duringthetripartitereviewsand in the countryvisits,andcould includedonorsandbeneficiaries.At the workshopsjoint exercisescould

44 AnEvaluationof theUNDP—WorldBank WaterandSanitationProgram

The Program’s work plan looksonly one year ahead eventhough many individual projectshave a longer time horizon.One reason is the uncertainlyof financing

diagnoseregionaltrendsandprobabilitiesandtheir possibleimpacton the Program.Workshopsessionscould alsojointly createPro-gramoptionsanda mapof decisionsforeclosedandkeptopenbyvariousoptions.The outputfor eachregionshouldbeasuccinct

description of preferred activities and their justification. This infor-mationshouldthenbedistributedamongall regions.

Theseconsultationscould befollowed by a final priority-settingexercise involving Programmanagersandthedonors.At leastonekey person should participatein eachregionalsession.Thismanagermight thenbeaskedto participatein thedraftingor reviewingof thefinal strategydocument.The Programshouldprovide feedbacktotheregionsanddonorson how the informationgatheredin theregional consultations was usedin draftingthe final strategy.

Systems

The Program’smanagersrequireseveralsystemsto managethe Pro-gram. Theseareawork planning,budgetingandmonitoringsystem;aprogrammonitoringandevaluationsystem;andapersonnelman-agementsystem.Thissectionreviewseachof thesesystems.Muchof the informationusedin reachingthe findingsherewasgathereddirectly by the EvaluationTeam.However,this sectionalsodraws

heavily on the observationsand conclusions of an internalauditofthe Programconductedby theInternalAudit Departmentof theWorld Bank during1993_9S.1

Work planning, budgeting and monitoring system

Soonafterthe Programwasrestructuredinto its presentform, man-agementestablisheda comprehensivesystemof work planningandprogramming,monitoringandevaluation.Developmentof theworkplanwas to takeplaceoveraperiodof abouttwo months,beginningwith an initial requestfor annualwork programssentout in March

of eachyear.The requestlaysout the framework,broadprinciplesandthe focusthattheProgramwantsto emphasizein the followingyear,andprovidesagrid of criteria1to be usedin selectingactivitiesfor the Program.Theprocessculminatesin thepublicationof theProgram’sWork Planfor the year.This documentsummarizesthestrategyfor eachcountryandlaysout the agreedtasks,the levelofstaff resourcesassignedto eachandthe sourcesof financing.Thedraft is preparedmainly by countryteams,thenreviewedat regional

AnEvaluationof theTJNDP—WorldBankWaterandSanitationProgram 4S

There is no review at the endof the fiscal year to evaluateprogress against the plan andlink it with the budget

headquarters and, subsequently, Programheadquarters.It reflects

informal consultationswith local repiesentativesof donors,govern-mentagenciesandBanktaskmanagersthattakeplacethroughoutthe year,but areintensifiedat the timeof work programming.Theprocessalsoincludesformal meetingsin Washingtonin April withrepresentativesof Bank operatingdivisions.

During the tripartite reviewscountryrepresentativesrepeatedlyacknowledged being consulted on thework programsfor their coun-tries.Theyalso madethepoint that theywould prefer to beconsultedat theoutseton the contentof thework programs.Theconsultations carried out prior to establishingtheir planswerenot

perceivedto havebeenadequate.Specifically, theyfelt that theywerenot involvedearlyenoughin the process,that the processwas tooformal with too muchtime spenton presentationsratherthanoninteraction,debateanddiscussionandthattheycould not seehowtheir contributionsinfluencedthe settingof priorities. The RWSGsmaynot havesatisfiedcountryexpectationsbecausetheydid notconsultbroadlyenoughor becausetheprocesswasnot clearenough.Regardless,at the conclusionof all of the tripartitemeetings,theRWSGstaffcarriedoutaparticipativeprocessof consultationat thecountryandregionallevel beforedeterminingthecontentof futurework plans.Theyare to be commendedfor their immediateunder-standingandreaction.

Althoughthe presentprocessof planningandprogramminghasgenerallyservedtheProgramwell, oneaspectof the processwarrantsattention:thetimehorizonof the planningcycle.Operationalunits

in mostinstitutionsandenterprises,including the UNDPandtheWorld Bank,havea three-yearrolling plan,a five-yearperspective,or

somesortof masterwork program.In contrast,the Program’sworkplanlooksonly oneyearahead,eventhoughmanyindividual pro-jectshavealonger timehorizon.Thereappearto beseveralreasonsfor this. Oneis theuncertaintyof financing.A secondis thatthe Pro-gramis basicallyacollectionof projects,manyof themvery short-term,reflectingthatdonorshavepreferredto financeprojects rather

thancoreactivities.Longer-rangeplanningmightallow resourcesto be usedmore

effectively andefficiently. This is thecasein theimportantareaofstaffingandpersonnelmanagement.Anotherbenefit is thatit wouldrequireProgrammanagersto considermorecarefully which activitiesare droppedwhennewonesarestarted.The strategicplanningprocessrecommendedabovewill setthe frameworkfor suchlonger-termwork programming.Thenextwork programmingexercisecould

follow thestrategicplanningexerciseandcoverathree-yearperiod.

46 AnEvaluationofthe UNDP—WorldBank WaterandSanitationProgium

There is a risk that the identilyof the Program as being in theBank but not of the Bank canbe lost

Every programmustassessits own performance.Two differenttypesof indicatorsare usedfor thispurpose.At onelevel, inputsandprogress toward completing specificactivitiesor outputsaremoni-tored.At another,indicatorsareusedto monitorprogresstowardachievementof developmentobjectives.Monitoring inputsin mostorganizationsis doneon amonthlybasiswith quarterlyreportingonprogressmadetowardachievingobjectives.Ar the time of thereviewby the World Bank’s internalauditors,thework programfor eachcountryin the Programconsistedof asetof individual tasks,calledproducts,with ataskleader,an identified setof activitiesandoutputsandthe staff resourcesestimatedfor each.Eachtaskwasassignedoneof severalactivity codesthatcharacterizethe typeof activity(sectorimprovement,humanresourcedevelopmentandsoon) andthe typeof agencyto which it is linked (the Bank,adonor,atrainingnetwork institution andsoon). Staffmembersrecordedtheir timemonthlyagainsteachtaskcodein the sameway asregularstaffof theWorld Bank.

Theinternalauditorsnotedthat,althoughthesystemis detailedandcomprehensive,severalareasdeservedmanagementattention.For manyactivitiesthereare no milestonesor key datesto provideyardsticksfor measuringprogressor the completionof an activity.Evenfor the activitiesthathavesuchelementstheseelementsdo notappearin the work program.Individual projects,apartfrom periodicreportsthataremadeto donorsregardingfinancialconditions,arenot the subjectof formal supervisionreports.While budgetresourcesin generalaretightly monitoredandcontrolled,onegapnotedwasthatboth headquartersandRWSGmayincurexpensesagainstsomebudgetitems,eachwithoutknowing what theotheris committing.The samelevel of monitoringandcontrol doesnot appearto be trueof the main operationalinput: professionalstaffweeks.Although theBank’s ManagementInformationSystem(MIS) containstimerecordsfor eachstaffmember,the auditorsdid not observe(with

someexceptions)thatthe resultsaresystematicallyreviewedandanalyzedto seehowtheycomparewith the plan.The categoriesthemselvescould be morehelpful.An exampleis given of a demon-strationprojectbeingimplementedin preparationfor aprojectto befinancedby theWorld Bank. But its codingdoesnot give anyindica-tion of its link to theBank—importantinformationfor the Programto try to quantify.The codingsystemdoestrackthis link whenassis-tanceto theBank is providedduring implementationof aBank-financedproject.Thereis no reviewat theendof the fiscal yeartoevaluateprogressagainstthe planandlink it with the budget.Fol-lowing the 1995Programmanagersmeeting,for example,anaction

AnEvaluationof the UNDP—WorldBankWaterand SanitationProgram 47

The Program should takeresponsibility for helpingestablish baseline criteria to beused in determining where thesector is today and in monitoringits movement over time

planwasproduceddefininganumberof responsibilities.But, field

managersarenot surehowprogressagainstthis planshouldbe mon-itoredor how achievementshould be rewarded.

The 1994 integrationof thePrograminto theWaterandSanita-tion Divisionof theWorld Bank’s Transportation,WaterandUrbanDevelopmentDepartment(TWUWS~hadseveraladvantages,includingthemorerigorousmanagementof the Bankandthe flexi-bility of beingpart of an organizationwith staff from manydisci-plines. But, thereis a risk that theidentity of the Programas being

in the Bank but not of the Bank could be lost. Forexample,the mostrecentstatusreporton thework programof TWUWS3 doesnot

separatelyidentify the inputsandoutputsof the Program.It wouldbe usefulto be ableto determinewhatthe Programis accomplishingon its own andhowmuchcross-supportthereis betweenthePro-gramandthe Division, as well as otherBank units. PotentialPro-gramdonors,as part of the GlobalWaterPartnership,will wanttobeable to determinethis.

Program monitoring and evaluation

As noted earlier, the Program does rLot haveasetof developmentorimpact indicators to measure progress against ultimate goals and

objectives such as capacity-building, sustainability, community par-ticipation, structured learning and so forth. Thereareat leastthreepossible reasons why. First, the Program’s strategy and immediateobjectives were not defined well enough to permit the selection ofcriteria that could be monitored. Second, sector operators have not

agreed on commonor standard criteria for monitoring access to safewatersupplyandsanitation,despitean acknowledgedneed.Third,many Program activities are research anddevelopmentprojects.Indicatorsfor learningprogramswill be different thanthosefor aconstructionor trainingproject.

Nevertheless,theremustbe away for participants,particularlyrepresentatives of funding sources, to determine whether all of this

activity is making a difference in the lives of the disadvantaged.

Continued funding for the Programwill ultimately depend onwhether it developsa set of criteria with which to measure

progressagainstobjectives.Perhapsmoreimportantly, investmentfunding in the sector may weaken unless countries candemonstratethat theyhavereliableassessmentsof thepresentsituationandamethodof determiningimprovementsmeasuredagainst clearly defined criteria.

48 An Evaluationof the UNDP—WorldBankWaterandSanitationProgram

The staff needs to know thatthe Program cares about themand that they will be givenopportunities to develop andgrow

The recommendationmadeaboveconcerningstrategydevelop-ment should result in a clearer and more focused set of objectives

against which to establish indicators.Much work hasbeendonebythe Bankandothersectoragenciesin definingdevelopmentindica-tors for thewater supplyandsanitationsector.4The Programcould

makeit a highpriority to ensurethe acceptanceof suchcriteria bythe internationalcommunityandto encouragetheir use by govern-

ments. These criteria could be used to collect information as partofthe povertyassessmentstudiescurrentlybeingundertaken,manywith World Bankfinancing. Manyof theProgram’sactivitiesareof ademonstration nature or intended for project development. While

criteria for research and development activities are different, they

canbe developed.

Personnel management system

Under the present UNDP—WorldBank arrangement the Programisbound to follow all of the World Bank’s personnel policies andprac-tices. This is unfortunate because the Program is structured quite

differently than the Bank. Most of its staffarelocatedin the fieldandmostarenon-regular.Conditionsof employment,includingtypesof contract,careerplanning,incentives,etc.,shouldbe devel-opedin this context.Poorlysuitedpersonnelpoliciesarepartlyresponsiblefor high staff turnoverandfeelingsof insecurityamongthose who remain. Given theplannedrestructuringof the Program

as apartof the Global WaterPartnership,this governancerelation-ship mightchange.If so, the Programwill be ableto developits own

personnelpoliciesandpractices.

Work processes and procedures

The abovereviewcoversthe majorsystemsthatsupportProgramoperationsat headquartersandin the regionalandcountryoffices.Othersystemsmight berequiredthatareuniqueto certain functions

at headquartersandin the field. A full reviewshouldcoverthese.

Staff and skills

Severaladditionalissuesmustbe addressedwith respectto staffand skills. A principal problem is that there is not enoughstaffat

AnEvaluationof the LJNDP—WorldBankWaterandSanitationProgram 49

There is a consensus amongbeneficiaries and donors thatthis regional structure is animportant asset and theEvaluation Team agrees

headquartersandin someRWSGsto meetcurrentcommitments.The additionalwork required i:o meet the requests of the benefi-

ciariesandthe recommendationsof the EvaluationTeamwouldmake the problem worse. The staff shortage is largely due to a

shortfall in available resources, arising because of a reduction in

funds available from UNDP. Program management hasbegunaddressing the immediate gap through intensive fundraising. It

now appears that the minimum level of funding required for 1996

will be achieved.Management must still carry out a careful assess-mentof whatcan be donewith theseresourcessothat staff will

not be demoralizedby impossibleexpectations.The workprogrammingprocessrecommendedearliershouldhelpin thisrespect. The issue of the optimal size of the Program is discussed

later in this chapter.The valueaddedby theProgramdependsdirectly on the skills

andexperienceof its staff. Most beneficiarycountries have sanitaryengineers, social scientists andrepresentatives of the other

disciplinesfound in the Program.TheyexpecttheProgramto con-tributethe personalexperienceof its staff, accessto theaccumulatedexperienceof the Programas awholeandthe ability to apply thisknowledgeandtheseskills rigorously to the situationat hand.TheProgram’scredibility thusrestson havingahigh caliberstaff. But,

there is some concern over this. One reason is that many positionsare fundedthroughpartially tied bilateralsourcesthat limit choicesin recruitment. Another is the high turnover rate. Until financing

and personnel policy issues are resolved, maintenance of the highest

standardsduringrecruitmentwill remainthe only meansto ensureagenerally high caliber staff.

The Program’sdecisionto drop someactivities,suchashand-pumpdevelopmentandPROWWESS,becausetheyarenow “main-streamed,”hasaddedto the issuecf staffskills. As activitiesaredropped,somestaffwith expertisein thesefieldshavemovedon toassignmentsoutsidetheProgram.But, clientsstill seekthis expertisefrom Program offices. Indeed, the Programitself still needstheseskills on occasion. Care must be taken in recruitmentto makesurethat skills neededby the Programaremaintainedor that thePro-grammaintainsaccessto themwhen the programsarebeingcarriedon by oneor moreof its partners.This is not alwayseasy.Newdirectionsfor the Programsometimesrequire differentskills. Forexample,the emphasison community participationrequirestheskills of afield anthropologist, while the emphasis on structured

learningrequiresamoreacademicapproach.

50 An Evaluationofthe UNDP--WorldBank WaterandSanitationProgram

The most important findings:the Program needs a morestrategic focus lo evolvetowards a true partnership andadjust its planning,programming, budgetingand monitoring arrangements

Staff development through training anddevelopmental assign-ments is one way to broaden the knowledge and experience of staff

members. It would help motivate the staffwhereotheremploymentconditions fail. Staff need to know that the Program cares about

themandthattheywill begiven opportunitiesto developandgrow.Programmanagersrecognizedthisandadopteda planof actionattheir meeting in May 1995. This planneedsto be followed up.Onemeans of developing staff, transferring knowledge andperhapscre-ating moreinterestingconditionsof employmentis to rotate staffamongassignmentsin the RWSGsandthe countries.Thereis scope

for this. Another way of developing staff,assuggestedby theinter-nalauditors,is to recruit, train andmentor youngerprofessional

staffwhobring newskills, suchas thoserequiredto facilitatecom-munity participation.

Staff recruited by the Program should have mastered at least one

of theinternationallanguagesusedin their regionandshouldhavethecapacityto learnothers.The previousevaluationnotedthatcapacityto work in French was missing in the RWSGin Abidjan.This has been corrected. Still, RWSGstaff and representatives of the

countries informed the Team that there is a paucity of communica-

tion and documentation from headquarters in French. Similarly, inEastAfrica the RWSGis not staffed to serve the francophonecoun-

triesin the region.

Management style

The EvaluationTeamheardonemessagerepeatedly—aneedtomake the Program a true partnership in which the beneficiaries,

other ESAs and donors feel they are respected as equals. While not

wantingto managethe Program, beneficiaries and the support com-munity wantto be involved in settingits strategicdirectionandinsetting priorities. But, the general perception is that the Program

managersarereluctantto learnfrom the experienceof othersandtodisseminate this learning. The 1991 evaluationrecommendedthat

one way to recognize the diversity of stakeholders would be tochangethenameof the Programto theJointProgramfor Water

Supply and Sanitation. This was not done. But as part of the newGlobalWaterPartnershipthe Programwill becomea truepartner-ship. A changein namewill signal this clearly.

The previous evaluation also recommended the creation of a

steering committee with a number of management responsibilities

AnEvaluationoftheVNDP—WorldBank Waterand SanitationProgram SI

for the Program.Thisconceptwasrejectedby donors,who felt thatmanagement should be left to those responsible. Instead, an Advi-

soryCommitteewas established, which meets annually to review

progress of the Program. An expanded role for this Committeewould alsosignalcommitmentto partnering.

A closer relationship between the Program and the Collaborative

Council on Water Supply and Sanitation would also help change the

perception that the Program does not learn from others. The Coun-

cil includes most of the Program’s partners in the process of learninghowto bettermanagethewaterandsanitationsector.Betweenbi-

annualmeetings,anumberof Councilcommitteesandstudygroupsexplore sector-related issues. Somestaffof the Programarepartici-

pating in these study groups. And al leastonerepresentativeof theCollaborative Council is invited to meetings of the Program’s Advi-

sory Committee. The Program should examine other means todemonstrate its commitment to the Council.

Whenthe Program was first established, donors were anxious tohelp ensure access to water supply andsanitationfor thepoor as

quickly as possible. Program management was relatively free toundertakealmostanyactivity thatseemedto leadto improvedaccess.Subsequently,requestsfor assistancebeganto comefrom thefield. These were frequently generated by civil servantswho spokefor the beneficiaries without knowing their realneeds.Morerecently,clearcriteria for selectingProgramactivitieshavebeendeveloped and administered centrally. Although thesehavehelped

focusthe Program,somestaffmembersandoutsidersbelievethatthe Program is now somewhat out of touchwith people’sneeds.

Reorienting the relationship between headquarters andtheRWSGS,andbetweenthe Programand theotherpartners,is funda-

mental to achieving the Program’s objectives. The proposedapproach to setting the Program strategy and the recommendedwork planning, budgeting and monitoring system andprocessshould help. The relationships can be further improved by delegat-

ing decision-making to the lowest level in the organization at whichit can be exercised responsibly.

Structure

The Programhasdevelopeda highly decentralizedorganizationalstructurethatincludesasmallandshrinking core of administrativeand technical staff at headquarters, five regional managers together

52 AnEvaluationof the (JNDP—WorldBankWaterandSanitationProgram

Beneficiaries, other ESAs,donors, N60s and even taskmanagers of the World Bankand staff of the regional bureausof UNDP usually judge theperformance of the Programby the competence andresponsiveness of the RWSGs

with regional staff, and teams in each of the major focuscountries.The organizationchartin Annex S showsthestructureof thePro-gram andcurrentcountriesof concentration.A list of theprincipaldisciplines and nationalities of the staff at the close of the evaluation

is includedin AnnexS. The principal rolesof headquartersstaff are:leadingthe strategicplanningandprogrammingandbudgetexer-

cise;providingadministrativeand technicalbackstopping;fundrais-ing; donorliaison andrelatedactivity (includingmarketingtheProgramwithin theWorld Bank);overseeingthe learninganddis-seminationprocess;andfacilitating exchangesof personnelandlearningamongregions.The RWSGstaffcarryout the daily opera-tions of the Program.This structureis generallyworkingwell. Thereis a consensuswithin the Programandamongthe beneficiariesanddonors that this structure is an importantassetof the Programandshould not be changed. The Evaluation Team agrees (subject to

action being taken on earlier recommendationswhich wouldstrengthenthe RWSGs).

The allocationof resourcesto thesecountriesreflectsthework

load involved and the relative costs of supervision. Countries thatdo not havetheir own countryteams,areusuallyhandledthroughthe regional team, which is usually larger andmoreflexibly staffedthan individual country teams. During the tripartite reviews the sug-

gestionwasmadethatthe Programdesignatecountrycontactper-sonswho areknowledgeableaboutthe Program’sactivitiesandcould thusserveas alink to the Program.The EvaluationTeamagreesas long as the chosen person is well-connected and respectedin the sector.

Beneficiaries, other ESAs,donors, NGOsand even task managers

of theWorld Bankandstaffof the regionalbureausof UNDPusuallyjudge the performance of the Program by the competence andresponsiveness of the RWSGs.The competence of the RWSGscould

be improvedwith additionalfunding,as discussedbelow.Anotherimprovementwould beto transfersomeof the functionsandactivi-ties of TWUWS andthe Programto the RWSGS.In doing so, thestaffcarryingout thesefunctionswould be closer to the beneficiaries andtheir needs,provideagreatervariety of disciplinesin the regionsandenabletheRWSGSto be more flexible in respondingto needs.

StrengthenedRWSGscould becomecentersof excellenceforsomespecializationsas well as centers for publication and dissemina-

tion of information.Theyshouldalsocarryout fund raisinganddonorliaison activitieswithin their region,be active in regionalorga-nizationsandwork closelywith the regionaldevelopmentbanks.

AnEvaluationof theUNDP—WorldBankWaterand SanitationProgram S3

Principal issues and summary recommendations

The generalconclusionis thatwhile Programmanagementis ade-quate,therearea numberof areasin which it canbe improved.Themostimportantfindingsare: (i) the Programneedsmorestrategicfocus; (ii) it needsto continueto evolve towarda “truepartnership;” and (iii) ~t needsto adjustits programmingandbud-getingarrangements.Following is a listing of recommendationsdealingwith issuesraisedin this chapter.

Recommendations

Mission1. It is recommended thatProgrammanagementreviewthe mis-

sion statementpreparedby the EvaluationTeamandconfirm itsvalidity or modify it in consultationwith all of thepartnersin theProgramstrategy.

2. It is recommendedthatthe Programmanagersestablishthestrategyfor thenextphasethroughan iterativeparticipativeprocess,beginningwith a tentative,centrallydeterminedstrategybasedoninformationgatheredby thetripartitereviewsandthis evaluation,andby makingoptimum use of the Program’scomparativeadvan-tage.Countryandregionalconsultationsto establishspecificregionalneedswould usethisdraft strategyasbackgroundinforma-

tion andsuggestchangesthatmight bettersatisfytheir needs.Finally, managementshouldfinalize the strategyin consultationwith

the principal partnersof the Program.

Workplanning, budgetingandmoniloringsystem3. It is recommendedthatProgrammanagement:

• Make a greater effort to record and analyze how staff time is

spent. In this it should revamp activity codesto makethemmoreuseful (Management has already set out to do this in response tothe internalaudit report).

• Conductreviews,at leastsemi-annually,but preferablyquarterly,of performanceagainstinput andoutputindicatorsandactivities.Thesecanbe primarily for interrLal use,althoughtheymight becited informally in discussionswith theWorld Bank,IJNDPandother partners.

• RequireeachProgrammanagerto evaluateperformanceof hisstaff in partusingtheresultsof thesereviews.

54 AnEvaluationof the VNDP-.WorldBank WaterandSanitationProgram

4. It is recommendedthatthe inputsandoutputsof theProgrambe identifiablewithin the work program,budgetandreportsof the

Division.

ProgrammonitoringandevaluationS. It is recommendedthatthe Programtakeresponsibilityfor

helpingto establishbaselineCriteria to be usedin determiningwherethesectoris todayandin monitoringits movementover time. It

shoulddo this by:• Developinga clearsetof monitorableindicatorswith which to

measureprogressagainstobjectives.• Makingoneof its highpriority activitiesthe upgradingof systems

to measureaccessto safewatersupplyandsanitationby thepoorin rural andmarginalurbanareas;encouragingotherinternationalagencies working in the sector to adopt a similarapproach;andadvocatingthattheyadoptandusetheseindicators.Oncesuchcriteria exist, theProgramshouldconductwith its

partnersaparticipatoryself-evaluationof progressagainsttheindica-tors for eachof its objectivesduringtheannualwork planningexer-cise.This evaluationshouldincludethe numberof personsservedforeachdemonstrationor large-scaleinvestmentproject.The Program’sAnnual Reportshouldincorporatethe findingsof thisexercise.

Personnelmanagementsystem6. It is recommendedthatthe Vice President,Environmentally

SustainableDevelopment,requestthatthe SeniorVice President,ManagementandPersonnelServices,designatearesourcepersontoassisttheProgramin designingpersonnelpoliciesandproceduresmoresuitableto its organization.The UNDP andthe Bank couldthenbe askedto put thenew policies into effectfor the remainder

of the currentproject.

Otherwork processesandprocedures7. It is recommendedthat the Programmanagerarrangefor a

reviewof work processesat headquartersandin the regionstodetermine whether different work methods andprocedurescould

improve efficiency.The OrganizationandBusinessPracticesDepart-ment (OBP) might be able to assistwith this exercise.

Staffandskills8. It is reconirnendedthatProgrammanagerssystematically

reviewprofessionaldevelopmentneedsandopportunitieswith each

AnEvaluationoftheUNDP-WorldBankWaterandSanitationProgram 55

staff member and jointly agree on an appropriate programbynolater than the next staff performance review.

9. It is recommendedthatProgrammanagementreviewlanguagerequirements for staffing in each region and develop plans to meet

theserequirementsthroughrecruitmentor training. It is furtherrecommendedthatlanguageskills at headquartersbedevelopedsothatheadquarterscancommunicatedirectlywith the regionalstaffs.

Managementstyle10. It is recommended that the Program be named as the Water

Supply andSanitation Program of the Global Water Partnership.

11. It is recommended that the role of the Advisory Committee

be expanded to approving the strategic plan for the Program and

reviewing progress against the objectives each year.

12. It is recommended that the Program encourage its staff to

participate in the committee work of the Collaborative Council by

planningandbudgetingtime for this activity. The formal relation-ship with the Councilshouldbe strengthenedperhapsby formalrepresentation on each other’s governingor advisorybodies.

13.To achievethe senseof teamworkthatis soughtwithin theProgram,it is recommended that the Programmanagerinvolvemanagers and staff of the regional offices and headquarters in

designingand implementingwaysto respondto recommendationsto strengthen the management of the Program.

Structure14. It is recommended that opportunities for further decentral-

izationto the RWSGsbe examinedin the reviewof work processes.Considerationshouldbe given to usingthe RWSGsas facilitatorsinestablishingtheregionalgroupsunderthe GlobalWaterPartnership.Decision-makingauthorityshouldbedelegatedto the regionalman-agerson issuesfor which theycanexercisethis authorityresponsi-bly, thus enhancingtheprestigeandeffectivenessof theregionalgroups.

Notes

1. Reporton an Auditof theUNDP—WorldBankWaterandSanitationProgram, SouthAsiaOperationalFunctions,ConfidentialReport;InternalAudit Department,World Bank; September27,1995.

56 An Evaluationofthe UNDP--WorldBankWaterandSanitationPrognim

2. Annex4 presentsthe criterianow usedin selectingactivities

for the Program.3. TWUWSFY96 Midyear Review; memorandumto Anthony

Pellegrmnifrom PaulaStone;December11, 19954. See,for example,IndicatorsProgram,MonitoringHumanSet-

tlements—KeyIndicators,AbridgedSurvey;Programmeof the UnitedNationsCenterfor HumanSettlements,Nairobi;March 199S.

AnEvaluationoftheVNDP-WorldBankWaterand SanitationProgram 57

Chaptor B

Program FiNancing

Althoughthe Programis an organicandintegratedentity in theoperationalsense,funding by the IJNDPremains tied to the origi-

nal, individual, regionalandmterregionalprojectcomponents.Atthe sametime bilateralsupportis governedby amultiplicity of indi-vidualprojectagreements.Thesefundingarrangementshavegreatlycomplicated financial management andaccounting.The level offundingpeaked in 1991 andhassincedeclinedrapidly, largelybecauseof severeUNDP financialdifficulties (table 1 andfigure 3).From1992 to 1994disbursementsfrom UNDP droppedby about$4.5million, adeclineof nearly50%anda lossof about30%of theProgram’stotal resourcesfrom the 1992 level. During thatperiodbilateral funding increased, but not nearly enough to offset the loss.In 1995,outof total fundingof $11.1million, $6.4million camefrom bilateral donors,$3.7million from UNDPand$0.9 millionfrom the World Bank.

Within thefunding categories,therearealsoconstraintsregard-

ing the useof funds.In the past,funding for thecoreactivitiesofthe Programcameprincipally from UNDP’s interregionalbudget.Thisamountdroppedfrom $4.7million in 1992, to $1.1million in1994, with aslight recovery to $1.3 million in 1995.The balanceofthe UNDPcontribution comes from its regionalbudgetor CountryIndicativeFunds.Thisamountis generallyintendedto supportpro-jectscarriedoutat thecountrylevel, althoughsomeof it maysup-portstaffof the RWSGs.Bilateral donorshavediffering approachesto allocating their funds. In general, they have preferred to supportspecificprojectsratherthancoreactivities.In fact,mostof theirfunding is allocatedby countrydesksthathaveno interestin coreactivities.Thispracticeis in keepingwith a worldwide trendtoshowconcreteresultsto taxpayers.

A significantportionof the World Bank’s contribution is

earmarkedfor direct support to the Bank’s operational activitiesor

58 An Evaluationof the UNDP-WorldBankWaterandSanitationProgiam

From 1992 to 1994,disbursements from IJNDPdroppedabout $4.5 million, adecline of nearly 50% and aloss of about 30% of theProgram’s total resources at the1992 level

is the passingthroughof feesearnedfor the managementof trustfunds.In calendaryear1995,out of the$947,000coming from theWorld Bank, $149,000was for activitiesdirectly supportingWorldBank operationsand$443,000representedfeesreceivedby theBankfor managementof trust fundsthatsupportthe Program.Thusthe Bank’s real contributionto the Program’sactivitieswasonly$355,000or 3.2%of its total budget.’This compareswith $1.0mil-lion (7.7%) in 1993.

Programmanagers have taken steps to bring expenditures in line

with reducedfunding andsomeof the majorbilateraldonorshavepicked up the costs of positions previously funded by the UNDP.Nevertheless, activities could not be cut back far enough or fastenoughto bridgeafunding gap.As aresult,as of early1995 theProgramfaceda substantialshortfall for its regional staffandopera-tions in bothAfrica andAsiaduring 1996.The Programhasexplorednew waysto accessfundingfor activities throughtheendof 1996.It hasapproachedmanyof its donorsto obtainadditionalmoney or permission to use funds already approved or earmarked

more flexibly. Mainly as a result of new commitments of support

from afew donors,the Programhasthe financingneededto con-tinue activitiesatthepresentlevel to theendof 1996.Howeverthecorefunctionno longerhasthe critical massrequiredto assureahigh quality Program.

In recentyearscorefundinghasnot fully coveredcoreactivities,includingmanagementandtechnicalsupportstaffat headquartersandthe regional level, plus someactivitiesat thecountrylevelincluding the team leaders. As a consequence, individual positions

andactivitiesat headquarters and in the regions have been increas-

ingly fundedfrom bilateralsources.Thesecommitmentsareoftenrelatively short-term andtied to thefunding cycle of individualdonors.EvenUNDP corefundingis tied in this manner.Oneconse-quenceof this arrangementis that in 1996the UNDP funding cyclewill endat thesametimeas anumberof bilateral funding arrange-ments.Only bridgefinancingby somebilateraldonorshaskepttheProgram functioning.

The internal audit report notes that the present financial arrange-ment hampers the Program’s effectiveness. Amongother difficulties,

it shortensthework programmingcycle and,mostimportant,requiresthat valuableprofessionaltimeandtechnicalresourcesareused in seeking funding sources. Many high-level staffat headquar-

tersandin the field arefundraisers,beginningwith the Programmanageratheadquartersandincluding regionalmanagersand

AnEvaluationofthe UNDP—WorldBankWaterandSanitationProgram 59

o 0 00 0 0 0 0 0 0 00 0 0 0 79 499 494 476 455 93 4900 0 0 0 0 0 0 0 0 0 860 0 0 0 150 131 36 34 35 0 480 0 0 0 0 0 0 0 0 503 3590 0 0 0 0 40 519 237 388 460 3720 0 00 0 0 0 0 0 0 00 0 0 0 0 0 91 24 60 80 45O 0 00 0 0 0 0 0 0 00 0 00 0 0 0 0 0 0 00 0 00 0 0 0 0 0 0 00 0 0 0 0 0 148 281 323 663 6050 0 00 0 0 0 0 0 0 00 0 0 0 0 0 196 258 401 895 1,0190 0 0 0 0 0 78 191 96 0 0

o 0 0 11 2 30 11 54291 133 83 92 931 29 0 4,152409 137 224 326 686 393 399 2,660123 146 126 0 111 134 0 1,075

15 0 0 0 0 35 80 99266 99 50 66 0 0 44 2,341

0 0 0 0 0 87 26 11335 0 0 0 0 42 231 608

0 0 0 0 343 1,111 1,402 2,8560 0 0 0 122 267 235 624

1~l 699 1,839 2,062 1,519 653 302 7,2058’O 1,276 854 1,348 1,474 972 1,554 10,259

0 0 0 0 202 329 880 1,411344 3~8 1,094 953 904 1,175 1,107 8,714

0 99 180 193 180 0 151 1,166

Agwe3

Donors’ cant r~ot1ons((ISdoIIo,~)

20,000

15,000

10,000

5,000

01978 1979 1980 198I 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995

loWe 1Programdisbursementsby donoi~1978-95(US$thousands)

1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 Total

Bllaten~AgenciesAu~CanodoDenmo6FinlandFranceGennonyIrelandItalyJ~onLuxem~urgThe NethedondsNorweySwedenSwitzerlandUnited Kingdom

Total Vonars 0 0 0 0 229 670 1,563 1,501 1,758 2,695 3,024 2,230 2,907 4,450 5,051 6,472 5,256 6,421 44,229

WIDPInterregionolRegionalAfiicoRegioiiolA~cCountry ProiectsOtherRegronal

3,3311,886

01,540

138

3,8952,139

01,838

147

3,5322,1211,3843,810

243

4,670 1,298 1,058402 1,024 916328 641 1,319

3,205 2,033 1,985445 150 79

1,219561723

1,12352

19,0639.0984,395

15,6441,254

Total IJNDP 51 415 822 2,135 3,818 4,054 3,936 2,306 4,820 4,637 11,805 6,894 8,119 11,150 9,049 5,145 5,351 3,738 88,250

WorldBank”(Fee Income)(CAM)’

0 0 0 0 0 0 178 37 113 0 0 0 575

—

636

—

1,061 1,002 843— 423 513— 0 0

355443149

4,8001,378

149

Total World Honk 0 0 0 0 0 0 178 37 113 0 0 0 ~15 636 1,061 1,425 1,356 947 6,328

GnmdTotd 51 415 822 2,135 4,048 4,723 5,677 3,844 6,691 7,332 14,829 9,124 11,~01 16,235 15,161 13,042 11,968 11,106138,807* The World Book’s conhuliutions represent several funding mechanisms (ptiinonlyfee4ncome a rnirnstrritive-hudget grant support which have evolved aver time). Pre-1990 datacould not he found for every year.a. Fee income: revenueearned from nmnnojng funds for other donors.b. CAM payment received from other ports of the World Bo,k for supportprovided to operufional lending activitres.

Total World B~*

60 An Evaluationofthe UNDP--WoddBankWaterandSanitationProgram

Activities could not be cut bockfar enough or fastenough tobridge a funding gap

countryteamleaders.Audit teaminterviews indicatedthatasmuchas 50 percent of key managers’ time is spent on direct fundraising or

in donor-relatedactivities.Another impact of the weak financial support is low staff morale,

high turnover and difficulties in developing andimplementing a

long-range human resources plan.Individual staffmembersmustfrequently be financed with project activity funds of relatively shortduration.This limits thetermsof employmentbecauseWorld Bank

policy, by which the Programis bound,requiresthatcontractsbecoterminouswith funding.An experiencedstaffmemberwith much-neededcoreskills might haveto be supportedwith a seriesof shortcontracts, some for no more than six months.

Theinternalauditorsnotedthat staffuncertaintycausedby shortcontracts is worsened by the perceived lack of commitment from the

World Bank. Staff members in the field told the auditors that they

could live with relatively short-term cpntracts provided they had a

sense that the Program would continue. But they felt that the Pro-gramis not valuedby the Bank. For example,theypoint to the lackof anymention of the Program in the documentsdescribingprojectsto which Programstaffhaveprovidedassistance.Theyareawarethatthe Bank’sreal contributionto the Programsince1992 hasfar fromoffset the decrease in overall core funding, despite the fact that thisdecrease began before the present round of World Bank downsizing.TheEvaluationTeamheardfrom otherpossibleandactualdonors

that lack of World Bank support, in addition to cutbacks made by

UNDP, hasaffectedtheir willingnessto supportthe Program.The internal audit report recommended that the Vice-President,

Environmentally Sustainable Development (ESD), make the neces-

sary arrangements to commit the Bank to provide greater financialsupportto the Program,recognizingits synergywith the new direc-tion Bankoperationsaretaking. The reportsuggestedthat this com-mitmentcould takethe form, of amongotherthings,financingorallocatingBank positionsto the slotsrequiredfor themanagersofthe RWSGs.TheEvaluationTeamsupportsthis recommendation andproposes that Bank support for core activities be pooled with that of

othersto avoid identifying sources of funding with individuals.

Recommendations

Thesefinancingissueswill alsotrouble proponentsof theGlobalWater Partnership.A suggestionmadeto planners of the Partnershipwasthatit finance its activitiesthroughawaterfund.2 Suchafund

AnEvaluationofthe UNDP—WorldBankWaterandSanitationProgram 61

As much as 50 percent of keymanagers’ time is spent ondirect fund-raising or in donor-related activities

wouldbe atarm’s lengthfrom anyspecificdonorandcouldattractfunding from differentsources.At thesametimedonorsto the fund

could earmark the specific elements of the work program to which

theywanted to contribute. Finally, it was argued,the fund arrange-mentcould helpprovidean elementof continuityto the principalthrustsof thePartnership,suchas theProgram.Developingcountriesor their institutionswouldsubmitthe proposalsfrom which theactiv-ities thatthewaterfund would SUpport would be selected.The size ofthe waterfundcould bebasedon anticipatedneedsoveraperiodoftime, sayfour years(the EvaluationTeamfeelsstrongly thatthepre-sentsize of theProgramis the minimumneededto be effective).Donorswould committo providetheirpro ratashare of the adminis-

trative costs. They would choose the projects that they wished to

support from the agreed upon overall program. Establishingamoremeaningful consultation process is likely to mean an increase in fund-

ing opportunities. Several donors said they would be willing to put

moremoneyinto theProgramif this approachis taken.The GlobalWaterPartnershipwill not be formally established

until August1996.The Swedishaidagency,Sida,hasagreedto pro-vide fundingfor the Secretariat.At this stageproponentshavenot

discussed the funding mechanism by whichthe Programwill beincorporated into the Partnership. In the meantime, the Program

must find financing for the next stage of its activities. A Programstrategyandwork programdevelopedfrom thegroundup by con-sensuswith all of thepartnerswill facilitate this process.

It is recommendedthatthe Programmanagersestablishafundingpool similar to the suggested water fund—with the size based on an

agreed upon strategy andwork program.A percentageof all fundingshould be set aside for core costs, as determined by an agreed upon

budget. As the principal core function is the distillation anddissemi-nation of worldwide learning in the sector, this share might be called

the Learning Fund. The remainder of the funding could beearmarked by the various donors to the elements of the Program

that they choose to support. Given the impact of the Program on the

capacity of countries to implement and sustain large investment pro-

jects, the World Bank should play a major role in raising the finan-cial supportrequiredfor theProgramto succeed.

Note

1. This doesnot recognizethevalueof anumberof overheadssuch as office facilities and publications provided by the Bank,and

62 An Evaluationofthe UNDP—WorldBankWaterandSanitationPmgrain

paideither from its administrativebudget,or from aportionof thefee-incomeretainedby theBank for administeringthecontributionsofotherdonors

2. Governancefor theWaterPartnership;a DiscussionPapercon-tributedto theStockholmmeetingby ‘William Cosgrove;11/30/95.

AnEvaluationoftheUNDP-WoddBankWaterandSanitationProgram 63

Chapter 9

Conclusions and Recommendations

The EvaluationTeamfound that the Programhasastrategicadvan-tagethatis recognizedby beneficianesanddonorpartners.It agreeswith bothpartiesthat the Programshouldbe continued.The Pro-gramperformsa uniquefunction. It is focusedon the provision ofwaterandsanitationservicesto thepoorin ruralandmarginalizedurbanareas.It is interregional,trusted,acceptedandconsideredindependentby nearlyall partners,yet seenas closeenoughto theWorld Bank to influencethis major lender.Its regionalstructure

makesit accessibleto countries.It is crediblewhenit adviseson pol-icy anddisseminatesinformation.And it can influenceinformal sec-tor coordinationat all levels.

The designersof thecurrentphaseof the Programsetobjectivesfor it thatwerebroadlydefined,ambitiousandclearlybeyondtheProgram’sability to achievealone.Methodsto measureits impactwerenot clearlydefined.Andsincechangesthathavetakenplaceinthe sectorarethe resultof manyforcesandthecontributionsofmanyfactors,it is not easyto separateout the specificcontributionsof the Program.

By its mission,muchof whatthe Programdoesis relatedtolearningthroughresearchandexperience,bothin its own andoth-ers’projects.Objectivesandindicatorsfor this typeof activity aredifferentfrom thoseof projectsdesignedto providegoodsor ser-vices.At the outsetof its work theEvaluationTeam designedasetofindicatorsagainstwhich it couldmeasurethe Program’sprogress.During the evaluationthe evaluatorsalsogainedexperience.Theindicatorsthattheywould selectnowaredifferentfrom thosecho-

senin the beginning.Today,the Teamwould attemptto separateobjectivesandactivitiesinto two categories:projectsdesignedtoproducedefinableoutputsandimpactsandprojectsdesignedtolearnwhat is neededandhow to achieveit. The indicatorsfor thefirst categorywouldbe morequantifiablethanthosefor the second

64 An Evaluationofthe 1JNDP--WorldBankWaterandSanitationProgram

category.This approachis recommendedfor the future.Program activities have contributed both directly andindirectly

to projects that provide improved access to water andsanitationforthepoor andto thosethat createcapacityto carry on this effort.Theyhavealsocontributedto learningaboutnew approachestoprovidingservicewhich canbeusedatthe governmentandcommu-nity level. TheProgram’sstaffhasaccomplishedthis in spiteof the

difficulties of working with continuously, andsometimesunexpect-edly, reducedresources.

The Program has hod ftsgreatestsuccesseswhensectoragenciesand NGOs have beeninvolved in prior pilot and/ordemonstrationprotects

Capacity-building: institutions, people, technologyan~research

National and local institutional capacity in different countries hasbeen strengthened to varying degrees by the Program.The Programhashad its greatest success when sector agencies andNGOshavebeeninvolved in prior pilot or demonstrationprojects.Severalprob-lemsare still common.In manycountriesit is not certainthat thecapacityof the communityto acton its own is sustainable.Skills andincentives are missing for the continuing operation andmaintenanceof thesystemsthathavebeeninstalled.In manycountriesthereisstill amultiplicity of governmentagenciesactingin the sector,con-tributing to inconsistencyandconfusionin policiesandpractices.

WhereNGOshavebeenactive in developingsystems,thereis alackof intermediation with the government agencies whose support will

be needed for sustainability. In some cases,moreeffort mustbemadeto sustain awareness of the value of sensitivity to issues of gen-derandhumandiversityin general.There is still aneedto createandstrengthenexistingnational training institutionsandnetworks.

Investments in the sector arelagging.Yet the capacityof commu-nitiesto build, operate,maintainandsustainwaterandsanitationsystems for the disadvantaged continues to be the major constrainthinderinglarge-scalesustainableinvestments.Building this capacity

is a slow process. More effective and innovative techniques must bedevelopedto speedit up.

Effectiveness of policies, strategies and plans

Principal findings

Most countrieshavemadesignificantprogressin shiftingto demand-driven sectorpoliciesandstrategy.The extentto which thesepolicies

An Evaluationofthe VNDP—WorldBankWaterandSanitationProgram 65

andstrategiesareimplementedvarieswidely. At least13 countries

havesoughtassistancefrom the Programto developnational,provin-cial or municipal sectorpolicies.Throughthis assistance,thePro-gramhashadan impacton policy directionsin thesecountries.

Capacity-buildingis a slowprocess, but it is essential tosustainable future large-scaleinvestments in the sector. Nowthat thereare more large scaleinvestment projects underwu~it is importantthat they bemonitored by the Program forlessonsteemed

Future issuesThe reasonswhy the new policiesandstrategiesarenot beingimple-

mentedvary by countryandneedfurtherstudy.Governmentsandthe Programhavenot given ahigh enoughpriority to hygieneedu-cation,betterhygienepracticesandaccessto sanitation,including

solidwastemanagement,especiallyfor pen-urbanareas.

Impact on domestic and external investment

Principal findingsThe impacton investmenthasbeenmixed. Severaltaskmanagersinthe World Bank havefound the Programhelpful in preparingpro-jects.Othershaveacknowledgedthe impactof thework of the Pro-gramon their own thinking. But thereis little recognitionof theProgramin the Bank’s operationsdocumentsalthoughtheProgram

has been associated significantly with more than $2 billion of Bank-fundedprojects.The influenceof the Programis often indirectas inSouthAfrica, which is not acountry of concentration,butwherethe Program’sapproachesare influencingsome$275 million ofinvestments for community water arid sanitation. Some investment

projectshavebeenfacilitatedbecausetherearestrategiesin placetowhich the Programhascontributed,suchas in Benin andGhana.InsomecasessuchasBolivia, Nepal,PakistanandSri Lanka, theWorld Bank is learning from demonstration projects in which the

Programis involved. But thereareothercountries,suchasUganda,wherescalingup hasnot yetfolloweddemonstrationalthoughthe

governmenthasexpressedcommitmentto theapproach.In addi-tion, manyof theWorld Bank andregional banktaskmanagersdonot look to the Programasa sourceof informationandassistance.Thosein theBankwho haveusedthe Programhavemostoftenlearnedaboutwhat it can do from the RWSGs,ratherthanfromheadquarters.

Future issuesProviding input duringthe preparationof investmentprojectsisneeded,but thereis dangerthatthe: ProgramWill becomeaservicecenter.It mustmaintainits identity andindependence.The

66 An Evaluationofthe LTNDP—WorldBank WaterandSanitat:onProgi’am

The Programstill is trying to dotoo much. Partof the reasonisthat its initial objectivesweretoo broadand ambitious

approachesthat theProgramhas identified as helpful often require

changes in the procedures and in the attitudesof the lenders andthecountries. The Program could do moreto fosterpolitical will amongthedonorsandthe countriesto supportdemand-driveninvestment.In manycasesit is questionable whether the capacityyetexiststosupportsustainable,large,demand-driveninvestment.The Programcanhelpto developthe understandingamonggovernmentsanddonorsthat,at leastup to now, no onehasfound away to shortenthe time requiredfor the capacity-buildingessentialfor sustainableprojects.Finally, theProgram’ssponsorsanddonorsmust insiston asystemthatWill enablethemto determinewhatprogress,if any,isbeingmadeagainstthe sectorobjectives.

Systematic learning and exchange of information

Principal findingsThere have been some successes in systematically approaching the

learningprocess.Theseincludea workshopon the subjectspon-soredby RWSG-EAPin 1994andsystematicapproachesplannedorunderwayin Indonesia,Nepalandthe Philippines.But theEvalua-tion Teamgenerallyfoundsystematiclearningandexchangeofinformationto be the weakestelementof the Program.The Pro-gram hasyet to definesystematicor structuredlearning or action

researchin conceptualor, moreimportantly,operationalterms.Many of the casestudieshavebeendocumentedbut arenot cultur-ally transferable.There is a biastowardproducingreportsin Eng-lish andthereis a needfor informationin otherinternationalandlocal languages.Yet theTeamfound thatthereis awealthof learn-ing that hasnot beendocumented.A researchprojectshouldnot beconsideredcompleteuntil its findings havebeendocumented,dis-seminatedandeffectively communicated.Onereasonfor the failureto documentis that local staff arereluctantto sharethe lessonsthattheylearnedfor fear of criticism or rejectionby Programheadquar-tersandBank staff, whoareperceivedto havehigherstandards.

Programstaffdo not learnenoughfrom othersoutsidethe Pro-gramor enoughaboutactivitiesbeingcarriedout elsewherewithinthe Program.Therehavebeensomelocal initiatives to overcomethis in India andin EastandWestAfrica. But, Programstaffcontin-uesto move in smallorbits.Internationallyknowledgeablestaffarefew in number.Over-dependenceon personalor individualnetworkingleadsto a lack of continuity andbreakdownsin informa-tion transfer.

An Evaluationof theUNDP—WorldBankWaterandSanttationProgram 67

Future issuesNowthat there are more large-scale investment projects underway,it is importantthat they be monitored by the Program to distill

lessons learned. Methods of disseminatingsuchlessonshavebeentraditionalanduncreative.Oncethe Programdevelopsan approachto bettergatheringandsharingof lessonslearned,it shouldbe moth-toredandevaluatedregularlyto ensureits effectiveness.

Sector coordination and collaborationIt is recommendedthattheProgram becontinued, Principalfindings

Governments are responsible for formal sectorcoordinationat themaintainingits distinct idenmy nationallevel. But, mostareexperiencinggreatdifficulty in fulfilling

within the GlobalWater this task and would benefitfrom ouisidesupport.Somesectorcoor-

Partnership dination is alsoneededatthe regionallevel. In all countrieswherethe Program hasbeen active and, especially, where it hashad acon-

tinuouspresence,it haspositively affectedsectorcoordinationbyproviding forumsfor informal andnon-threateningcommunicationsbetweenESAsandNGOs,andbetweentheseorganizationsandgov-ernmentagencies.The Programhasbeenlesssuccessfulas aregionalcoordinator.Its recordwith regard io directcollaborationwith

other agenciesis also mixed. SomeESAs havereportedaseriouslackof willingnesson thepart of the Programto participatein interna-tional workinggroups.The recentrecordshowsthat,whatevertheinitial causefor thisunhappiness,the Programnowmakeseveryeffort to collaboratesystematicallywithin the limits of its resources.

FutureissuesThe role of the Program in formal and informal sector coordinationat both thecountryand regionalle%el requiresbetterdefinition.

Program management

Principal findingsThe Programstill is trying to do too much.Partof the reasonis thatits initial objectivesweretoo broadandambitious.At the sametime,budget cuts have resulted in inadequate resources to support several

coreandfield activities. In 1995 amajoreffort wasmadetostrengthenthework planning.budgetingandmonitoringsystem.The criteriaintroducedat thar time for selectingcountries of con-

68 AnEvaluatsonofthe UNDP—WorldBankWaterandSanitationProgram

centrationandspecificprojectsshouldhelpsignificantly. It wouldstill beuseful to reviewtheProgram’sstrategicfocus,keepingin

mind theProgram’scomparativeadvantage.Although the Programhasbeen evolvingto recognizethe rolesof thedonorsthat supportits activitiesandthe countriesthatbenefitfrom them,Programman-agersshould involve bothdonorsandbeneficiariesto agreaterextentin definingstrategyanddevelopingwork programs.

Programmanagementneedsto be fully reviewedandstrength-ened.Staffing, systemsandprocedures,managementstyleor cultureandstructureshouldbe selectedto supportthe redefinedstrategicdirection, to respondto client needsatthe regionallevel andtoensurethatauthorityis delegatedto the lowestappropriatelevel.

Future issuesThe Programneedsto be strengthenedas quickly as possibletoensure effective use of resources in the ongoing phase. At the sametime it must begindiscussions to raise financing for the next phase

of activitiesin a mannerthatwill ensureboth the adequacyandcon-tinuity of funding. As leaders of the flagship program of the Global

Water Partnership, the Program managers must define its relation-shipsin a way thatgainsdonorconfidence.

Corroboration of conclusions by questionnaire

Fromthe 100 responsesto the closeendedquestionsaskedin thequestionnaires distributed by the Evaluation Team, between 63 and

75 percent of the participants in the surveyresponded positively

with respect to the Program achieving its objectives (see figure 2).But, therearethreeareasin which the Programis seenas contribut-ing much less: the developing of low-cost technology, providing

publications that are useful and positively affecting sector policies.

Rightfully, the Programis not currentlyinvolved in developmentoflow-cost technology. It is focused instead on keeping abreast of

developments and keeping the field informed. The low rating givenheremightbe linked to therelatively low ratinggiven to providingpublicationswhich areusefulin thework of the respondents.Respondentsmaysimply feel that theyarenot hearingenoughfromtheProgramaboutlow-costtechnology.

Thelow ratingin the areaof communicatinglearningconfirmsthemessageheardconsistentlythat theProgramis not doing an ade-quatejob of communicatingwhatis beinglearned.The lower rating

AnEvaluationoftheVNDP—WorldBankWaterandSanitationProgram 69

givento the Program’srole benefitingsectorpolicies is probablyderived from the indirect link that is generally seen between the

Program’s activities and changes in sector policy. There aremanyfactors at work that ultimately convince a government to introduce

new policies. The Program’s influence is only one.

Recommendations

It is recommendedthatthe Programbe continued,maintainingits

distinctidentity within the GlobalWaterPartnership.The Programhasbeentransformingitself while helpingto transformthesector.This is commendable. It should continue to evolve in response to

changesin the environment,thussettingan examplein the sectorofcontinuous improvement. Nevertheless, this evaluation provides an

opportunityfor stock-takingandsignificantcoursecorrection.Throughout this report a number of recommendations aremadeforimproving the Program and its approach to the sector. Although these

were made in the context of specific subject areas, such as capacity-

building or effectiveness of sector policies, they are interrelated.The first threeof the following recommendationsareintendedto

improvetheway the Programachievesits immediateoperationalobjectives. The last two will help the Programdo this, but concerninternal changes in the management and financing of the Programtoensureits efficiencyandsustainability~It is, therefore,recommended:

• That the Programcreate,or causeto be created,programsorprojects supporting direct user-to-user assistance or mentoring.

Thesewould bring togetherthosewhohavealreadybeenthroughtheprocessof introducingwatersupplyandsanitationservices,usingthe approachesrecommendedby the Program,with others who are beginning the process. Users should manage

theseprograms.The RWSGs,ITN centersor otherNGOsandESAs mightplay facilitatingroles. The Programshouldrecom-mendmoreselectiveuseof individual capacity-buildingapproaches which include an emphasis on the role of WID,

appropriate technology, training and focus on policies andstrate-giesbasedon specificcountryneeds.

• That the Programassistgovernmentsin identifying the obstaclesto implementingnew policiesanddeterminewhat role it mightplay in assistinggovernmentsto overcomethem.Casestudiesshouldbe developedbasedon successfulexamplesof demand-driven,bottom-upsectorpoliciesandstrategies.Theseshouldbe

70 AnEvaluationof theUNDP—WorldBankWaterandSanitationProgram

promulgatedthroughsuchmeansastheProgram’sIndiaThinkTank, theannualWestAfrican Retreat,workshopsandseminars.Studytourswith follow up andsupportshouldbe arrangedforsectorpoliticiansandadministratorsto visit countriesthat havesuccessfullyput suchpolicies into effect.The Programshouldprovidepolicy adviceandtechnicalassistancethatgive higherpriority to hygieneandsanitation.

• Thatthe Programmakegreatereffortsto learnfrom theexperi-enceof othersandto documentanddisseminatelessonslearned.It shouldmakemoreinnovativeandcreativeuseof availableinformation-sharingmethodsandtechnologies,for example,aninteractivee-mail/dialoguesystemamongregionsandportable

audio-visualdemonstrationsusingvideo cassettes.• ThattheProgrammanagershouldconductafull reviewofthe

managementof theProgram.A reviewof the purpose,strategyandpriorities for operationsandresearchshouldbe conductedjointly with theProgram’spartners,includingbothdonorsand

participatingbeneficiaries.Staffing,structure,systemsandproce-duresshouldflow from thestrategyandpriorities.Greaterdecentralizationfrom headquartersto the RWSGsandamorecol-laborativerelationshipbetweentheProgramandtheotherpart-ners,arefundamentalto achievingthe objectivesof theProgram.

• Thatthe Programmanagersseekto establishafunding pooi simi-lar to the suggestedwaterfund.The sizewould bebasedon theagreedstrategyandwork program.A percentageof all fundingwould be setasidefor corecosts,asdeterminedby an agreeduponbudget.As the principalcorefunctionis the distillation anddisseminationof worldwidelearningin the sector,this sharesetasidemight becalledthelearningfund.The restof thefundingcould beearmarkedby the variousdonorsfor the elementsof theProgramtheychooseto support.Given the impactof thePro-gramon thecapacityof countriesto implementandsustainlargeinvestmentprojects,the World Bank shouldplay amajor role in

raisingthe financial supportrequiredfor the Programto succeed.The Programis amodel for the GlobalWaterPartnership.Its

strategicor comparativeadvantagesarethoseneededby thePartner-ship as anadvocateof integratedwaterresourcemanagement.Asnotedabove,the EvaluationTeamrecommendsthatthe Programretain its distinct identity as the programwithin theGlobalWaterPartnershipthat is dedicatedto ensuringthe provisionof safewatersupplyandsanitationfor disadvantagedruralandurbanpopulations.Basedpartly on evaluationof theProgram,adiscussionpaperwas

An EvaluationoftheUNDP—WorldBankWaterand SanitationProgram 71

submittedto thefoundingconferenceofthe Global~ter Partner-ship. This paperrecommendedthatthe governanceof thePartner-ship encompassatrue partnership,decentralizationto the regionaland the country level, respectfor theautonomyof thedonorsandthebeneficiarycountriesandtransparencybasedon clearobjectivesandmonitoredcriteriaandreporting~

72 AnEvaluationof theUNDP—WorldBankWaterandSanitationP~ognvn

Annexes

Annex 1

Status of Recommendationsfrom 1991 Evaluation

A teamof externalevaluatorscarriedout the last evaluationof theUNDP—WorldBankWaterandSanitationProgramin 1990—91.TheirSummaryof PrincipalFindingsandRecommendationscontains54 recommendationsaddressedto theProgrammanagers anditssponsors.Manyof thesehavebeenimplemented.Giventhe numberof recommendationsandtheevolutionarynatureof theProgram,

othersarestill in the processof beingimplemented.Finally,Programmanagementhasnot actedon afew recommendationsthat theydo

not considerappropriate.The readeris referredto the originalreportfor acompletelisting of recommendations.The following

sectionscommentonly on thoserecommendationsthatarestill beingimplementedor havenot beenactedupon.Theyarepresentedundertheheadingsin whichtheyappearedin thereport.

Chapter 2: An emerging identity

a) Changethetitle of theProgramto “Joint Programfor WaterandSanitation.”

• This change was not made because the Program managers didnot considerthewordjoint to be appropriate.However,theyagreedthatamorerepresentativetitle wasdesirableandareseekingamoreappropriatetitle.b) Link theProgrammorecloselywith themonitoringprogram

currentlybeingdevelopedby UNICEF in collaborationwith WHO,with CESI, andwith theInternationalLaborOrganization’s(ILO’s)networkof training programs.

An Assessmentof theUNDP—World Bank WaterandSanitationProgram;ReportofanIndependentTeam;May, 1991

AnEvaluationoftheUNDP-WorldBankWaterandSanitationProgram 75

• CESIno longerexists.Efforts areongoingto collaborateon

monitoring.However,thecurrentevaluationteamdid not seeanintegratedmonitoringprogramat eitherthecountryor globallevel.Similarly, it did not find atthecountryor regionallevelsanintegratedlisting of training programsofferedby ILO, the 1TNs,andothersprovidingtrainingin the sector.c) Establishcloserlinks with otherUN agencysectorstaffalready

workingin the participatingcountriesanddrawupon the agenciesfor short-termconsultantservices.

• This is beingdonequite systematicallywith WHO andUNICEF. In Chinatherehasbeencollaborationwith UNTCDC.In Kenyathe Programhasworkedwith ILO. Thereseemsto belittle collaborationwith PAHO. Thus therehasbeenprogressandcooperativeefforts arecontinuouslyincreasing.d) Increasethe participationof PROWWESSin Programactivi-

ties as ameansof expandinglinks with NGOs(aswell asenhancingwomen’s participationat variouslevels).

• PROWWESShas been fully integrated into the Program.

However as a result of budget cuts activities in this areahavebeen cut back.

e) Expandcontactswith regional developmentbanks(followingpolicy discussionsin the CollaborativeCouncil).

• Contactswith theAsian DevelopmentBank(ADB) havebeengoodover the years.Representativesof ADB told the Teamthattheyhavebeenevenbetterunderthe new Programmanagement.ADB is experiencingorganizationandfinancingproblemsthathaveaffectedthe possibilitiesof workingmore

closelywith them.Therehasbeenlittle contactwith 1DB. Coop-erationhasbeenwantingin the past.The new Programmanage-ment in Bolivia is expectedto addressthis issuethereandinneighboringcountries.In general,Programmanagersbelievethatthey havebegunto improverelationshipswith 1DB.

Chapter 3: The financial base

a) UNDP shouldcontinueto providecoresupportfor thePro-gramat not lessthanthe existinglevel andthe World Bankshouldassume significant financial responsibility toward the core costs.

• UNDP funding for the Programhasbeenthe victim of generalbudget cuts in all UNDP programs.The World Bankhas

provided additional funding in recent years. Some of this is

76 AnEvaluationofthe UNDP-.WorldBankWaterandSanitationProgram

throughtheCAM process, that is, compensation to theProgramfor services.This hasnot providedthe Programwith theresourcesit needs to maintain its work program. Staffing at head-

quartersand the field is nowless thanthe critical level required.b) A Joint TaskForceshouldbe setup by UNDP andthe Bank to

negotiate an appropriatedivision of corecostsupport.• Bank management declined.

c) UNDP’s regionalbureaushouldcontinueandexpandtheirsupport, including the broadening of operations in Latin AmericaandtheArab states.

• Supporthasnot expandedoverall. Theydid providethecoremoneyfor theCentralAmericanandAndeanNetworks.The Pro-gramwas involvedin only oneregionalprojectin the Arab states.d) UNDP shouldconsiderconsolidatingall of its supportinto an

overall indicativeplanningsubventionfor coresupport(both atheadquartersandatthe field level) on a rolling basisovera five-yearperiod,allocatedwith the minimumpossibledocuments.

• It is not knownwhetherthis wasconsideredbut it wasnotundertaken.Onthe contrary,fundshavebeenwithdrawnwithshortnoticeafter theProgramhadpreviouslybeenadvisedoftheir allocation.The three-yearwork planandbudgetcurrentlybeingpreparedwill provideanotheropportunity.

Chapter 4: Interchange with sponsors

A small informal ReviewBoardcomprisingrepresentativesof sup-portingdonorsshouldbe establishedwith the following functions:

a) providefor candidandunrestrictedexchangeof views;b)sanction important policy statements; c) verify the need for signifi-cantchangesin managerial and programming arrangements; d) con-

sidernew initiatives, takingaccountof theneedfor coordinationwith Boardmembersownprograms;ande) helpmobilizecontinu-ing supportfor theProgram.

• An AdvisoryGrouphasbeencreatedto which Programman-agementreportsannually.Althoughthe Groupprovidesfeedback

to the Program’smanagersas providedunder(a) above,it hasnoformal termsof referenceanddoesnot appearto fulfill theotherfunctions recommended. The Team understands that the Nordic

donorsin particulardid not wishto createthe moreformalreviewstructure.Theypreferredto leavethe responsibilityforthese actions to Program management.

AnEvaluationofthe IJNDP—WorldBankWaterandSanitationProgram 77

Chapter 5: The World Bank linkages

b) Cooperationbetweenthe Programandthe Bank’s technicalandoperatingdepartmentsin theare~sof work programming,sec-tor work, projectpreparationandpolicy guidanceto governmentsshouldbe continuedandexpanded.

• Goodprogresshasbeenmadefor somecountries.However,the suggestedapproachcannotbe saidto havebecomegeneral-ized.The Programcould be moreproactive.

Chapter 6: Management and staffing

a) Considerationshouldbe givento carryingoutamorecompre-

hensiveorganizationandmanagemeitt studyof theheadquartersstructurethantheteamwasableto conduct,to identify possibleeconomiesandwaysof improvingmanagementefficiency.

• While a management study was not done, the recent internal

auditdoescoversomeof thisareaandmakessomesuggestions.b) Theproblemsrelatingto conditionsof serviceshouldbe

addressed as a matter of urgency. The Team proposes that the Bank’sVice Presidentfor PersonnelandAdministrationbe requestedtoundertakeareviewof thecurrentsituationfor thepurposeof devel-oping asystemthatwould reducethe existinganomaliesto theabsolute minimum consistent with legal requirements governing

World Bank administration.• It is not knownwhetherthe‘VPA everreceivedthis recommen-

dation.It hasnot beenundertaken.c) As a minimum, all staffmembers,beforesigningtheir

contracts,shouldbe informedveryclearlyof the termson whichtheyare beingengaged.In addition, field staffshouldbe told theterms of employment of other members of the RWSGS.

• Although this is not beingdonesystematically,it is not con-sideredas big aproblemaspreviouslythoughtbecausethe Bankhasreducedthenumberof categoriesof staff, makingthe sys-temmoretransparent.Theredoesremainthe problemof radi-cally differentcompensationfor locally recruitedandinternationallyrecruitedstaff whomayhavesimilar qualifica-

tionsandresponsibilities.

78 AnEvaluationof the UNDP-•WorldBankWaterand SanitationProgram

Chapter 7: Field structure

a)Althoughthe compositionof the variousRWSGswill necessar-ily vary in accordancewith the regionalsituationandProgrampri-orities, membershipmight typically include: engineer(rural watersupply),economist/financialanalyst,communitydevelopmentspe-cialist, humanresourcesdevelopmentspecialist,engineer(sanitationandwastemanagement),oneperson-yearof consultancy(various),andapublichealthspecialistin WestAfrica.

• Many of thesepositionsareprovided,but in ascaled-downversionto fit the budget.The resultis that in someoffices,clearlyin EastAfrica, acritical massis no longerpresent.b) The humanresourcesspecialistwould be responsiblefor

expandingactivitiesaimedatcapacity-buildingandfor relationsbetweentheRWSGsandthe ITN centers.

• Whentherewerehumanresourcesspecialistsin the teams,

theyhadthis responsibility.As thestaffingof the RWSGshasdecreasedandthe ITN centershavebecomemoreindependent,therehasbeenatendencyfor this relationshipto weaken.

c) In ‘West Africa half the membersof the RWSGshouldbeFrenchspeaking.

• The targethasbeenexceeded.The needis now for morestaffwho speakEnglish.

d) More nationalsfrom the respectiveregion shouldbe included

in the RWSGs,andin-countrystaff should,to theextentpossible,benationalsof thecountryconcerned.The situationvariesby region,but in generaltherearemorenationalsin the regionalstaff. Moststaff in thecountryofficesarenationalsof the country. [Note thatthesefigureshaveyet to be verified from thestaffing lists.]

e)Budgetaryresourcesshouldbe increasedto providefor addi-tional countryProgramstaff,someof whomcould assistaclusterofneighboringcountries.

• This wasdonetemporarily.Today,theremight be fewer

resourcesfor this purposethanwhenthe recommendationwaswritten. It is necessaryto keepin mindthe distinctionbetweenProgramstaffandprojectstaff. The latterareon specificassign-mentsandleavewhenthattaskis completed.1) In Latin Americatwo modestoffices—RWSGsin embryo—

should beestablished:onefor CentralAmericaandtheCaribbeanregionbasedin GuatemalaCity andone for SouthAmerica,basedin Bolivia.

AnEvaluationoftheUNDP-Wo&BankWaterand SanitationProgram 79

• An office wascreatedin CentralAmericaandthe responsibilityfor it hasbeen transferred to UNICEF. There is an office in

Bolivia, but it is basicallyanationaloffice as opposedto an officeservingSouthAmerica.

Chapter 8: Strategy and effectiveness

b) Organizedconsultationsshouldbe heldwith all governmentswith regardto the proposedcountrywork programfor their coun-

triesprior to its finalization.• Consultationsdo take place.I-[owever, thereis an issueaboutwho representsgovernments.Governmentrepresentativesmetbythe evaluationteamfeel thattheseconsultationsprior to workprogrammingstill needto he moreorganized.c) Futureemphasisshouldbe placedon analyzingthe lessons

learnedfrom Program-sponsoreddemonstrationwork, aswell asfrom the experienceaccumulatedby otheragenciesandprograms,andon disseminatingthe informationthusgatheredasa steptowardscalingup to major investmentprojects.Organizationof rural watersupply demonstration projects should be de-emphasized unless there

is somecompellingreasonto conductone, in collaborationwithgovernment,wherenoneexists.

• Analysisof demonstrationwork, and especially of the experi-encegainedby otherprogramsandagencies,still doesnotreceiveadequateattention.Staff arebeingrecruitedto addressthe issueof disseminationof lessonslearned.Nevertheless,

scaled-upprojectsarebeingdesignedto incorporatemanyof thelessonslearned.d) Work in low-income,congestedurbanareasbothon the urban

fringeandwithin citiesshouldbe expanded,alongwith intensifiedresearchanddevelopmentaimedat building a wider rangeof low-costintermediatetechnologies,with high priority on sanitation.

• TheProgramis nowbeginningto addresstheseissues.How-ever, resourcesavailablefor this purposeareminimal comparedwith thesize of theneed.f) The Programshoulddefinemoreclearlyits approachto

capacity-buildingandto theprioritiesit shouldfocuson in thisimportantarea.

• This remains to be done. Someguidancewasgiven by theAdvisory Committeein Delft.

go AnEvaluationofthe UNDP-World BankWaterandSanitationProgram

Chapter 9: International training network (ITN)

b) The assessment process for establishing a Network center

shouldbeorientedto the objectives,andthe selectionofinstitutions,key staff,andpreparationof work programsshould

follow accordingly.

• The principal selection criterion now being used is whetherthereis local demand.The remainderof the aboverecommenda-

tion flows from this.e) Globalcollaboration,as well as ITN expansion,shouldbe the

responsibilityof asmallandnon-bureaucraticcoordinationunit atWorld Bank headquarters.But for collaborationactivitiessuchas

training,exchange,research,documentation,informationandsoforth, this unit shouldcooperatewith appropriateinternationalinstitutions,thusestablishingalink with the entirewaterandsanita-

tion community.• Oneperson was given the abovetaskas afull-time assignment.However,theeffort wasdropped,presumablybecausetherewasno demandfor the serviceor approach.

Chapter 10: Program priorities in the 1 990s

c) Revitalizethe ITN alongthe linesdescribedin Chapter9 asa

critical componentof future strategicplanningandas aninstrumentto supportlocally generatedinitiatives basedon Programprinciplesandpriorities.

• As notedearlier,this effort wasdropped.Thelocal ITN centersarenowlargely supportedby bilateraldonors,with the emphasison their financialautonomy.Wheretheyaremostsuccessful,it ispreciselybecausetheysupportlocallygeneratedinitiatives. How-ever,links to the Programarenowalmostalwaysmadeon an adhocbasis.d) Acceleratepublicationof Programfindings.Thesepublications

fall into two categories: materials needed as soonas possibleby oper-ationalstaffand the ITN (guidelines,casestudies,designs,sample

questionnairesandsoon)andbasicresearchfindings.The longprocessof researchandpeerreviewfor thelatter shouldnot beallowedto jeopardizeoperationsby delayingthe former.Preparationandpublicationof theformermightbe entrustedto the 1TN centers.

• Thereis almostunanimousagreementamongclientsof theProgramthat thisareacontinuesto beneglected.Gathering,eval-

An EvaluationoftheCJNDP—WorldBankWaterand SanitationProgram 81

uatinganddisseminatinglessonsIi as beenneglectedbecauseofoperationalprioritiesandbudgetrestriction~s.A communicationsspecialist, who will address the issue, is now joining the Program.No systematic attempt hasbeenmadeto usethe ITN centersforthispurpose.e) Prioritizetheresearchanddevelopmenteffort to supportthe

agreedstrategy.More systematicandoperationallyfocusedresearchon bothhardwareandsoftwareis vital. Much moreof this shouldbe undertakenthroughthe rejuvenatedITN, which will bein mostdirectcontactwith field needs.

• Therehasbeena clarification of objectives.In 1995aprior-ity-settingprocesswasestablishedfor thework program.Struc-tured learninghasbeenadoptedas theguiding principleto

research.However,little attempthasbeenmadeto transfertheactual research to others, such as the ITN, which hasnot beenrejuvenated.

f) Developanddisseminatemodelsfor institutionalsystemsthatensure sustainable service. These models shoulddrawon arangeofinstitutions (public sector agencies, NGOs, the private sector and

usercommunities)andshouldemphasizeinvolving thecommuni-ties themselves.

• The work programinvolvesthesetypesof activities.How-

ever,it hasnot beencarriedout, evaluatedanddisseminatedsystematically.g) Make greateruseof socialmarketingapproachesandtech-

niquesto ensurethatscaling-upactivitiesaretruly demanddriven.Effective demandapproachesshouldbe basedon the customerselec-

tion of servicelevel and thewillingnessto pay.• This recommendationis interpretedby the Programmanagersto meanadvocacyor selling the conceptsof the recommendedapproach.Therecan be no doubtthat the principleunderliestheProgram’sactivities.Fieldexperienceis varied. Projectdevelop-ersoftenhavedifficulty acceptingthe slownessandapparentcostof the approach.Evenwherecountrieshaveagreedto adoptthestrategy,projectscontinueto be carriedout thatdo not respectit.The advocacyrole of the Programcould be reinforced.h) Expandactivitiesin low-incomeareasnotadequatelyserved

by existingconventionalsystemsandestablishedutilities, andsup-port thiseffort with expandedinnovational researchon appropriateurban technologies,institutionalarrangementsandlinkageswithconventionalurbansystems.

• The Programis beginningto addressthis recommendation,

especiallyin theareaof sanitation,for example,in Ouagadougou,

82 An EvaluationoftheUNDP-WorldBankWaterand SanitationProgram

Burkina Faso.Programmanagersconsiderthatthe supplyof

waterin theseareasis aquestionthatis bestaddressedby thewaterutility specialists.i) Establish closer cooperation with the Urban Development Divi-

sion (IN1JRD)of the Bank’sInfrastructureandUrbanDevelopmentDepartment (PRE)complexto link Programactivitieswith overall

urban policy and planning.

• Program managers have found this difficult, especially withtheir limited staffing.

k) Thenextdecadewill seeincreasingproblemsin the urbaninfrastructure in developing countries, especially in the megacities.As a complement to the urban component of the program, consider

broadening the scope of the Asia SectorDevelopmentTeaminto anUrbanEnvironmentalServicesTeam(UEST),andexpandits activi-tiesbeyondAsia to Latin Americaandselectedmajor cities in AfricaandtheArab states,with core fundingfrom UNDP’s DGIP andRegionalBureau.

1) It is not recommended that the Program and the proposed

UEST, if established,be mergedandbroughtunderasinglemanage-ment.Eachprogramshouldhaveits own specific identityandfocus,as well as diverse staffing and management requirements.

m) If broadenedin the mannersuggested,considerlocatingthemanagementof UESTin theUrban DevelopmentDivision of theBank’s PREcomplexto facilitatecoordinationwith the Programandwith urbanpolicy research.

Thesethreerecommendationsareinterrelated.It is not cleartowhomtheyshouldhavebeenaddressed.Programmanagementhasnot actedon them.

AnEvaluationof the UNDP—WorldBankWaterand SanitationProgram 83

Annex 2

UNDP—World Bank Water and Sanitation Program:Logical Framework Analysis

Annex table21LogIcal frameworkanalysiS

End-of-projectsituation Indicator Source/meansof verification

1. Notional andlocal capacitystrengthenedfrom communitytommistenallevel

1! Appropnatemodelsof institutionalframeworkindudmgrole of private sectorandnationalresearchinstitutions12 Beneflcionesacquireskills requiredby porticipotoryapproach.1.3 Greaterattentionto genderissues.1,4Creationand supportof national training networkshikedto regionalnetworks15 Recruitmentof nationalstuff for in-countryandregionalposts.

11 ObtainSof organizationswith increasedcapacityand meansiS.1.2 Selectedverification with beneflcianes.13 Selectedverification with beneficiaries.L4 Obtain lists of networkscreatedandsupported;verify with networkmembers.1.5 Examinestaff lists of cunentarid previousyears

2. Countrieshavemadesignificantprogressin shifting national sectorpoliciesand strategiesto becomedemand-driven,bottom-up

21 Involve privatesector,NGOs, communitiesandusers,especiallywomen,in developmentandmanagementofservces22Decision-snakingat lowest possiblelevel23Financingthroughcost recovery74 Developmentanduseof affordable technologies.

2.1 Newpalicy/irategydocumentexistswhich includethese.22 Sectar/pra~ectreports,S consultselectedbeneficiaries.23 Questionnaire24 Individual rrnetingsor focusgroups covermgbeneficiaries,NGOs,ESAs,privatesectorandgovernmentagenciesatall levels.

3.1 New optionsfor provision ofservicestestedm large-scaleprotectsand leamingdissemmated.

311 Largescaleprotectsincludecomponentswith neworganizationand/ortechnicalapproaches.3.12 Evaluationof new approachesbuilt into pra~ects.313 Lessonslearneddisseminatedwithin country,regionand‘mtenegianallyas appropriate,as well as tofinancingagenciesanddonors.

311 Cibtamlist of protectsfrom regionalprogram.3.12Reviewapprarsalreportsfor projects.3.13Regionalmanagerprovideslist of meansused(publications,wcrktheps,tramftl2 maten&s,etc.) and recipients

3.2 Qualitative improvementsmdesignand implementationoflarge-scaleinvestmentsprolects.

321 Programidentifiesandprovidesmputsto designandimplementationof investmentprolectsfundedby World Bank,regionaldevelopmentbanksandotherdonors to improveaccessby poor throughdemand-dnvensectorpolicies3,22 Proiectsaremoresustainableas resultof Programinvolvement

321 Obtainlist of investmentprotectsto which inputshavebeenmadebyProgram.Verify throughappraisalreports,interviewswith projectTuskManagers3.22Seekopinions of govemments,task managers,benefldrsiesSothersexercisejudgment

4.Systematiclearningprocessdevelopedandinstitutionalized,

4.1 New approachesto sectordevelopmenttestedandmonitoredatcountrylevel, nesulisfed backinto sectordevelopmentprocess41 Cross-fertilizationencouragedbetweenregions,for example,by exchangeof national staff

- 4.3 thamfromprojectsondmeseorchoutsideProgram

41 Obtain list; of examples,relateddocuments,andmechanismsusedtofeedresultsba±into sectordevelopmentprocessfrom regionalmanagers.42. Ask for mechanismsused for rnteriegionalaass4ertillzatiankichSgstafF lists with explanatorynotes43 lnterviewiof headquartersandregionalmanagers.

84 An EvaluationoftheLINDF—WorldBank WaterandSanitationProgram

Anneorahle21Logical frameworkanalysis(cont.)

S-of-projectsituation Indicator Source/meansof veriftcatlon

5. Sectorcoordinationwill havebeengreatlyimproved

51 Sector-coordinationmechanismsformalizedamongnationalagencies52 Formal andinformal coordinationand communicationmechanismsdevelopedat country level amongESAsandNGOs, andbetweenthem andgovernment.53 Formalandinformal regionalcoordinationandcommunicationmechanismsdevelopadto involve countriesandotherpartners.

51 Ask governmentsfor documentsfarmallzmgmechanisms.5.2 Ask countryrepsandregionalmanagersfor documentsfor formalmechanismsandexamplesof Informal ones.53 As above.

6.EffIcient, cost-effectiveorganizatiorand managementstructureatheadquartersand field levels.

6.1 Clear objectivesand strategydevelopedin participatorymannerand welkommunicatedto all levels62 Work planningand monitonngsystemsin placeto ensurefocuson pnarrtmes.63 Monagemeatandstaff performanceappraisalsystemmplace relatedto Programabtectives.64 Managementstyleencouragestwo-waycommunicationsbetweenandwithin all levels,andcreativity.65 Rolesandresponsibilitiesof headquartersand regionalofficesareclear,and appropeateto mandateand work plan6.6 Numbersand skills of staffareappropnateto work plan67 Staff retentionensurescontinuityof skills and knowledgeof program.68 National staffwith gendersensitivityfill mostpesitiansat headquarters,regionaland countrylevels.69 Efficient and effectivemechanismsore in place to raisefundsand apply themto nationalandregional programsandprojects610 PartnersaresatisfiedthatProgramhas utilizedtheir strategicadvantage(value-added)

6.] Assessclarity of obiectrvesandstrategy,assessprocessfor theltdevelopmentwith respectto participationatall levels.Determirewhetherknowledgeof existingobjectivesandstrategyobtainsatall levels.62 Examine processfor settingpnionties,and for preparingSmonrtonngthe work plan; relateexistingwork plansto Programobjectives63 Consult managementand staffatall levels,64 Consult managementSstaff atall levels.6.5 Consult managementandstaffatof levels andcomparewith strategy66 Seekthe views of managementandstaffatall levels,comparework planwith staff numbersand profiles6.7 Examinestaffingrecordsfar tomoverwith respectto numbersS slulls.68 Examine staff recordsoverrecentyearsand documenttrends69 Assessexisting andpossiblealternativemechanismswith ProgrammanagersandInternalAudit.610 Interviewswith managersof Bank,IJNDP,major danars,andpossiblemqordonors who do not contribute

AnEvaluationoftheUNDP—WorldBankWaterand SanitationProgram 85

UNDP—World Bank Water and~Sanitation ProgramExternal Evaluation

WorkProgramSummary(July 1995--January1996)Literature review

Prepare regional(tripartite review)meetingsHeadquarters interviews

BankandUNDPRegional meetings(Including prep. country meetings)

Delhi

JakartaNairobiAbidjan

Country meetingsDonor consultations

Headquarters2ndRound Intei-viewsEvaluation team working session and

presentation of findingsto UNDP-WB

Draft final report

Final report

July 20—Aug 10Aug 21—31Aug 21—31

Sept18—19Sept21—22Sept28—29Oct2—3

Oct 10—31Nov 1—15Nov 1—15Nov28—30

Dec15Jan15

86 An Evaluationofthe UNDP—WorldBank Waterand Sanitat:onProgram

Mnex table 3]UNDP Headquarters and New York

Name

Cait~nWiosenChnstian lehembeeBen BrownRoberto IentanFirouz SobhoniTim RothermelFrank HariveftPiet KiopAnenne NaberClaude SauveplaneJacob BurkeRichardRiegerDr. Michael SacksHelk PerrattGaunsonkor GhoshBrenden BoyleMkhoel Sointl.ot

Annex 3

Division

RBARBARBAPSEEDSEEDSTAPSDSTAPSOSlAPS0STAPSOUNDDSMSUNDDSMSRBASBPPS/STAPSDBPPS/STAPSDUNICEFUNICEFUNICEF

Aiinex table 32

World Bank Headquarters and Washington DC

Name Division

TWUWSTWUWSTWUWS‘WUWSPNUWSTWUWSTWUWSIWV DR1INUDRAFIESEDIE NEA2EMAF5INAFIESA FT FSEDIENSA2AW

l1]ura Fngenli AF2PHChristopherCouzens SA2AWJohn Shepherd AFIEIMoryvanne Flessis-Fraissord AF3INXavier Legrain SA2AWDung Nguyen IADDR

List of Persons Met

Bran GroverBruce GrossLaurle EdwardsJenn~orSaraWendy WakemonMike C,omMarl DhokoiAmii FehxLevis Poullquen[ernieObengTaiiy Van VugtEdouard MotteMberto HarthJean DoyenD~dGreyFroncois Mono-PatorniKarl Kiernec

AnEvaluationof the TJNDP—WorldBankWaterand SanitationProgram 87

Annextable 32WorldBank HeadquartersandWashingtonDC (ant.)

Name Division

SteveWerssmon Al 2ElDavid Hnwarth H~GLlDiannMartin ACCTFAnupom Klianna M3lNGroemelee SAIINleslieRigaud GEETauno Skytta OEDD3Jan Janssens AFSINOlga Boemeke OPRSLRaymondReid PAIlO

Annex table 33South Asim Tri-partite reviewmeeting,September18—September19, 1995,New Delhi,India

Name Title Organization

6, Haldia JointSecretory MOER.S. Sharma Director (IC) MOEManaranianKumar Deputy Director MOEPlC Sivonondon Jomt Secretary MORAEDr. S.K. Biswas Adviser MORAEB.S. Minhnxs Joint Secretary MOUAEDr. S.K. Shukla Adviser CPHEEO/MOUAElilak Hewawasam Director CommunityWS Ptogrom,Govemmentof Sn LankaDC. Pyakur’yal DirectorGeneral Dept of WS & S,Govemmentof NepalIan Curtis First Secretory/WSO ODAJensBterre Counsellor (Dev), New Delhi DANIDACaretBrands First Secretory DGIS (Netherlands)OveBjerregaard Depuly ResidentRepresentative UNDP IndiaB. Radhokrrshnan ProgramOfficer LINDP, IndiaManel deSilna Sr National ProgramOfficer UNDP, Sn LankaZafan lqbal AssistantResidentRepresentative IJNDP,PakistanM. Alorn Ptolect ImplementationOfficer ADB, IndiaRupert talbot Chief, WSS UNICEF, IndiaB.B. Samantha Project Caordiaator,WSS UNICEF, IndiaIC. Pospisilik Regional Director WHO, Indiaken Bcinerjee World Honk, BangladeshJasliiTenzing World Bank, NepalSheeloPatel Director NGOs, SPARC, IndiaBnan Grover ProgramManagerRobert Boydell RegionalManagerReklio Dayal RegionalProgramAdviser RWSGSA, IndiaL PannerSelnamKS. Romasubban HRD Adviser RWSGSA, IndiaSusanIumqaist Consultant RWSG SA, IndiaK.M. Minnatulloli Team Leader RWSGSA, PakistanZahfr Sadeque Team Leader RWSG 5k BangladeshPrem Ral Goutam Consultant RWSG SA, Nepal

88 An Evaluationofthe UNDP—WorldBankWaterand SanitationProgram

Name

Dr. SoutsakhoneChantaphoneDr. PhanSi ItemBaasanhuDamboEllen PascuaWilem Haihorulfan MarsIi Reggy Irwoni BusroniMichael SteinbergHansVS~Hildo WinartoNeil CollinsRobertKim FadeyY.D. MotherImam KrtsmantoFrancoisBinderJonC. KampFntz lobusMinoR De BressenBananaSrahaanIbrahimlahabBnan GroverRobertBaydellJerrySilvermanRichardPollardMary JuddAlfred IambertusNguyenCong Thanh

Organization

MoPH, Lao PDRVice Ministry of Construction,VietnamMongoliaDIIG, PhilippinesBina Swodayo,IndonesiaBino Swodaya,IndonesiaOECF, IndonesiaDANIDA, IndonesiaDutchEmbassyAusAidAusAidWHO, IndonesiaUNICEFBappenos,601SwissEmbassyUNDP, IndonesiaUNDP, IndonesiaUNDP, IndonesiaUNDP, IndonesiaFAOWB, WashingtonRWSG SA, IndiaRWSG EAP, IndanesiaRWSG EAP, IndonesiaRWSG EAP, IndonesiaRWSG EM IndonesiaRWSG EM Vietnam

Annex table3.4EastAsiaand thePacificTri-partite reviewmeetIng,September21, 1995,Jakarta,Indonesia

An Evaluationofthe UNDP—WorldBankWaterand SanitationProgram 89

Annex table35EasternandSouthernAfrica: Trl-partite reviewmeeting,September18—September19, 1995,Nairobi, Kenya

Name Title Organization Cauniny

D N Stower DeputyDirei tar WaterDevelopment,Ministry of lendReclamation Kenyai.E. Alela Director Urban Development,Ministry of Local Government KenyaFW Mama Sr UrbanPlanner Urban DevelopmentMinistry of Local Government KenyaVictor M Serugendo Town Clerk Kampla CountyCouncil UgandaP Kahangiere Director Directorateof WaterDevelopment,Ministry of Natural Resources UgandaSimon S Mambali Commissionerfar WaterAffairs Ministry of Water Energy andMinerals TanzaniaGeorgeNhunharna Coordinator NationalCoordinationUnit, NationalAction Conirnittee forWater andSanitation ZimbabweC C. Govatr Chief Water ResourcesOfficer WaterDepartment,Ministryof lrngatronandWater Development MaloiviMo I Mosisso WaterSupply& SewerageAuthonty EthiopiaDr Paul Taylor ExecutiveDirector Instituteof Water& SanitationDevelopment ZimbabweKkiamisi Chome Deputy ExetutrveDirector KenyaWaterfor HealthOrganization KenyaMargaretMwangala Executive Director KenyaWaterfor HealthOrganization KenyaMatthewN. Kanuki Director Networkfar Water & SanitationInternational KenyaI. Islamshah Head SwissDevelopmentCorporation KenyaSaraWakeham Coannotor Unft II, HABITAT KenyaAageKruger First Secretory RoyalNorwegianEmbassy ZambiaAte Grrmo Alemayehu UNDP ZambiaFredenckLyons Res. Repre’entatrve UNDP KenyaNancyMango Dep. Res Rep. UNDP KenyaKhadila lbrahim Program&‘st UNDP KenyaSalameMwendo ProiectOfficer UNDP KenyaKen Moskall ProlectOfficer UNDP TanzaniaKnstin Sverdrut ProgramOfficer UNDP TanzaniaGeorgelrmalima NationalPiogramOfficer UNDP MalawiMoflano Asst. Res Rep UNDP UgandaBruceGross Sr Tech Asst. World Bank WashingtonDCTore bum Reg.Manciger RegionalWater & SanitationGroup, World Bank KenyaMukami Kanuki RegionalSpecialist RegionalWater& SanitationGroup, World Bank KenyaLed Hommelgaord Planning Engineer RegionalWater & SanitationGroup, WandBank KenyaRoseLidonde Asst. ProgramOfficer RegionalWater & SanitationGroup, World Bank Kenya

90 AnEvaluationof the UNDP—WorldBankWaterandSanitationProgram

Name

J. Bazrlle-FinleyA CisseR. RadjiO.ClssokaBruceGrossMaui MathysMvii ManouSavinaSylvie DebomyEustacheOuayoroYves DegiacomiA D1oukaM. SakhoA TaupeH. OuadraogoM. A. CongaM L SalrfuH OuedroagoC. deRooyM JensensP BerthetThomasAvidA. MadyG KopieuA. BodarouEugeneLarbi

Organization

UNDP, AbidjanUNDP, AbidjanWorld Bank,AbidlonWorld Bank,AhidjanWorld Bank,WashingtonDCRegionalWater& SanitationGroup,World Bank,AbmdjanRegionalWater& SanitationGroup,World Bank, AbidjanRegionalWater& SanitationGroup,World Bank, AbidjanRegionalWater& SanitationGroup, World Bank, AbidjanRegional Water& SanitationGroup,World Book, AbidlanMinistry of Equipment,AbidlanMinistry of Equipment

Waler & Sanitation NationalAgency, Burkina FasoWater & SanitationNationalAgency, GhanaWasteManagementDept,Kumasi MetropolitanAuthonty, GhanaADB, AbldlanUNICEFDANIDA, CotonouFrenchCooperationFrench Agency far DevelopmentASAPSU,AbidjanCREPA-CI,AbidjanEsseimellos,CotonoriTREND, Ghana

Annex table36WesternAfrka: Tri-partite review meeting, Abkljan, Côte d’Ivolre

Title

ResidentRepresentativeRegionalProgramOfficerProgramOfficerProgramOfficerSr. Tech.Asst.OfficerRegionalManagerProiectOfficerProjectOfficerProlectOfficerProiectOfficerDir of WaterDeportmentDir ofWater DepartmentHydraulics Dept. DirectorChief of SanitationDeptExGeneralManagerDeputyDirectorDivision ChiefPeg ProgramsCoordinator

ConsultantManagingDirector

Annex table37Country meeting,India

Name Title Organization

V M. Ramaprasad DeputyAdviser Ministry of UrbanAffairs & DevelopmentS Mendrrottu DeputySecretary Ministry of Rural Areas & DevelopmentDr. Corolla do Costa Joint Dvector ACDILPS Reddy Administrator PaniimMunicipal CounolPR Mohon Consultant OMNagobhusiian Consultant OMKullappo Consultant OMA Dasgupto Director DevelopmentDialogueDr. V Lokshmipotfmy Professorof IIRO & M OsmanraUniversityDr. ShyalamaAbeyratne Chief Advisor IRS & WS ProiectCarel do Groat IRS & WS ProiectPM.K. Namboodiri Deector Instituteof Rural DevelopmentPV. Baa ExecutiveSecretary Municipal IndustrialtnistLP Selvam Team Leader RWSG-SA,World BankKS. Rarnasubbon SectorSpecialist RWSG-SA,World BankA Da~upta Consultant RWSG-SA, World BankM. Rovichandran Admin Secretary RWSG-SA,World Bank

An Evaluationof theUNDP—WorldBankWaterand SanitationProgram 91

Annex toNe 3.8

Country meeting,Pakistan

Name

ArifHasanMohammed Mixtazolmnza ElahiSaleem Rowlsmat Am SoeedSimi KamalSiddique RhanSyed Mubbashe AhabMamariyun SabirAminiid-Din AhmedMir NaeemuffahJohn CollettMuhanimed AslamAttmur-Rehman(mdi. A & NaseemYosmeen lawaidAllah JavayaOaziAzmatlsoAbdol Aziz KhenSyed Mazhar Alt Shah BukhaaSaeedA KhonRota Rehon AishodKM MinatullahMoliho Hussein

Title

ConsultantDeputy Director (Planning)Acting DGProject Cootdinotoi

Managing DSctorDoector General

Assistant DirectorDeputy DirectorWSS Specralrst

Protect DirectorChief Executive

Chief ExecutiveChief EngineerDirector GeneralDeputy DirectorEngineerTeam leoderConsultant

Organization

OPPIGRDDPMU

SPORoasta DevelopmentConsultantIGRDDPHEDLGRDDMinistry of LGRDDMSUGilgitPHEDPHEDNRSP

BRSPPHEDRRDLocal Bodies & Rural DevelopmentRWSG SA, PakistanRWSGSA, PakistanEnterpäse Consulting (pvt) Ltd.

Anneu tableS?

Country meeting, Indonesia

Name

Dr. Marttuni Budi SaIfaS lanaMinalli de BressierNugorohMednlRisyanalswadiJuh Saemuot SlametNashihin MasonWillem HarhansRachmadiIndrawati

Organization

National Planning Board Office (Bappenas)UNDPUNDPBAPPENASBAPPE HASMinistiy of Public WorksWarld BankInstitute of Technology DepartmentCAREBina !wadaya (NGO)Rudft Public Works

92 An Evaluationof the UNDP-WorldBank WaterandSanitationProgram

Annex table 3.10Countrymeeting,Philippines

Atmoz JerefeAsefaGemedalege H. Ry&andMeselechSeyoumTesfu EstiftmosAmho YesusMetafenoYitborek TessemoLokechHoleFoote FayissoPleroBucoBeleteMulunehEndashowfodesseGedlonMowMesfln LemmaEstifonosleroiGebre SelossieOkugogzhiYohonnesGebremedhinAkole ~2fleTeshomeRegossoGemoAtemoyehuRoseUdondeTore 1mm

Organization

tnt Holistic ApproachMinistry of HealthNorwegianChurchArd/EWaterActionWaterAidMetofenaConsultingMUrbanDevelopmentProjectMinistry of Water ResourceOromro WMERDBItabon EmbassyPnvoteConsultantESRDFPrivateCorisultont/WBUNICEF WESMinistry of WaterResourceWorld Bank, EthiopiaMinistry of WaterResourceU/Dev SapportSn.Mirestry of HealthUNDPRWS&EA, World BankRWS&EA, World Bank

Name Title Organization

Ellen Poscua Asst.PlogromManager 01116RogelioOcampo Head,PlonnmgDivIsion 01116RebeccaRovolo ProtectOfficer, PHI Project DIIJGElizabethYopa Protect Officer, FW4SP 01116RoissaSegowa DevelopmentManagementOfficer 0116RogelioBores Project Director, PMO-RurotWaterSupply OPWHLombertoAbrecea Chief MechanicalEngineer PMO-Rurol WaterSu* DPWHMolncsBogodlon ExecutiveDirector PENMoribel Belizano OperationsOfficer World Bank, Manila OfficeThelmoCruz ProgramManager UNDPEribertoColubaqoib ResidentMonoger Cart Bra GroupA/S (Local ConsiSt)CharlesPendtey World Bank Consultant LGUUWSSPKennedyTuzoon CommunityDevelopmentSpecialist I.GUUWSSPTeamCormeloGendrano ResidentOfficer TutungonSaTubrganFoundation(P160)Santanulahin UNV Speootist PHI Project,RWSGEAPCesorYniguez TenorLeader PHI Project,RWS6EAPEloriro I Abesomis PPDOStaffYolandok SonJuan SupervisingSanitaryInspector IPHORomeoS. froncrsco EngineerCrisontoS. Sevillo

DEO/DPWHMLGOO/DILG

Annex table3 11Country meeting, Ethiopia

flame

An Evaluationof theUNDP—WorldBankWaterandSanitationPmgrarn 93

Annex table 112

Country meeting, Uganda

None

Ion DolonC.B JespersenSven JacobiTheresaMcDonnellMonica PlenrhrmWr. FellousJM.R KrrzoMorcello OchwoSamwelMutonoMosesLavaAyub KutanKokernboFredAndreaBouerWilsonWomimbiDavid MukarnaMuoizi Rwondume1e~HommelgoordRoseLidonde

Orgonizotion

GOALRJWASADANIDAIrish AidWaterAidUNICEFKUPPKUPPDWDDWDKUPPKUPPUNDPRUWASARUWASAKCCRWS6~EA,World BankRWSG-EA, World Bank

Andre loupeStanrslousNkwornSherrySuggsDr SauleymoneDiollnKay E. EhlersFlemmrngNrcholsJ LugrosVouquelinAdnenAffogboloGeorgesAbolloKono MereFoustin DohitoVeronique GnanihJ EudesOkounde

Philip AnglarsProf Alfred MondionogniJeanBaptisteAchidiAgnrolo ClaudioneBadarouDo SrlvnHelgo FrnkBettino HorstrnnnnLambertKotyChristian LoupedoBachirOloudeArrrrdou Roufoi lamomaBemordGnontannou

DirectorResidentRepresentative

ProgramCoordinator

DirectorDirectorGeneralRegionalDirectorDirectorCoordinatorof ProjectApplicatorAdministrator

Development& Ironing ConsultantDevelopment& Training CorrsuttjatDirectorSeniorConsultantProjectni-ChargeChiefBiologistDirectorGeneralProtectin-ChargeDirectorDirector

Organization

Directionde l’HydrauliqueUNDPUSAIDUNICEFl6lPDANIDACFDDirectionde ‘Hygiene etde l’AssornissomentSIBEAUDirectiondel’Hydroulique, BayconENERDASCPPSConsult,Cobrnet d’IngenrereConsedet AssistanceTechniqueCEDACEDASTRA HMESSENTIELLESUNDPIGIPGTZAgenced’ExecuhondesTrovouxUrbainsWorld BonkSERHAUSacreteBenroorsed’Electrrcito at d’EouCEO,Survie deIa Mere et do l’Enfrmnt, Dassa-Zoume

Annentoble313

Country meeting, Benln

Name Title

94 An Evaluationofthe UNDP--WorldBankWaterandSanitationProgram

Anneutable314Country meeting, Ghana

None Title Organization

EustocheOuoyaro ProtectsOfficer RWSG WA, World BankS.Edem Asimoh PlanningOfficer CWSD-GWSCThomasKirke SeniorAdvisor CWSD/DANIDAElse Moller Nielsen Inst. Secr. Donish EmbassyPhilippe Collrgnon ProtectOfficer C F.DEmmanuelM Bawa ProtectOfficor UNICEFDr. J G Mooney Head of EnvironmentalQuality Civil Eng. DeportmentHenryNoye-Mortey Director/EnvironmentalHealth Division Ministry of Local Govemment& Rurol

DevelopmentSamuelAkwer Allotey Env HealthTechnologist Ministry of Local GovernmentKworne Aswonoteng Env HealthTechnologist Ministry of Local GavemmentDivine Dugbartey AnsI. Public HealthEng. Ministry of Locol GavemmentEugeneLarbr Monoging Director TREND GroupPaulineFoFi Neehes ProgramOfficer UNDPCotinsAnnoh Consultant World BankED. Monnor DeputyMinister M W.HNA. Asmoh Chief Mechonicol Engineer WM D./A.M A5k Darkwen DeputyDirector Water M W.HDI F. Agyopong DeputyDirector Housing M.WJf.

Annentable315Country meeting,Bolivia

Name Title Organization

Walter Franca ResidentRepresentative UNDPAntonio Vigilante DeputyResidentRep UNDPGerordaBerthin Protect Officer UNDPlsabellet~rordat-Berg ResidentRepresentative World BankFemandoTomeyo NationalSecretory Urban AffairsJorgeColderon National Directorate Water& SanitationJoseDecker Director PROSABAR ProtectFidel Herrera Director IPTKRicardoGonzales Technical Advisor PROSABAR ProtectVirginia Chumocem SacrolAdvisor PROSABAR ProjectMarco Quirago TechnicalAdvisor PROSABAR ProjectRamirolporre Head Basic SanitationUnitCustovoClovqo Ministry of Health

AnEvaluatron ofthe (JNDP—WorldBank WaterandSanitationProgram 95

AnnextaSe316

Bilateraldonors

None Organization Country

KlausKrosse GE? CastaRicaWhom Ankersnrit DGIS HollandHughesLa Mosson FormerlyCaisseFrancoise FrancePiersCross Mvula Trust SauthAfricaMonoGtodrtsch NORAD NarwayIngvar Anderson SIDA SwedenAtistarr Wray DDA United KingdomPaulPeter SDC SwitzerlandGuy Carrier CIDA CanadaHookvon Schatk FormerlyDGIS Netherlandslos C Baye-Molter DANIDA DenmarkHennngJensen DAJ4IDA DenmarkEddyPerez EHP U.SA.

96 An Evaluationofthe UNDP—WorldBankWaterandSanitationProgram

Annex 4

UNDP—World Bank Waterand Sanitation Program:

Criteria for Selecting Projectsof Opportunity

The following criteriashould beappliedwhenselectingprojectsofopportunity within which the Programwill work.

Project characteristics

• Vehiclefor structuredlearning in the Water andSanitationSector—The projects’ objectives should besuch that theyprovideavehiclefor carryingout our mandateof improving knowledgeandpracticein the waterandsanitationsector,particularly in the

theme areas established by the Informal Institutions Group estab-

lished under TWUWS.

• Adaptiveandflexible design—Theprojectsshouldbe basedon aflexible and adaptive design that will use a learning processapproach during their implementation.

• High potentialimpacton futurepracticeandprojects—Thepro-jectsshouldhavehighpotentialfor havingan impacton futureprojects and sector initiatives both in the countryandglobally.

• Sufficientresources—Theprojects should have sufficientresources assigned to them to ensure a high standard of prepara-

tion and supervision. Within each project sufficient resourcesshould be allocated to manage the learning process.

An Evaluationofthe UNDP—WorldBankWaterand SanitationProgram 97

Project justification

• High potentialimpacton sectordevelopment—Theprojectshould have a high potentialfor havingan impacton improvingservicedeliveryandsustainabilityin the country.

• Collaborativesectoroperatingdivision (SOD)—Theprojectshould have a task manager anddivision chiefwho arewilling tocollaboratewith the Programand who supportinnovationandsystematiclearning.Theyshouldalsobe willing to takeastandon principles and maintain support throughout the projects’evolution.

• Cooperativegovernment—TherecipientGovernmentshouldsupport thelearningapproachandprinciplesembodiedin theprojectdesign.It shouldalsoprovideareceptiveenvironmentinwhich reformscanbe initiated.

Project location

• Basedin a counttyofconcentration—Theprojectshouldbelocatedin acountryof concentrationfor TWUWS in which suffi-cient resourcescanbe broughtto bearto providethe high level

of strategicsupervisionrequiredto ensureasuccessfulconclusionandahighstandardof documentation.New countriesof concen-trationcanbe identified in a dynamicprocessas newprojectsofopportunitypresentthemselves.

98 AnEvaluationof the UNDP—WorldBank Waterand SanitationProgram

Annex 5

Organization of UNDP—World Bank Water andSanitation Program and Countries of Concentration

(as ofNovember1996)

EastendSouthernAfricaN~UrobI

L (I2~~~~)Etluopu~TanzaniaUganda

(World Bank a tke .x.cvttagagency)(l3steHm.mberi)

RwSOSouthAsiaNewDe~

(34~affmembers)BangkdeshIndiaNepalPakistanSnLanka

EastAsia andthePacificJakarta

(10 staff members)IndonesiaLao PORMangokaPhdippinanVietnam

RWSGAndean Ro~en

LoPez(8staffmembers)

BobeaEcuador

Mnex table51Programstaffing

ProgramMonagenMnanishatooBudgetAnalystCommunicotionSpecialistComn~undyDevelopmentSpeanlistBudgetAssistantRumi WS & S SpecialistSeniorTechnkalAssistanceOfficerOpemnomAssistantSupportStaff

~~damR.~oeRWSG AndeanNetwork Coordinator(vacant)

SupportStaffBolMa Engineer

4

UNDP-WorldBank Water and 5a~tat1oaProgramWasWugto~,DC

~WSG

Want Africa

(10st~tbers)BeamBuikma FosoGhwm

Location DesaipifonInternational

staffRegional

staffNational

staff

Ha

2

AnEvaluationoftheUNDP—WorldBank WaterandSanitationProgram 99

Annex tnkde 5.1

frternatlonal Wonallocation Description staff staff

Nationalstaff

Ecuador Chief Technical AdviserTechnical Support AdviserCommunity Development SpedalrstSocial Development Spedalist

East and Southern Africa RegionRWSG Regional Manager I

Assistant Program OfficerRegional Specialist 1Water & Sanitation Specialist 1Support Staff 5

Ethiopia Country Officer 1

West Africa RegionRWSG Regional Manager 1

Community Development Specialist 1Community Water Supply SpecialistSanitary Engineer 1Urban Planner 1SupportStaff 5

GhanaBeninGuinea

East Asia wid Padflc RegionRWSG Regional Mannger 1

Program Officer 1Support Staff 5

China TranslatorLao PDR Chief Technical Adviser 1Philippines National Country Officer 1

United Nations VolunteerIndonesia National Country Officer 1Vietnam National Country Officer

South Asia RegionRWSG Regional Manager

Regional Program AdviserSupport Staff 5

India Team leader 1ConsultantHRDAdviser I

Bangladesh Team Leader 1Consultant 1Protect Officer 1Technical Assistant Officer 1HTMP Coordinator & 2 Assistants 1SupportStaff 3

Nepel JAKPAS ChiefTechnical Adviser 1.IAKPAS Consultants 6SupponrStaff 3

Pakistan team Leader 1Engineers 2Support Staff 3

100 An Evaluationofthe UNDP--Worid Bank Waterand SanitationProgram

Annex 6

Summary of Recommendations

Chapter 2: Building National and Local Capacity

It is recommended that:

• The Program assist governments in developing capacity-building

strategieswhich shouldinclude: aregularreview of capacity-

building needs at all levels, identification of institutions andorga-nizationsthat canbestfulfill theseneedsandcoordinationoftheir activities.

• The Program adopt a new strategy toward the ITN program,

basedon the concept that each ITN center developsits ownobjectives,guidingprinciplesandbusinessplans,andthattheProgram ensure that the ITN centersareincludedin thedevelop-ment of nationalcapacity-buildingstrategies.

• The ProgramassisttheITN centersin identifying suitableorgani-zationsfor externalsupport,assistanceandadvice.

• Modelsbe developedthatdemonstratehowto increasethecapacityof beneficiariesto repairandmaintaintheir systemswith

minimal externalassistance.• Models should also be developed on how to make optimal use of

the NGOsystem without creating parallel structures.• The Program focus on making the link betweenpilot projects

andlarge-scaleinvestmentprojectsby implementingdemonstra-tion projectsandparticipatingin the preparationandevaluationof large-scaleprojects.

• The Programensurethatnew developmentsandlearningat alllevels, around the world are captured and fed into the sector.

• The Programcontinueto emphasizetheneedto involve thediverseelementsof the community,especiallywomen,in theplanningandoperationof waterandsanitationsystems.

AnEvaluationofthe UNDP—WorldBankWaterandSanitationProgram 101

a TheProgramseekwaysto apply the participatory approachinthe sectorata nationalscale:.

Chapter 3: The Effectiveness of Policies, Strategiesand Plans

The EvaluationTeamrecommendsthat:a The Programidentify on acountry-specificbasisthe reasonswhy

new policies are not always being implemented and identify the

mostappropriaterole theProgramcanplay in eachcase.

• The Programdevelopas soonas possibleagroupof casestudiesbasedon countriesthathaveimprovedthe well-beingof theircitizensin general,especiallythe poorest,as aresultof havingdevelopedandappliedpolicies,strategiesandplansbasedon theDublin principles.

a The Programcontinueto help countries develop and apply poli-cies,strategiesandplansbasedon the Dublin principles.

• The Programsponsorwell-thought-outstudytours for politiciansandseniorsectoradministratorsfrom countrieswishingto adoptnew policy frameworksto countriesthathaveone.Thesecouldbe followed by acontinuingmentoringprocesswhereeffectiverelationshavebeenestablished.

a Given the enormous disparity betweenthe numberof peoplewhohaveaccessto potablewater andthosewho haveknowledgeof appropriatehygienepracticesandaccessto adequatesanita-tion facilitids, the Programassigna higher priority to hygieneandsanitationthanit hasin the past.

• The Programrecognizethat individual capacity-buildingthemesor solutions,suchas Womenin Development(WID), technology,training,policies,strategiesandthe Programitself aretools thatarenot universallyapplicable.

Chapter 4: Impact on Domestic and External Investment

Thereis astrongneedfor demonstrationprojectsto testcommunity-wideapplicationof waterandsanitationsectorpolicies andapproaches.Theseareexpensiveto implementandoftenexceedlocal financingcapacity.But theyareessentialprecursorsto thelargeinvestmentprojectsthatlenderssuchas the World Bankareorganizedto make.The Programshouldadvocateto the WorldBankandthe regionaldevelopmentbanksthatwater andsanitation

102 AnEvaluationofthe IJNDP--WorldBankWaterand SanitationProgram

demonstrationprojectsbe financedandmonitoredunderproject

preparation funds.TheProgramshouldgive greaterpriority to micro-levelprepara-

tion of investmentprograms,which includespreparingprojectpro-posalsandfinancingschemes,developingpilot demonstrationprojectsandupscalingpilot anddemonstrationprojects.This canbeaccomplishedby providingtraining or capacitydevelopmentin thevariouscountriesregardingtheseaspects,ratherthanrelying on the

Program’sregionalofficesto undertakethem.The regionalstaffshouldnot bein-houseconsultantsfor the

countriesin which theyarelocated.Theymust continueto focusonthe objectivesof the Programwithin eachcountryand leaveoncethey haveachievedtheseobjectives.Otherwise,the Programwill bespreadtoo thinly andwill lose its uniquepositionandcompetence.

Chapter 5: Systematic Learning and Exchange ofExperience and Information

The Programshoulddevelopa strategyfor systematiclearningfromexperiencewithin andoutsidetheProgram.This shouldtakeaccountof thedifferent requirementsat the levelsof pilot, demonstrationandlarge-scaleinvestmentprojects.It should includeat least:

• A definition of the Program’sapproachto structuredlearningasan operationalconceptandprocess;and

• A conceptof how to ensurethat learningexperiencesfrom aroundthe world arecapturedandcommunicatedas rapidly as possible.The Programcould makemoreinnovativeandcreativeuseof

availableinformation-sharingmethodsandtechnologies,such as aninteractivee-mail dialoguesystemamongregionsor portableaudio-visual demonstrationsusingvideocassettes.It couldbe usefulfor

the Programto maintainanelectronicbulletin board,the equivalentof the telephoneyellow pages,with the namesandcoordinatesofwho knowswhat.

The Programshoulddevelopa systemfor measuringthenumberof usersandusefulnessof the informationthat is beingdisseminated.

Chapter 6: Sector Coordination and Collaboration

The EvaluationTeamrecommendsthatthe Programshouldmoresystematicallyandconsistentlymakeuseof its independentimagebybeingmoreactive in sectorcoordination.This could include:

An Evaluationofthe VNDP—WorldBankWaterandSanitationProgram 103

a Assistinggovernmentsin the developmentandnormalizationofcoordinationmechanismsat the nationallevel, particularlyfacili-tating andenhancingcoordinatiorLbetweengovernmentagenciesactive in thesector.

a Providing forumsfor informal andnon-threateningcommunica-tion mechanismsamongESAs, NGOsandthe privatesector,andbetweenthesegroupsandgovernmentagencies.

a Taking alead role in initiating anddevelopingformal andinfor-mal coordinationandcommunicationmechanismsat theregionallevel,especiallyin connectionwith theestablishmentofthe regionalorganizationsof the GlobalWaterPartnership.Thispositionmight includeclearlydefininganddisseminatingastate-ment of its own role in the sector,thusencouragingthedefini-

tion of the respectiverolesof theotheragenciesin thesector,includingUNICEF, WHO andthe regionalbanks.The Programshould enhanceits collaborationwith otheragencies

andorganizationsactive in the sectorby regularlyinvolving all part-nersin definingProgramprioritiesandactivitiesandworking moreactivelywith otheragenciesandorganizationsin internationalwork-ing groups,including thoseof the CollaborativeCouncil, thataredealingwith specificsectorissuesanddevelopingtools for the field.

Chapter 7: Program Management

Mission(i) It is recommendedthatProgrammanagementreviewthemis-

sion statementpreparedby theEvaluationTeamandconfirm itsvalidity or modify it in consultationwith all of thepartnersin the

Programstrategy.(ii) It is recommendedthatthe Programmanagersestablishthe

strategyfor thenextphasethroughan iterativeparticipativeprocess,

beginningwith a tentative,centrallydeterminedstrategybasedoninformationgatheredby thetripartitereviewsandthis evaluation,and by makingoptimumuseof theProgram’scomparativeadvan-tage.Countryandregionalconsultationsto establishspecificregional needswouldusethis draftstrategyas backgroundinforma-tion andsuggestchangesthat might bettersatisfytheir needs.Finally, managementshouldfinalize the strategyin consultationwiththe principal partnersof the Program.

Workplanning, budgetingandmonitoringsystem(iii) It is recommendedthatProgrammanagement:

104 AnEvaluationof theUt’IDP—WIorldBank WaterandSanitationProgram

• Make a greatereffort to recordandanalyzehow staff time isspent.In this it should revampactivity codesto makethemmoreuseful(managementhasalreadysetout to do this in responsetothe internalaudit report).

• Conductreviews,at leastsemi-annually,but preferablyquarterly,of performanceagainstinput andoutput indicatorsandactivities.Thesecanbe primarily for internaluse,althoughtheymight becited informally in discussionswith the World Bank, UNDPand

otherpartners.• RequireeachProgrammanagerto evaluateperformanceof his

staff in part usingtheresultsof thesereviews.(iv) It is recommendedthatthe inputsandoutputsof thePro-

grambe identifiablewithin thework program,budget,andreportsof the Division.

Programmonitoringandevaluation(v) It is recommendedthatthe Programtakeresponsibilityfor

helpingto establishbaselinecriteria to be usedin determiningwherethesectoris todayandin monitoringits movementover time. Itshoulddo this by:• Developinga clearsetof monitorableindicatorswith which to

measureprogressagainstobjectives.• Makingoneof its high-priority activitiesthe upgradingof systems

to measureaccessto safewatersupplyandsanitationby thepoorin ruralandmarginalurbanareas;encouragingotherinternational

agenciesworking in thesectorto adoptasimilarapproach;andadvocatingthattheyadoptandusetheseindicators.Oncesuchcriteria exist,conductingwith its partnersa participa-

tory self-evaluationof progressagainstthe indicatorsfor eachof itsobjectivesduringthe annualwork planningexercise.This evaluationshouldincludethenumberof personsservedfor eachdemonstrationor large-scaleinvestmentproject. The Program’sAnnualReportshouldincorporatethe findings of thisexercise.

Personnelmanagementsystem(vi) It is recommendedthat the Vice President,Environmentally

SustainableDevelopment,requestthatthe SeniorVice President,ManagementandPersonnelServices,designatea resourcepersontoassistthe Programin designingpersonnelpoliciesandproceduresmoresuitableto its organization.The UNDP andthe Bankcouldthenbe askedto put thenew policiesinto effect for the remainderof thecurrentproject.

AnEvaluationofthe UNDP—WorldBank WaterandSanitationProgram 105

Otherwork processesandprocedures(vii) It is recommendedthat the Programmanagerarrangefor a

reviewof work processesat headquartersandin the regionstodeterminewhetherdifferentwork methodsandprocedurescouldimproveefficiency.The OrganizationandBusinessPracticesDepart-ment (OBP) might be ableto assistwith this exercise.

Staffandskills(viii) It is recommendedthatProgrammanagerssystematically

reviewprofessionaldevelopmentneedsandopportunitieswith eachstaff memberandjointly agreeon an appropriateprogramby nolater thanthenextstaffperformancereview.

(ix) It is recommendedthat Programmanagementreviewlan-guagerequirementsfor staffing in eachregionandthatplansbedevelopedto meettheserequirementsthroughrecruitmentor train-ing. It is further recommendedthatlanguageskills atheadquartersbe developedsothatheadquarterscancommunicatedirectly with

theregionalstaffs.

Managementstyle(x) It is recommendedthat theProgrambe namedas the Water

SupplyandSanitationProgramof the GlobalWaterPartnership.(xi) It is recommendedthatthe role of theAdvisory Committee

be expandedto approvingthe strategicplanfor the Programandreviewingprogressagainstthe objectiveseachyear.

(xii) It is recommendedthatthe Programencourageits staff toparticipatein the committeework of the CollaborativeCouncil by

planningandbudgetingtime for this activity. The formal relation-ship with theCouncilshouldbe strengthened,perhapsby formalrepresentationon eachother’sgoverningor advisorybodies.

(xiii) To achievethe senseof teamworkthat is soughtwithin theProgram,it is recommendedthatthe Programmanagerinvolve man-agersandstaffof the regionaloffices andheadquartersin designingandimplementingwaysto respondto recommendationstostrengthenthemanagementof the Program.

Structure(xiv) It is recommendedthatopportunitiesfor furtherdecentral-

izationto the RWSGsbe examinedin the reviewof work processes.Considerationshouldbe given to usingthe RWSGsas facilitators inestablishingthe regionalgroupsunderthe GlobalWaterPartner-ship. Decision-makingauthorityshouldbe delegatedto the regionalmanagerson issuesfor which theycanexercisethis authority

106 AnEvaluationof thet)NDP—WorldBankWaterandSanitationProgram

responsibly,thus enhancingthe prestigeandeffectivenessof theregional groups.

Chapter 8: Program Financing

It is recommendedthatProgrammanagersestablisha fundingpoolsimilar to thesuggestedwaterfund—withthe size basedon anagreedstrategyandwork program.A percentageof all fundingshouldbesetasidefor corecosts,as determinedby an agreedbud-get. As the principal corefunction is the distillation anddissemina-tion of worldwide learningin the sector,this sharemight be calledthe learningfund.Theremainderof the funding could be earmarked

by thevariousdonorsto the elementsof the Programthat theychooseto support.Giventhe impactof the Programon the capacity

of countriesto implementandsustainlargeinvestmentprojects,theBankshould contributesignificantly moreto the Program,say 15%of the total cost.

Chapter 9: Conclusions and Recommendations

It is recommendedthatthe Programbecontinued,maintainingitsdistinct identitywithin the GlobalWaterPartnership.The Programhasbeentransformingitself while helping to transformthe sector.This is commendable.It shouldcontinueto evolve in responsetochangesin the environment,thussettingan examplein the sectorofcontinuousimprovement.Nevertheless,this evaluationprovidesanopportunityfor stock-takingandsignificantcoursecorrection.Throughoutthis reportanumberof recommendationsaremadeforimproving the Programandits approachto the sector.Althoughtheseweremadein the contextof specificsubjectareas,suchas capacity-building or effectivenessof sectorpolicies,theyareinterrelated.

The first threeof the following recommendationsareintendedto•improvethe way the Programachievesits immediateoperationalobjectives.Thelast two will helpthe Programdo to this, but con-cern internalchangesin the managementandfinancingof thePro-gramto ensureits efficiencyandsustainability.It is, therefore,recommended:• That theProgramcreate,or causeto be created,programsor

projectssupportingdirectuser-to-userassistanceor mentoring.Thesewould bringtogetherthosewho havealreadybeenthroughthe processof introducingwatersupplyandsanitation

AnEvaluationof theTJNDP—WorldBankWaterandSanitationProgram 107

services,usingthe approachesrecommendedby the Program,with otherswho arebeginningthe process.Usersshouldmanagetheseprograms.The RWSGs,fl’N centersor otherNGOsandESAsmightplay facilitating roles.The Programshould recom-

mendmoreselectiveuseof individual capacity-buildingapproaches,suchas emphasizingthe roleof WID, appropriate

technology,training andfocuson policiesandstrategiesbasedonspecificcountryneeds.

• That the Program assist governments in identifying the obstaclesto implementingnewpolicies anddeterminewhatrole it mightplay in assistinggovernmentsto overcomethem.Casestudiesshouldbe developedbasedon successfulexamplesof demand-driven,bottom-upsectorpoliciesandstrategies.Theseshouldbepromulgatedthroughsuch meansas theProgram’sIndia Think

Tank, the annualWestAfrican Retreat,workshopsandseminars.Study tourswith follow-up andsupportshouldbe arrangedforsectorpoliticiansandadministratorsto visit countriesthathavesuccessfullyputsuchpolicies into effect.The Programshouldprovidepolicy adviceandtechnicalassistancethat give higherpriority to hygieneandsanitation.

• That theProgrammakegreaterefforts to learnfrom theexperi-enceof othersand to documentanddisseminatelessonslearned.It shouldmakemoreinnovativeandcreativeuseof availableinformation-sharingmethodsand technologies,for example,aninteractivee-mail/dialoguesystemamongregions,andportable

audio-visualdemonstrationsusingvideo cassettes.• Thatthe Programmanagershouldconducta full reviewof the

managementof the Program.A reviewof the purpose,strategyandpriorities for operationsandresearchshouldbe conductedjointly with the Program’spartners,includingbothdonorsandparticipatingbeneficiaries.Staffing,structure,systemsandproce-duresshouldflow from the strategyandpriorities. Reorientingthe relationshipbetweenheadquartersandthe RWSGs,andbetweenthe Programandtheotherpartners,is fundamentaltoachievingthe objectivesof the Program.

• Thatthe Programmanagersseekto establisha funding pool simi-lar to the suggestedwaterfund.The size woulabc basedon theagreedstrategyandwork program.A percentageof all fundingwould besetasidefor corecosts,as determinedby an agreedbudget.As the principal corefunction is the distillation anddis-seminationof worldwidelearningin the sector,this sharesetasidemight be calledthelearningfund. The restof the fundingcould be earmarkedby the variousdonorsfor the elementsof the

108 AnEvaluationof the UNDP—WorldBankWaterand Samtat:onProgram

Programtheychooseto support.Giventhe impactof thePro-gramon thecapacityof countriesto implementandsustainlargeinvestmentprojects,the World Bank shouldcontributemuchmoreto the Program,say15% of total costs.TheProgramis amodelfor the GlobalWaterPartnership.Its

strategicor comparativeadvantagesarethoseneededby thePartner-ship as an advocateof integratedwaterresourcemanagement.Asnotedabove,the EvaluationTeamrecommendsthat the Programretain its distinctidentity as the programwithin the GlobalWaterPartnershipthatis dedicatedto ensuringtheprovisionof safewatersupplyandsanitationfor disadvantagedrural andurbanpopulations.Basedpartlyon the evaluationof the Program,adis-cussionpaperwas submittedto the foundingconferenceof theGlobalWaterPartnership.This paperrecommendedthat thegover-nanceof the Partnershipencompassatrue partnership,decentraliza-

tion to the regionalandthe countrylevel, respectfor the autonomyof thedonorsandthe beneficiarycountriesandtransparencybased

on clearobjectivesandmonitoredcriteria andreporting.

An Evaluationof theUNDP—WorldBankWaterandSanitationProgram 109

I