29th-2 nd /03/2009 conference afrea - nonie - 3ie cairo , egy pt
DESCRIPTION
29th-2 nd /03/2009 Conference AfrEA - NONIE - 3ie Cairo , Egy pt Olivia Nakayiza & Luc Vanhoorickx – Save the Children in Uganda. Programme Overview. - PowerPoint PPT PresentationTRANSCRIPT
Experiences with Lot Quality Assurance Sampling Cairo – April 2009
29th-2nd/03/2009 Conference AfrEA - NONIE - 3ie Cairo, Egypt
Olivia Nakayiza & Luc Vanhoorickx – Save the Children in Uganda
Programme Overview Save the Children in Uganda (SCiUG) is a unification of Save the Children
Norway (Managing Member), Save the Children Denmark, Save the Children USA, Save the Children UK, Save the Children Sweden and Save the Children Italy
In 2008:• SCiUG programmes in over 26 districts
in 4 regions of Uganda• SCiUG works with 59 partners including
local NGOs, CBOs and local government.
• Total 313 staff as per December 2008• Reached 213,051 children
(116,137 girls & 96,915 boys)
Strategic Choices
In the year of unification, Save the Children in Uganda (SCiUG) developed a strategic framework that builds both on the strengths of participating members as well as on emerging analyses of the child rights situation that calls for more integrated programmingSCiUG has also continued to scale up its emergency response capacity to deal with emergency contexts, focusing primarily on emergency education and child protection
Community Based Family Planning Program Background
Increased voluntary use of key family planning services and RH behaviors among women and men of RA
A 4 year USAID/FLEXIBLE Fund Community Based FP/RH- Luwero, Nakaseke and Nakasongola Districts in Uganda
Women of Reproductive age (15-49) 68,826, Married WRA 33,036, Men 68,423. Total # of beneficiaries 170,285
Partnerships: District, MoH, etc
FP/RH Strategies and Intermediate results
Partnership Defined Quality (PDQ) process……
Behavior Change Communication, …
community-based services to increase access to FP methods ,...
Build capacity of district staff, …
strategic objective : “Increased and sustainable use of key family planning services and RH behaviors among women and men of reproductive age”.
INTERMEDIATE RESULTSIR-1: Increased knowledge, interest and use of FP/RH services
IR-2: Improved quality of FP service delivery by providers at the facility and community level
IR-3: Increased access of communities to FP services and information
IR-4: Improved social and policy environment for FP/RH services and behaviors
Results FrameworksWhere is results monitoring?
Goal
Strategic Objective
IR 1 IR 2 IR 3 IR 4
Strategies
Activities
Impacts
Outcomes
Inputs & Outputs
Time
Longer Term
Medium Term
Shorter Term
Check change in knowledge, behavior, status
RESULTS MONITORING:
Monitoring & Evaluation => LQAS
Higher level monitoring
Measure outcome indicators : compare with baseline & final targets
=> information needed = population/beneficiary based
=> sampling needed !
Effects of Behavior Change campaign: … behaviors
LQAS = Lot Quality Assurance Sampling
“determination of the quality of a lot by sampling”
comparison between areas (lots) possible small sample size, typically 19 per area
From the Monitoring we knew that we were generally carrying out the activities as planned, but wanted to Monitor&Evaluate whether it all had the desired effect/result/outcome/impact
Lot = sub county from a district
What is LQAS?
•A sampling method that: Can be used locally, at the level of a “supervision area,” to identify priority areas or indicators that are not reaching average coverage or an established benchmark
•Can provide an accurate measure of coverage or health system quality at a more aggregate level (e.g. program area)
For example, what LQAS can: just by sampling 19 women in a targeted population, at least 92% of the time LQAS will determine correctly whether yes or no these women have adopted the family planning method.
LQAS: dichotomic - 19Lot Quality Assurance Sampling in general small sample of 19 provides an acceptable
level of error for making management decisions; at least 92% of the time, it identifies whether a coverage benchmark has been reached or whether a supervision is substantially below the average coverage of a program area.
Samples larger than 19 have practically the same statistical precision as 19. They do not result in better information, and they cost more
Only if it’s above/below target
Decision Rule
LQAS cannot be used for coverage estimates in lot / district!
Some indicators that were measured
• Contraceptive Use (CU) or Contraceptive Prevalence Rate (also known as Met Need)Percentage of women of reproductive age (WRA) 15-49 who are married or in union using (or whose partner is using) a modern method of family planning
• Percentage of women of reproductive age (WRA) currently married or in union who are fecund (not pregnant or unsure if they are pregnant and not sterilized) who desire to have no more children or postpone childbearing, and who are not currently using a method of family planning (Unmet Need for Family Planning)
• Percent of Demand (Met need + Unmet need) satisfied• Knowledge about family planning methods:
FP METHODS THAT WERE MEASURED• Depo-Provera• Pills• Condoms• Norplant and Tubal Ligation
LQAS in Family Planning project
Effects/outcomes of project interventions:… behaviors
% of women of reproductive age who report having access to FP services% of women of reproductive age who were counseled about the birth spacing
Sampling: Systematic Random Sampling from …• 5 lots were chosen for different districts• 2-step random sampling procedure. Communities were selected using
systematic random sampling. HH selected using classical random sampling
Data collection tools: Adapted from FlexFund survey guidelinesData analysis: Standard LQAS tables in Excel worksheets for analysis & graphs
As the mid-term evaluation of the Family Planning project
Process
5 “supervision areas” Random samples of 19
from monitoring records of women of reproductive age (WRA)
Interchanged extension workers for data collection
Minimal data-entry where PDA were used
Instant results Fast feedback
LQAS Summary Tabulation
Indicators
Priorities where Target ( T ) and/or Coverage ( C ) is not achieved
TargetLQAS Coverage
95% CI
Lot 1 Lot 2 Lot 3 Lot 4 Lot 5
% of women 15-49 years who are not pregnant or are unsure, who are using a modern family planning method (Contraceptive Prevalence Rate or Met Need) OK C OK OK OK 50% 68.9% 10.8%% Women of reprod. age know about at least three methods of family planning (Knowledge about Family Planning Methods) OK OK OK OK OK 70% 100.0% 0.0%% of women of reproductive age that receive counseling about birth spacing C OK C OK OK 60% 68.8% 8.7%% of women of reproductive age who report having access to FP services OK OK OK OK OK 70% 96.0% 3.3%% of women of reproductive age who report discussing FP with a health worker or family planning worker/promoter T & C OK T OK OK 70% 67.4% 9.5%% of WRA (15-49) currently married or in union who are fecund (not pregnant or unsure if they are pregnant and not sterilized) who desire to have no more children or postpone childbearing, and who are not currently using a method of family planning (Unmet Need for Family Planning) N/A N/A N/A N/A N/A 30% 34.2% 13.6%Total Demand for Family Planning 93.1% 4.9%Percent of Demand Satisfied 72.6% % of sexually active respondents who report discussing FP issues with their spouse or (cohabitating) sexual partner in the past 12 months OK T T & C OK OK 60% 54.3% 10.2%% of respondents that lives within 5 km of a family planning service delivery point (SDP), [among women who know where to obtain a method] (Proximity to Family Planning Service Delivery Point) C C OK OK OK 30% 73.0% 8.7%
FP LQAS OUTCOMES LQAS Family Planning '07
68.9%
100.0%
68.8%
96.0%
67.4%
34.2%
93.1%
72.6%
54.3%
73.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
ContraceptivePrevalence Rate
Knowledge aboutFamily Planning
methods
Counseling aboutbirsth spacing
Access to FamilyPlanning services
Discussing FamilyPlanning withhealth worker
Unmet Need forFamily Planning
Total Demand forFamily Planning
Percent ofDemand Satisfied
Discussing FamilyPlanning with
spouse of sexualpartner
Proximity toFamily Planning
Service
LQAS Coverage
Target
Average Coverage
LQAS Health outcomes
48.0
%
96.9
%
96.0
%
36.5
%
84.9
%
79.9
%
78.4
%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
Daily water use percapita above 20l
Appropriate handwashing behavior
Bring their childrento appropriate
provider with fever
% of womenreporting currentuse of moderncontraceptives
Givingcomplementaryfoods at 4 to 6
months
Give more fluidsduring diarrhea
episode
Know 2 or morecauses for growth
faltering
Baseline
MTE
FY07
Final Target
After the quantitative survey…
• LQAS gives the numbers (it’s quantitative) Identifies the problem How big the problem is Where the problem is
• BUT NOT why we have the problem… Hence the need for qualitative research
/discussions to o Explain quantitative resultso Identify the reasons whyo Provide recommendations on program design
and strategies
Lessons Learned (1)
• Internal interviewers were used but swapped to reduce biasQuantitative LQAS survey replicated yearly
• Although some statistical analysis is not possible with LQAS, indicators can be analyzed further to reveal additional informatione.g. analyze which ‘part’ of knowledge / behavior adopted => focusing messages
Lessons Learned (2)• Results discussed w/ program staff during
participatory feedback meetings: discussions & decision making w/officers + review meeting w/extension staff
• Guided by principal areas for improvement identified in LQAS surveys: provide recommendations to guide the team in second half of the project and inform planning
PDA Experience/Lessons learnt (3)
• PDA to be easy to carry in the pocket of the riding gear
• PDA contained automatic skip patterns
• Of completeness of the data, all required questions were responded to
• a reminder in most required questions in the PDA which could beep
• Once an electronic questionnaire was filled in, data is instantly stored and possibilities were built-in to review the data
• data is clear in terms of spellings and uniformity
• Data entry time is obviously reduced to zero
Experiences with Lot Quality Assurance Sampling Cairo – April 2009
Thank You !
LQAS & the MTE