Data Quality Assessment of PEPFAR ART Sites

In Nigeria

Final ReportFebruary 17, 2006

Nigeria/Monitoring and Evaluation Management Services in collaboration withMEASURE Evaluation

What is Data Quality?

Data Quality fundamentally means

that the information collected as

part of a program’s monitoring and

evaluation system adequately

represents the program’s activities.

Adequately Represents?

The information is accurate and

reliable. Accurate information is

interpreted as measuring what we

intend to measure (that the

information is correct).

Role of Data Quality in the Emergency Plan

PEPFAR emphasizes data quality because the Emergency Plan is explicitly evidence based and results oriented

Good data are needed to inform the design of interventions and monitor and evaluate the plan’s quantitative progress toward pre-determined treatment, prevention, and care targets

Hence, if the effectiveness of the emergency plan is evaluated on the basis of numbers, then any doubt about those numbers makes the entire program vulnerable to criticism

Data Quality Assessment of PEPFAR ART Sites in Nigeria

Assessment PurposeAssessment Purpose

Determine quality of data used to report on PEPFAR ART indicators

Examine data collection, management and reporting procedures

Ascertain site capacityfor M&E

Document degree of datasharing and harmonization among PEPFAR, GON, and other HIV/AIDS Programs

Assess cross-agency data comparability

Methods and Facilities

Field Visits – August 30 – September 2 2005

Seven out 9 PEPFAR’s targeted States covered

16 facilities out 23 assessed Qualitative and Quantitative techniques

Mapping of patient flow within facility Structured interview Observations Accuracy checklist Geographic Information System (GIS)

coordinates

Nnewi

ART Assessments took ART Assessments took place:place:

Data Quality Assessments Data Quality Assessments MeasureMeasure

ValidityValidity: Does data adequately represent performance?

ReliabilityReliability: Are data collection and management procedures consistent?

PrecisionPrecision: Do data have sufficient detail to present a fair picture of performance?

TimelinessTimeliness: Are data current and collected with required frequency?

IntegrityIntegrity: Are data free of manipulation for political, professional or personal reasons?

DQA MethodsDQA Methods

Mapping

Questionnaire

Interviews

Participant observation

DQA Findings: DQA Findings: SuccessesSuccesses

M&E personnel were well trained and highly motivated

Written/posted M&E guidelines and flow charts at many sites

All sites had or were about to implement an electronic data

reporting system

Well coordinated ART patient administration, testing, treatment, counseling and tracking systems

DQA Findings: DQA Findings: ChallengesChallenges

ART eligibility guidelines not uniform among sites

Physicians complained about number and redundancy of required forms

Forms and databases not uniform across IPs, preventing unified reporting system

Tensions between PEPFAR IPs providing services at same site

DQA Findings: DQA Findings: Challenges (cont.)Challenges (cont.)

Outsourcing of lab tests hampered document monitoring

Lack of coordinated supervisory system led to fragmented data monitoring and processing delays

Inadequate Staff and equipment

ConclusionConclusion

The provision of ART services was a ‘work in progress.’

Most sites had only just started operating when assessments occurred –two clinics had not yet opened

PMM harmonization across all ART sites in progress

For the complete ART Assessment report, log on to: www.nigeriamems.com

Updates after Assessment

USG-SI-Advisor convened SI-working group

USG through IPs have engaged GON on needs for harmonization of tools

USG-supported GON in the harmonization process

Stakeholders’ meeting held Current PMM tools under review and

awaiting approval by GON New harmonized tools to be piloted in

scale facilities

DISCUSSION and

QUESTIONS