Surveillance, Monitoring & Evaluation (SME) System Assessment in the GMS

------Findings & Recommendations

Surveillance, Monitoring & Evaluation (SME) System Assessment in the GMS

------Findings & Recommendations

Bayo S. FatunmbiMonitoring & Evaluation, ERAR-GMS

ERAR–GMS M&E activity (Workplan, 2014)

• Principle: – Evidence based system strengthening

• Key objectives:– To give an overview of the different aspects of

surveillance systems in the six different countries,– To highlight gaps and examples of good practice,

and chart key indicators and system characteristics.

– To guide future efforts to strengthen malaria surveillance across the GMS.

Methodology:

• Rapid ‘desktop’ analysis in selected countries (May 2014)– review of available policies, plans and other documentation

• Detailed review of the malaria SM&E status of the different malaria control programmes was conducted between October 2014 and February 2015. – Key Informants Interviews (using modified M&E assessment tool) and – Observation (direct and indirect) during field visits to different

administrative levels of the surveillance systems• Data analysis and validation• Reporting:

– A detailed and fully referenced report is available for each individual country in addition to a regional summary.

Thematic areas

Malaria context

Organizational structure and human resources

Stratification and targeting

Passive case detection and reporting

Reporting, data management including analysis

Case Notification, case and focus investigation

Response mechanisms

Cross border activities and coordination

Operational research

Others

Malaria surveillance guidance documentation and indicators in the GMS, 2014

Vietnam Myanmar Thailand Lao PDR Cambodia ChinaAvailable documentation            Surveillance plan/strategy l l

Elimination plan/strategy l l

National Strategic Plan Malaria l l l l l l

Monitoring and evaluation plan l l l l l

Malaria surveillance SOPs l l l

Well-defined indicators l l l l l l

IndicatorsAccess to early diagnosis l l lPrompt and effective treatment l l lLLIN and IRS coverage l l l lStratification status of admin areas l lElimination status l l lKnowledge and behaviour change l lMalaria mortality l l l l l lAnnual Parasite Index l l l lSurveillance and epidemic response l l l lDrug availability l l lPrivate sector l l

Some countries have surveillance and response tools, some do not have. Majority of the available documents are outdated. Existing malaria indicator framework is weak in terms of malaria elimination.• need updating to reflect the mood of the day – accelerating towards malaria elimination. • Indicator standardization and harmonization across the region is required urgently.

Organisational structure of malaria surveillance – System is weakIndicator Vietnam Myanmar Thailand Lao PDR Cambodia China

Malaria Surveillance within the wider health system

Vertical MIS in place l l l l lEssential malaria data from MIS linked/integrated to HMIS l l lCommunity-based malaria report systemIn place l l l l l lHarmonised within country l l lFully linked within MIS l lOrganisational structure for surveillance:Clearly documented l l l lRoles and responsibilities clearly defined l lAnnual updated register or database of health facilities/reporting units clearly exists

l l l l l l

A complete register/database of village volunteers and mobile volunteers for health facility/district exists

l l l

Identified focal SM&E person at NMCP l l l lSufficient SM&E capacity and staff time at NMCP l l lTraining and SupervisionSM&E training plan in place lFrequency of training available every 2 years not regular not regular every 2 years annuallyMajor constraints reportedLack of sufficient training l l lLack of proper staffing l lLack of sufficient supervision l lTraining materials availableFor community-based personnel l l l l lFor data management, analysis, interpretation l l l l l lFor case investigation, ACD, outbreak response lCoordination mechanismsMechanism for collaboration on policies and strategies for malaria control between NMCP and relevant government sector

l l l l l

Mechanism for collaboration on policies and strategies targeting vulnerable populations between NMCP and different health programmes

l

Mechanism to review SM&E data, coordinating external technical support

l l l l

Malaria stratification and PAR targetingIndicator Vietnam Myanmar Thailand Lao PDR Cambodia China

Formal stratification exists l l l l l l

Updated in previous five years l l l

Incorporates malaria case data l l l l l

Village level stratification l l l l

Specific SOPs for stratification l l l

Stratification used to estimate PAR l n/a l l l

As at time of assessment, no published evidence of current efforts to target and strengthen malaria surveillance in specific high-risk groups was available. However, several innovative partners’ efforts across the sub-region were reported during Yangon, Hanoi and Phuket MMP meetings. • There is a need to carry out a more extensive survey of stakeholders and partners to

systematically document relevant approaches to learn from best practice in the region.

All six countries have a formal system for malaria stratification in place, but there is considerable variability in the methods and data used. Incidence-based stratifications incorporate a variety of thresholds to define individual strata in different countries. • There is a need for standardising classification schemes across the region.

Passive case detection components within NMCPs (1): Health Facility based

  Vietnam Myanmar Thailand Laos Cambodia ChinaPublic health facilities            Policy for always testing before treatment

l l l l l l

Occasions when treatment may happen without test

l * l l l l

Separate system for HIS/other nationally collected data

l * l l l

Residence (village level) captured l l l l lCase classification: local/imported captured

l l

Written SOPs for data collection available l l l l lMonthly case reporting aggregated l l l lElectronic aggregation l lFully electronic reporting system lVillage disaggregated data available at national level

l l

Adequate systems in place to check completeness of health facility data

  l partial n/a

Adequate systems in place to check completeness of health facility data

    l   partial n/a

TrainingTraining includes data entry * l l * lTraining includes data manipulation (indicator calc)

* l * *

Training includes data management l * l l *

Passive case detection components within NMCPs (2): Community based

  Vietnam Myanmar Thailand Laos Cambodia China

Community based systems VHVs in some remote areas

VHVs VHV, VMWs VMW and VHV VMW NONE

Standardised case management guidelines available

l l l l n/a

Do testing l l l l l n/aDo treatment l l l l n/aSystem for recording malaria test and diagnosis

l l l l l n/a

Data combined with public health facility data

l l l l * n/a

Case classification: local/imported captured l n/a

Written SOPs for data collection available l l l l n/a

Indicator: which units did not report l * n/a

Standardised arrangements for dealing with late reports

* n/a

Initial training for community workers l l l l l n/a

How regular is training for community workers

* * annual 2 years n/a

Incentive structure in place l l some areas only l n/a

VHV monthly salary/incentive l l l l n/a

Approximate/example salary amount 40 USD n/a 20USD 12 USD 10 USD n/a

Transport subsidies   l l     n/aTargeting of VHV system stratification strata 1+RAI A1-A2 areas strata 2-3 distance from

health facility, enlarged spleen rate and distance to forest

n/a

VHV coverage review policy l l n/a

Confirmatory diagnosis of malaria as a surveillance tool: quality assurance processes in the GMS

Indicator VN MM Thai Lao PDR Cambodia China

Microscopy            

Microscopist paid specifically to prepare/read slides l          

Routine testing of microscopists – reference slides/slide bank

l l      

Is there a refresher training for microscopists that do not pass routine testing?

l l l l l

Is there a system of accreditation of microscopists (WHO ECA accreditation)?

l l l   l

Is a sample of slides routinely submitted for cross-checking in a reference?

l l l l   l

RDT    

Is there a system for training and re-training of health workers in use of RDTs?

l l l l l

All malaria endemic countries in the GMS have adopted the policy of confirmatory diagnosis of suspected malaria in the national and regional malaria strategies. However, the quality assurance system remains a suspect in many countries.• QA diagnosis resources within and outside the sub-region should be optimized

Key indicators of malaria surveillance in the private sector

Indicator VN MM Thai Lao PDR Cambodia China

Microscopy            

Microscopist paid specifically to prepare/read slides l          

Routine testing of microscopists – reference slides/slide bank

l l      

Is there a referesher training for microscopists that do not pass routine testing?

l l l l l

Is there a system of accreditation of microscopists (WHO ECA accreditation)?

l l l   l

Is a sample of slides routinely submitted for cross-checking in a reference?

l l l l   l

RDT    

Is there a system for training and re-training of health workers in use of RDTs?

l l l l l

In all countries it is currently legal for some or all entities in the private sector to diagnose and treat malaria. Registration of private facilities varies between the countries. The private sector is extensively used in most countries especially by migrant populations.• Efforts to capture private sector data should be intensified.

• Data entry forms should incorporate private data. • Supportive supervision and regulatory measures institutionalised by authorities at

all levels

Key indicators of management and analysis of malaria dataIndicator Vietnam Myanmar Thailand Lao PDR Cambodia China

Use of database for reporting l l   l lCase-based data entered into electronic system l l   l lData entered into electronic system at low levels (below province)

l l l   l l

Electronic system covers whole country     l   l lRoutinely integrates data from community l l l l l n/aRoutinely integrates data from private sector     l l lRoutinely integrates data from other (military, worksites etc.) l l l lSOP for data quality exists   l l   l lMechanism to monitor timeliness of reporting   l l   l lMechanism to monitor completeness and mistakes l l l l l lMechanism for addressing missing data l     lDesignated staff for monitoring data quality l l l l l lAdequate IT infrastructure at national level   l   lAdequate IT infrastructure at lower levels l   l   l lUse of SMS/internet/email for reporting l   l l l lAdequate backup systems   l     lFocal points for data analysis exist at different levels l l l     lAutomated analysis used l l l l lAnalysis at peripheral level l l l      Analysis at province/district levels l l l     lAnalysis at national level l l l l l lGeographic information systems used     l l l lFormal feedback system in place     *   lSpecific training on data management and analysis l l   l

Situation varies in countries. Poor performance on some indicators cuts majority of GMS countries e.g. formal feedback mechanism, backup systems, analysis and use of data for action at lower levels, IT infrastructures. • Policy intervention may be needed in some countries to strengthen data management, especially on

information sharing for cross border and regional level actions

Case notification, case and focus investigationIndicator Vietnam MM Thailand Lao PDR Cambodia ChinaCase notificationMalaria a notifiable disease l l lCase-base reporting implemented l l l lIf yes, is it nationwide RAI areas l lFirst level of notification township township townshipTimeline for notification/reporting within 2

weeksdaily daily

Non-public sector sources included lCase investigationStandard system in place l lRecording of number of cases investigated in place

  l l l l

Standardised malaria case investigation form

  l l l

Locations mapped as part of investigation

  l l l

Ecological/entomological data routinely collected?

  l l

Roles and responsibilities defined   l l lTraining materials, SOPs in place   l l lSupervision systems in place  Incentive mechanisms in place   l

System of incentives differ. How to sustain incentives for active case detection? Experience across countries will be useful

Response mechanisms (1)Indicator Vietnam Myanmar Thailand Lao PDR Cambodia China

Formalised system to connect surveillance system outputs to responses

l l l l

System functional l n/a l n/a l

Outbreak detection, investigation and control

Outbreak detection included in NSP l l l * * l

Outbreak detection system in place l l l l l

Outbreak detection/guidance in place l l l l

Does a national outbreak plan exist l l

Is outbreak plan adequately resourced n/a n/a * n/a n/a

Software/electronic system for detecting outbreak at national level

l l (xl) l

Outbreak detection system effectively implemented l l

Roles and responsibilities for outbreak detection and response documented?

l l *

Roles and responsibilities for outbreak detection and response clear and implemented?

l *

Outbreak investigation SOPs in place for those (formally or informally) implementing

l *

Outbreaks consistently identified and investigated l

Plans for development of improved tools and systems

l l

Emergency stockpiles of insecticides and antimalarials

l l l l l

Emergency funding in place * l l

Emergency funding sufficiently flexible n/a n/a n/a n/a l

Response mechanisms (2)Indicator Vietnam Myanmar Thailand Lao PDR Cambodia China

Other responsesACD response l l l lACD during/following outbreak investigations l n/a l l n/a l

ACD is a routine response l n/a n/a lACD methodology is standardised l n/a l n/a lVector control as a response l n/a l l n/a lVector control response is routine and defined l

IRS response * * l lIRS is a routine response * n/a * n/a lLLIN distribution as a response * n/a * l n/aLLIN is a routine response * n/a * n/aClear and standardised response deployment system/thresholds in place for ACD

l n/a n/a l

Clear and standardised response deployment system/thresholds in place for vector control

n/a n/a l

Lack of staff to respond to available data raised as an issue

l l l

Tracking of migrants raised as issue l lReactive case detection (RCD)RCD used in country l l lFormalised system for RCD exists l n/a n/aRCD only in context of outbreak control l n/a n/aRCD used in foci investigations (active/inactive)

n/a n/a n/a * l

Other case detectionAny further case detection activities outside outbreak control/response

l l l l l

Border screening RAI areas l n/a l lMass blood surveys l limited areas RAI areas n/a l

Mobile malaria clinics l

Operational research structures and prioritiesIndicator Vietnam Myanmar Thailand Laos Cambodia China

Operational researchOR coordination mechanism in place l l l l n/aOR priorities focusing on vulnerable populations l l l l l

Evidence that OR outputs feed into control/elimination strategies

n/a l l l n/a

Current operational research prioritiesImproved diagnostic techniques for surveillance l

G6PD tests and rollout of Primaquine (PQ) lDrug discovery lApproaches for elimination, alert, response, modelling, etc.

l l l l

Molecular surveillance for drug resistance l l lMMPs and border areas l lVector control (outdoor transmission) lNew mHealth evaluations l

Recommendations (Regional level)

1. Develop adequate and up-to-date guidelines and policy documentation at country and regional level

2. Improve case-based automated data management3. Strengthening linkages with the private sector4. Harmonize elimination and resistance SM&E

indicators5. Strengthen SME capacity at all levels of NMCP6. Avoid multiple reporting systems and transition to

case-based reporting and investigation systems only where viable

Perspective

• Diverse and rapidly changing surveillance situation in GMS countries: – many documents, strategies and plans are actively

under development, with countries at different stages of positioning themselves for elimination

• This assessment should be regarded as a working document, a ‘baseline’ or starting point, from which countries can be best assisted to transition their surveillance from malaria control into an elimination setting.

Way forward

• Publication– WHO Technical Report (Approved IPPF by WPRO)– Peer review papers (during dissemination in Nov 2015)

• Dissemination meeting planned for 13 November 2015

• Update vital national malaria tools– NSPs, M&E Plans, Regional SME Framework, SOPs, etc.– Resource mobilization for SME system strengthening

• Support countries to implement recommendations and assess performance in system strengthening

Acknowledgement• National Malaria Programs in the Greater Mekong Sub-region (including

collaborating sectors and programs)– Kingdom of Cambodia– People’s Republic of China– Lao People’s Democratic Republic– Republic of the Union of Myanmar – Kingdom of Thailand, and – Socialist Republic of Vietnam

• Malaria Consortium (WHO technical partner): conceptualization, field work, data management, reporting

• Development Partners:– Financial support from Bill & Melinda Gates Foundation (BMGF) and Australia Department

of Foreign Affairs & Trade (DFAT)• Other technical partners at regional and country level • Expert reviewers – internal and external to WHO for • WHO colleagues (at all levels for technical inputs)