1st part making sense of data in emergencies
TRANSCRIPT
1
1st PartMaking Sense of Data in Emergencies
• The impact of emergencies on information• Lack of information• Information and decision-making• Features of information• Sources of information flaws• Making sense of poor data
S.C. October 2003
2
How do emergencies impact on information?
• Routine information systems collapse:– Poor communication,– Reduced access limited coverage
• Population movements lack of reliable denominators
• Collection & analysis of data clash with other priority activities
• Increased background noise
3
Lack of information?
• Lack of information as the strongest indicator of a crisis
• Lack of information as the best piece of information for crisis decision-making
but
Lack of information is not = to lack of data
4
From facts to decisions in an IDEAL world
Judgement / decisions / actions assessing options /
Decision-making attributing values
Facts/events deaths collection / recordingData nr. of deaths countingIndicators mortality rates in sorting, selecting
a pop.,area, period calculatingInformation rates by sex, age, analysis
provenience, etcKnowledge / trend analysis, interpretation
Understanding comparison withother areas, benchmarks,
contextual information
Steps example intelligence at work
5
From facts to decisions in the REAL world
• Decision-making involves political and personal values, especially in complex emergencies:
• “priority setting is ultimately concerned with power relations and value judgements” (Green & Barker, 1988)
6
(Non)-information and (non)-decision-making
• Informational exclusions
• Non-decision making as a frequent strategy in difficult contexts
• The extreme case of non-decision making:“paralysis by analysis”
• Decision-making without information
7
Features of information in emergencies
• Timeliness • Validity • Relevance• Precision:
– “precise quantification of incidence and prevalence rates rarely influences the choice of programmes offered by the health services” (Sandiford, 1992)
• Cost
8
The opportunity cost of information in emergencies
Cost
: ca
pacities
& o
ppor
tunities
9
Validity and Precision
C. Precision only
A. Both validity and precision B. Validity only
D. Neither validity nor precision
Jekel, 1996
10
Bias versus Lack of Precision
• Assuming that in the area X in August 2002 the “real” (unknown) CMR is 1.6/10,000/day
– Valid (unbiased) & precise estimate: 1.5 (1.4 - 1.7)
– Valid & imprecise estimate: 1.4 (1.0 – 2.2)
– biased estimate: 0.6 (0.3 - 0.8)
– imprecise estimate: 1.2 (0.8 - 1.9)
– biased & imprecise estimate: 0.6 (0.1 - 1.2)
11
Common sources of information flaws
• Inconsistent denominators• Biases • Inconsistent levels of aggregation • “Brand”, bad indicators• Incomplete reporting• Over-reliance on quantitative techniques
and/or presentation tools, applied to poor data
12
Making sense of poor data
• Linking indicators related to the same areas and produced by different sources,
• comparing them for reliability,• checking suspect data at their sources,• comparing them with other standards,
triangulating them with other sources,• inquiring why certain data are not reported,• discarding irrelevant and/or clearly unreliable
data,• validating data, information and interpretation
with knowledgeable people
13
Statistics, epidemiology and intelligence
• “For scientists, linked to respectable statistical methods, complexities can be an unmanageable nightmare: for, if they simplify them until they are measurable, they destroy the complexities ..” (Chambers & Toulmin, 1989)
• Statistical and epidemiological techniques can only rarely be applied in the messy context of an emergency,
• however, they are crucial, together with field experience, for developing hunches for figures and guiding the analyst in gradually making sense of poor data
14
2nd part:Health Needs Assessment
in Emergencies
• NA as a function of health intelligence• NA: meaning, subject matter, & scope• Definition and classification• Key questions that NA address in an emergency• Preparing for a NA• An example of template• tips
15
Main functions of Health Intelligence
Assessment on-off past, present &future
Evaluation on-off past (present)
Monitoring continuous past, present &future
Function Periodicity Time scale
16
Needs Assessment in Health
What are we talking about?• Needs? • Risks / capacity?
What is the subject matter?• Health status of the affected population?
• The determinants of ill-health?
• The performance of health services?
17
Needs Assessment in Health (cont.d)
Context:• NA in emergency?• NA in recovery / reconstruction?
Scope:• area-based?• Country / sector-wide (CAP, reconstruction)?
18
What a RHA is and for what is used?
“collection of subjective and objective information in order to measure damage and identify those basic needs of the affected population that require immediate response”
From: RHA protocols for emergencies, WHO, 1999
19
Health assessment and information systems
• Routine:– Surveillance Systems– Health (Unit-managed) Information Systems– Civil registration (vital statistics)
• Non-routine:- Rapid Health Assessments (RHAs)- Surveys
20
Non-routine data collection methods in emergencies
Type When What How
Rapid reconnaissance
Immediately after a disaster
A quick, preliminary inspection of the disaster area
• Satellite imagery • Overflights • Mapping • Drive/walk through
RHA As soon as it is
possible to go to the area
A quick collection of information to confirm the emergency, measure the impact, identify health needs and guide response
• Visual inspection • Analysis of records • Interview of key
informants • Rapid surveys
(MUAC, etc)
Surveys When the situation stabilises and response has been activated
A detailed study in which information is systemati-cally collected in a sample of population (morbidity, mortality, nutrition, KAP)
• Probability sampling • Non-probability s.
21
HypotheticalTimeframe
Alert Decision 1
• Whathappened?
• Where?• RHA?• When?
Pre-assessmentsitrep
??
Preparation
• To do what?(ToRs)
• Which team?• Where?• How?
( methods)• How? (logistics)
few hours/days
RHADecision 2
• Emergency?Y/N
• What needs?• What
constraints?• What local
resources?• Response?• Which one
(which resources)?
• What evolution?
1-4 days
Emergency
0
Analysis
• Action?• If yes,
which one,whichresourcesrequired?
possiblyduring RHA
Referencevalues
Sitrep
• Presentsmainfindings
• Givesrecommen-dations:
• What to do?• Why?• By whom?
1 day
ActionDecision 3
• Who doeswhat?
• When?• How?• With whom?
??
Political/financialconsiderations
Needs assessment and decisions: the time sequence
22
Which are the key questions in a RHA?
• Is there an emergency or not?
• What is the main health problem?
• What is the existing response capacity?
• What decisions need to be made?
• What information is needed to make these decisions?
23
Which information?• The population:
– numbers, characteristics, & trends– morbidity and mortality
• The vital needs:– security– food– water– shelter & sanitation– clothes and blankets– domestic utensils and fuel– health care
• The support systems:– information– logistics– coordination– resource flow
• Other relevant contextual issues
24
Main steps of a RHA
• Set the assessment priorities
• Collect the data:
– reviewing existing information
– inspecting the affected area
– interviewing key people
– carrying out a rapid survey
• Analyse and interpret the findings
• Present results and conclusions
25
Preparing for a RHA
• What should I know before going to the field?
• What methods are appropriate, considering:– the given emergency context, and
– the security, time, logistic, technical constraints?
• What composition of the NA team?
• Which logistics, communication & transport?
26
What is available in the EHA webpage?
• Short introduction with objectives• SitrepTemplate• Instructions• Reference Values
27
28
29
Advantages of using a standard template
• Quicker and comparable analysis
• Ensures all important items of information are included
• Consolidates information from different sources into a single document
• But, remember, the standard template is just a guide and can be adapted to your needs
30http://www.who.int/disasters/sitrep.cfm
EHA first sitrep
31
Reference Values
32
RHA: a few tips (1)• Concentrate on your sector, but don’t lose sight of
the context
• Concentrate on the NOW, but look at the past(WHY?) and think of the future (...WHAT IF...?)
• use local knowledge:– “a good rapid assessment is one that accesses as broad a
cross-section of local knowledge as possible” (Collins, 2001)
• Don’t create excessive expectations! A NA, as a rule, should be followed by response
• Share with your team, report to your HQ, but leave something for who remains in the field
33
RHA: a few tips (2)
• Don’t be too ambitious: time is short
• Being roughly right is generally better than being precisely wrong or precisely late
• Beware: wrong conclusions from the RHA can do more harm than not taking any action
34
What is needed for a RHA?• Clear lines of authority and reporting• Partnerships• Division of responsibilities and agreed procedures• Maps• Transport• Radio or mobile/sat phones• Tent, food?• Security clearance• Qualified personnel• Interpreters (if no local collaborators are part of the
team)• Data collection forms, containers for specimen, other
equipment• Guarantee of follow-up (response, other assessments)
35
CMR: reference values
• “normal” in developed countries: 0.2 per 10,000 per day• “normal” in developing countries: 0.5 “ “ “• crisis (under control): < 1.0 “ “ “• very serious: > 1.0 “ “ “• out of control: > 2.0 “ “ “• famine, major epidemic,
catastrophic: > 5.0 “ “ “• Ajiep, South Sudan, in 1998: 26.0 “ “ “• Rwanda, 1994: 19.4-30.9 “ “
36
Defining humanitarian needs
• basic human needs (“food is a basic need”)
• lack of - or deficit in - the above(“these people need food”) -norm / capacity, not risk
• need for relief assistance (“these people need food aid”) -response
Adapted from Darcy & Hofman, ODI, 2003
37
Defining needs• Need for a health care intervention, an intermediate
state (a product of derived demand in economics)• Need as the capacity to benefit from health care
(economist’s view concerned about the change in state), implying that an effective intervention must be available
• Need for a desiderable end state (e.g. health)• Normative need: I.e. defined by professionals (an
example of a socially constructed need)• Felt need/wants: individually experienced• Demand: expressed need• Comparative need: against a reference group• Need as a deficit: the difference between a norm -
2,100 Kcl- and the reality (the actual intake of calories)
38
Defining risk
• Hazard (threat) x vulnerability• it is a mix of probability and consequences:
• risk of an earthquake is the probability of its occurrence
• risk from an earthquake refers to its consequences (death, injury, etc)
From R.Doran, 2002
39
3rd part:The practice of health needs assessments.
Lessons from Southern Iraq
• The environment for NA• The outbreak of Nas• Assessing for what?• Recurrent flaws in Nas• Lessons learned• Conclusions
40
The environment in Iraq in the aftermath of the war
• Limited humanitarian space• No major humanitarian crisis under way • High-visible crisis, whose political overtones
sidelined health-related considerations • Limited knowledge of the health sector by most
participants• Unique features of the crisis and of the health
sector, not anticipated by newcomers
41
An outbreak of needs assessments?
• Many agencies embarked in needs assessments, which became their many and often only activity
• 1,600 health facilities (75% of the entire network) were inspected between April and June
• Coverage was very uneven. Many facilities were assessed several times, whereas ¼ of the network remained inaccessible
• A new geography of accessibility: little attention to systemic issues and administrative division of the country
• The supposed scope of the needs assessments shifted towards lying the grounds for the formulation of reconstruction plans
42
Assessing for what?
• Different instruments and criteria were applied, generating a wealth of data impossible to aggregate
• Compiling these data was compared to ‘herding cats’(and generated an analogous outcome)
• The yield of this impressive activity was remarkably modest in terms of understanding the overall situation of the health sector.
• The discussion about reconstruction, started in June, relied on approximate estimates based on heroic assumptions, formulated in the absence of hard data, despite the previous enormous fieldwork investment
43
Some recurrent flaws in needs assessmentsmay be recognised across crises:
– To apply a micro approach to macro issues– To opt for rapid assessment techniques because of
their apparent simplicity or ease of application– To mistake rapid needs assessments with the data
collecting tools designed to contribute to them– To apply a short-term, survival-oriented perspective
to problems calling for long-term, recovery-oriented solutions
– To jump to assess end-results of long chains of events
– To assess needs with already a response in mind– To limit only to those needs that are "easy" to
assess
44
Lessons learned from the Iraq crisis
• Tap local knowledge from the beginning of the crisis• Approach issues systemically rather than piecemeal• Concentrate on a few selected critical areas calling
for urgent intervention• Work inside the system to the maximum extent• Verify in the field whether the identified priorities
are truly such• Co-opt NGOs to support this strategy• Advocate for restraining from hurried-up initiatives
or for measures not called for by true needs• Start long-term processes, such as human resource
development, as soon as possible
45
Conclusions
– In many complex situations, the primary source of intelligence is the unused information, scattered around in the health environment and waiting to be tapped
– No formalised, ‘technical’ approach can substitute common sense, a solid cultural background, field experience and familiarity with the issues at stakes