context and mixed methods final - université de montréal · benefits of using a logic model ......

Putting together multiple interventions, context and mixed

methods

Nancy Edwards, RN, PhDUniversity of Ottawa

January, 2008

Addressing critical challenges in the field of community health research

Explore contextIntroduce Multiple Intervention Program (MIP) Framework Public health toolsPromising options

Recognizing context

Researcher’s Context:Disciplinary roots and traditionsMethodological and theoretical familiarityPrevailing values and priorities (historical and contemporary)Why do you study what you do in the way you do it?

Disciplinary “Treatment” of Context(Edwards & Clinton, PHAC Context Background Paper, 2008)

“harness” context as a critical element of intervention strategies (e.g., organizational psychology, political science) acknowledge influence of context with focus on proximal contexts (e.g., behavioural and child psychology)treat context as a confounder or nuisance variable (e.g., epidemiology)

Implementation Context(Greenhalgh, Robert & Bate, 2004)

Inner program context - proximalOuter program context - distal

Inner Program ContextProgram parametersFunding requirements and constraintsCapacity to deliver interventionCompeting priorities among those responsible for program implementationLeadership and championsOrganizational mandates and change processesResults-based management, accountability

Outer Program ContextCommunity readinessCommunity leadership Social structures and disparitiesExisting infrastructure (built environment, legislation) Buy-in, strengths and working relationships among partner organizationsPolicy directions, policy windowsPolitical ideologies

Interventions are products of their context (Pawson, 2006)

Contextual layers:individual capacities of key actorsinterpersonal relationships supporting the interventioninstitutional settingwider infrastructure system

“Interventions are continually affected by the interaction of these different layers” (p. 31).

What produces change?

“It is not programmes that make things change, it is people, embedded in their context who, when exposed to programmes, do something to activate given mechanisms, and change” (Pawson & Tilley, 1997)

“Generative Mechanisms”(Poland, et al., in press)

Act in each context, although differently between sites

Key, enduring features of context: tension between agency (people) and social structure (norms, resources etc.) power relationsprocesses of emplacement (location in time and space)

An intimate knowledge of context and dominant forms of knowledge

“The Long Exile” by Melanie McGrathForced relocation of Inuit People in 1953“All manner of people were consulted about what to do with the Inuit except the Inuit people themselves.” “But the culture of the Inuit and their way of living was finely-tuned to the terrain….. In order to hunt on the land, you have to know the land and the Inuit did not know the land where they were relocated.”

Context IS:(Edwards & Clinton, 2008)

Multi-level (socioecological models)Dynamic rather than staticPervasive, relevant context is not defined by scope of programContemporary and historicalKnowledge of context is deeply intimate

Social Justice and Public Health Core Competencies(Edwards & Davison, CJPH, In Press)

B Rosenkrantz – early on “public health was closely allied with social reform, without any apologies”Mid-20th Century - shift to reductionist thinkingIncreased demand for empirical evidenceSocial justice thinking considered to be “overly subjective”

Canada’s draft public health core competencies (2007) “bereft of a social justice orientation”

Elements of Context in MIP Framework

Contextual influences:Examine interconnections among determinantsConsider deeply embedded determinants

Contextual fit:Identify conditions required for contextual readiness

Adaptation to dynamic community context:Optimize synergies between interventions and context Examine differential impacts and spin-offs

Sample MIP Propositions (Riley & Edwards, Under Review)

The timing, the effort and the features of the intervention strategies are tailored to the implementation context

Interventions are continuously adapted to the contextual environment while maintaining integrity with theoretical underpinnings

Public Health Tools

Systematic reviewsLogic models

Systematic reviews – synthesis tools(Armstrong, Waters et al., 2008; Edwards & Clinton, 2008)

RCTs generally strip away context

Analysis of context cannot exceed descriptions of context provided in primary research papers

Settings - most common description of context in systematic reviews (from analysis of >25 Cochrane reviews)

The Public Health Regime(Asathan & Halliday, MillbankQuarterly, 2006)

“Decontextualized systematic reviews”Need to use a wider analytical framework in the field of health inequalitiesSome interventions work for certain conditions and in certain contexts but not in others – “conditionally successful” (Pawson & Tilley, 1997)“Public health regime – specific legislative, social, political and economic structures that have an impact on public health & the appropriateness and effectiveness of interventions adopted”

Addressing Context:Additional Requirements for Structured Abstracts?

What was the contextual readiness for the intervention and how was this assessed? What were the contextual eligibility criteria that guided sample selection?What were the dynamic features of context and how were these monitored and/or leveraged during the implementation period?

Logic Models DescribeWhat the program is supposed to do

Components and activitiesWith whom

Target groupsWith what

Resources (financial, human, physical)And why

Outcomes desired

Benefits of Using a Logic Model

Integrate what is known about cause and effect relationships (determinants)Systematically organize program design informationProvide means to incorporate relevant evidence from literatureSupport results-based management

Logic Model Elements

Components – what are the main sets of closely-related activities?Activities – what things are done, what services are delivered?Targets – for whom are activities designed?Short-term and long-term outcomes –derive from objectivesWhere oh where is context???

Logic models – planning toolsLinear orientation – if inputs provided in correct dose, sequence and intensity then outputs, outcomes and impact will resultProcess evaluation focuses on outputs, outcomes and impact rather than on throughputs and their interaction with contextThere is no explicit place for context descriptors within a logic model

Promising Possibilities

Source: http://www.follenderwerks.com/galleries/the_monarch_butterfly/images/monarch24.jpg

Whole systems change

“The uneven, nested cycles of adaptation that evolve within closely coupled, complex socio-ecological systems over time” (Marck, Higgs, Edwards & Molzahn, 2006; Gunderson & Holling 2002; Holling, 1998).

Theoretical basis for whole systems adaptation and change: Analytic and integrative science

Two fundamentally different but critically related forms of scientific thinking are important to understand population health interventions:

analytic sciences of parts (e.g. field of biomedical science)integrative science of the integration of parts (e.g. field of ecology)

Differentiating between analytic and integrative science research questions

Analytical research questions: “Which population health intervention works best?”

Integrative research questions: “What population health interventions are different sectors of our society most willing and able to support and how do these social values shape policy?” “How does the dynamic partnership between hospitals and communities shift when national policies actively discourage deliveries by Traditional Birth Attendants?”

Integrative Sciences

Contextual influences on changeThroughputs of change processesRipple effects occurring as a result of changeFeedback and adaptive processesInterplay among nested layers of a system that support or impede change

Mixed Methods

Paradigmatic and philosophical tensionsMixed research designs Mixed data collection methods A weak link: mixed methods analysis and interpretation

Looking back and looking forward: “Intervention” research

Emphasis to date:Fidelity of interventionStandardized protocolsControl co-intervention, reduce contaminationAttribution – what active ingredient(s) yield a particular effect?

Future emphasis:Responsive to community & to participants (contextual fit) Adaptation of intervention to dynamic context and to feedback processes (contextual adaptation)Longer-term spin-offs, scaling-up, sustainability (contextual uptake)

The roots of positive results:Population Health Programs versus MIP Trials

Population health programsAdaptabilityLong-term strategyGauge and respond to community readiness, changing capacity, policy windowsDynamic, real-time adjustments to changing context Horizontal and vertical scaling-up/positioning

MIP TrialsAttributionShorter-termStandardized protocolsActive ingredientsGuided by integrated theory

What can be learned about context?

Pop health changes-ve

Pop health changes +ve

MIP trial –veMIP trial +ve

Discrepant Findings: Opportunities to understand context

+ve pop health program, -ve MIP trial Temporal and dynamic contextual fitLeveraging context (context an active ingredient)Synergistic action with long-term investment-ve pop health program, +ve MIP trialBarriers to scaling-up (community readiness, infrastructure & policy support)Need for cultural or contextual adaptation as intervention is scaled up

Scaling-up Health Interventions in Poor Countries (Hanson, Ranson et al., 2003)

Identified constraints to implementation and scaling-up of 49 priority interventions selected on basis of: burden of illness, technical efficiency, feasibility of delivery, cost-effectiveness, demand characteristics

Examples of interventions: skilled birth attendants, insecticide treated bed nets, tobacco tax, antiretroviral treatment and home-based care for HIV/AIDS

Scaling-up Health Interventions in Poor Countries (Hanson, Ranson et al., 2003)

Constraints (context): SystemsProcessesIncentivesValues or normsSociopolitical environmentAbsorptive capacity

People’s Republic of China

Yunnan Maternal &

Child HealthProject,Yunnan

Province,China

Beijing

Yunnan Maternal and Child Health ProjectYunnan Maternal and Child Health ProjectPurpose:Purpose: To improve the quality of village life and

promote the development of productivity and social prosperity in poor ethnic minority

counties in Yunnan Province

Participatory training for grassroots workers

Essential MCH equipmentStrengthen MCH referral

mechanismsParticipatory monitoring

and evaluation

Co-constructing the language to describe core project elements

Sharing experienceModeling different forms of inquirySurfacing underlying assumptions

Threats to Project Sustainability(Edwards & Roelofs, CJPH, 2006)Uneven support for the training innovation across system levelsWork unit leaders expressed doubt about the utility and appropriateness of participatory approaches for village health workersSupervisory approaches mirrored the authoritative and didactic approaches to training that were standard practiceSignificant national reforms increased work demands and threatened job security

Developing Management Systems with Cross-Cultural Fit (Edwards & Roelofs, Intl J Health Plann Mgmt, 2006)

Systems dimensionsAdministrative and financial systemsGovernance structuresCommunication systemsMonitoring and reporting systemsHuman resource infrastructure

Areas of assessmentCultural contextCapacityRapid systems changeAccountability

Cross-cultural fit of management systems - examples

Where do the negotiation and expression of dissenting views happen: during public meetings, in private meetings or behind closed doors? How is approval for a change in policies communicated between different levels of the system? How is the balance between responsibility and accountability shared between managers and working level staff?

Yunnan, China – Lessons Learned

Authentic participatory approaches must be infused through all levels of the work including management structures, decision-making processes & evaluation approachesA deep mutual understanding of an intervention must be co-created by partners Never assume the intervention has “taken hold”

Context QueriesIntervention design: How would those who intimately know the context(s) advise us to adapt the intervention? What elements of the intervention can or should be adapted for a dynamic context?

Evaluation: What is the interface between the inner and the outer context and the intervention? How has context shaped the intervention? How was the intervention adapted to the context?

Context QueriesIs the intervention and/or the context the “active ingredient”?What assumptions are we making about the context for intervention implementation?What contextual conditions are necessary for:

Authentic community engagementProgram implementationSustainabilityScaling-up

Expand our lexicon of intervention studies

Questions of attribution:EfficacyEffectivenessEfficiency

Questions of adaptation:Contextual fitContextual adaptationContextual uptake

ImplicationsA shift towards adaptation queries requires fundamental rethinking of some of our most commonly used public health toolsBoth analytical and integrative science questions need to be posedMixed methods must extend to analysis and interpretation phasesMixed methods may be especially helpful in addressing vexing questions about context