The Meaning of “Evidence” and Strategies for Incorporating Evidence into Practice

Using Concept Mapping

“Both/And” Research Conference

University of Victoria, May 10, 2013

Bernie Pauly and Marjorie MacDonald

Overview of RePHS

Principal Investigators: Marjorie MacDonald, Trevor Hancock and Bernie Pauly, University of Victoria

ON Academic Lead: Ruta Valaitis, McMaster University

ON Decision-maker Lead: Heather Manson, PHO

Funder: CIHR

Time Frame: January 2009 – January 2014

Purpose: To examine the implementation and impact of public health renewal processes using two public health programs as exemplar cases - chronic disease prevention/healthy living and sexually transmitted infection prevention across the two provinces.

Authors – on behalf of the REPHS Team

University of Victoria

• Marjorie MacDonald

• Bernie Pauly

• Diane Allan

• Laura Tomm Bonde

• Trevor Hancock

McMaster University

• Ruta Valaitis

• Danielle Hunter

BC Ministry of Health

• Warren O’Briain

Public Health Ontario

• Heather Manson

Concept Mapping

• Concept mapping is a structured conceptualization process used to develop a framework of participants’ understanding of a particular concept and the processes related to the concept

• Done in 3 phases:

– brainstorming

– sorting and rating

– interpretation

Phase 1: Brainstorming

• Prompt: Effective strategies to support the integration of evidence into public health practice and/or decision making are: …….

• 18 people from BC and 55 people from ON responded to the brainstorming survey

– BC: all roles and 4 of 6 HAs + Ministry represented

– ON: all roles and 6 of 6 HUs + Ministry + PHO represented

• Brainstorming statements were synthesized resulting in 27 statements

Phase 2: Sorting and Rating

• 14 BC and 21 ON respondents

• All roles (FLS, Managers, Executives) represented in 35 respondents

• BC: 5 of 6 HAs + Ministry + Other represented

• ON: 5 of 6 HUs + Ministry + PHO represented

Multi-Dimensional Scaling and Cluster Analysis

• Sorting data entered into analysis program for concept mapping (Ariadne)

• First, data are entered into similarity matrices and then analyzed using multidimensional scaling

• Then the program conducts a cluster analysis to group the statements into related clusters as reflected in a point

Concept Map

Cluster 1: Fostering innovation, learning and an evidence-informed public health culture

1: Involve all levels of public health practitioners in the process of creating, gathering, and integrating evidence into practice

4: Ensure support and/or commitment in place from leadership to integrate evidence

6: Ensure formal processes and structures for integration of evidence are consistent across the health unit/health authority

7: Have flexibility and be willing to challenge standard practice and change practice as needed

11: Foster an evidence-informed organizational culture

16: Incorporate evidence into program planning and evaluation in an iterative fashion

Cluster 1 continued

18: Value all forms of research as evidence

22: Promote external partnerships/collaborations in the development and use of evidence

23: Promote reflexivity in public health practice

25: Promote greater visibility of public health

26: Ensure that innovation can still happen within evidence-based practice

Cluster 2: Building staff capacity

2: Provide education, professional development, staff training, and/or skill building specific to research skills for staff

14: Ensure accessibility of knowledge brokers or content experts

19: Ensure staff has time to seek out and integrate evidence into practice and decision-making

27: Integrate research skills into all public health practitioners’ role/practice/scope

Cluster 3: Structural supports for developing, accessing and using evidence

3: Create government funded evidence-informed resources specific to public health

9: Provide access to librarian support

12: Provide examples of how evidence can be integrated into and impact practice

13: Provide access to relevant literature, resources, and/or databases

15: Ensure some staff members are equipped with advanced skills in research

17: Establish internal forums or networks to enable staff to share and review evidence

20: Make evidence understandable

24: Provide financial resources for research

Cluster 4: Supporting relevant, public health specific evidence

5: Utilize standard models/frameworks/tools for gathering and/or integrating evidence

8: Review, appraise, and incorporate multiple sources of evidence

10: Generate research that guides public health practice

21: Create or establish context specific evidence

Pattern Matching: Importance and Feasibility

Importance

Rank Mean

1 Cluster 2 4.19

2 Cluster 1 4.16

3 Cluster 3 4.09

4 Cluster 4 4.01

Feasibility

Rank Mean

1 Cluster 3 3.38

2 Cluster 1 3.09

3 Cluster 4 3.07

4 Cluster 2 2.95

Importance and Feasibility: BC and Ontario

Cluster 1 Go-Zone

Cluster 2 Go-Zone

Cluster 3 Go-Zone

Cluster 4 Go-Zone

CONCLUSIONS

• This study, grounded in the experiences of both practitioners and decision makers, provides some concrete direction about possible actions to support evidence-informed practice that are identified as both feasible and important

Next Steps

• Analyze interview and focus group data from front line public health practitioners, mid-level managers, and senior executives about:

– understandings of the meaning of evidence,

– barriers and facilitators to implementation of strategies to support evidence use

– What strategies are being used and how

– Explanations for differences between BC and Ontario, and between importance and feasibility ratings