PowerPoint Presentation

Building Evidence in PracticeModule 3Elaine A. Borawski, PhD

1If We Want More Evidence-Based Practice

We Need More Practice-Based Evidence.~ Lawrence W. Green, DrPHUniversity of California at San FranciscoOverviewUnderstand what evidence-based meansBecome familiar with evidence-based programs and how to locate themUnderstand why evidence-based practices are important in public and community healthBecome familiar with strategies for selecting an appropriate evidence-based practice for your own project.

3QuestionWhat do you think of when you hear the term evidence-based?

4AnswerAn evidence-based program has been:Implemented with a groupEvaluatedFound to be effective.

Evidence based first used as evidence based medicine i.e. best medical practices

What we mean by evidence based program has more to do with social science.Found to be effective by evaluationPrograms and evaluations reviewed by experts (peer reviewed)

5What is Evidence?Surveillance DataSystematic Reviews of Multiple Intervention StudiesAn Intervention Research Study Program EvaluationWord of MouthPersonal Experience

OBJECTIVESUBJECTIVE6How do we define evidence in science?Scientific evidenceisevidencewhich serves to either support or counter ascientifictheory or hypothesis. Suchevidenceis expected to be empirical evidenceand in accordance withscientificmethod.

What does it mean to have empirical evidence?Empirical evidenceis a source of knowledge acquired by meansof observation or experimentation.

Why the Fuss? More Federal funders are requiring program planners to use evidence-based programs.Some consider evidence that is proven through research (explicit).Some consider evidence that is derived from experience or practice (tacit).The best evidence may be a combination of research and practice.

9TYPES OF EVIDENCETYPE IEstablished through observational research. Provide evidence for a link between a preventable risk factor (e.g., sedentary lifestyle) and some specific health outcome (e.g., obesity). EXAMPLE: Individuals who exercise regularly have less cardiovascular disease, diabetes, and obesity. Many studies have confirmed this relationship.However, does not tell us, HOW to best help people to establish regular exercise routines. 10Community gardening and cornerstore project. These are two projects

Is there Type II evidence that these programs or approaches will change the eating behaviors.? Not yet.

What we know is this:Poor physical activity and poor diet are highly correlated with certain chronic conditions such as diabetes and heart disease.Individuals from impoverished areas have higher rates of chronic diseases such as diabetes and heart disease. They also tend to have poorer, less varied diets and report less physical activity than those living in more affluent areas.Individuals from impoverished areas have poor access to fresh fruits and vegetables and higher access to fast food outlets. They are also less likely to have access to environments conducive to physical activity.

These are all examples of Type I evidence.

TYPE II: Evidence-Based PracticesEstablished through clinical or behavioral trials. A specific intervention or approach has been found to effective in changing a behavior that has been linked to a specific health outcome EXAMPLE: A number of studies have shown that walking programs that include the use of incentives, motivational tools such as pedometers, and social support are more likely to result in participants adopting a regular walking schedule TYPES OF EVIDENCE11Research tells us that individuals with controlled diabetes are better at identifying carbohydrates in foods than those not in control (Type I)How do we take this information and turn it into an intervention? Carb-counting educational intervention then compare carb knowledge AND A1c levels 2 months later with group that did not receive the intervention.If it works, thats Type II evidence.If it doesnt theres still Type I evidence just not an evidence-based practice.12Heres some examples.

Sources for EvidenceRead, read, readAcademic journalsGovt websites (CDC, NIH)Advocacy websitesLay journals/mags that summarize researchGuide to Community Preventive Services http://www.thecommunityguide.org/index.html

13Now I know some of you are thinking.this is too much like more school.

Just like doctors much continually learn new diagnoses, treatments,

Lawyers must keep on top of the newest laws

As health educators, you must stay on top of the evidence in your specific area.

Lay jornals/mags like Prevention magazine;

http://www.thecommunityguide.org/index.htmlCovers a variety of public health topics14The Guide to Community Preventive Services website is one of the most useful that youll ever find.

Here are some screen shots

Recommendation SummariesDetailed Summaries of Studies Conducted15

In seven of eight studies reviewed, increases in the price of tobacco products results in decreases in both the number of people who use tobacco and the quantity they consume.

Systematic reviews at your fingertipsSee original publications for additional information

16Lets check out some evidence http://www.thecommunityguide.org/index.html

17Now I know some of you are thinking.this is too much like more school.

Just like doctors much continually learn new diagnoses, treatments,

Lawyers must keep on top of the newest laws

As health educators, you must stay on top of the evidence in your specific area.

Lay jornals/mags like Prevention magazine;

Your ExperienceWhat has your experience been with evidence-based programs?Where have you heard of them before? Have any of you used these programs in the past?Are any of YOUR programs considered evidence-based?

18Advantages to UsingEvidence-Based ProgramsEffective in the study populationsCost effectiveShorten the time it takes to develop a program Reduce the time it takes to research a communityHelp narrow the evaluation.

19Barriers to Evidence-Based ProgramsMay limit my/our creativity.Take too much time and/or money.Often difficult to replicate in community settings (translation).Too scientific. My community is unique. An evidence-based program will not be as appropriate as if I developed the program myself. I do not know what evidence-based programs are or where to find them.

Evidence based means that the program and evaluation have been peer reviewed. This allows the program to be described as effective.

20Finding an Evidence-Based Program21ObjectivesBe able to find evidence-based program resources.Know how to use search options to narrow your program choices and find out what programs will and will not work with your community. Alternative Sources for Evidence-Based ProgramsTalking With the Principal InvestigatorFinding an Evidence-Based Program: Case Study.

22Selecting Evidence-Based Community ProgramsPeer reviewed literature and researchNational Registry of Evidence-Based Programs and Practices (NREPP) http://www.nrepp.samhsa.govResearch-Tested Intervention Programs (RTIPs)http://rtips.cancer.gov/rtips/index.doGuide to Community Preventive Serviceshttp://www.thecommunityguide.org/index.htmlEvidence-basedTailored to the target populationConducive to health gainsModifyable Technologically and logistically feasibleReasonable costPolitically feasibleAddresses societal priorities

23Criteria for Selecting a ProgramThinking about your organization and the target population for your project:Was the program conducted with people who had similarSocioeconomic status ResourcesEthnicity Traditions PrioritiesCommunity structure and values.Is the program appropriate for the age of your audience?

Choose a program that is well-matched with: Your health topic (e.g., breast or cervical cancer, nutrition, physical activity)What your audience is already doing about the health issue.

Remember you will be adapting the program not changing it, so you need a program in line with your health topic.

24Criteria for Selecting a ProgramContext for interventionCoverage across the range of populations or setting involved in a health concernKnowledge of what populations or settings involved in a health concernKnowledge of what populations interventions will be effective for and under what conditionsRole of race, ethnicity, and cultureStaff creativity, experienceBalancing fidelity and adaptation

(Allensworth & Fertman, 2010)

25Criteria for Selecting a ProgramThese strategies can include:Giving informationEnhancing skillsImproving the services and/or support systems that existChanging incentives or barriers that maintain the problemPromoting accessMaking suggestions for policy changes.

26ResourcesRemember to avoid a program that takes more resources than you have. Different evidence-based programs will take different amounts of money, labor, and/or time. Whenever you can, speak with the team that developed the program or product in which you are interested. They can share information about the program that may be helpful.

27Evidence-Based Practice in Action: LifeSkills Training 28What is LifeSkills Training?Substance Abuse Prevention and Personal Development CurriculumDeveloped by Dr. Gilbert Botvin in late 1970sIdentified as a Program that Works by the Centers for Disease Control and Prevention Effective substance abuse prevention focuses on changing behavior

29The 6 Fundamentals of LSTThere are three Domains of Cognitive Behavioral TheoryLST is an evidence-based programLST changes thinking and behavior (cognitive-behavioral skills)Booster sessions increase effectivenessInteractive teaching methods enhance learningLess is more

30Brief overview of the major components of LST program and how it is structured. Remember our goal is to focus on and be effective in changing behavior. The 6 fundamentals of LST provide an understanding of how LST addresses the reasons and needs that you identified as important for students. Core components of the program. Explains how the program is different from other programs and why it works.

3 Cognitive Behavioral DomainsPersonal Self ManagementGeneral Social SkillsSocial Resistance Skills

Interactive Teaching SkillsFacilitationCoaching FeedbackBehavioral RehearsalsSummary of Evaluation ResultsMiddle/High School CurriculumReduces substance abuse by up to 87%Effective in reducing tobacco, alcohol, marijuana, inhalants, narcotics & hallucinogensEffects last for at least 6 yearsEffective in reducing aggressive / violent behaviorResearched and proven effective with African-American, Hispanic, White, Urban, Suburban and Rural Youth

31For nearly two decades, LST has been rigorously evaluated in federally funded research studies. These studies have consistently shown that the LST program maintains and reduces problem behaviors, whether implemented by teachers, counselors, health professionals, or peers.

LSTs remarkable reductions in risky behaviors are consistent among students of different racial, ethnic and geographic backgrounds. These results have been published in major scientific journals and presented at national and international conferences over the past 20 years.

LST is a research proven, skills based universal prevention program, that has gathered an impressive body of evidence supporting its effectiveness.Summary of Evaluation ResultsElementary School ResultsReduces annual smoking rates by up to 63%Increases self-esteemFindings show that the evidence-based LST model is equally effective at the Elementary level32Recent pilot studies in 20 elementary schools have shown that the LifeSkills Training curriculum is effective in reducing substance use at elementary level. Most significant among the findings is a 64% reduction in smoking.

Further results show an increase in self-esteem. Individual level results also found decreases in normative expectations for drug use and increases in both skills and substance use knowledge.Summary of Evaluation ResultsEvidence of EffectivenessEffectiveness established through rigorous scientific inquiryDurability of effectsLearn, retain, transfer

Ease of ImplementationFits with existing infrastructures

33Building EvidenceTypes of Evaluation

Formative Evaluation Asks: is an element of a program or policy (e.g., materials, messages) feasible, appropriate, and meaningful for the target population?Often, done in the planning stages of a new programCan be used to examine contextual factorsWhat are the attitudes among school officials to the proposed healthy eating programAre there certain schools that have healthier food environments that others?

3535Process Evaluation Assesses the way a program is being delivered, rather than the effectiveness of a program. Asks:

Are all activities being implemented as planned?Is the intervention feasible and acceptable?Are you reaching intended recipients?

Provides shorter-term feedback on program implementation, content, methods, participant response, practitioner response.3636

California Local Health Department Funding by Core Indicator2001/042004/07LicenseSSDOutdoorCess.BarsSpo.Cess.OutdoorLicense37In 1998, smoke-free bar law went into effect.In 2002, California launched STORE campaign one goal was to increase retail licensing ordinances. 37These pie charts reflect the percent of funding that Californias 61 local health departments have collected directed to the 13 core indicators in their 2001/04 comprehensive tobacco control plans and in their 2004/07 plans. The data from these charts is drawn from Californias Online Tobacco Information System.

The charts reflect positive changes over time. For example, the smoke-free bar law went into effect in 1998. In the 2001/2004 Plans, local health departments were putting 19% of their funds into achieving compliance with this law, but by 2004, the need to direct funds towards this state law dropped to 6% of funding.

In 2002, the California Tobacco Control Program launched the STORE Campaign. One of the goals of that campaign was to increase local tobacco retail licensing ordinances that earmarked funding for enforcement. We can see that local health departments dramatically increased the funding that they were directing to local tobacco retail licensing from 2001 to 2004.

Types of EvaluationImpact and outcome evaluations only after program has had adequate time to yield quantitative changes.Amount of time needed depends on nature of program and changes expected.Each type of evaluation required different designs and data collection methods.38Impact Evaluation Asks: Has the initiative been successful in achieving intended outcomes?long-term or short-term feedback on knowledge, attitudes, beliefs, behaviorsuses quantitative or qualitative dataalso called summative evaluationprobably more realistic endpoints for most public health programs and policies3939Source: California State Board of Equalization (packs sold) and California Department of Finance (population).U.S Census, Tax Burden on Tobacco, and USDA.. Note that data is by fiscal year (July 1-June 30).Prepared by: California Department of Health Services, Tobacco Control Section, February 2006.

Packs/PersonCalifornia and U.S. minus California adult per capita cigarette pack consumption, 1984/1985-2004/2005US minus CACalifornia$0.25 tax increase$0.02 tax increase$0.50 tax increase0501002001504040Number of cigarettes sold per fiscal year is reported by the Board of Equalization. Number of packs of cigarettes sold per fiscal year per adult decreased by 45% from 1988, the beginning of the Tobacco Control Program, to the present. In 1987/88, per capita consumption was 126.6 packs of cigarettes, and in 1997/98, 69 packs per year. Outcome Evaluation Provides long-term feedback on health status, morbidity, mortality, and QOL.Often difficult to attribute to a particular program because it takes so l on for the effects to be seen and because outcomes are influenced by many factors.Usually requires experimental or quasi-experimental (rather than observational) approaches to link program influences on outcomes.Usually relies on quantitative methodsOften used in strategic plans4141Acute myocardial infarction Death Rate per 100,000 (2008-2010) 42

42Program Evaluation Designs43rateCan we conclude that the intervention is effective? Intervention initiated43Program Evaluation Designs44rateCan we conclude that the intervention is effective? Intervention initiated44How about now..assuming the trend line at the top tracks individuals NOT exposed to the intervention.Indicators for Impact and Outcome EvaluationHealth indicators measure extent to which targets in health programs are being reached.For evaluation purposes, indicators are not goals in themselves should not be confused with program objectives and targets.Indicators provide valuable benchmarks.Program evaluation aimed at impacting these benchmarks require shorter-term (intermediate) markers. i.e., # of smoke free campuses45

Questions?46ReferencesPowerpoint presentation adapted from Using What Works: Adapting Evidence-Based Programs to Fit Your Needs. U.S. Department of Health and Human Services. National Cancer Institute. 2006.http://cancercontrol.cancer.gov/use_what_works/start.htmHealth Promotion Programs: From Theory to Practice. Carl I. Fertman (Editor), Diane D. Allensworth (Editor), Society for Public Health Education ISBN: 978-0-470-24155-4

47Chart1138.6165.3133.1161.3127.9157123.3152.5109.6147.1101.5139.395.1133.591.6129.785.5126.180.7122.978.9123.276.1120.473.8119.570.5116.662.1111.954.6108.450.7103.947.7101.545.394.844.291.643.587.7

CAUS-CA

Sheet180/8181/8282/8383/8484/8585/8686/8787/8888/8989/9090/9191/9292/9393/9494/9595/9696/9797/9898/9999/0000/0101/0202/0303/0404/05CA160156149142139133128123110102959286817976747162555148454444US-CA183180173167165161157153147139134130126123123120120117112108104102959288