Evaluation Practice

Stephen Gilson, PhD.sfgilson@astos.org

Elizabeth DePoy, Ph.D. edepoy@astos.org

What is Evaluation Practice?

• Evaluation Practice is the purposive application of diverse sets of evidence to the definition and clarification of social problems, to the identification of what is needed to resolve them, and of the way in which and extent to which problems have been resolved.

STEPS

• Clarification– Problem– Need Goals and Objectives

• Reflexive Intervention• Outcome Assessment• Dissemination

Clarification Stage

• Problem• Need• Goals and Objectives for Practice

• There is a significant distinction between problem and need.

Definition of Problem

• A problem is defined as a conceptual value assertion.

Definition of Need

• A need is a systematic, evidence-based claim linked to all or part of a problem, which specifies conditions and actions necessary to resolve the part of the problem to be addressed.

Nature of Evidence

• Unlike many empirically-based practice models, the nature of evidence is not rigid and prescriptive.

Final Step of Clarification

• The final phase of clarification involves setting goals and objectives that proceed from a clear statement of need.

• Goals and objectives provide the structure for interventions and posit desired intervention outcomes at one or more time intervals.

Goals

• Broad statements about the ideal or "hoped for."

• They look forward in time to specific "desirables."

• They also look back to the value context because they emerge directly from the statement of what is needed to remediate a problem.

Objectives

• Operationalized goals.

• Applying the concept of operationalization, objectives identify how abstract goals will be met.

Reflexive Intervention

• The thinking and action strategies during prevention and intervention

• The provider engages in systematic monitoring of practice, of resources, and of use of self as well as scanning other internal and external influences that impact the practice process and outcome.

• The scholar/professional who applies the principles of reflexive intervention carefully scrutinizes the implementation of goals and objectives through systematically monitoring intervention.

• In Evaluation Practice, the scholar/professional is aware of the evidence that supports his/her decision making, carefully looks at all practice to obtain clarity about what was done, and considers feedback from engaging in practice action processes.

Outcome Assessment

• Outcome assessment is a set of thinking and action processes conducted to ascertain and document what occurs as a result of being voluntarily or involuntarily exposed to a purposive prevention or intervention process, and to assess the worth of an action.

• Evaluation practice highlights the importance of assessing both planned and unintended occurrences.

Dissemination

• Evaluation Practice does not end with summative strategies. Rather dissemination, an essential step, advances prevention and intervention knowledge through sharing the entire Evaluation Practice process from problem definition through outcome assessment.

• Dissemination is purposive and thus should be shaped by an understanding of what the audience will value as believable and important, and by assuring that the information that is shared is accessible and anchored on evidence that is relevant to the receiver of information.

Implementing Evaluation Practice

Problem Mapping

A thinking technique used to expand and contextualize an initial problem statement.

Problem Mapping

• The problem map is constructed by stating the initial problem presented by the individual, group, community, system as conceptualized by that entity, and then asking two questions two analytic questions: “what causes the problem?” and “what are the consequences of the problem?”

• The same two questions are repeatedly posed for the causes and consequences that are articulated.

Illustration of Evaluation Practice

Alcohol abuse

Access to alcohol Poor social skills Genetic predisposition

Social context

Marketing of alcohol Cultural sanction

Dysfunction DependenceUnderemployment

PovertyPoor health

Need

• Prevention• Social marketing• Access policy change

Goal

• Implement relevant and responsive community programs to change sanction access and marketing messages

Objectives

• Change social context• Decrease access to alcohol• Change marketing messages

Reflexive Intervention

• Coordinated community prevention and social marketing

• Cost-benefit analysis

Outcome Assessment

• Community and social support for prevention• Prevention-positive marketing messages and

outcomes• Policy change• Access changes

Dissemination

• Presentation of formative and summative assessment at local, regional and national venues

References

DePoy, E. & Gilson, S.F. ( 2008) Evaluation Practice. Boston MA: Taylor & Francis.

Extensive Bibliography listed in text.

Websites• http://www.ahcpr.gov/clinic/epc/• http://www.mco.edu/lib/ebm.html#Cochrane• http://healthlinks.washington.edu/ebp• http://www.cche.net/usersguides/main.asp• http://www.shef.ac.uk/uni/academic/R-Z/scharr/

triage/docs/systematic/1.htm