Cornell UniversityProgram Planning and Evaluation

From Theory to Practice and Back: Finding New Ways to Link Research and Practice

Jennifer Southwick BrownPh.D. Candidate

Department of Human DevelopmentMay 31, 2007

• Evaluation Team Leaders– Bill Trochim, Jennifer Southwick Brown,

Sarah Hertzog, Claire Lobdell, Cath Kane • Evaluation Planning Partnerships

– 7 sites in Cornell Cooperative Extension• New York City, Tompkins County, St. Lawrence

County, Jefferson County, Chenango County, Onondaga County, Ulster County

– Museum of the Earth, Cornell Center for Materials Research

Acknowledgements

• Why did we read the Renger & Hurley (2006) article?

• Systems Evaluation– From logic models to Pathway models– Analysis of evidence

• The Netway: Creating a tool for the future

Overview

• Program theory– Theory of change

• What we’re going to do, with whom, in what context, to effect what outcomes

– Logic Model• Visual representation (typically columnar) of the

inputs, activities, outputs, short- and long-term outcomes associated with a program

• Basic theory

What do we mean by theory?

• Activities, Stakeholders, Participants, Outcomes, Goals

• How can we achieve a link between research and practice?– Planning and Evaluation/Feedback

What do we mean by practice?

Youth Leadership Academy

Public Presentation

Program

Developing a Program Model: NYC 4-H After-School Clubs and Special Interest Projects

Context / Assumptions

Inputs Activities

Students make public

presentations to a group

Outputs

Youth interest in science

Increased interest in science

education

Develop leadership

skills

Short Medium Long

Outcomes

Curricula

FTE Extension Educators

Partnerships (community, sponsors)

State & National 4-H

Initiatives

Volunteers

Agilent Science & Tech

Volunteer Orientation &

Trainings

Students complete

engineering kits

Increased rates of higher

education & career

goal attainment

Funder mandates time & budgetary constraints

Staff is proficient at train-the trainer skills

Increased rates of higher

education & career

goal attainment

Short-Term OutcomesOutputsActivity

Developing a Pathway Model

Agilent Science & Tech

Public Presentation

Program

Volunteer Orientation &

Trainings

Youth Leadership Academy

Legend:

Students complete

engineering kits

Students make public

presentations to a group

Youth interest in science

Develop leadership

skills Increased interest in science

education

Medium-Term Outcomes Long-Term Outcomes

Articulating Causal Relationships

Students complete

engineering kits

Youth interest in science

Increased interest in science

education

Analysis of Evidence: Finding the “Golden Spike”

Students complete

engineering kits

Youth interest in science

Increased interest in science

education

Increased rates of higher

education & career

goal attainment

• The Netway is an internet-based database tool for monitoring and tracking program activities and participants

• Features of the Netway– Stores program information– Converts logic models to pathway models– Produces evaluation plans– Stores participant information– Provides logic model, pathway model, &

participant reports

The Netway

http://www.extensionnetway.net/

The Netway: Tracking Programs

The Netway: Logic Model

The Netway: Pathway Model

The Netway: Evaluation Plan Report

• Program– Description– Questions– Activities

• Methods– Sampling– Measures– Design– Analysis

• Reporting

The Netway: Develop Evaluation Plan

The Netway: Participant Report

• Interactive user database– Practitioner Practitioner: Connecting

similar programs, activities, outcomes, & measures

– Practitioner Researcher: Connecting research to programs

• Referral Linkages• Social Networking• Interactive Web Graphics

The Netway in the Future

The Netway in the Future: Referral Linkages

The Netway in the Future: Networking

The Netway in the Future: Academic Portal

The Netway in the Future: Student Portal

The Netway in the Future: Visual Networks

http://www.touchgraph.com/

The Netway in the Future: Visual Networks

http://www.touchgraph.com/

The Netway in the Future: Visual Networks

The Netway: Research to Practice AND BACK

• Pilot testing and development with the 7 CCE sites, Museum of the Earth, and CCMR

• Designing and populating the researcher portal

• Finding gaps in the research literature• Adding additional portals

Online Resources

• Research Methods Knowledge Base http://www.socialresearchmethods.net

• Penn State Cooperative Extension (website) Extension and Outreach Program Evaluation:http://www.extension.psu.edu/evaluation/

• University of Wisconsin Extension (online course) Enhancing Program Performance through Logic Models: http://www1.uwex.edu/ces/lmcourse/

• Cornell Website:http://evaluation.cce.cornell.edu

Initiation Development Maturity Translation &Dissemination

Farmer’s Market

EFNEP

Project Bridge

Program Lifecycle

Initiation Development Maturity Translation &Dissemination

Farmer’s Market

Program Phase

Evaluation Phase

Project Bridge

EFNEP

Program & Evaluation Alignment