evaluating progress on implementation of the safe routes ......intermediate outcomes •increased...

Evaluating Progress on Implementation of the Safe Routes to School initiative in Lawrence, Kansas

Vicki Collie-Akers, Charlie Bryan, Johana Bravo, Michael Showalter, Daryl Stewart, Chris Tilden, Vince Romero, Dee Vernberg

Overview

■ Describe supports and approaches used through the National Center for Safe Routes to School (SRTS)

■ Describe the comprehensive approach for evaluation used at the local level in Lawrence, Kansas

■ Share current data and related analysis

Evaluation Approach and Resources ■ National Center for SRTS recommends and supports key evaluation

measures – Student Travel Tallies – Parent Surveys

■ Data collection tools offered – Descriptive analysis – Automated reports

Evaluation Approach at Lawrence, Kansas

• LiveWell Lawrence: Multi-sectoral community coalition established in 2008

• Community health assessment completed in 2012

• Community Health Improvement Plan completed in 2013

• Qualified staff support at the Lawrence-Douglas County Health Department

• Existing data and evaluation resources

• Critical community partners

Inputs Strategies/ Activities Short-term Outcomes

Risk Factor-Related Population-Based Strategies

• Implement policies & practices in three public school districts to promote physical activity among school children (K-8) through the Safe Routes to School Program

• Include interested private schools in implementation of SRTS

• Provide technical support to schools involved with SRTS

• Educate community on physical activity with audience-specific public education messages

Infrastructure & Other Support • Maintain staff and coalition engagement in

selected activities • Assure collection of all evaluation measures • Conduct communication activities to

promote project efforts

• Number of communications

• Fidelity of SRTS model

• Number/ type environmental changes implemented

• Number/type of educational, encouraging, enforcement, and equity approaches implemented

• Number/ type of barriers and perceptions of encouragement reported by parents

Intermediate Outcomes

• Increased daily physical activity among children, youth, and families.

• Proportion of children walking or biking to and from schools

• Improved driver behavior

Long-term Outcomes

• Reduced prevalence of obesity by 3%

Impact

• Improved quality of life

• Premature deaths averted

• Medical costs averted

Evaluation Questions

■ To what extent are SRTS elements being implemented? – How does implementation vary across the district, between

buildings?

■ To what extent do key intermediate outcomes change throughout implementation? – Walking and biking to and from school – Parent perceptions of barriers – Driver behavior

■ How are different measures related?

Key Measures

•Number/ type environmental changes implemented •Number/type of educational, encouraging, enforcement, and equity approaches implemented •Number/ type of barriers and perceptions of encouragement reported by parents •Proportion of children walking or biking to and from schools •Proportion of drivers yielding to pedestrians in marked crosswalks

Data Collection

■ Quality data collection supported by: – Availability of codebooks including definitions, scoring criteria* – Training including practice – Quality assurance through inter-observer agreement

To what extent are SRTS elements being implemented? ■ Illustrative examples:

■ Developmental activities/ community actions (n=93)

• 9th & New York stop signs added to list of city improvements

• SRTS team submitted a city crossing policy

■ Services provided (n=27) • 1200 students participated in

Bike to School Day

0

10

20

30

40

50

60

70

80

90

100

Sep-

14

Nov

-14

Jan-

15

Mar

-15

May

-15

Jul-

15

Sep-

15

Nov

-15

Jan-

16

Mar

-16

May

-16

Jul-

16

Sep-

16

Nov

-16

Jan-

17

Mar

-17

Num

ber

of a

ctiv

ities

Distribution of activities to enable implementation over time

Services Provided (n=27)

Development Activities & Community Actions (n=93)

To what extent are SRTS elements being implemented?

■ Illustrative examples of community changes (n=18)

• BLAST program implemented at 4 elementary schools

• Two speed humps installed around Woodlawn Elementary to moderate traffic

• Signage installed and painting completed at roundabout proximal to Cordley Elementary

0

10

20

30

40

50

60

70

80

90

100

Sep-

14

Nov

-14

Jan-

15

Mar

-15

May

-15

Jul-

15

Sep-

15

Nov

-15

Jan-

16

Mar

-16

May

-16

Jul-

16

Sep-

16

Nov

-16

Jan-

17

Mar

-17

Num

ber

of a

ctiv

ities

Distribution of activities to enable implementation over time

Services Provided (n=27)

Development Activities & Community Actions (n=93)

Community Changes (n=18)

Intensity of Efforts

0

1

2

3

4

5

6

7

8

2014 2015 2016 2017

Intensity of Changes Over Time ■ Intensity calculated based on scores for reach, strategy, and duration

– 67% had a reach less than 6% of all students; 27% had a reach of greater than 20% of all students

– About 38% of changes were continuous in duration; 33% occurred more than once; 27% were one-time events

– 77% of activities were focused on provision of information or services; 33% focused on modifying access, barrier, opportunities, or policies

One-time addition bike safety at an elementary

school during field day

Completed SRTS plans for each

school

To what extent do key intermediate outcomes change throughout implementation?

■ Parent perception – self-report survey; 1x year*

■ Driver behavior – direct observation; 2x year

■ Travel tallies – self-report; 2x year


0 10 20 30 40 50 60 70 80 90 100

Safety of Intersections and Crossings

Amount of Traffic Along Route

Distance

Speed of Traffic Along Route

Weather or climate

Violence or Crime

Time

Sidewalks or Pathways

Adults to Bike/Walk With

Convenience of Driving

Child's Participation in After School Programs

Crossing Guards

Barriers reported by parents

2015 (n=567)

2014 (n=703)


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Cordley ES Deerfield ES Eudora ES Kennedy ES Langston HughesES

New York ES Prairie Park ES Woodlawn ESPerc

enta

ge o

f dri

vers

not

yie

ldin

g to

pe

dest

rian

s

Axis Title

Percentage of drivers not yielding to pedestrians

Fall 15 Spring 2016 Fall 2016

Parent Perception & Driver Behavior

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

PR

OP

OR

TIO

N P

AR

ENTS

CO

NC

ERN

WIT

H

SAFE

TY I

NTE

RC

ETIO

N A

ND

CR

OSS

ING

% OF DRIVERS YIELDING AT CROSSWALK

Proportion concern issue with safety or intersections and crossings and actual driver yielding behavior

Parent Perception & Driver Behavior

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

PR

OP

OR

TIO

N O

F P

AR

ENTS

CO

NC

ERN

W

ITH

CR

OSS

ING

GU

AR

DS

% OF DRIVERS YIELDING AT CROSSWALK

Proportion concern issue with crossing guards and actual driver yielding behavior


0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Fall 2014 Spring2015


Fall 2016

Percentage of students reporting walking/ biking - AM

Broken Arrow ES

Cordley ES

Deerfield ES

Eudora ES

Eudora MS

Hillcrest ES

Kennedy ES

Langston Hughes ES

Liberty MC MS

New York ES

Pinckney ES

Prairie Park ES

Quail Run ES

Sunflower ES

Woodlawn ES

MEAN

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5



Fall 2016

Perc

enta

ge o

f stu

dent

s re

port

ing

wal

king

/bik

ing

Percentage of students reporting walking/ biking - PM

Broken Arrow ES

Cordley ES

Deerfield ES

Eudora ES

Eudora MS

Hillcrest ES

Kennedy ES

Langston Hughes ES

Liberty MC MS

New York ES

Prairie Park ES

Sunflower ES

Woodlawn ES

MEAN

0

0.1

0.2

0.3

0.4

0.5

0.6

Fall 2014 Spring 2015 Fall 2015 Spring 2016 Fall 2016

Woodlawn Elementary

Drivers not yielding Children walking/ biking

0

0.1

0.2

0.3

0.4

0.5

0.6

Fall 2014 Spring 2015 Fall 2015 Spring 2016 Fall 2016

Langston Hughes Elementary

Drivers not yielding Children walking/ biking

Bike to School

Day

School Specific Analysis

Bike to School

Day

Bike rodeo

BLAST program

Bike to School

Day

Bike to School

Day

Speed Humps

Installed

Challenges and Strengths

■ Challenges experienced – Fatigue – Weather – Variation in context – Getting the protocol right

■ Strengths – Clear protocols for data collection and quality assurance – Multi-level evaluation to examine how we are moving multiple

needles

Next Steps for Evaluation

• Continue thorough documentation of SRTS implementation

• Enhance training, support, and communication for travel tallies

• Continue parent surveys, strengthen communication

• Continue driver behavior observation, refine method for challenging contexts

evaluating progress on implementation of the safe routes ......intermediate outcomes •increased...

Documents