evaluation guide for pedestrian safetyevaluation guide for pedestrian safety about the evaluation...

Evaluation Guide for Pedestrian Safety

Accreditation | 2Evaluation Guide for Pedestrian Safety

Authors

Safe States Alliance • Jamila Porter, DrPH, MPH, Director of Programs and Evaluation• Ina Robinson, MPH, Evaluation and Technical Assistance Coordinator

Research and Evaluation Group • Shenee Bryan, MPH, MPA, Director of Programs and Evaluation• LaTasha Barnwell, MPH, Research Associate

Transtria • Laura Brennan, PhD, MPH, Chief Executive Officer• Brandye Mazdra, MPH, Director of Projects and Operations

Advisors to the Authors

• Ross Brownson, PhD, Bernard Becker Professor of Public Health, Washington University in St. Louis• Sarah Strunk, MHA, Strategic Advisor, Active Living By Design• Ian Thomas, PhD, State and Local Program Director, America Walks• Denise Yeager, MPH, Healthcare Program Consultant and Operation Analyst, Georgia Division of Public Health, Injury Prevention Section

ACKNOWLEDGMENTS

The Evaluation Guide for Pedestrian Safety was developed by the Safe States Alliance, with support from the National Highway Traffic Safety Administration (NHTSA). Content and structure for the guide were provided by Research and Evaluation Group and Transtria. Web development, design,

and hosting of the guide were provided by Banyan Communications.

Evaluation Guide for Pedestrian Safety Table of Contents | 3

TABLE OF CONTENTSAbout the Evaluation Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Create Your Evaluation Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Resource Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Section 1: Evaluation Purpose and Partners . . . . . . . . . . . . . . . . . . . . . . . 12Step 1: Convene partners to plan the evaluation . . . . . . . . . . . . . . . . . . . . . 12Step 2: Identify your population of interest and your

vision for change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Step 3: Consider the intervention stage and corresponding

types of evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Step 4: Determine evaluation audiences and the relevance

of the evaluation to these audiences . . . . . . . . . . . . . . . . . . . . . . . . . . 21Step 5: Create a clear, concise purpose statement . . . . . . . . . . . . . . . . . . . 23Step 6: Design your collaborative, including a governance

structure and process, partner responsibilities, and budget and time constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Section 2: Describing the Intervention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Step 1: Describe your Pedestrian Safety Intervention . . . . . . . . . . . . . . . . 31Step 2: Create a Logic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Section 3: Evaluation Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Step 1: Develop evaluation questions aligned with the

evaluation purpose statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Step 2: Identify the population or subpopulations,

intervention goals and objectives, and types of evaluation for each evaluation question . . . . . . . . . . . . . . . . . . . 42

Step 3: Specify the evaluation design to maximize causal inferenceand internal validity for each evaluation question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Step 4: Determine sampling strategies to maximize external validity in the selected population or subpopulations for each evaluation question . . . . . . . . . . . . . . . . 53

Section 4: Methodology and Data Collection . . . . . . . . . . . . . . . . . . . . . . . 59Step 1: Identify data sources or design data

collection methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Step 2: Create a matrix to connect your evaluation

questions, indicators, and data sources . . . . . . . . . . . . . . . . . . . . . . . 64

Section 5: Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Step 1: Convene data analysis partners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Step 2: Identify variables and themes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67Step 3: Organize data and ensure data quality . . . . . . . . . . . . . . . . . . . . . . . . 70Step 4: Agree on data analysis methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72Step 5: Interpret results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Step 6: Create a data analysis plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Section 6: Reporting and Dissemination . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79Step 1: Identify audiences and develop a dissemination plan . . . . . . . . . 79Step 2: Select key messages for specific purposes . . . . . . . . . . . . . . . . . . . 80Step 3: Select communication methods and create evaluation

products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80Step 4: Monitor Dissemination Efforts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87Evaluation Templates Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Tools and Resources Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

About the Evaluation Guide | 4Evaluation Guide for Pedestrian Safety

ABOUT THE EVALUATION GUIDEThe Evaluation Guide for Pedestrian Safety is a robust and comprehensive resource designed to help you and your partners effectively evaluate your pedestrian safety interventions.

How You Can Use This GuideYou can use this guide to:• Plan your evaluation of a pedestrian safety intervention using a helpful, step-by-step process.• Obtain useful tips to help you:

• Collaborate successfully with partners.• Infuse principles associated with social, health, and economic equity into your evaluation.• Implement and manage your evaluation process effectively.• Capitalize on assets and overcome anticipated barriers during the evaluation process.• Access templates, tools, resources, and examples from the field to inform and enhance your evaluation.

Why Evaluation Is EssentialThere are a variety of reasons that it is essential to evaluate pedestrian safety interventions. Evaluation can:

1. Improve interventions by identifying what works to increase pedestrian safety.�Evaluation�findings�can:• Reflect strengths and weaknesses of your partnership’s structure and processes.• Strengthen the design and implementation of your pedestrian safety intervention.• Highlight the reach of your pedestrian safety intervention to vulnerable populations.• Demonstrate impacts of your pedestrian safety intervention on population health behaviors, outcomes, and quality of life.• Assess sustainability of your pedestrian safety intervention over time.

2. Build support for collaborative partnerships that can mobilize resources for pedestrian safety initiatives.Evaluation partnerships can:• Raise awareness and visibility of pedestrian safety interventions.• Leverage political and community support, funding, and resources for pedestrian safety interventions.1


3. Establish an evidence base for scale up of effective pedestrian safety policies and systems. Systematic�reviews�of�evaluation�findings�can: • Facilitate state, regional, and local decision-making about priority pedestrian safety interventions. • Provide state, regional, and local model practices or implementation guidelines.

4. Make the connections between pedestrian safety and population health explicit. 2,3 “Field-building”�based�on�evaluation�findings�(and�research)�can: • Provide data support to show pedestrian safety interventions increase active travel. • Show how increases in active travel increase the proportion of the population meeting physical activity recommendations and

decreases in auto use reduce sedentary behaviors.• Demonstrate that increases in physical activity and reductions in sedentary behaviors reduce morbidity and mortality (e.g., obesity,

cardiovascular disease, cancer).• Draw connections between active travel/reduced auto use and cleaner air, community development, economic growth, sustainability,

and other related outcomes.

“Evidence-based public health utilizes the current best available evidence to make decisions in the public health service,

and also to develop action plans, public health programs, and policies for addressing public health issues.” 4

FOOTNOTES1. Redmon, T., D. Gelinne, L. Walton, J. Miller. Jan/ Feb 2012. Spotlight on pedestrian safety. Publication Number: FHWA-HRT-12-002 75(4).

Retrieved from: https://www.fhwa.dot.gov/publications/publicroads/12janfeb/03.cfm 2. U.S. Centers for Disease Control and Prevention. 2010 CDC Recommendations for Improving Health through Transportation Policy.

Retrieved from: https://www.cdc.gov/transportation/docs/final-cdc-transportation-recommendations-4-28-2010.pdf 3. U.S. Department of Transportation. Walking and Biking are Good for Public Health. Retrieved from: https://www.transportation.gov/safer-people-safer-streets4. Brownson, R. C., E. A. Baker, A.D. Deshpande, K. N., Gillespie. Evidence-Based Public Health. 3rd Ed. New York: Oxford University Press; 2018.

Retrieved from: https://global.oup.comacademic/product/evidence-based-public-health-9780190620936?cc=us&lang=en&


What Interventions You Can EvaluateThe Evaluation Guide for Pedestrian Safety is designed to evaluate pedestrian safety interventions, specifically pedestrian safety action plans (PSAPs) and pedestrian safety education campaigns and promotions.

• Pedestrian Safety Action Plans (PSAPs) create a framework for state and local officials and a variety of partners to address pedestrian safety problems in specific geographic areas to identify, implement, and evaluate optimal solutions to improve pedestrian safety.

• Pedestrian safety education campaigns and promotions are coordinated efforts designed to improve pedestrian safety for a defined population by targeting knowledge, attitudes, awareness, beliefs, behaviors, and/ or social norms related to pedestrian safety. These efforts can vary in complexity depending upon a variety of factors, such as duration, resources, and message.

Pedestrian safety interventions are frequently comprehensive.• PSAPs are implemented as part of larger city, county, or regional efforts that include a variety of different

pedestrian safety interventions, such as engineering initiatives, educational efforts, or enforcement practices.

Pedestrian safety interventions can also be standalone.• Policy, practice, or environmental changes can be independently designed and implemented,

particularly when resources and support are not available. • Educational and promotional efforts, such as social marketing campaigns, media advocacy,or

training programs, can be implemented to increase awareness and generate action related to comprehensive pedestrian safety interventions.

PSAPs and education campaigns and promotions often incorporate multiple intervention strategies to improve pedestrian safety. • These intervention strategies can be categorized across nine domains, including the traditional “E’s”

(engineering, education, enforcement, encouragement, equity, evaluation), as well as others that include advocacy, collaboration, and land use planning and policy.

• The table below describes each domain and provides examples of common PSAP and education campaign and promotion interventions for each domain.


Nine Pedestrian Safety Domains, Descriptions, & Intervention ExamplesDomain Description Intervention Example

Advocacy To generate political will and community support Decision-maker engagement and support (elected officials, appointed officials, community leaders) through presenting to city/ county/ state councils/ committees, adopting principles for practice among appointed officials, etc.

Collaboration To partner and organize to create change Provide training and other types of technical assistance to increase the knowledge, skills, and abilities of individual community leaders or champions to engage in certain partnership and community activities

Engineering To design sites and streets, intersections, sidewalks/crosswalks/signage, or timed traffic lights

Improvements to sidewalks, bike lanes or streets for traffic calming on routes to schools, child care agencies, parks and recreation facilities, and worksites

Education To increase educational outreach to communities School districts implement Safe Routes to School program/ Walking School Bus program

Enforcement To develop active transportation and pedestrian safety policies and enhance local law enforcement of these policies (speed limits, yielding to pedestrians in crosswalks, and proper walking and bicycling behaviors) and community or site enforcement of active transportation and pedestrian safety

Government permits authority to local cities and counties to pass a local resolution or ordinance to support active transportation or pedestrian safety

Encouragement To host events inspiring communities to try something new, which often results in the development of ongoing programs to encourage walking and bicycling

Social marketing campaigns (systematic application of marketing, along with other concepts and techniques, to achieve specific behavioral goals for a social good (e.g., Take A Walk, Feet First, Travel Smart)

Equity To support safe, active, and healthy opportunities for children and adults in low-income communities, communities of color, and beyond

Policy that supports development, funding, or maintenance of active transportation and pedestrian safety facilities and design features in affordable housing/ housing authority locations

Evaluation To assess the need and scope for effective pedestrian interventions.

Parent surveys help pinpoint reasons why parents drive their children to school instead of allowing them to walk or bicycle, and the changes that could encourage active school transportation options.

Land Use Planning &

Policy

To design and develop communities (permission, protection, and use of land)

Small area plan incorporating active transportation or pedestrian safety provisions (e.g. neighborhood, corridor)

For additional descriptions and examples of pedestrian safety domains, see Pedestrian Safety Strategies and Domains.


Who’ll Enjoy Using This GuideA range of different professional and lay audiences can use this guide.5,6,7,8 The table below highlights some of the ways these audiences can use this guide.

You’ll enjoy using this evaluation guide if you’re a:

Because you can contribute to an evaluation of a pedestrian safety intervention by:

Public health practitioner • Identifying or collecting primary or secondary data related to pedestrian safety behaviors, policies, practices, environments, and educational or promotional efforts

• Analyzing the magnitude, trends, and patterns of pedestrian fatalities, injuries, and associated risk and protective factors

• Examining trends among subpopulations to convey connections to the social determinants of health and indicators of health inequities

Transportation, highway safety, or planning professional

• Prioritizing pedestrian safety policies and improvements for evaluation• Identifying existing and needed data to use for evaluation (e.g., pedestrian counts, traffic flow data, road safety

audits/ assessments, crash reports).• Designing policies, practices, or environments based on evaluation findings (e.g., urban growth boundaries,

roundabouts, countdown timers)

Law enforcement professional • Identifying pedestrian safety practices ready for evaluation• Identifying existing data to use for evaluation (e.g., traffic code violations, crime rates)• Designing equitable and community-centered policies, practices, and environments based on evaluation

findings (e.g., warnings vs. citations)*

Elected�or�appointed�official • Voting in formal policies, such as Vision Zero policies, that provide unambiguous direction to city bureaucrats and engineers (e.g., eliminate traffic deaths by 2030).

• Serving as a champion to rally support for evidence-based pedestrian safety policies and improvement projects• Allocating funding to pedestrian safety policies and projects through budget line-items, earmarks, tax incentives,

or related approaches

Administrator,�teacher,�or�staff�member in a child care center, school, or other educational setting

• Demonstrating the value of Safe Routes to School initiatives to students, families, and communities• Serving as a champion for safe environments to support active children• Linking pedestrian safety initiatives to improved physical activity and educational outcomes in children

Administrator,�provider,�or�staff�member in a health care setting

• Connecting pedestrian safety initiatives to reductions in use of emergency services and associated health care costs in communities

• Engaging health care providers in educational approaches to increase safe driving practices and implications for individual and community health


You’ll enjoy using this evaluation guide if you’re a:

Because you can contribute to an evaluation of a pedestrian safety intervention by:

Member or employee of a community group or advocacy organization

• Identifying evidence-based pedestrian safety policies, practices, environmental strategies, and educational and promotional efforts

• Increasing equitable access to safe pedestrian accommodations among vulnerable populations (e.g., lower-income communities, people with disabilities)

• Educating city council representatives, other elected and appointed officials, and community leaders on the benefits of pedestrian safety initiatives

Media and communications professional

• Identifying existing media (e.g., television, radio, social media) from which evaluation data can be accessed• Assessing reach to vulnerable subpopulations (e.g., children, elderly, people with disabilities, lower income

populations)• Assessing gains in public support for pedestrian safety initiatives• Identifying education messages, audiences, and communication methods for dissemination of evaluation findings

Community member • Communicating pedestrian safety concerns• Volunteering to serve on committees or support data collection, interpretation, or dissemination efforts• Helping to garner community support and recruiting participants for the evaluation

There are also many other potential users of this guide, given that:

• Traditional designs for communities and transportation systems focused on automobiles are increasingly being replaced with “smart growth,” “new urbanist,” and “Complete Streets” approaches that balance the needs of drivers with those of pedestrians, bicyclists, and other road users (e.g., wheelchairs, scooters, strollers).

• Interdisciplinary policies, practices, environmental strategies, and educational and promotional efforts to increase pedestrian safety are gaining momentum.

• Technology and social media are altering the communication environment and require new approaches to education and promotion.• There is increasing recognition of the influence of social determinants on systems and outcomes that are central to social, health, and

economic equity.

FOOTNOTES5. U.S. Department of Transportation National Highway Traffic Safety Administration. 2014 Pedestrian Safety Enforcement Operations: A How-To Guide

Retrieved from: https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/812059-pedestriansafetyenforceoperahowtoguide.pdf 6. Boston Department of Transportation. 2001 Access Boston 2000-2010 Pedestrian Safety Guidelines for Residential Streets: Safety Guidelines for Residential Streets.

Retrieved from: http://www.bu.edu/police/files/2016/07/pedestrian_safety_guidelines.pdf7. World Health Organization 2010 Pedestrian safety: A road safety manual for decision-makers and practitioners.

Retrieved from: http://apps.who.int/iris/bitstream/10665/79753/1/9789241505352_eng.pdf8. Leah Shahum. July 21, 2016. Vision Zero, Equity & Law Enforcement. Retrieved from: http://visionzeronetwork.org/vision-zero-equity-law-enforcement/.

Evaluation Guide for Pedestrian Safety Create Your Evaluation Plan | 10

CREATE YOUR EVALUATION PLANThe Evaluation Guide for Pedestrian Safety is organized into six sections that correspond with the primary elements of the Centers for Disease Control and Prevention’s (CDC) Framework for Program Evaluation9:

In this area of the guide, you’ll find sequential, step-by-step guidance and instructions that can help you create an evaluation plan for a pedestrian safety intervention.

FOOTNOTES9. Centers for Disease Control and Prevention (CDC), Program Performance and Evaluation Office (PPEO). A Framework for Program Evaluation. Retrieved from:

https://www.cdc.gov/eval/framework/index.htm

Evaluation Guide for Pedestrian Safety Resource Center | 11

RESOURCE CENTERThe Resource Center provides a variety of resources to support your evaluation planning process; these are referenced throughout the “Create an Evaluation Plan” section of the guide. You can navigate to each resource using the icons below.

Glossary – Visit the Glossary to look up new terms and their definitions. Throughout the guide, words in the glossary are light blue.

Tips for Success – Check out these helpful tips to enhance your evaluation planning process.

Examples from the Field – Learn about noteworthy field work in each section of the guide.

Templates – Download forms and templates that can help you create elements of your evaluation plan.

Tools & Resources – Find helpful tools and resources that can inform and inspire your evaluation efforts.

Section 1: Evaluation Purpose and Partners | 12Evaluation Guide for Pedestrian Safety

SECTION 1: EVALUATION PURPOSE AND PARTNERSCongratulations! You’ve decided to evaluate your pedestrian safety intervention.

Your evaluation will help to…improve pedestrian safety in your community or state ANDcontribute to a growing pedestrian safety movement nationally and internationally BY identifying what works to effectively prevent pedestrian injuries and fatalities.

Put your feet to the pavement!

This section highlights your first steps to design your evaluation plan — convene your partners to discuss the fundamental components of your evaluation (population, intervention, audiences) in order to create your evaluation purpose statement and design your collaborative.

Step 1: Convene partners to plan the evaluationPartners provide a variety of perspectives, helpful ideas, and useful resources to ensure the success of the evaluation.

Start your journey!

Reach out to existing colleagues and make a list of their contacts and networks. AND Investigate previous pedestrian safety initiatives and evaluation efforts in your community or state.

This information can help you determine what individuals and agencies to engage as well as well as what successes or challenges emerged as part of previous related efforts.


What partners?Invite people from all walks of life to the table, including funders, decision-makers, advocates, implementers, evaluators, and members of the population of interest to discuss their respective interests and concerns related to pedestrian safety.

Identify those with wisdom and past experience working on pedestrian safety as well as those who are new to pedestrian safety, who offer fresh perspectives on how to create positive change in your community or state.

Key partners for evaluating pedestrian safety initiatives include representatives from:

• Local, regional, and state government agencies• Academic institutions• Health care organizations• Civic, community, and advocacy groups• Funding organizations• Offices of elected and appointed officials and other policy- and decision-makers

How to engage partners?Identify partners’ interests and concerns related to pedestrian safety.

Varying viewpoints from diverse partners across different sectors and disciplines (e.g., design and planning, transportation, health, economics, and environmental science) can inform and enhance the breadth and depth of the purpose of your evaluation.

Engagement of members of the population of interest may require a strong partnership commitment to hiring residents, people who grew up in the community, and/or those who have the cultural and linguistic characteristics of the individuals living in this population.

Tip for Success


Consider the “added value” of your pedestrian safety initiative to a wide range of potential partners concerned with different assets or goods affecting sustainability in your community or state, such as:

Source: Adapted from Forum for the Future, https://www.forumforthefuture.org/project/five-capitals/overview.

Generate a list of these individuals or organizations and their assets.

Example Relationship to pedestrian safety

Environmental protection of renewable and non-renewable resources�(natural�capital)

More pedestrians and fewer drivers reduce the carbon footprint.

Health promotion and education(human�capital)

Moderate-intensity physical activity, such as walking, improves health and well-being.Healthy, knowledgeable, and skilled people contribute to a productive workforce.

Networks, trust, and norms of reciprocity�(social�capital)

More pedestrians and fewer drivers increase opportunities for social interaction, stimulate the local economy, and enhance crime prevention.

Infrastructure and technology(manufactured�capital)

Pedestrian-oriented community and street design increases walking and decreases driving.

Wealth accumulation(financial�capital)

Walking is the simplest and least expensive mode of transportation (individual).Pedestrian infrastructure costs much less than automobile infrastructure (community).

Partner Assessment and Engagement Plan templateA template to guide partner engagement and assessment of assets for evaluation.

Template


Partners’ assets can offer insight into potential roles and responsibilities of partners as well as in-kind resources to support planning and implementation of the evaluation.

Types of Partners Examples Assets

Government agencies(local,�regional,�and�state)

• Public health• Design and planning• Transportation• Transit services • Highway safety• Traffic engineering• Public works• Law enforcement

Pedestrian Safety Action Plans• Access to city officials and city council agendas• Access to community and business leaders• In-kind support (e.g., staff support, meeting space, data)• Best practices in road design, traffic safety, and law enforcement

Educational and Promotional Efforts• Access to community events or calendars• Access to communication/ public affairs staff• Best practices in health education and injury prevention• Existing networks and systems for community education and outreach

Academic institutions • Centers for applied science/ practice across disciplines (e.g., public health, planning, law. environment)

• Knowledge of theory, best practices, and evaluation and research methods• Ability to collect and analyze data• Connections with students or volunteer support

Health care organizations • Emergency response• Hospitals

• Data on pedestrian injuries and fatalities

Civic, community, and advocacy groups

• Advocacy organizations• Coalitions• Business districts• Walk/ bike clubs• Youth groups and senior citizen

centers• Municipal advisory groups• Neighborhood organizations

• Authentic voice on pedestrian concerns and danger areas• Knowledge of and influence on civic and community leaders• Access to community listservs and grassroots outreach channels• Source of volunteer support for events and outreach


Types of Partners Examples Assets

Funding organizations • Government (e.g., federal or state agencies, local budget and tax incentives)

• Non-profit (e.g., national or state associations, community development corporations)

• Private (e.g., foundations, businesses)

• Source of funding for pedestrian safety projects as well as events or campaigns• Links to technical assistance and resources• Opportunities for community development and capacity building

Offices�of�elected�and�appointed�officials�and�other policy- and decision-makers

• City or County Councils• Local, state, and national-level

elected and appointed officials (mayors, county executives, school district administrators)

• High visibility “champions”• Formal commitments, such as city council resolutions

Source: Adapted from Watch for Me – NC Bicycle and Pedestrian Safety, Education, and Enforcement Campaign: Project Summary and Evaluation Final Report. 2014. http://www.watchformenc.org/wp-content/themes/WatchForMeNC_Custom/documents/WFM_FinalReport_2014.pdf

Create a brief partnership profile to share with potential new partners or to use as a tool for recruitment of new partners.

Tip for Success

New Orleans, LA partnership profileAn example of a partnership profile.

Example from the Field


NC Watch 4 Me Community and Partner ProfilesExamples of different sized communities across North Carolina (e.g., 100,000+ population, < 25,000) participated in the Watch for Me NC Campaign including descriptions of partnership profiles and key outcomes.


With your partners, decide whether it makes sense to conduct the evaluation yourselves or to hire an external evaluator.

Step 2: Identify your population of interest and your vision for changeTo ensure your evaluation plan has a clear focus and direction, define the population of interest and the pedestrian safety vision for this population.

Define your population of interestThe�population�of�interest�for�pedestrian�safety�initiatives�may�be�defined�in�several�ways,�including:

• Geographically, such as single or multiple states, single or multiple counties, metropolitan areas, cities or municipalities, zip codes or school districts, census tracts or neighborhoods.

• By institutions or organizations, such as government agencies, worksites, churches, schools.• By socially-defined or virtual groups, such as walking clubs or internet communities.• A combination of the groups above.

Consider priority subpopulations in this population as well. For instance, lower-income neighborhoods and people of color are disproportionately impacted by pedestrian fatalities and serious injuries.1, 2

NC Watch 4 Me Scaling and Focusing Intervention for Target PopulationsLessons learned for scaling and focusing interventions to target populations.



Revisit your partnership’s pedestrian safety vision and missionRefer to your partnership’s vision and mission to align partners’ intentions for increases in pedestrian safety in the population of interest.

Your partnership’s vision and mission may continue to evolve as new partners join or as the evaluation plan unfolds.

If you and your partners need to create a vision and mission, start by discussing how the population of interest will look different (e.g., improved health and quality of life, increased walking for transportation and use of public transportation, increased educational attainment and productivity) and what factors or conditions (e.g., mixed-use development, pedestrian-oriented street design, traffic calming, awareness of the benefits of walking) need to be changed in order to achieve these population changes.

To increase partnership momentum to evaluate your intervention, align your vision and mission with other national, state, or local partners and initiatives as a movement (e.g., Vision Zero, http://visionzeronetwork.org/).

Step 3: Consider the intervention stage and corresponding types of evaluationIdentify your intervention stage:

Planning – You and your partners are in the process of designing your intervention goals, objectives, activities, partners’ roles and responsibilities, and resources needed to carry out the intervention.

Implementation – You and your partners are engaged in intervention delivery.

Enforcement/maintenance – You and your partners are trying to sustain the intervention beyond the original timeline for intervention delivery.

FOOTNOTES 1. Morency P, Gauvin L, Plante C, Morency C. (2012). Neighborhood Social Inequalities in Road Traffic Injuries: The Influence of Traffic Volume and Road Design. American

Journal of Public Health 102(6):1112-9. DOI: 10.2105/AJPH.2011.300528. 2. Coughenour C, Clark S, Singh A, Claw E, Abelar J, Huebner J, et al. (2017). Examining racial bias as a potential factor in pedestrian crashes. Accident Analysis and Prevention

98:96-100. https://doi.org/10.1016/j.aap.2016.09.031.


Depending on your intervention stage, your evaluation plan may address one or more of the following four evaluation types:

1. Formative evaluation Planning stage ! Formative evaluation

Formative evaluation is frequently referred to as “community assessment”, or the examination of a community’s assets, needs, current resources, strengths, and challenges.

Asset mapping documents the resources and supports that already exist in your community.

Needs assessment identifies the changes need to occur in the community to improve health.

A comprehensive formative evaluation can address all of the following:

• decisions about where to focus resources and interventions in order to maximally benefit the community;• understanding of the relationship between pedestrian safety interventions, pedestrian and driver behaviors, and health and related

outcomes;• characteristics of the partnership and the population prior to intervention to identify factors for tracking or monitoring change; and• contextual factors that can influence the intervention or evaluation (e.g., concentrated poverty or areas with a small local tax base, rural

communities or areas with a small population density).

2. Process evaluation Implementation stage ! Process evaluation

Process evaluation helps you and your partners determine how well the intervention is working.

Factors such as feasibility, cost, reach, and unanticipated barriers can positively or negatively affect implementation fidelity and the population’s satisfaction with the intervention.

Process evaluation involves an assessment of how well intervention activities are carried out (e.g., policies, media, partner or community meetings), including evaluation of both the partnership and the pedestrian safety intervention.

3. Impact evaluation If intervention implementation is partially or fully complete, impact evaluation demonstrates how and to what degree intervention objectives have been attained.


4. Outcome evaluation If intervention implementation is fully complete, outcome evaluation demonstrates how and to what degree intervention goals have been attained.

Type of Evaluation Example Questions Example Types of Documentation

Formative evaluation • Will state, regional, and local departments of planning and transportation require developers and engineers to adhere to pedestrian level-of-service (LOS) guidelines to prevent pedestrian injuries and fatalities?

• Interviews with appointed officials and staff in departments of planning and transportation

• Policy scans to identify existing guidelines

Process evaluation • What resources (personnel, space, equipment) and costs (in-kind and monetary) are incurred as a result of developing, implementing, and enforcing pedestrian LOS guidelines for developers and engineers?

• Review of meeting minutes, time sheets, and budget or expense reports

• Implementation tracking forms or checklists

Impact evaluation • Will the pedestrian LOS guidelines decrease the number of collisions involving pedestrians in three years? (policy objective)

• What were some of the intended and unintended consequences?

• Inventory of new or modified developments or streetscapes adhering to the pedestrian LOS guidelines

• Crash reports for these areas• Focus groups with local residents to assess related consequences

Outcome evaluation • Will the pedestrian LOS guidelines reduce the proportion of pedestrian injuries by 75% in five years? (policy goal)

• Crash reports or crash maps over 5-10 year time frames• Hospital records


Multi-component and complex pedestrian safety interventionsYou and your evaluation partners may have different policy, practice, environmental, programmatic,or promotional components in various intervention stages, thus multiple types of evaluation may be occurring simultaneously.

“Downstream” and “upstream” intervention componentsYou and your evaluation partners are likely to evaluate your policy, practice, environmental, programmatic, or promotional components or your “downstream” intervention strategies.

To fully understand the change process and outcomes, it is also helpful to evaluate your complementary “upstream” strategies and activities leading to these primary intervention components (e.g., community development and organizing, advocacy).

Visit Intervention Strategies for a list of potential pedestrian safety strategies.

Step 4: Determine evaluation audiences and the relevance of the evaluation to these audiencesWho will use the evaluation findings?

Your audiences represent the same interest groups as your partners from Step 1, including funders, decision-makers, advocates, implementers, evaluators, and members of the population of interest.

Examples of how findings will be usedReflect on findings from a formative evaluation to work with your partners to develop or refine intervention approaches.

Identify ways you and your partners may use findings from a process evaluation to improve the delivery of the intervention.

Consider how funders and decision-makers may use the results of your impact and outcome evaluation to assess the return on their investment or allocation of funds, respectively.

How will findings be used?Most audiences will be interested in the results of an impact and outcome evaluation to determine the “value” of the pedestrian safety intervention strategies for reducing pedestrian injuries and fatalities, as well as improving health and quality of life.


Partners Perceived Purpose of an Evaluation Interests

Evaluators and researchers To determine the effectiveness of local policy, environment, and system changes

• Behavior, health outcomes• Reliable, valid tools• Evaluation design

Policymakers and practitioners

To identify the changes with the greatest impact, relevance, feasibility, and sustainability

• What works, where, when, how and why• Resources and assets• Costs and challenges

Advocates and community leaders

To inform local decision-making, document successes, and improve quality of life

• Local representation and participation• Health equity• Improved quality of life

Research and evaluation partners:

Assess the effectiveness of pedestrian safety action plans and educational and promotional intervention approaches on improvements to a range of short-term, intermediate, and long-term impacts and outcomes. Use reliable and valid quantitative tools and measures, as well as a rigorous study design and execution to ensure confidence in the findings from the evaluation.

Policymakers and practitioners:

Identify the intervention approaches with the greatest impact, relevance, feasibility, and sustainability (RE-AIM framework) by assessing what works, where it works, when it works, how it works, why it works (or why not), and how much it costs. Specifically, policymakers often seek evidence that their constituents are being served, stories of positive examples of change, and publicity that will get them re-elected.

Incorporate mixed-methods (which combine quantitative and qualitative data) and ensure that the data is representative of the population of interest and corresponding subpopulations to ensure confidence in the findings from the evaluation.


Advocates and community leaders:

Determine the practical implications of findings from the use of different intervention approaches to inform state or local decision-making, highlight state or local successes, and secure more funding to sustain the changes over time.

Track intended and unintended results, practical considerations (resources, costs), assets used, and challenges encountered using simple, efficient measures serving multiple purposes (advocacy, marketing, cost-benefit analysis) that are easily translated to the interests of local audiences (decision-makers, business owners).

These interests are not mutually exclusive, yet it can sometimes be difficult to meet the needs of allpartners. Therefore, it is important to be clear about what your evaluation is or is not going to accomplish.

Revisit the list of partners’ interests and concerns related to pedestrian safety from Step 1 for additional potential uses of the evaluation findings.

Step 5: Create a clear, concise purpose statementInitially, your evaluation purpose statement can be general, such as: “the purpose of this evaluation is to measure the impact of the pedestrian safety initiative.”

From your work in Steps 1-4…

Your purpose statement should have evolved to include the following elements:

• A specific reference to your population and subpopulations of interest;• A connection to the partnership’s vision and mission; • Relevant formative, process, impact, and/or outcome evaluation components; and• Uses of the evaluation findings by priority audiences.

Consider the range�of�factors�or�conditions�affecting�pedestrian�safety�in�the�population�of�interest,�your�priority�factors�and�conditions, and “low hanging fruit”, or factors and conditions that can be more readily addressed by your partnership, including:

• Partnership strengths, weaknesses, opportunities, and threats;• Organizing and advocacy activities;• Changes in policies, practices, and environments affecting pedestrian safety;


• Educational and promotional efforts increasing pedestrian safety knowledge and skills in the population of interest; and • Health behaviors, health outcomes, and related social, economic, and environmental outcomes.

To increase health equity, pay particular attention to how health disparities, inequities, and social determinants of health may affect these factors and conditions.

WHO Commission on SDOHProvides background on equity, inequity, and disparity.

Tools & Resources

Unnatural Causes documentaryProvides background on equity, inequity, and disparity.

NACCHO Health Equity and Social Justice CommitteeProvides background on equity, inequity, and disparity.

RWJ Commission on SDOHProvides background on equity, inequity, and disparity.


Work with partners to obtain buy-in for the evaluation purpose across multiple sectors and disciplines (e.g., public health, transportation, planning, law enforcement, education, academia).

Update or refine the evaluation purpose after following the steps in subsequent sections, adding new partners, or adapting the intervention.

For instance, in Section 2, specification of your intervention goals and objectives as well as your associated intervention strategies and activities can help to refine the evaluation purpose statement.

Step 6: Design your collaborative, including a governance structure and process, partner responsibilities, and budget and time constraintsA clear governance structure and processes for collaborative engagement will support partnership cohesiveness and effectiveness throughout evaluation planning and implementation.

Federal Highway Administration, Office of Safety’s Pedestrian Safety Strategic PlanExample Pedestrian Safety Strategic Plan.


Governance PrinciplesProvides guidance on developing a governance structure and process for partnerships.

Tools & Resources


Governance structureReflect on the history of your partnership, the purposes for partners’ participation, the maturity of the initiative, and the degree of formality of the governance mechanisms for interaction.

Establish a governance structure that supports shared decision-making and accountability, including:

• Identifying a lead agency (fiscal accountability and coordination).• Assigning a representative set of core partners to lead the partnership, such as a steering committee (shared decision-making authority).• Gathering input and feedback from an extended network of partners varying in skills, expertise, or resources. This extended network of

partners can also be organized into subcommittees or workgroups to facilitate progress on multiple evaluation goals and objectives at the same time.

Consider�the�following�options�for�organizing�your�partners�and�determine�the�best�fit�for your collaborative work, including:

• A hierarchical organizational structure with regular committee and subcommittee meetings involving all partners.

• A management team of core partners that meets regularly and invites partners to participate as needed or available.

• A “revolving” partnership that is project-oriented, permitting partners to spend time focused solely on what interests them and to participate in focused meetings with relevant partners to discuss these specific projects or activities.

• An informal collaboration with open meetings as needed among staff and/or volunteers.

Governance processesSeveral processes can facilitate good governance practices in your partnership, including:

• Developing strong leadership and group management skills among partners.• Forming groups (e.g., committees, task forces) to distribute the workload.• Creating partnership principles or a partner orientation manual.

• Document and share the partnership’s history.• Establish mechanisms for all members to voice their opinions and share their expertise.• Require partners to have consistent meeting attendance.• Increase partners’ cultural competency.• Address the needs of representatives of lower-income neighborhoods, including selecting convenient times


and places for meetings as well as providing meals, childcare, and transportation assistance to attendees.• Providing training and orientation to new partners.• Pooling resources across the partnership (e.g., staff, funding, skills).• Maintaining records on the partnership’s processes, decisions, and accomplishments.• Developing succession plans to support leadership transitions.

The�effort�to�delineate�roles�and�responsibilities�is�an�ongoing�process�through�the�life�of�the�partnership�for�the�following� primary reasons:

• As you and your evaluation partners move through design and planning stages into implementation of the evaluation plan, partners’ respective roles and responsibilities will likely shift over time.

• To promote shared governance, regular leadership transitions can foster greater inclusion and representativeness of partners.• Leadership, staff, or volunteer turnover is common in partnerships, thus it is helpful to have multiple partners assigned to different roles

and responsibilities to increase institutional memory and partnership sustainability.

With these changes in mind, work with your partners to assign roles and responsibilities for evaluation planning and implementation

Critical evaluation skills and capacities of partners include previous experience in research methods or evaluation (e.g., design, data collection methods, analysis and summary).

Supplemental skills and capacities may include having talented networkers, diplomatic personalities, good listening skills, passion about pedestrian safety issues, perseverance, dedication, adaptability.

Assess your partners’ evaluation skills, readiness, and capacity for evaluation as well as assets and available funds or resources to support evaluation efforts.

Identify potential needs for training or technical assistance.

Evaluation Partner Planning Grid TemplateA template to guide assignment of roles and responsibilities for evaluation.

Template


Local champions are sometimes absent from initial planning efforts. You and your evaluation partners may consider investing time in recruiting these partners for a variety of reasons.

Ensure each organizational partner has more than one person involved in the partnership to maintain institutional memory in the event of staff turnover.

Tip for Success

FL Pedestrian and Bicycle Strategic Safety Plan - Partner RolesAn example of the types of partners and their roles in a pedestrian safety evaluation.


NY DOT Steps to form evaluation teamAn example of the steps taken in to form a successful evaluation team.

NC Watch 4 Me Partner InputAn example of how successfully integrate partners into the overall evaluation planning process.


These individuals can be very valuable to generate enthusiasm for your work behind the scenes, in the public eye, or with populations of interest.

These individuals are visionary, charismatic, energetic, passionate, well-known and respected, well-connected with a strong network of resources, trusted by the partners and the population of interest, competent, persistent, and politically savvy or in a position of power or influence.

Frequently, these “sparkplugs” are asked to:

• engage many different communities and audiences; • inspire movement from vision to action;• leverage funding;• stand up to established thinking and lead advocacy efforts;• maintain and support direct and indirect participation of the community; and• connect the right people to the right opportunities.3

Once you and your evaluation partners have delegated responsibilities to different individuals, you are ready to begin to outline tasks and timelines for these respective duties. A template is available to assist you and your partners through this process.

FOOTNOTES3. Baker EA, Wilkerson R, Brennan LK. Identifying the Role of Community Partnerships in Creating Change to Support Active Living. American Journal of Preventive

Medicine2012;43(5S4):S290-S299.

NC Watch 4 Me Community ChampionAn example of how to successfully utilize community champions in an evaluation.


Gantt Chart and Timeline of Evaluation Activities TemplateA template to guide the development of a timeline for evaluation.

Template


Institute shared leadership or periodic leadership changes to bring new energy to the evaluation, overcome bureaucratic or regulatory roadblocks, or realign partners with the vision over time.

Tip for Success

Enhance partnership sustainability by increasing consistency and follow through among leaders and staff, offering revolving partnership membership, creating long range plans, or involving representatives from local government.

In addition, you and your evaluation partners will need to agree on a budget for the expenses associated with the evaluation activities and revisit this budget as new partners join or the evaluation plan is adapted.

Sample BudgetProvides guidelines for developing a budget.

Tools & Resources

Section 2: Describing the Intervention | 31Evaluation Guide for Pedestrian Safety

SECTION 2: DESCRIBING THE INTERVENTIONThis section provides steps on how to create an intervention description and logic model to ensure partners’ have a mutual understanding of the intervention and a “map” to focus the evaluation on the intervention inputs, activities, outputs, and outcomes.

Step 1: Describe your Pedestrian Safety InterventionThe description of your pedestrian safety intervention should include:

• Intervention goals and objectives• Populations of interest • Risk and protective factors that will be addressed by the intervention

Intervention goals and objectives Identifying and articulating your pedestrian safety intervention’s goals and objectives early in the evaluation planning process can ensure you and your evaluation partners have a clear understanding of what you are evaluating.

Begin by having partners clarify what the intervention is to achieve in their own words to ensure multiple perspectives are captured and the evaluation plan is comprehensive. Engaging partners in this manner also ensures that evaluation findings will be useful and applicable to partners and audiences (described in Section 6).

Consider the following questions with your evaluation partners:

1. What are the intervention goals and objectives? 2. Are these goals and objectives timely? 3. Who will be interested in the goals and objectives being achieved? 4. What evaluation resources are needed/ available to assess these goals and objectives?5. How can the evaluation take into consideration the historical, political, and environmental contexts influencing these goals and

objectives?


If�you�and�your�evaluation�partners�decide�to�refine�the�goals�and�objectives�for�evaluation�purposes,�remember�the�following:�

Goals are: Objectives are:• Broad • Narrow• General intentions • Precise• Abstract • Concrete • Intangible • Tangible• Generally difficult to measure • Measurable • Time bound

Goals are driven by the ultimate vision of a successful intervention when all conditions are optimal. Goals describe the long-term, desired condition once the intervention is complete.

In your evaluation plan, your goals should include:

1. Your population of interest and relevant subpopulations; AND2. Your desired intervention impact; OR3. Your outcomes intended to be achieved.

The following are a few examples of sources of pedestrian safety goals that can be useful to you and your evaluation partners:

Sources of Intervention Goals

Description Examples

Pedestrian Safety Action Plans�(PSAPs)

Evaluators commonly adapt and modify existing intervention goals outlined in the PSAP to align with the evaluation effort.

• Identify and prioritize candidate project corridors through a data-driven process to reduce fatally and severe injury pedestrian crashes on all public roads throughout the state

• Establish a governance and management structure to facilitate coordinated implementation of pedestrian safety initiatives statewide and gauge the success of the Pedestrian Safety Action Plan

Pedestrian safety education campaigns and promotions

Review intervention implementation plans for educational campaigns and promotions. Evaluators commonly use these plans to create, modify, or adapt goals.

• Mobilize the local community and increase awareness and action related to pedestrian safety

• Inform the public or policymakers about specific pedestrian safety issues and concerns


Sources of Intervention Goals

Description Examples

Literature review and evidence based strategies

Conduct a simple, or extensive, literature review to identify anticipated evidence-based outcomes that can potentially be seen in your demographic.

• Foster behavioral change among users of public-right-of-way to promote an environment of mutual respect, courtesy and acceptance

• Increase the amount and quality of pedestrian space in the downtown area

Organization’s strategic plan Align the intervention goals with the overall strategic plan, organizational mission, or agency vision.

• Improve transportation system infrastructure (through the implantation of strategic countermeasures and construction of new transportation facilities) to optimize the safety for all users.

• Manage the objectives of efficient automobile travel and pedestrian safety mobility through land use strategies.

Review intervention objectives to ensure they are measurable and describe the desired outcome as well as how and when the results will be achieved. Likewise, make sure the objectives are clear and concise so the definition of “success” is obvious to everyone involved. Objectives should be measured against pre-established and data-driven benchmarks (e.g., data from NHTSA’s Fatality Analysis Reporting System) set by your team (elaborated in Sections 4 and 5).

The objectives should be created using the SMART objective framework outlined below:

Specific A specific objective will identify the setting and activity the caused the desired change. Additionally, it will indicate how the change was implemented and clearly demonstrate what was done to facilitate the impact.

Measurable A measurable objective requires a quantifiable activity that resulted in the desired change. It implies that baseline data is required so that result can reflect the positive and/or negative impact of the proposed intervention.

Achievable An achievable objective is feasible and considers the availability of resources, the scope of the intervention, and is attainable within a bounded timeframe.

Relevant A relevant objective relates to the goals and reflects program activities appropriately. The evaluation objective has an overall effect on the desired change.

Time Bound Identifies when the objective will be accomplished using a specific, reasonable timeframe


The SMART objective framework can be applied to objectives associated with process evaluation, impact evaluation, and outcome evaluation that will guide and inform the evaluation plan. Populations of InterestThe populations of interest can include pedestrian sociodemographic groups (e.g., older pedestrians 55+ years, children aged 5-9 years, those in poverty)and residents or commuters in a specific location (e.g., street, corridor, campus, neighborhood, county). If the populations of interest are not clearly specified in the intervention plan for each of the goals and objectives, your evaluation plan should specify the sociodemographic group(s) and location(s) of the intervention to guide the data collection methods (Section 4) and the unit of analysis (Section 5) in the evaluation plan.

Tailoring your evaluation plan to your priority population or subpopulations can increase confidence that your results are generalizable to these and other similar populations.

Below are some things to consider with respect to your priority population or subpopulations:

• What are key characteristics of the population and subpopulations? • What are the rates of pedestrian injuries and fatalities in the population and subpopulations? • What factors affect pedestrian safety in your population and subpopulations and what is their independent or collective impact

(good and bad)? • What related policies, practices, services, facilities, or educational and promotional efforts are currently being provided? Does this

intervention complement or duplicate any of these? • Does the intervention address pedestrian safety priorities and needs identified by the population of interest?• What are national, state, and local policy priorities for pedestrian safety and how do these correspond to your population or

subpopulations?


The�table�below�provides�some�example�characteristics�to�include�in�your�population�profile:

Risk and protective factorsRisk and protective factors identified during the intervention planning phase will focus the evaluation on known safety problems and allow the evaluation partners to select an evaluation design and methods to best understand the influences of risks and protective factors in the context of the intervention (see Sections 3 and 4).

Refer to the intervention plan to identify risk and protective factors. Typically, these are grouped into two categories: personal factors and environmental factors. These two categories take a multi-level approach to understanding what will have a positive or negative influence on pedestrian safety in your population of interest.

Characteristics of population profile

Geography What geographic areas will the evaluation plan cover (e.g., street, neighborhood, city, county)?

Size How many people are impacted by this intervention (e.g., population density)?

Age What is the age distribution of this population?

Gender How many males versus females?

Income What is the income distribution of this population? What is the percent below the federal poverty level (FPL)?

Ethnicity and religion What is the racial and ethnic composition of this population? What are this population’s cultural norms and values?

Population trends or context What communication approaches are most appropriate for this population (e.g., social media, billboards, TV)? What is the historical context affecting your population or subpopulations? What political or economic influences influence your population’s access to information and resources?

Language and literacy What proportion of the population speaks English as a second language? What is the reading level or what literacy considerations need to be taken into consideration for this population?


Tabled below are examples of common personal and environmental risk and protective factors applicable to pedestrian safety intervention evaluations:

Step 2: Create a Logic Model A logic model is often used in an evaluation to visually depict the “theory of change” behind how an intervention is intended to influence outcomes. Generally, a logic model is read by column from left to right and it includes directional arrows and lines to illustrate causal relationships between the intervention inputs, activities, and outputs and the short, intermediate, and long-term outcomes. Although the logic model is frequently a linear concept map, it can be complex in terms of the number of relationships among inputs, activities, and outputs and the resulting outcomes as well as the potential feedback and influence of the outcomes on the inputs, activities, and outputs.

Categories Types Examples of Risk and Protective Factors

Personal Knowledge and skills • Understanding of pedestrian safety laws near school zones

Experience and history • Drug/alcohol use by motorists and pedestrians• Literacy rates of program participants

Support and services • Roadway nighttime lighting, lack of signage and walkways

Environmental Access, barriers, and opportunities

• Land use (commercial complexes located immediately across from residential areas with high-volume multi-lane roads separating them)

• Designs (multiple turn lanes and wide vehicle turning radii)• Wide roads (four or more lanes)

Policies and living conditions • Enforcement of speed limits in high motor vehicle volume areas

To read more about risk and protective factors, read “Section 2. Understanding Risk and Protective Factors: Their Use in Selecting Potential Targets and Promising Strategies for Intervention”

Tip for Success


Logic models should reflect process evaluation, impact evaluation, and outcome evaluation elements. The process component of the model will describe what is needed in the planning phase of the intervention, while the outcome elements in a logic model demonstrate the intended effect or goal with respect to a given time period.

Other elements of the logic model include ‘assumptions’ and ‘external environment’

Here is an example of a simplified logic model that illustrates the theory of how specific pedestrian safety interventions leads to intended short-term and long-term outcomes. The goal of the simplified logic model is to communicate with other stakeholders and partners the intent of the interventions and create the framework for the evaluation.

Activities Short-Term Outcomes Long-Term Outcomes

Educational Campaigns and Promotions / Encouragement • Implement walking program or club to support pedestrian safety• Implement wellness program that educates attendees about the

safe walking

• Increase knowledge and awareness of pedestrian safety programs

• Improved pedestrian behaviors

• Reduced pedestrian crash rate • Reduced intersection pedestrian

crash rate (per crossings)• Reduced pedestrian injuries• Reduced pedestrian deaths• Reduced percent of pedestrian

fatalities• Improved livable, walkable

communities

Enforcement • Issue citations for both the driver and pedestrian at crosswalks• Provide High-Visibility Enforcement (HVE) on driver compliance

with pedestrian right-of-way laws

• Change in motorist cited for not yielding to pedestrians

• Change in motorist cited for speeding in pedestrian/school zones

Engineering • Create longer delays for pedestrians at intersections• Create pedestrian facilities (sidewalks, walkways, adequate lighting)• Constructing medians to control turning movements.• Install pedestrian signals (i.e., walk/don’t walk messages,

symbolic hand/walking man messages)

• Decrease in vehicle volume • Reduction in the number of travel lanes • Improved pedestrian crossing behaviors• Improved in motorist driving behaviors• Improved visibility of pedestrians and

motorists• Decrease in average vehicle speeds


Activities Short-Term Outcomes Long-Term Outcomes

Land Use Design, Planning, & Policy • Convert auto-oriented strip development into more accessible

land use patterns more suitable for pedestrians• Improve existing local street connectivity and circulation by

adding sidewalks, paths, stairs/ramps, gates, etc. to link dead-end streets and cul-de-sacs

• Change in walking distance• Increase in number of route choices• Decrease in the number of wide cross

streets

• Reduced pedestrian crash rate • Reduced intersection pedestrian

crash rate (per crossings)• Reduced pedestrian injuries• Reduced pedestrian deaths• Reduced percent of pedestrian

fatalities• Improved livable, walkable

communities

Advocacy • Decision-maker engagement and support (elected officials,

appointed officials, community leaders) through presenting to city/ county/ state councils/ committees, adopting principles for practice among appointed officials, etc.

• Increase knowledge of pedestrian safety issues and concern among elected/appointed officials and community leaders

Collaboration • Provide training and other types of technical assistance to

partners to engage in partnership and community activities

• Increase knowledge, skills, and abilities of individuals implementing pedestrian safety programs

Equity• Support policies for the development, funding, or maintenance

of active transportation and pedestrian safety facilities and design features in affordable housing/ housing authority locations

• Decrease pedestrian crash rates in low income geographical areas

Section 3: Evaluation Design | 39Evaluation Guide for Pedestrian Safety

SECTION 3: EVALUATION DESIGNThis section provides steps to develop evaluation questions in order to specify your evaluation design. It is important to work closely with your evaluation partners during the design phase to ensure there is buy-in for the approach that you choose. You should refer to your evaluation purpose statement (see Section 1) as you develop your evaluation questions.

Step 1: Develop evaluation questions aligned with the evaluation purpose statementWith your evaluation partners convened,

A. Brainstorm a list of evaluation questions aligned with your partnership’s evaluation purpose statement. Several approaches can be used to structure a brainstorming session with your partners, including more formal (nominal group technique, Delphi method) or informal methods.

In the idea generation phase, encourage your partners to consider the following categories of evaluation topics:

• Partnership strengths, weaknesses, opportunities, and threats;• Organizing and advocacy activities;• Changes in policies, practices, and environments affecting pedestrian safety;• Educational and promotional efforts increasing pedestrian safety knowledge and skills in the population of interest; and • Health behaviors, health outcomes, and related social, economic, and environmental outcomes.

You can also refer partners to the following resources generated in Sections I and II to stimulate ideas, including:

• Evaluation purpose statement;• Population and subpopulations of interest;• Partnership’s vision and mission;• Intervention goals and objectives;• Partnership’s logic model; and• Evaluation audiences and relevant of the evaluation findings to these audiences.


Make sure partner concerns are included to increase applicability of the evaluation results to pedestrian safety improvements.

Tip for Success

Sample Evaluation QuestionsSample process and outcome evaluation questions for PSAPs or education and promotion campaigns.


Process Questions(logic�model�input,�activities,�outputs)

Outcome Questions(logic�model�short-,�intermediate,�long-term�outcomes)��

• What intervention components were adopted, implemented, or enforced fully, partially, or not at all? Why?

• Were participants satisfied with the intervention components? Which ones? Why?

• Was there variation in implementation across subpopulations? Which subpopulations?

• What were assets or facilitators of intervention delivery? How can assets or facilitators be harnessed?

• What were barriers or challenges to intervention delivery? How can barriers be overcome?

• Who engaged, dropped out, or did not engage? Why?

• Did pedestrian safety training significantly improve knowledge and skills among participants (e.g., law enforcement officers, traffic engineers, city planners)?

• Did outreach to city council members (e.g., data on pedestrian fatalities, citizens’ testimonies on walkability) significantly influence their support of local ordinances and regulations to improve pedestrian safety?

• Did the Safe Routes to School education campaign significantly increase awareness of safe walking routes among students and safe driving practices among parents?

• Did traffic calming devices significantly reduce traffic speeds? • Did the worksite “distracted driving” campaign significantly increase safe

driving behaviors among employees? • Did the pedestrian safety initiative significantly reduce pedestrian injuries

and fatalities?

Sample Process and Outcome Evaluation Questions for PSAPs or Education and Promotion Campaigns4

FOOTNOTES4. Adapted from: HM Treasury 2011. The Magenta Book: Guidance for evaluation. Retrieved from: https://www.gov.uk/government/uploads/system/uploads/attachment_

data/file/220542/magenta_book_combined.pdf


B. Prioritize the list of evaluation questions using criteria agreed upon by partners.In the prioritization phase, the following questions may stimulate your partnership’s selection of criteria for prioritization:

• Are the evaluation questions important to all partners or subsets of partners?• Are the evaluation questions relevant to the audiences for the evaluation?• Are the evaluation questions aligned with the evaluation purpose statement and the

intervention goals, objectives, and logic model?• Are the evaluation questions feasible to address within the current timeline, funding,

and resources? • Will the evaluation questions inform either actionable improvements to the partnership or

the intervention or evidence-based practices for the field?

As partners go through the prioritization process, be sure to track and explain the reasons why evaluation questions may be combined, refined, or discarded in relationship to the evaluation criteria. It is critical to share this rationale with partners to ensure all individuals feel their input and ongoing participation is needed and valued, particularly for those with little to no evaluation experience.

Review your final list of evaluation questions with all partners. Check that each question is specific, measurable, and achievable and that the partners responsible and the timeline are assigned, consistent with the SMART guidelines described in Section 2.

FOOTNOTES5. Adapted from: Salabarría-Peña, Y, Apt, B.S., Walsh, C.M. Practical Use of Program Evaluation among Sexually Transmitted Disease (STD) Programs, Atlanta (GA): Centers for

Disease Control and Prevention; 2007. Retrieved from: https://www.cdc.gov/std/program/pupestd/Step3_0215.pdf

Worksheet to Prioritize Evaluation Questions templateA template for rating the relevance of each evaluation question and making decisions about including vs. discarding questions based on specific criteria.5

Template


Step 2: Identify the population or subpopulations, intervention goals and objectives, and types of evaluation for each evaluation question Now that you and your evaluation partners have identified and prioritized the evaluation questions, it is time to identify the population or subpopulations, intervention goals and objectives, and types of evaluation corresponding to each question.

A simple table can be used to link all of these elements as follows:

Your partnership can draw on the initial populations and subpopulations and types of evaluation identified in Section 1, including formative, process, impact, and outcome evaluation, as well as the intervention goals and objectives from Section 2.

You and your evaluation partners will need to identify whether the evaluation questions are aimed at assessment (pre-intervention, formative evaluation), implementation (intervention delivery, process evaluation), or outcomes (during or immediately following the intervention, impact, or outcome evaluation), or some combination of these. Again, impact and outcome evaluation correspond to the outcomes identified in your intervention objectives and goals, respectively.

Crafting evaluation questions that bring together formative and impact or outcome types of evaluation set the stage for evaluation designs with both baseline and follow-up measures. Combining these methods will increase confidence that changes in the outcomes being measured are, to some degree, due to the intervention strategies being evaluated.

Evaluation question Population or subpopulations Intervention goals and objectives Types of evaluation

From Section 2, Step 1 Updated from Section 1 From Section 2 Updated from Section 1

Attribution Contribution

Caused the observed outcomes• Are the outcomes of interest attributable to the intervention?• Are the outcomes of interest changing as a result of the

intervention?• Did the intervention cause the outcomes of interest?

Helped to cause the observed outcomes• Is the intervention contributing to the outcomes of interest?• Are the outcomes of interest changing?• Is there evidence that the intervention helped achieve (or was part of what

caused) the outcomes of interest?


When you evaluate pedestrian safety interventions, such as educational and promotional campaigns and PSAPs, it is challenging to determine if the intervention components caused the change in the outcomes being measured. This is due to a number of factors, including multiple complementary intervention strategies, longer intervention duration, and fluctuating environmental, social, and cultural factors affecting pedestrian safety that can be challenging to account for in the evaluation. While you and your evaluation partners are designing an evaluation plan to demonstrate the changes in outcomes are fully attributed to your pedestrian safety intervention, it is more likely that the intervention components contributed to the changes in outcomes. See figure6 highlighting differences between attribution and contribution in evaluation.

Consider the following example:

FOOTNOTES6. A. Almquist (2011). CDC Coffee Break: Attribution versus Contribution. National Center for Chronic Disease Prevention and Health Promotion, Division for Heart Disease and Stroke

Prevention, Evaluation and Program Effectiveness Team. Retrieved from: https://www.cdc.gov/dhdsp/pubs/docs/april_2011_cb.pdf

Sample Evaluation Question

Population or Subpopulations

Intervention Goals and Objectives

Types of Evaluation

Did outreach to city council members (e.g., data on pedestrian fatalities, citizens’ testimonies on walkability) significantly influence their support of local ordinances and regulations to improve pedestrian safety?

City council members

Goal: To develop and enforce local policies in the next 12 months to reduce pedestrian injuries and fatalities in the next 3 years.

Objective: To increase city council members’ knowledge and awareness of escalating rates of pedestrian injuries and fatalities and associated ordinances and regulations to improve pedestrian safety in the next 6 months.

Ormative Evaluation: • Conduct a survey of city council members’ knowledge and awareness of rates of pedestrian

injuries and fatalities in their jurisdiction and potential policies to improve pedestrian safety.• Implement a policy scan to determine existing pedestrian safety ordinances and regulations

and interview staff from local planning, transportation, and law enforcement agencies to assess implementation and enforcement of these policies.

• Identify baseline rates of pedestrian injuries and fatalities prior to intervention implementation.

Process Evaluation:• Assess whether all city council members were present to receive data on pedestrian fatalities

and citizens’ testimonies on walkability.• Assess whether competing priorities or other barriers interfered with city council members’

opportunity to adopt new or modified policies.• Assess whether staff from local planning, transportation, and law enforcement agencies had

the knowledge, skills, and resources to implement and enforce the new or modified policies.

Impact Evaluation:• Re-administer the survey of city council members’ knowledge and awareness and assess

changes after 6 months of intervention implementation.

Outcome Evaluation:• Assess the number of new or modified pedestrian safety ordinances and regulations adopted

by city council and implemented and enforced by local planning, transportation, and law enforcement agencies after 12 months of intervention implementation.

• Assess changes in rates of pedestrian injuries and fatalities after 3 years of intervention implementation.


Alignment of process and impact/ outcome evaluation questions creates the opportunity to examine exposure to the intervention and/or dose of the intervention hand-in-hand with the changes observed, increasing the ability to make causal inferences about the intervention’s influence on the outcomes. In other words, the process evaluation data can provide useful descriptive data about how the intervention succeeded or failed to affect the outcomes observed.

Step 3: Specify the evaluation design to maximize causal inferenceand internal validity for each evaluation question The next step is to create your evaluation design(s) with respect to one or more of your evaluation questions and the associated types of evaluation. As noted in the previous step, you want to maximize your ability to say that the pedestrian safety intervention caused the changes in outcomes observed. Although many pedestrian safety evaluations will not be able to show full causality, or attribution, your evaluation can show that the combination of intervention activities contributed to the observed outcome of interest. Causal inferences increase confidence in the internal validity of the relationship of the intervention to the outcomes observed. Rigorous evaluation designs help to minimize threats to internal validity.

A. Select your evaluation designThere are three basic categories of evaluation design: experimental, quasi-experimental, and non-experimental. The key characteristics of each design are shown in the following table:

Experimental Designs Quasi-experimental Designs Non-experimental Designs

Attribution Attribution or Contribution Contribution

• Randomization• Includes a control group• Helps establish cause and effect• Most rigorous type of design

• No randomization• Includes a comparison group• Individuals in comparison and intervention

groups should have similar characteristics other than intervention exposure

• No randomization• Does not include a comparison group• Most feasible/ practical to implement• Does not establish cause and effect


Evaluation Design IllustrationsThis table shows descriptions, examples, and graphic depictions of each evaluation design along with causality and strength of attribution.

Tools & Resources

Evaluation Design Illustrations7

FOOTNOTES7. Adapted from: Measurement, Learning & Evaluation (MLE) Project (evaluation component of the Urban Reproductive Health Initiative). Retrieved from:

https://www.urbanreproductivehealth.org/toolkits/measuring-success/types-evaluation-designs

Causal Evidence Evaluation Design(examples) Description Design Illustration

Experimental (randomized controlled trial or RCT)

Individuals from the evaluation population are randomly assigned to experimental (intervention) or control/ comparison groups for group equivalence and data is collected from both groups before and after the intervention.

Quasi-experimental (e.g., non-randomized trial, before-and-after study, time series)

Individuals in the experimental (intervention) group are compared to individuals from another, similar “comparison group” or data is collected from individuals in the experimental group before and after the intervention. Baseline observations are the “comparison group.”


Causal Evidence Evaluation Design(examples) Description Design Illustration

Non-experimental, observation, or descriptive (e.g., post-test only, case study)

Only the exper

evaluation guide for pedestrian safetyevaluation guide for pedestrian safety about the evaluation...

Documents