leveraging the knowhow2go network survey in evaluation and program
TRANSCRIPT
Leveraging the KnowHow2GO Network Survey in
Evaluation and Program Improvement Efforts:
The Case of the Washington State College Access
Network
American Evaluation Association
Hilary Loeb [email protected]
Kelly Bay-Meyer [email protected]
Lee Lambert [email protected]
October 25, 2012 1
Presentation Overview
Mission and work of Washington
College Access Network (WCAN)
WCAN evaluation planning
Implementing the KnowHow2GO survey
Reflections about institutionalizing the
KnowHow2GO survey
2
WCAN Mission
WCAN strives to improve access to and
preparation for higher education
through a network of community
organizations that collaborates to
promote and support the use of best
practices, leverage training
opportunities and public policies
ensuring the success of each student.
3
WCAN Logic Model
4
WCAN Theory of Change
5
Local college access
networks in every
region of the state.
Short Term Goal
Coordinated delivery of
early college
awareness information
to low-income
students.
Intermediate Goal
Increase in the number
of low-income first
generation student
graduating from
college.
Long Term Goal
• Socioeconomic status no longer a predictor for educational outcomes.
• 60% of Washingtonians with high-quality degrees and credentials by the
year 2025.
WCAN’s Long Term Outcomes
About CSF’s Research and Evaluation
Department
Small team of doctoral-level staff and
university interns
Supported by operational and grant
funding
Influenced by utilization-focused
evaluation and appreciative inquiry
Commitment to mixed-methods inquiry
6
Early Implementation Evaluation
Questions
1. What have been the patterns of growth (membership, network, capacity, etc.) in WCAN since the inception of the Network?
2. How have WCAN and partner organizations built awareness of college access among students and organizations in Washington?
3. What have been the patterns of growth in the College Bound Scholarship since the launch of WCAN?
7
Consultation with external and internal advisory group
was a critical step in designing the study.
Early Implementation Inquiry Methods
and Data Sources
8
Triangulation
across
Data Sources
Focus Groups
Grantee
Reports
College Bound
Scholarship Data
Ongoing Discussion of Findings
KnowHow2GO Surveys
KnowHow2GO Network Survey Design
and Data Collection
Developing customized questions about policy context
Administering over Survey Monkey to selected members
Sending email reminders and making phone calls critical to boosting response rate
Providing opportunity to enter into drawing for small incentive
9
KnowHow2GO Network Survey Analysis
and Reporting Descriptive statistical analysis looking at differences
between 2010 and 2011 administrations
Findings summarized in grant reports and proposals
User-friendly reports shared with team
10
Leveraging the KnowHow2GO Network
Survey
Survey adapted for the District of
Columbia Double the Numbers initiative
2012 process improvement in data
collection to provide weekly updates
about responses
11
Leveraging the KnowHow2GO Network
Survey
2012 survey adapted by WCAN team
and is currently in field
Shortening the survey
Crafting items in which specific progress is
attributed to WCAN
12
Reflections about Institutionalizing the
KnowHow2GO Survey
Survey is critical tool in defining annual progress of newer program at CSF
Need to provide ongoing support to team about technical aspects of administering survey
Triangulation through multiple data sources enhances confidence in larger evaluation findings
Eager to benchmark work with college access networks in other states
13