gk-12 program evaluation: summary of findings · 2019-11-25 · 3. evaluation data sources,...
TRANSCRIPT
GK-12 Program Evaluation: Summary of Findings
Annual GK-12 PI Meeting March 28, 2010Beth C. Gamse
111
Overview of Presentation
• Description of Evaluation
• GK- 12 Projects and Participants – GK-12 Fellows – K-12 Teachers
• Comparing GK-12 Fellows to Their Peers
• Former Fellows … After GK-12
• Questions/comments
22
Evaluation Purposes
• To describe the experiences of past and present GK-12 participants, including Fellows and K-12 teachers
• To compare the educational experiences of current GK-12 Fellows with other students in their home departments
• To document the education and early career outcomes of past GK-12 Fellows
33
Evaluation Data Sources, Respondents, and Response Rates
• Surveys for Principal Investigators, GK-12 Fellows, K- 12 Teachers, Faculty Advisors and comparison students, all administered in 2008-09 academic year
• On-line surveys customized for each respondent group
• Response rates:– 81% PIs – 89% current Fellows – 61% current comparison students – 51% former Fellows – 53% Teachers – 50% Faculty Advisors of former Fellows
44
About the Fellowship
• The next set of slides provides data about the GK-12 Fellowship, using data primarily from the Fellows and the Principal Investigators
– Who the Fellows are – Why Fellows apply– How Fellows are prepared– The benefits of participation
5
Graduate Program Discipline, by Fellow Type
4%
4%
4%
6%
7%
8%
10%
15%
27%
6%
5%
3%
8%
7%
7%
8%
17%
28%
0% 5% 10% 15% 20% 25% 30%
Physics
Computer and Information Sciences
Agricultural Sciences/Natural Resources
Mathematics
Geological andEarth Sciences
Ocean/Marine Sciences
Chemistry
Engineering
Biological Sciences
Current FellowsFormer Fellows
Source: Fellows Survey
Back to# 35
66
Criteria Projects Use to Select Fellows
Criteria Percent
Interest in working with K-12 students 96%
Interest in GK-12 fellowship 96%
Interest in conducting outreach to the public 83%
Ability to conduct high quality research 76%
Nomination by faculty member 67%
Prior teaching or K-12 classroom experience 24%
Financial need 8%
Source: PI Survey
77
Fellows’ Reasons for Applying to GK-12
Reasons (Top 6) Former Fellows
Current Fellows
To gain financial support for my education 89% 92%
To develop my teaching skills 73% 83%
To share my knowledge of STEM 68% 80%
To work with school-age students 59% 63%
To have new experiences 55% 66%
To develop my communication skills 53% 80%
Source: Fellows Survey
8
Gender, by Fellow Type
Former Fellows Current Fellows
Gender
Female54%
Male43%
Chose Not to Report
3%
Female51%
Male46%
Chose Not to Report
3%
Source: Fellows Survey
9
Race/Ethnicity, by Fellow Type
Race/Ethnicity
White and/or Asian
80% Under- represented
Minority 14%
Chose Note to Report
14%
White and/or Asian
78%
Under- represented
Minority 15%
Chose Note to Report
7%
Former Fellows Current Fellows
Source: Fellows Survey
1010
Targeted Capacities/Skills of GK- 12 Projects
Target skills/abilities (Top 5) Percent
Ability to teach STEM research and concepts to K-12 students 97%
Ability to generate K-12 students’ interest in STEM research and activities 92%
Ability to develop instructional materials about STEM concepts and methods 83%
Ability to work on a team 78%
Ability to explain STEM research and concepts to public (non-technical) audiences 75%
Source: PI Survey
1111
Topics Covered in Training of GK-12 Fellows
Topics Percent
GK-12 project goals 100%Instructional techniques 95%How to work effectively with teachers 90%Student cognition (how students learn, students’
learning needs and abilities) 90%
How to communicate STEM content 89%How to develop age-appropriate lessons and
activities 88%
Student diversity and equity issues 78%Classroom management strategies 78%Importance of STEM professional involvement in
K-12 education 70%
Source: PI Survey
1212
Common Activities to Prepare Fellows for K-12 Assignments
Activities Former Fellows
Current Fellows
Review GK-12 materials prepared by GK-12 PI, other GK-12 staff 69% 68%
Attended meetings with GK-12 PI and/or other staff 67% 68%
Attended a 1–2 week introductory workshop 64% 74%
Attended an ongoing seminar 58% 54%
Review materials prepared by teachers in participating schools 32% 29%
Source: Fellows Survey
13
GK-12 Fellows’ School Assignments
1%
1%
28%
31%
38%
1%
1%
11%
31%
55%
0% 10% 20% 30% 40% 50% 60%
An entire school district
A science museum orother educational
organization open to thepublic
Multiple schools
Multiple teachers in asingle school
A single teacher in asingle school
Current FellowsFormer Fellows
Source: Fellows Survey
14
GK-12 Fellows’ Grade-Level Assignments
0%
21%
47%
42%
1%
31%
47%
53%
0% 10% 20% 30% 40% 50% 60%
Preschool/center-based early child
care
Elementary school
High school
Middle school
Former FellowsCurrent Fellows
Source: Fellows Survey
1515
Fellows’ Most Common Activities in K-12 Settings
Activities (Top 7) Former Fellows
Current Fellows
Presented lessons 91% 94%Designed new lessons, activities, or
modules 89% 90%
Led small group activities or discussions 82% 84%
Modified existing lessons, activities, or modules 80% 82%
Planned, coordinated, facilitated inquiry-based learning activities 76% 78%
Demonstrated scientific procedures, tools, and techniques 73% 82%
Developed educational resources 64% 59%
Helped K–12 teachers understand and use technology 50% 56%
Source: Fellows Survey
1616
Benefits of GK-12 Experience to Fellows
GK-12 has…Percent
(who agree or strongly agree)
Increased my interest in outreach to K–12 students or teachers during my career 88%
Helped me to better manage my time 67%
Motivated me to complete my graduate degree 65%
Gave me a deeper understanding of content in my graduate field of study 45%
Led me to consider a career as a K–12 teacher 31%
Source: Fellows Survey
1717
Faculty Members’ Perceptions of GK-12 Benefits
GK-12 has improved Fellows’ ability to …Percent (who
agree or strongly agree)
Teach STEM to K-12 students 83%
Work on a team 82%
Develop teaching materials for K-12 80%
Facilitate group discussions 79%
Generate K-12 enthusiasm in STEM 75%
Teach STEM to undergraduate students 75%
Source: Faculty Advisor Survey
1818
About Our K-12 Partners
• The next set of slides provides data about the experiences of the K-12 Teachers, using data primarily from the Teachers
– Who the Teachers are – Why Teachers participate – Supports provided to Teachers– The benefits of participation
1919
Teachers’ Experience
Source: Teachers Survey
2%15%
27%56%
1-2 years 3-5 years6-10 years 11-plus years
Years of K-12 Teaching Experience
55%35%
10%
1-2 years3-5 yearsMore than five years
Years Participated in GK-12
2020
Teachers’ Educational Attainment
Source: Teachers Survey
36%
61%
3%
Bachelors Masters Doctoral
Highest Degree Earned
21
Teachers’ Current Grade Level Assignments
Middle Only35%
(n=258) Elementary Only21%
(n=153)
Not Currently Teaching
3%(n=25)
Multiple Grade Levels
6%(n=41)
High Only35%
(n=254)
Source: Teachers Survey
2222
Teachers’ Reasons to Participate in GK-12
Reasons Percent
To help my own K-12 students 90%
To deepen my STEM content knowledge 75%
To learn more about mathematics/science research 49%
To help Fellows develop their communication/teaching skills 42%
Source: Teachers Survey
2323
Projects’ Targeted Topics for K-12 Teachers
Our GK-12 Project is designed to enhance … Percent
Teachers’ STEM content knowledge 97%
Teachers’ skills and techniques for STEM instruction 87%
Teachers’ interest in different instructional strategies 80%
Access to university resources for teachers 81%
Teachers’ participation in professional development about STEM education 80%
Source: PI Survey
2424
Supports GK-12 Provides to Teachers
Types of Support Percent
Opportunities to consult with Fellows for planning 95%
Stipend for participation 92%
Equipment and materials provided to schools 75%
Access to university resources (e.g., library, laboratory, equipment) 73%
Opportunities to present about GK-12 activities at professional conferences 65%
Loaned equipment and materials 64%
Continuing education credits 55%
Stipend for attending professional development workshops and trainings outside GK-12 42%
Source: PI Survey
2525
Contact between Teachers and GK-Fellows
Source: Teachers Survey
16%
65%
14%5%
Daily or almost daily Once or twice a weekOnce or twice a month A few times a year
262626
Teachers’ Content Knowledge Influenced by GK-12
Source: Teachers Survey
Topics/Content Knowledge PercentHands-on & manipulative instruction in math & science 37%
What Math/science researchers do 29%
Math/science content you teach 27%
Math/science content more advanced 24%
Current math/science research findings 19%
Math and science technology related instruction 18%
Student misconceptions in math/science 16%
2727
Comparing Current GK-12 Fellows and Their Peers
• The next set of slides provides information about the graduate school experiences of current Fellows in comparison to other current graduate students in the same departments and institutions
– Why graduate students apply for Fellowships – Types of activities in graduate school – Perceived benefits of participation– Volunteer and work activities in graduate school– Funding and other honors/awards– Current career goals
2828
Why GK-12 and Peer Students Applied for a GK-12 Fellowship
To gain financial support for education
To share my knowledge of STEM* (PhD only)
Faculty member encouraged me to apply
To develop my teaching skills
To have new experiences
To work with school-aged students* (PhD only)
To enhance my CV/resume
To develop my communication skills* (PhD only)
To develop my teamwork skills*
Another graduate student encouraged me to apply* (Master’s only)
I was interested in a teaching career*
Source: Fellows Survey* Statistically significant (p<.05)
2929
Differences between GK 12 and Peer Students in Types of Graduate School Activities
Explain STEM concepts to public*
Present work at a professional conference*
Develop instructional materials for undergraduates* (Master’s only)
Generate undergraduate interest in STEM*
Work on a team*
Lead a team*
Facilitate group discussions*
Conduct supervised research* (Master’s only)
Conduct independent research*
Develop research/technological agenda*
Source: Fellows Survey* Statistically significant (p<.05)
3030
Differences between GK-12 and Peer Students about Perceived Benefits of Their Graduate School Education
• Explain general STEM concepts to public audience*
• Teach STEM concepts and methods to K-12 students*
• Develop instructional materials about STEM concepts and methods for K-12 students*
• Generate K-12 interest in STEM activities*
• Teach STEM concepts and methods to undergraduates*
Source: Fellows Survey* Statistically significant (p<.05)
3131
Differences between GK-12 and Peer Students’ Volunteer and Work Activities
• Volunteered in an educational setting with elementary, middle or high school students*
• Volunteered to work with children or youth in a non- educational setting* (PhD students only)
• Worked in an occupation unrelated to my graduate program to earn extra money*
Source: Fellows Survey* Statistically significant (p<.05)
3232
Differences between GK-12 and Peer Students’ Receipt of Funding and Other Honors/Awards
Master’s Fellows GK-12 Non-GK-12
Research award or honor* 25% 15%
Teaching award or honor* 9% 3%
Research funding on own 23% 17%
Doctoral Fellows GK-12 Non-GK-12
Research award or honor 35% 29%
Teaching award or honor* 19% 9%
Research funding on own 34% 28%
Source: Fellows Survey* Statistically significant (p<.05)
3333
GK-Fellows’ Current Career Goals
Master’s Fellows GK-12 Non-GK-12
Primarily research-related employment 65% 66%
Primarily teaching-related employment* 28% 17%
Other employment 43% 44%
Doctoral Fellows GK-12 Non-GK-12
Primarily research-related employment 84% 89%
Primarily teaching-related employment* 51% 35%
Other employment 21% 19%
Source: Fellows Survey* Statistically significant (p<.05)
3434
Former Fellows, After GK-12
• The next set of slides provides data about the experiences of the GK-12 Fellows after they had completed their fellowship experiences
– Degree completion status of former Fellows – What former Fellows do now – Subsequent graduate study for former Master’s students– Perceived benefits of GK-12
35
Current Status of Former Fellows, by Fellowship Degree Level
91%
3%6%
58%
31%
11%
0%
20%
40%
60%
80%
100%
Graduated Still enrolled Left degree program
% Master’s% Doctoral
Source: Fellows Survey
3636
Former Fellows, After GK-12
• Median time to degree for PhD graduates is 6 years
• No differences in time to degree for female and male students
• 40% of former Master’s Fellows subsequently enrolled in other graduate programs
• 81% of former PhD Fellows and 67% of former Master’s Fellows found jobs within 3 months
• Nearly 75% of former PhD Fellows and 25% of Master’s Fellows work in higher education
37
Primary Discipline Former Fellows Use in Current Work
5%
6%
6%
6%
9%
14%
22%
0% 5% 10% 15% 20% 25%
Ocean/MarineSciences
Science Education
Mathematics
Geological andEarth Sciences
Chemistry
Engineering
Biological Sciences
Source: Fellows Survey
Go toAll Fellows
38
Former Master’s Fellows Subsequent Graduate Study, by Program Type
Ph.D. 73%
M.A./M.S. 15%
M.D. 4%
J.D. 2%
M.B.A. 2%
Ed.D. 1%
Other3%
Source: Fellows Survey
39
Former Fellows’ Current Employment Setting, by Degree Level
2%
11%
12%
72%
14%
18%
38%
24%
0% 20% 40% 60% 80%
K-12 school ordistrict
Government
Industry/business
College or university
% Master's% Doctoral
Source: Fellows Survey
40
Employment of Former Fellows in Government Positions
Federal Dept or Agency
41%
Federal Laboratory
21%
Other7%
Foreign Government
1%Federal
Legislative or Judicial Branch
1%
State or Local Government
29%
Source: Fellows Survey
4141
Former Fellows’ Perceptions of Program’s Benefits
Experience benefited their ability to… (Top 5) Former Fellows
Explain STEM research and concepts to public audiences 90%
Facilitate group discussions 89%
Work on a team 83%
Lead a team 79%Present own work/my work at a professional
conference 54%
Source: Fellows Survey
4242
Other Takeaway Findings
• GK-12 experience improved Fellows’ communication, teaching, and teamwork skills
• 90 percent of Fellows believe K-12 teacher—researcher interactions provide reciprocal benefits
• GK-12 Fellows more likely to consider teaching-related employment
• Majority of teachers reported positive effects of GK-12
• Over 80% of PIs plan to sustain GK-12 elements
• Much more detail in the forthcoming Final Report!
43
Next Steps
• Complete final comprehensive report (April)
• Prepare final dissemination report (late spring)
4444
Questions and Comments
• Questions about the evaluation from the audience– What else would be of interest to GK-12 PIs and
participants?– How consistent are national findings with local evaluation
findings?– What would you suggest to NSF about coordination
between local and national evaluation activities?
• How would you think about defining a comparison group?
• How might NSF think about conducting a longer-term follow-up study?
45
GK-12 Evaluation Staff
• Beth Gamse ([email protected]; 617-349-2808)
• Kristen Neishi ([email protected]; 301-634-1759)
• Amanda Parsad ([email protected]; 301-634-1791)
• Carter Smith ([email protected]; 617-349-2543)
• Jennifer Carney ([email protected]; 301-634-1747)