![Page 1: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/1.jpg)
LANCET Makes an IMPACT
North Carolina Department of Public Instruction NC State University/Friday Institute for Educational
Innovation SERVE Center at the University of North Carolina-
Greensboro SETDA
![Page 2: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/2.jpg)
![Page 3: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/3.jpg)
EOG scores show stronger growth than comparison schools
![Page 4: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/4.jpg)
IMPACT students caught up within one year
Effect significant at p<. 0001, controlling for grade, race, exceptionality, Free/reduced lunch, sex, absenteeism
![Page 5: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/5.jpg)
Reading Growth 2003-2005, by Grade
EO
G g
row
th fr
om b
asel
ine
to Y
ear 2
Effect significant at p<. 05, controlling for free/reduced lunch, race, exceptionality, sex, absenteeism, parent education
![Page 6: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/6.jpg)
Higher achievement in 20 technology skills measured
![Page 7: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/7.jpg)
Significantly more positive about working with computers
![Page 8: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/8.jpg)
Student Use of Computers in Grades 3-5,
2004-05
![Page 9: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/9.jpg)
Teachers also grew significantly in their use of and attitudes toward computers.
![Page 10: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/10.jpg)
Teacher Technology Skills (NETS-T)
Note: Effects significant at p < .0001. Response options were 1 (not at all), 2 (minimally), 3 (confidently) and 4 (able to teach others). Analyses controlled for sex and age.
![Page 11: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/11.jpg)
Teacher Outcomes (attitudes toward technology)
• Perceived utility of IT • Email • Internet • Multimedia • Productivity-teacher • Productivity – student
IMPACT teachers showed stronger change in attitudes or more positive attitudes overall on:
![Page 12: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/12.jpg)
Teachers’ Activities of Instruction, Year 1- Year 3
Subscale IMPACT Time #1
IMPACT Time #6
Comparison Time #6 F (1, 197) η2
Comparison Time #1
Teacher-centered activities
4.33 4.22 4.49 4.03 14.57*** 0.07 Constructivism 3.17 3.52 3.26 3.24 9.85** 0.05 Technology
utilization 2.9 4.06 2.87 2.82 83.24*** 0.3
Note: All scales measured on a scale from 1 (never) to 6 (almost every day). Higher numbers indicate higher frequency of behavior. Wilks’ Lambda reported for F.
![Page 13: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/13.jpg)
Technology Facilitator models technology use more often
![Page 14: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/14.jpg)
Media Center Usage
![Page 15: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/15.jpg)
The more transformational a leader is, the more teachers improve their skills.
![Page 16: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/16.jpg)
Leadership Practices Inventory (LPI)
• Developed by Kouzes and Posner (1995) • 30 question survey • 10 point likert-scale responses (ranging
from 1-almost never to 10- almost always) • 360 degree assessment
– Leader fills out a self-assessment – 7 to 10 faculty/staff members fill out leader
ratings
![Page 17: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/17.jpg)
“Five Qualities of Exemplary Leaders”
• Challenging the Process: willing to take risks and question the status quo
• Inspiring a Shared Vision: setting a clear and stimulating vision for others to buy into
• Enabling Others to Act: encouraging and engaging in cooperative decision making
• Modeling the Way: matching actual practice to stated values
• Encouraging the Heart: giving positive feedback and recognizing others’ accomplishments
Kouzes and Posner (1995)
![Page 18: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/18.jpg)
IMPACT Leader Ratings • IMPACT principals who
were present for all three years of the grant were rated more highly in Year 3 than in Year 1 on all 5 constructs.
• These principals grew most in “Challenging the Process” and “Inspiring a Shared Vision.”
![Page 19: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/19.jpg)
Changes in Leadership Over Time
![Page 20: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/20.jpg)
![Page 21: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/21.jpg)
Leadership Team Ratings • IMPACT Principals
scored highest on “Inspiring a Shared Vision” and lowest on “Encouraging the Heart.”
• Media Coordinators and Technology Facilitators scored highest on “Enabling Others to Act” and lowest on “Inspiring a Shared Vision.”
![Page 22: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/22.jpg)
Leadership Team Ratings • On all 5 constructs, media coordinators out-
scored principals, in absolute terms. • On 4 of 5 constructs, technology facilitators out-
scored principals, in absolute terms. • These findings indicate that IMPACT teachers
value the leadership qualities of media coordinators and technology facilitators, and that these individuals are seen as better leaders, in some respects, than school principals.
![Page 23: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/23.jpg)
![Page 24: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/24.jpg)
Capacity for Applying Project Evaluation (CAPE)
www.serve.org/evaluation/capacity/
Formative Evaluation
![Page 25: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/25.jpg)
CAPE is…
A suite of resources, tools, and professional development activities, designed to help educators collect and use data to make decisions that will help them improve the implementation and impact of their technology projects.
![Page 26: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/26.jpg)
Introduction How to Use the
Resources
Resources Index of Materials
Examples Real Evaluation
Plans and Reports
The Report Communicating
the Results
Implementation Putting the
Evaluation to Work
Outcomes Goals,
Objectives, and Strategies
Data Sources Some
Examples
Theory Explaining How
Your Project Works
The Plan Basic
Components
Getting Started Planning for Evaluation
The CAPE Evaluation Framework
![Page 27: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/27.jpg)
CAPE Instruments and Protocols
• School Technology Needs Assessment (STNA)
• Professional Development Questionnaire (PDQ)
• Looking for Technology Integration (LoFTI) drop-in protocol
• Technology and School-Family-Community Partnership survey
![Page 28: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/28.jpg)
School Technology Needs Assessment (STNA)
![Page 29: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/29.jpg)
Online STNA
• Bar graphs • Repeated use
indicates changing needs over time
• Used in about 200 schools to date, with more than 7914 respondents
• Now in Version 3.0
![Page 30: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/30.jpg)
STNA Report
![Page 31: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/31.jpg)
Implementation
Note: mF(13,425) = 47.22, p < .0001, η2=.59. Response options ranged from 1 (strongly disagree) to 5 (strongly agree).
![Page 32: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/32.jpg)
Teacher Retention
![Page 33: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/33.jpg)
Teacher Retention, by Type, Year 1-Year 3
Category Comparison IMPACT
Administrators 58.8% 76.5%
Classroom teachers 69.3% 77.0%*
Special subjects teachers 76.8% 62.5%
Note: * significant at p < .05
![Page 34: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/34.jpg)
Teacher Retention, By Years of Experience, Year 1-Year 3
Note: Years in the profession was significant (Odds Ratio = 1.18, p < .03), and IMPACT was a near-significant trend (Odds Ratio = 1.52, p < .07).
![Page 35: LANCET Makes an IMPACT - NCWiseOwlit.ncwiseowl.org/.../Servers/Server_4500932/File/IMPACT/IMPACT_N… · LANCET Makes an IMPACT North Carolina Department of Public Instruction NC](https://reader034.vdocuments.net/reader034/viewer/2022042115/5e92e46caa83cf2179203b02/html5/thumbnails/35.jpg)
For more information, contact Frances Bradburn: [email protected]
In compliance with federal laws, N C Public Schools administers all state-operated educational programs, employment activities and admissions without discrimination because of race, religion, national or ethnic origin, color, age, military service, disability, or gender, except where exemption is appropriate and allowed by law.