LANCET Makes an IMPACT

North Carolina Department of Public Instruction NC State University/Friday Institute for Educational

Innovation SERVE Center at the University of North Carolina-

Greensboro SETDA

EOG scores show stronger growth than comparison schools

IMPACT students caught up within one year

Effect significant at p<. 0001, controlling for grade, race, exceptionality, Free/reduced lunch, sex, absenteeism

Reading Growth 2003-2005, by Grade

EO

G g

row

th fr

om b

asel

ine

to Y

ear 2

Effect significant at p<. 05, controlling for free/reduced lunch, race, exceptionality, sex, absenteeism, parent education

Higher achievement in 20 technology skills measured

Significantly more positive about working with computers

Student Use of Computers in Grades 3-5,

2004-05

Teachers also grew significantly in their use of and attitudes toward computers.

Teacher Technology Skills (NETS-T)

Note: Effects significant at p < .0001. Response options were 1 (not at all), 2 (minimally), 3 (confidently) and 4 (able to teach others). Analyses controlled for sex and age.

Teacher Outcomes (attitudes toward technology)

•  Perceived utility of IT •  Email •  Internet •  Multimedia •  Productivity-teacher •  Productivity – student

IMPACT teachers showed stronger change in attitudes or more positive attitudes overall on:

Teachers’ Activities of Instruction, Year 1- Year 3

Subscale IMPACT Time #1

IMPACT Time #6

Comparison Time #6 F (1, 197) η2

Comparison Time #1

Teacher-centered activities

4.33 4.22 4.49 4.03 14.57*** 0.07 Constructivism 3.17 3.52 3.26 3.24 9.85** 0.05 Technology

utilization 2.9 4.06 2.87 2.82 83.24*** 0.3

Note: All scales measured on a scale from 1 (never) to 6 (almost every day). Higher numbers indicate higher frequency of behavior. Wilks’ Lambda reported for F.

Technology Facilitator models technology use more often

Media Center Usage

The more transformational a leader is, the more teachers improve their skills.

Leadership Practices Inventory (LPI)

•  Developed by Kouzes and Posner (1995) •  30 question survey •  10 point likert-scale responses (ranging

from 1-almost never to 10- almost always) •  360 degree assessment

– Leader fills out a self-assessment – 7 to 10 faculty/staff members fill out leader

ratings

“Five Qualities of Exemplary Leaders”

•  Challenging the Process: willing to take risks and question the status quo

•  Inspiring a Shared Vision: setting a clear and stimulating vision for others to buy into

•  Enabling Others to Act: encouraging and engaging in cooperative decision making

•  Modeling the Way: matching actual practice to stated values

•  Encouraging the Heart: giving positive feedback and recognizing others’ accomplishments

Kouzes and Posner (1995)

IMPACT Leader Ratings •  IMPACT principals who

were present for all three years of the grant were rated more highly in Year 3 than in Year 1 on all 5 constructs.

•  These principals grew most in “Challenging the Process” and “Inspiring a Shared Vision.”

Changes in Leadership Over Time

Leadership Team Ratings •  IMPACT Principals

scored highest on “Inspiring a Shared Vision” and lowest on “Encouraging the Heart.”

•  Media Coordinators and Technology Facilitators scored highest on “Enabling Others to Act” and lowest on “Inspiring a Shared Vision.”

Leadership Team Ratings •  On all 5 constructs, media coordinators out-

scored principals, in absolute terms. •  On 4 of 5 constructs, technology facilitators out-

scored principals, in absolute terms. •  These findings indicate that IMPACT teachers

value the leadership qualities of media coordinators and technology facilitators, and that these individuals are seen as better leaders, in some respects, than school principals.

Capacity for Applying Project Evaluation (CAPE)

www.serve.org/evaluation/capacity/

Formative Evaluation

CAPE is…

A suite of resources, tools, and professional development activities, designed to help educators collect and use data to make decisions that will help them improve the implementation and impact of their technology projects.

Introduction How to Use the

Resources

Resources Index of Materials

Examples Real Evaluation

Plans and Reports

The Report Communicating

the Results

Implementation Putting the

Evaluation to Work

Outcomes Goals,

Objectives, and Strategies

Data Sources Some

Examples

Theory Explaining How

Your Project Works

The Plan Basic

Components

Getting Started Planning for Evaluation

The CAPE Evaluation Framework

CAPE Instruments and Protocols

•  School Technology Needs Assessment (STNA)

•  Professional Development Questionnaire (PDQ)

•  Looking for Technology Integration (LoFTI) drop-in protocol

•  Technology and School-Family-Community Partnership survey

School Technology Needs Assessment (STNA)

Online STNA

•  Bar graphs •  Repeated use

indicates changing needs over time

•  Used in about 200 schools to date, with more than 7914 respondents

•  Now in Version 3.0

STNA Report

Implementation

Note: mF(13,425) = 47.22, p < .0001, η2=.59. Response options ranged from 1 (strongly disagree) to 5 (strongly agree).

Teacher Retention

Teacher Retention, by Type, Year 1-Year 3

Category Comparison IMPACT

Administrators 58.8% 76.5%

Classroom teachers 69.3% 77.0%*

Special subjects teachers 76.8% 62.5%

Note: * significant at p < .05

Teacher Retention, By Years of Experience, Year 1-Year 3

Note: Years in the profession was significant (Odds Ratio = 1.18, p < .03), and IMPACT was a near-significant trend (Odds Ratio = 1.52, p < .07).

For more information, contact Frances Bradburn: fbradbur@dpi.state.nc.us

In compliance with federal laws, N C Public Schools administers all state-operated educational programs, employment activities and admissions without discrimination because of race, religion, national or ethnic origin, color, age, military service, disability, or gender, except where exemption is appropriate and allowed by law.