evaluation report year five: kenan fellows …self-efficacy in teaching changed from the beginning...

EVALUATION REPORT YEAR FIVE: KENAN FELLOWS PROGRAM

JUNE 2005 - JUNE 2006

PROGRAM RATIONALE AND OVERVIEW The State of North Carolina has a two-fold problem related to the instruction of science, math, and technology in its public schools: a statewide shortage of teachers and a need for relevant, meaningful curriculum. These factors have contributed to lower student achievement in science and mathematics in North Carolina and nationally. Recent results from state-to-state comparisons of a variety of indicators related to student achievement in science point to the critical need for programs in North Carolina and elsewhere in the U. S. which focus on improving science and mathematics instruction1. For example, recent National Assessment of Educational Progress (NAEP) results from 20052 showed that one-fourth (25%) of North Carolina’s fourth grade students and just 22% of eighth graders were considered proficient or advanced in science. In addition, the science scale score of North Carolina’s eighth graders was significantly lower than the national average. Contributing to this problem is the fact that many North Carolina students do not take science coursework that may lead to science proficiency. Just one-quarter of high school students enroll in advanced science courses (e.g., chemistry or physics) compared with 31% nationally, and percentages have declined since 1996.3

Unfortunately, many teachers in North Carolina lack the qualifications necessary to teach science and mathematics, and this likely contributes to the achievement problem. The most recent state indicators show that many North Carolina science teachers are not certified to teach science, and trends suggest a downward decline. Only approximately one-half (56%) of middle grades science teachers in North Carolina were certified in science in 2004, down from 60% in 1996; nationally 63% were certified to teach science in 2004. At the high school level, more than 80% of North Carolina science teachers were certified to teach science in all areas except earth science; however percentages still lag behind those seen nationally in all areas. The lack of qualified science teachers contributes to the problem of student achievement in science, as research has shown that teacher training in science education is of critical importance to student achievement. 1Blank, R.K. & Langesen, D. (1999). State indicators of science and mathematics education: State-by-state trends and new indicators from the 1997-98 school year. Council of Chief State School Officers, Washington, D.C.

2 National Assessment of Educational Progress (2005). Retrieved from: http://nationsreportcard.gov/science_2005/.

3 State Indicators of Science and Mathematics Education: 2006. Retrieved from: http://www.ccsso.org/projects/Science_and_Mathematics_Education_Indicators/.

Prepared by Donley Educational Evaluation Consulting, Inc. July 2006 1

For example, recent NAEP reports suggest that student achievement in eighth grade science is directly related to whether the student’s teacher majored in science education. One study also found that teacher expertise (defined as teacher education, licensing examination scores, and years of experience) explains a significant portion of the difference in student achievement in reading and mathematics4. In addition, the study argues that money spent on improving teacher qualifications produced greater increases in student achievement than money spent on any other single program.

Although a variety of professional development reform efforts have been made since the publication of A Nation at Risk in 1983, teachers mostly continue to teach as they have in the past. Often this is because both the quantity and quality of professional development provided have been inadequate5. School districts typically allocate only approximately one percent of their budgets to professional development, and less than half of teachers surveyed report that they received release time to attend training sessions. Also, few teachers believe that staff development improves their ability to implement new teaching methods and that they feel well prepared to implement higher standards. Many professional development efforts may primarily involve one-shot workshops that lack a connection to the real-world challenges teachers face in the classroom. In addition, the system of rewarding teachers with higher pay simply because they take graduate courses which may or may not be linked to school or student needs may contribute to the ineffectiveness of many staff development efforts.

In order to address the problem of quality in professional development of teachers, the National Staff Development Council recently provided revised standards for staff development that improves the learning of all students.6 These standards encourage staff development efforts that:

• are organized based on learning communities; • provide appropriate leadership and resources for continuous instructional

improvement; • allow teachers to apply research to classroom decision making and collaborate

with other teachers in their knowledge development; and, • encourage equity and family involvement in their improvement efforts.

The process of identifying and retaining qualified teachers, fostering their teaching and leadership skills through professional development based on national standards, and helping them develop meaningful and relevant curriculum is a challenge of great importance. In order to address this challenge, the Kenan Fellows Program for Curriculum and Leadership Development seeks to enhance teacher professionalism and leadership by encouraging teachers to develop novel curricular resources in collaboration

4 Ferguson, R. (1991). Paying for public education: New evidence on how and why money matters. Harvard Journal of Legislation, Volume 28.

5 Sparks, D. & Hirsh, S. (1999). A national plan for improving staff development. Retrieved from http://www.nsdc.org/library/NSDCPlan.html.

6 National Staff Development Council. (2001). NSDC Standards for Staff Development. Retrieved from http://www.nsdc.org/educatorindex.htm.


http://www.nsdc.org/educatorindex.htm

with the private sector, public schools, and institutions of higher learning. Fellows are encouraged to develop curricular resources that emphasize inquiry-based learning and helping students apply academic knowledge to authentic, real-world problems. The program was established in 2000 as a result of a community effort to address teacher retention and recruitment in the area of science, mathematics and technology instruction, as well as the need for relevant and meaningful curricula in these areas.

The Kenan Fellows Program is administered by the Kenan Institute for Engineering, Technology and Science at NC State University, whose goal is to seek out partners who share its vision and who have the capacity to help transform the results of university scholarship into an improved quality of life. The program is supported by grants from foundations, government organizations, corporations and individual partners. The North Carolina Department of Public Instruction (NCDPI) provides direction for curriculum development based on current classroom needs.

While a major program goal is to help teachers develop curriculum tools and resources, the Kenan Fellows program also focuses on process. The program places great emphasis on leadership development through ongoing professional growth based on national standards for staff development, and nurtures the art of teaching by providing opportunities for teachers to pursue creative and innovative ideas. The program allows teachers to experience learning in stimulating ways, leading to new discoveries and approaches to the art of teaching – all while teachers remain fully active in their classrooms. In addition, the program promotes National Board Certification by linking teacher-Fellows with local universities, thus satisfying a key requirement for certification.

Kenan Fellows program goals include:

• Developing teacher leaders in the classrooms across North Carolina;

• Advancing the art of teaching by fostering a focus on teaching based in inquiry, problem solving and real life application;

• Enriching teachers’ content knowledge in mathematics, science and technology through strategic partnerships with businesses and higher education; and,

• Developing innovative and relevant curricular tools and resources in partnership with the NCDPI to be disseminated to teachers and students across North Carolina.

Outstanding classroom teachers selected as Kenan Fellows engage in a two-year fellowship in partnership with scientists and university faculty. Fellows participate in two summer internships as well as special seminars and events throughout the school year to foster their professional growth. They develop a curriculum project in partnership with their University Mentor in an area that is deemed important by NCDPI; this project is then disseminated via a web site developed by the Fellow, and through presentations at state and national conferences. They also earn six graduate credits from NC State University through their participation as a Fellow, receive support to pursue National Board Certification, and participate in national and state conferences. Currently six


classes of Fellows have been selected (2001-2006), and two classes are actively participating (2005 and 2006).

EVALUATION METHODS

According to the Evaluation Plan7, the evaluator relied upon data supplied by the program in the form of surveys, program statistics and qualitative data. Data for this report are provided for the 2004 (who have recently completed their final year of the Fellowship) and 2005 (who have recently completed their first year of the Fellowship) classes of Fellows8. The Evaluation Plan provides a description of how data sources relate to program goals. The following instruments and procedures were used to assess the program for purposes of this report:

• Workshop Evaluations from the 2005 summer internship sessions that provided data on whether Fellows find sessions challenging and useful for application to the classroom;

• Impact Surveys completed by both classes of Fellows in fall 2005 that gauge Fellows’ perceptions of program impact related to key goals, including teaching/ leadership ability, comfort levels in developing and maintaining partnerships with the community, and relationships with University Mentors;

• Kenan Fellows responses to reflective prompts, were used to address various questions of interest to the program;

• Data on National Board Certification rates, attendance/presentations at conferences, and grants obtained by Fellows provided information on how the program has enhanced Fellows' leadership skills and how curricula have been disseminated;

• Teacher retention data provide an indicator of how successful the program has been at encouraging participants to remain in teaching;

• Evaluations from Fellows’ presentations at conferences provided data both on the quality of the presentations being made and the extent to which curricula are being disseminated to other teachers.

• Tracking of Fellows’ web site hits provided data on dissemination of curricula via the World Wide Web;

7 Donley, J. (2005). Evaluation Plan for the Kenan Fellows Program for Curriculum and Leadership Development.

8 Results for previous cohorts of Kenan Fellows are also included where appropriate.


• Teacher Leadership Survey9 data were used to detect changes to Fellows’ perceived teacher leadership skills from the beginning to the end of the fellowship; and,

• Professional Efficacy Survey10 data were used to determine whether Fellows’ self-efficacy in teaching changed from the beginning to the end of the fellowship.

The remainder of this report describes the 2004 and 2005 classes of Kenan Fellows, the summer internships, school-year training and other programming they have received, and how the program has impacted them in areas that pertain to key program goals.

PROFILE OF 2004 AND 2005 KENAN FELLOWS Selection Process Kenan Fellows are selected through a competitive application process that targets outstanding K-12 science and mathematics teachers in the Triangle area within North Carolina. Letters are distributed to principals, superintendents, and teachers in the public schools in November announcing an application deadline in January. Potential applicants propose innovative curriculum projects in the areas of science and mathematics, and proposals must demonstrate the project’s vision and impact throughout the state. The Kenan Fellows Board and NCDPI typically also encourage projects in more specific areas based on the state’s curriculum needs. For example, in 2005 a Fellowship was awarded to a high school teacher to conduct research that includes measurement of river discharge and groundwater levels, collection of water samples and analysis of their nitrogen concentrations (dissolved inorganic and organic species), interpretation of data, and preparation of results for presentation and publication. This research will lead to the development of web-based curricular materials that will serve as a resource for teaching and learning some of the newly required competencies in Earth science in North Carolina public high schools.

From the applications received, a Kenan Fellows selection team composed of university faculty, NCDPI representatives, and industry partners selects candidates for personal interviews. The rubric used for the Kenan Fellows selection process ranks the teachers on their leadership potential, content knowledge, initiatives taken to grow professionally, and recommendations from principals and colleagues.

Characteristics of Program Participants

Kenan Fellows constitute a group of teachers with excellent teaching and leadership skills; the Kenan Fellows program seeks to further enhance these teachers’ skills and

9 Survey adapted from a Teacher Leader Survey developed by Barnes, N. & Dozier, T., Center for Teacher Leadership, Virginia Commonwealth University, 2003.

10 Survey adapted from the Teachers’ Sense of Efficacy Scale (Tschannen-Moran & Hoy, 2001) and Teacher Self-Efficacy Scale (Bandura, undated).


disseminate the products of their work to other teachers in North Carolina. Kenan Fellows are required to make a two-year commitment to the fellowship program. Fifteen Kenan Fellows were selected in 2004 and all but two11 have recently finished the program, while the program included eleven Fellows for the class of 2005 who are currently in their second year of the program.

While the Kenan Fellows program has primarily attracted secondary teachers, elementary teachers have also been participating over the last two years. For the 2004 and 2005 classes, half taught at the high school level, while roughly one-quarter taught at either the elementary or middle school levels.

Elementary23%

Middle23%

High54%

Figure 1. Grade Levels Taught by 2004 and 2005 Kenan Fellows

Two of the six elementary Kenan Fellows were specialists with academically gifted populations, one was an elementary science specialist, and one was an instructional technology specialist. Five of the six middle school Fellows taught science, while one taught Language Arts. High school Fellows’ teaching disciplines are represented in the figure below. Nearly three-quarters (72%) of high school Fellows taught within the sciences, with biology being the most frequently represented discipline.

11 Two Fellows withdrew from the program temporarily but have since returned to finish their Fellowship during 2006-07.


Biology37%

Chemistry14%

Earth Science14%

English7%

Mathematics14%

Physics7%

Technology7%

Figure 2. Teaching Areas of High School 2004 and 2005 Kenan Fellows

The Kenan Fellows program has expanded its targeted geographic area and now includes more counties within North Carolina than in previous years. While at least three-quarters of Fellows taught in Wake County in the first three years of the program, the program now attracts teachers from a broader geographic region, and includes teachers from Guilford and Lee counties.

Figure 3. County of Origin, Kenan Fellows 2001-2005

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

67%

13%

13%

7%

92%

8%

27%

18%

9%

9%

27%

9%

71%

14%

14%

80%

20%


2001 2002 2003 2004 2005

Wake Durham Chapel Hill/Carrboro Orange Guilford Lee

PROFESSIONAL DEVELOPMENT PROGRAM

ACTIVITIES FOR 2005-06 The description of programming which follows contains information covering the summer 2005 internship through the end of the academic year 2005-06. The summer internship professional development experiences were designed to foster teacher development in each of the program's key goal areas: 1) developing teacher leaders in North Carolina; 2) advancing the art of teaching and curriculum development in the areas of science, mathematics and technology; and, 3) encouraging partnership-building between Fellows and university faculty and others in the community. A discussion of the professional development and participants’ reactions to the sessions is provided below.

Professional Development: Summer Internship Experience

As part of their program participation, each Kenan Fellow engages in a two-year fellowship that includes two summer internships focusing on leadership and curriculum development, goals that are central to program success. In 2005, these five-week summer internships included the following professional development sessions:

• Current State of Legislative Affairs Education in NC

• Students who Struggle with Learning (All Kinds of Minds seminar)

• Myers Briggs Type Inventory (MBTI)

• Science Leadership Institute

• Inquiry-Guided Instruction

• Paedeia Diagrams

• AuthorGen Software

• Service Learning

• Grant Writing

• Targeting Hard-to-Reach Students Through After School Programs

• Standards Based Curriculum Development

• National Board Certification process

Within the summer internship Fellows were also provided with ample opportunity to work with their University Mentor, consult with the NCDPI, and conduct library or web research for their curriculum project. Many Fellows also had opportunities to travel to engage in scholarly activities related to their project. For example, one Fellow traveled to Costa Rican rainforests as part of a Duke University trip focusing on environmental education. At the close of the summer session, Fellows (along with their University Mentors) presented their curriculum project in a poster session as part of the Kenan


Fellows Showcase. This event is typically attended by university faculty, NCDPI personnel and other interested members of the community.

Fellows completed evaluation forms for each professional development session provided. These surveys asked for Fellows’ perceptions of session quality as well as relevancy to their teaching. Fellows rated both the facilitator and professional development sessions favorably, as shown in the figures below, although results were not as favorable as in past years. At least 60% of Fellows rated these sessions as at least somewhat high in terms of usefulness of content, importance of training, applicability to their job, and overall session quality.

Figure 4. Professional Development Session Ratings, Summer 2005

5%8%

25%

35%

27%

6%

10%

25%

38%

22%

6%9%

24%

32%

29%

3%5%

26%

39%

27%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Useful content Necessity of training Applicability to your job Overall quality of session

Low Somewhat Low Average Somewhat High High

The facilitators who provided the sessions were rated positively, with at least 60% of Fellows assigning a rating of at least somewhat high in the areas of presentation skills, responsiveness to questions, providing balance between training and application, knowledge of subject and providing supplemental materials.


Figure 5. Facilitator Ratings for Professional Development Sessions, Summer 2005

4%2%

15%

46%

33%

2%

10%

19%

27%

42%

4%

10%

21%

42%

23%

0%

8%

13%

27%

52%

6% 6%

28%

33%

28%

0%

10%

20%

30%

40%

50%

60%

Presentation style Response to questions Balance between training andapplication

Knowledge of subject Supplemental materials

Low Somewhat Low Average Somewhat High High

In addition, when asked in fall 2005 about the impact of the summer internship experiences on their teaching, a full 94% agreed that it had positively impacted their teaching. Survey comments illuminate how these experiences have benefited Fellows' teaching, leadership and networking with others:

“I believe I now have the skills to more effectively reach across academic disciplines to integrate my instruction and content with other areas. I also have a much better understanding of my students' academic background and experiences. This greater understanding helps me to do a better job of preparing their instruction.”

“My fellowship has increased my awareness of educational issues and ways to promote my opinions as a teacher to affect the political process. I have reexamined the way I teach. I have dramatically increased hands on activities and made strong connections between my lessons and relevance to the real world. The information I have learned from the All Kinds of Minds Institute has helped me become more aware of student learning styles and enabled me to provide support to a number of students who are struggling with school success.”

“I am more willing to try new things. I am also more focused in my critique and subsequent rewriting of activities/lessons.”

“Kenan Fellows provided me with the incentive to build a strong network of people at Universities, museums, environmental organizations, and local businesses. These people helped me with funding, workshops, support


information, encouragement, and practical experiences. My network consists of over 35 individuals or organizations that help me with this project.”

“I am including greater opportunities for inquiry based learning. I have also used many of the skills acquired through professional development to "attune" students, and incorporate more "brain-based" strategies in my lessons. I am definitely more enthusiastic about teaching science. My Fellowship has reignited the spark!”

These comments suggest that the professional development provided through the Kenan Fellows program is consistent with national standards for professional development that encourage efforts which provide resources for leadership-building and continuous instructional improvement, and collaboration with other teachers in knowledge development.

Although Fellows participate in extensive professional development during their summer institute, additional program activities continued to take place during the school year. For example, the “Fireside Chats” provided an arena for Fellows to dialogue and network with state leaders in education. At one Fireside Chat Fellows met North Carolina Superintendent of Public Instruction June Atkinson to engage in dialogue regarding key issues germane to public education in North Carolina. Program staff also met regularly with Fellows and communicated information regarding grant opportunities, meetings and additional professional development they may want to attend. Fellows remained engaged in their work with their University Mentor through strategic planning, content area enrichment and joint presentations at conferences.

ATTAINMENT OF KEY PROGRAM GOALS

Development of Teacher Leaders in North Carolina

Leadership Behaviors

A key goal of the Kenan Fellows program is to develop a cadre of highly motivated teacher-leaders in classrooms in North Carolina. As discussed previously, professional development sessions were provided as part of the summer internship that focused on areas such as Educator Advocacy, NCDPI’s Science Leadership Institute, Grant Writing and Presentation Skills.

Fellows provide leadership data to the secure Kenan Fellows web site regularly in the areas of National Board Certification status, awards received, grant writing, professional presentations given and attended, and other leadership activities (e.g., curriculum writing). Leadership data include only activities undertaken by Fellows since becoming a Kenan Fellow. Data included in this report are the most current as of June 2006.


National Board Certification and Advanced Degrees

One way teacher leadership can be exemplified is by Fellows modeling the importance of acquiring the highest credentials possible in their teaching area to other teachers in their schools. Statewide, only approximately 11% of teachers hold National Board Certification in their teaching area.12 The Kenan Fellows program promotes National Board Certification by linking teacher-Fellows with local universities, thus satisfying a key requirement of certification. Fellows are encouraged but not required to pursue this certification upon entry to the program; many of the program components are aligned with expectations and requirements of the certification process. Fellows who have already achieved certification present information and encourage others to begin the process.

Leadership profile data show that approximately one-quarter (26%) of Kenan Fellows were Nationally Board Certified prior to their program participation. Since becoming Kenan Fellows, an additional 17% have either received this certification or begun pursuing it. Just over half of the 2004 and 2005 Kenan Fellows are not actively pursuing certification.

Figure 6. National Board Certification Status of Kenan Fellows as of June 2006

Before fellowship26%

Since beginning fellowship4%

Actively pursuing13%

Not currently pursuing57%

While some Kenan Fellows are not choosing to pursue national board certification, data from the leadership profile showed that nearly one-quarter (24%) have either begun pursuing, or achieved an advanced degree (e.g., Masters) since becoming a

12Results from the NC Teacher Working Condition Survey reported in: Report and Recommendations from the State Board of Education Teacher Retention Task Force, February,2005.


Fellow. This finding is impressive given that these teachers must combine these degree efforts with their already busy teaching careers and roles as Kenan Fellows.

Awards and Grants Received

Many Kenan Fellows received awards since beginning their Fellowship; approximately half (48%) of 2004/2005 Fellows have received an award such as "Teacher of the Year" or community service awards. Sessions on grant writing were also featured within the summer internship experience for Fellows. Program data collected show that approximately half (48%) of Fellows received grant awards since becoming a Kenan Fellow. Examples of grants received by Kenan Fellows include:

• NC Science Teacher Association Innovative Teaching Grants;

• Bright Ideas Grants;

• Durham Public Education Network Grant;

• Wake Education Partnership Food for Thought Grants;

• NC Beautiful Grants; and,

• NC Office of Environmental Science Project Tomorrow Grants.

Impact Survey results suggest that 2004 Fellows attribute increases in grant-writing capability to the Kenan Fellows program, with nearly three-quarters 73% reporting that the program has helped or will help them write grants to secure funding for their initiatives.

Presentations to Colleagues

Other ways that leadership is encouraged through the Kenan Fellows program is by promoting Fellows’ attendance and presentation at state and national conferences, and encouraging community and school presentations. These leadership experiences also provide opportunity for the dissemination of innovative curriculum resources for teachers in the surrounding community statewide, and nationally.

Almost three-quarters (72%) of Kenan Fellows have made professional presentations to colleagues since becoming a Fellow; in fact on average Fellows have made three presentations since joining the program. Many Fellows gave presentations at state and national conferences, including the North Carolina Science Teachers Association, the National Science Teachers Association, and the North Carolina Council for Teachers of Mathematics. Participant reaction to these conference presentations is provided later in this report.

Participation in Other Leadership Activities

Fellows documented via the secure program web site additional leadership behaviors (e.g., committee work, teacher-mentoring and curriculum writing) they had engaged in since becoming a Fellow. These data showed almost all (85%) had engaged in at least


one additional leadership activity, with an average of five activities per Fellow. Examples of other leadership activities include:

• Technology Planning Committee;

• Appointment to the North Carolina Environmental Education Advisory Board;

• Wake County Advanced Placement Professional Institute Task Force;

• Updated curriculum for Wake County’s Introduction to College Math course;

• Science Department Chairperson;

• Durham K-8 Science Leadership Team; and,

• End-of-Grade Science Test Item Writing.

Clearly, in addition to writing grants and giving professional presentations, Kenan Fellows are involved in many facets of education and serve as role models to their colleagues in the area of teacher leadership.

Changes to Fellows’ Leadership Skills

Kenan Fellows who began in 2004 completed a Teacher Leadership Survey just prior to, and at the end of, their Kenan Fellowship. The survey assessed Fellows’ perceptions of their skills and knowledge in areas crucial to teacher leadership roles within a school setting, and asked whether they had participated in a variety of teacher leader roles. Results from the survey showed significant changes in Kenan Fellows’ perceptions of themselves as teacher leaders, and in their perceived ability to engage in tasks that research has shown are central to being an effective teacher leader. Statistically significant13 gains (p < .05) were seen both in Fellows’ beliefs that others saw them as a teacher leader, and that they considered themselves a teacher leader. They also were clearer on what exactly is meant by the concept of teacher leadership. Clearly Kenan Fellows became more confident about themselves as teacher leaders, and believe that their colleagues also perceive these positive changes.

13 All statistical analyses were t-tests conducted on pre and post data.


Figure 7. Kenan Fellows’ Teacher Leadership Survey Results

3.83

4.67

4.17

5.00

4.00

4.83

1

2

3

4

5Strongly Agree

Strongly Disagree

I have a clear definition of a TeacherLeader.

I consider myself a Teacher Leader. Others consider me a Teacher Leader.

Pre Post

2004 Kenan Fellows also showed increases in their perceived ability to write curriculum within their content area (although the difference was not statistically significant) and to work collaboratively with educational stakeholders and understand the needs of and work with policymakers (p < .05), which are central goals of Kenan Fellows programming.


Figure 8. Kenan Fellows’ Teacher Leadership Survey Results: I Have the Knowledge and Skills to…

4.17

4.67

3.50

4.50

2.50

3.83

1

2

3

4

5

Strongly Agree

Strongly Disagree

Write curriculum for my contentarea

Work collaboratively with parents,business leaders, and othereducational stakeholders

Understand needs of policymakersand discuss policy with them

Pre Post

Another important facet of Kenan Fellows programming is to foster teachers’ ability to take what they learn from the professional development and curriculum development process and train other teachers to be more effective. Kenan Fellows showed significant gains in their perceived ability to provide professional development successfully and work with adult learners (p < .05). Modest gains were observed in their perceived ability to coach experienced and mentor new teachers, although it should be noted that most Kenan Fellows began the program already strong in these areas.


Figure 9. Kenan Fellows’ Teacher Leadership Survey Results: I Have the Knowledge and Skills to…

4.00

5.00

4.33

4.674.50

4.83

3.83

4.83

1

2

3

4

5Strongly Agree

Strongly Disagree

Provide effective professionaldevelopment for teachers

Coach experienced teachers Mentor new teachers Work with adult learners

Pre Post

Kenan Fellows also reported an increase in teacher leadership behaviors, although the difference for the leadership behaviors addressed on the survey was not statistically significant. The types of teacher leadership behaviors that evidenced the largest changes were use of data driven instruction, outreach to parents and communities and serving in an advisory capacity to a policy-making group. Again it is important to note that the typical Kenan Fellow begins the program already having served in a variety of leadership capacities within their school (e.g., coaching and mentoring of teachers and staff development); the Kenan Fellows program may actually serve to refine teachers’ skills within these areas as well as encourage them to expand their roles as teacher leaders.


Figure 10. Kenan Fellows Teacher Leadership Survey Results: Mean Number of Leadership Behaviors Reported at Beginning and End of Program14

5.33

7.33

0

1

2

3

4

5

6

7

8

Pre Post

Fellows’ Perceptions of Program Impact on Leadership

To further assess program impact in the area of teacher leadership, Fellows were asked several questions as part of an Impact Survey administered in fall of 2005. All or almost all Fellows agreed that the program helped them become an instructional leader in their school and demonstrate leadership by interacting with educational leaders and policy-makers, and most believed that the program had helped or would help them with grant-writing. While a majority of 2004 Fellows reported that the program had fostered the National Board Certification process, just slightly more than one-third of 2005 Fellows reported impact in this area.

14 The total possible number of leadership behaviors was 12.


Figure 11. The Kenan Fellows Program Has Helped Me To…

100% 100%

73%

63%

91%

25%

57%

38%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Perc

ent A

gree

/Str

ongl

y A

gree

Become an instructional leader inscience/math/technology in my school

Write grants to secure funding forfuture projects

Demonstrate leadership by interactingwith and influencing state and

national leaders and policy-makers.

Pursue or achieve National BoardCertification

2004 Fellows 2005 Fellows

Fellows also were likely to report that their colleagues perceived them as leaders in inquiry-guided instruction and use of classroom technology, two areas which were emphasized in the program’s professional development.

Figure 12. Other Teachers Consult Kenan Fellow for Advice or Assistance With…

80%

63%

73% 75%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Perc

ent A

gree

/Str

ongl

y A

gree

Promoting inquiry-guided instructional techniques in theirclassroom.

Integrating technology into their classroom instruction.



Enhanced leadership skills were also reflected in Fellows’ responses to open-ended items addressed on Teacher Leadership and Impact Surveys. Apparently although Fellows may have served in leadership roles before program participation, they now experienced increased opportunities and a sense of confidence in this area:

“The Kenan Fellows program has given me the confidence to voice my opinion and concerns regarding educational policy in the community and to better help shape policy at my own school.”

“Being a Kenan Fellow has opened up so many opportunities. I have become a teacher leader within the state and have assisted with writing curriculum for science content. Additionally, I had the opportunity to present science content at local, state, and national sessions. It was an honor to be a Kenan Fellow.”

Several also described how the program promoted teacher leadership by providing a supportive network of colleagues and encouraging contacts with other professionals:

“ [The Kenan Fellows program fosters teacher leadership by…] Being able to interact with strong teachers that share similar views and interests. The encouragement factor adds a huge boost to the summer.”

“Collaborating with other Fellows gives me ideas and support! The opportunity to present at conferences has really boosted my comfort level when speaking with other professionals. Most of all, the doors the Fellowship has opened as far as contacts is incredible! My mentor has been fantastic, and I am branching out to other professionals for ideas and opportunities. Things I would never have done or known about prior to my Fellowship!’

In fact, all 2005 and 90% of 2004 Fellows agreed on the Impact Survey that the opportunity to network with other Kenan Fellows has promoted their teaching and leadership skills. The evidence from teachers’ self-perceptions, presentations at conferences, grant writing and other leadership activities clearly show that the Kenan Fellows program has begun to achieve its goal of developing teacher-leaders who can positively impact education in North Carolina.

Program Statistics: Teacher Retention

Teacher retention in North Carolina has become a pressing concern in recent years. Recent statistics for the cohort of teachers who began teaching in North Carolina during 2000-01 show that only approximately half of teachers with previous teaching experience, one-third of new teachers with no experience and fewer than half of lateral entry teachers were retained beyond their third year15. In addition, the N.C. Center for Public Policy Research reports that teacher retention is probably most crucial in subject

15 Report and Recommendations from the NC State Board of Education Teacher Retention Task Force, February, 2005.


areas with the most acute shortages of qualified teachers, namely math, science, special education and foreign languages16

It is expected that through its emphasis on leadership development, partnership-building with university faculty and business leaders, and collegiality with other teaching professionals, the Kenan Fellows program will enhance the likelihood that participating teachers remain in the teaching profession. It is also hoped that Fellows will at least indirectly promote the retention of other teachers, as they promote professionalism and leadership within their own schools.

When examining Kenan Fellows’ retention in the teaching profession to date, 93% overall remain as active teachers in the classroom. Almost all Fellows who have left the classroom remain in the education field in some capacity: several have left the profession full-time but continue to work part-time with NCDPI or NC State on curriculum support documents or projects such as the Summer Leadership Institute. One 2004 Fellow left the profession temporarily but has since returned to teaching and his Kenan Fellowship. Therefore, while no longer in the classroom, most Fellows continue to serve in a leadership capacity within the field of science education.

Figure 13. Percentage of Kenan Fellows Still Teaching in the Classroom as of July 2006

88%

75%

100% 100% 100%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2001 Fellows 2002 Fellows 2003 Fellows 2004 Fellows 2005 Fellows

As part of the Impact Survey, 2004 Fellows were asked whether they had considered leaving the profession prior to becoming a Kenan Fellow, and if so, what impact their

16 N.C. Center for Public Policy Research (2004, August). Press Release: Center Says Shortage of Teachers and Impending Crisis – State Action Needed. Retrieved July 24, 2005.


participation had on their career decision-making. A number of these Fellows reported that they had considered leaving the profession, and that the Kenan Fellowship had influenced their desire to remain in the classroom:

“I had reached the point of being burned out and recognized that I could make more for the same amount or less work. The Kenan Fellowship has recharged me and allowed me to see the leadership qualities I have. It has also rekindled the teaching flame. I still don't know if I will retire from the profession, but I will definitely teach for a few more years. I had the opportunity to leave the classroom this summer. Even though it was a great opportunity, I didn't consider it for long because I have unfinished business in the classroom. I have started various things in the past year and a half that I would like to see through to the end.”

“Yes, the Kenan Fellows program has shown me that there is a way to be a leader in the classroom, school, and community without going into administration. I am not sure whether I will continue to remain in the classroom, but I have better learned how to tap the resources of teacher leaders through this program.”

“I have become re-energized. At 52, that is pretty impressive.”

Comments such as these illustrate the vital role that the Kenan Fellows program can play in building the leadership capacity of North Carolina’s teachers, and increasing the likelihood they will remain committed to the teaching profession.

Advancing the Art of Teaching

The Kenan Fellows program seeks to advance the art of teaching by enhancing Fellows’ use of inquiry-guided instructional techniques and technology, enriching their content knowledge, and fostering the development of innovative curricula for dissemination across North Carolina. Many of the professional development sessions offered through the summer internship focused on enhancing teaching skills and content knowledge. Some of the topics included Students who Struggle with Learning, Inquiry-Guided Instruction, Web-Based Lesson Plans, and Standards-Based Curriculum Development.

It is expected that through these programming efforts and the others previously described, Kenan Fellows might enhance their sense of professional efficacy. Therefore, a Professional Efficacy instrument was administered at the beginning and end of the fellowship to detect changes to Fellows’ efficacy with teaching. In addition, questions from the most recent Impact Survey yielded data on Fellows’ perceptions regarding program impact on their teaching. Upcoming evaluation data collection efforts will be expanded to include an analysis of program impact on student achievement using value-added measurement procedures17.

Impact Survey Results

17 These procedures will allow an examination of how much academic growth (relative to expected growth) a Fellow is able to make with students prior to, and at the end of, their Fellowship on state-mandated tests


Results from the survey in the areas of impacting teaching, curriculum development, and content knowledge show that Fellows perceive positive changes in all areas. All or almost all 2004 and 2005 Fellows believed that the program has helped them become a better teacher and enhanced their science, mathematics and technology knowledge. A majority of Fellows who began in 2004 also reported that they had significantly changed their instructional approach since beginning the program, while just less than two-thirds had detected significant changes; this is not surprising given that 2005 Fellows had less of an opportunity than 2004 Fellows to make instructional changes.

Figure 14. "The Kenan Fellows Program has…" 100% 100%

82%

63%

90%86%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Perc

ent A

gree

/Str

ongl

y A

gree

Helped me become a better teacher Significantly changed myinstructional approach

Enhanced my knowledge in the areasof science/mathematics/technology


A majority of 2004 Kenan Fellows reported noticeable improvement in their use of inquiry-guided instruction and classroom technology, while half of the 2005 Fellows had begun to notice significant improvement. Most Fellows also reported improved content knowledge and presentation skills; however, they were less likely to notice improvement in their knowledge of National Science Standards or Service Learning.18

18 National Science Education Standards, National Committee on Science Education Standards and Assessment, National Research Council, 1995. Full text available for searching at http://books.nap.edu/catalog/4962.html)


Figure 15. Percentage Reporting a Fair Amount or Great Deal of Improvement In…

73%

50%

70%

50%

40%

13%

40%

50%

60%

50%

64% 63%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Inquiry-guidedinstruction

Use of classroomtechnology

Service learning Knowledge ofNational Science

Standards

Presentation skills Knowledge withinmy content area


Kenan Fellows who began in 2004 reported significant improvement in their students’ achievement in, and appreciation for, science mathematics and technology; again 2005 Fellows were less likely to detect these changes. Most also noticed significant improvement in their students’ ability to use sophisticated technology and inquiry-guided learning techniques.

Figure 16. Since Becoming a Kenan Fellow, I Can See Real Improvement In…

91%

63%

91%

50%

73%

50%

60%

50%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Perc

ent A

gree

/Str

ongl

y A

gree

My students' appreciation forscience/math/technology

My students' achievement inscience/mathematics/technology

My students' inquiry-guidedscience/mathematics/technology

learning

My students' use of sophisticatedtechnology



Professional Efficacy Results

The Professional Efficacy survey results showed significant increases for Kenan Fellows in certain aspects of teaching efficacy. Kenan Fellows exhibited significant increases (p < .05) in both their efficacy with regard to instructional strategies and classroom management, and to teacher leadership; no difference was observed for student engagement.

Figure 17. Professional Effiacy Survey Results*, 2004 Kenan Fellows

55.0052.00

61.00

70.71

46.86

58.29

0

10

20

30

40

50

60

70

80

Efficacy in Student Engagement Efficacy in Instructional Strategiesand Classroom Management

Leadership Efficacy

Pre Post

*Note: The maximum number of points on all subscales was 80.

An inspection of item means within each subscale revealed that the most prevalent improvements within the instructional strategies and classroom management subscale stemmed from items dealing with gauging student comprehension and enhancing their memory for material, getting students to work together, providing appropriate challenges for very capable students, and controlling disruptive classroom behavior. The most significant increases within leadership efficacy occurred for items involving helping fellow teachers enhance their skills and influencing them to advocate for the teaching profession, enhancing collaboration between teachers and the administration and influencing policymakers about educational issues. All of these issues except classroom management are key focuses of Kenan Fellows programming; it may be that improvements in efficacy in other areas may indirectly also enhance Fellows’ confidence when dealing with classroom management issues.

Several comments from Fellows illustrate changes to their professional efficacy:


“I have stepped “outside the box” and become much more creative with my teaching. I am better able to help students with learning problems. I have a better relationship with my students because science class is fun and interesting.”

“This program has given me the confidence to strike out and share my knowledge and skills. I have sought opportunities that I would not have normally tried.”

State and National Impact: Curriculum Dissemination

While a major goal of the Kenan Fellows Program is to foster teaching and curriculum development, an equally important goal is to disseminate the innovative curricula developed by Fellows to other teachers. Each Fellow is expected to create a web site, including key components of their curriculum project and how the project is aligned with national and state standards.19 NCDPI provides guidance on web site development and on other statewide dissemination strategies as well. In addition, each Fellow presents at local, state, national and international conferences such as the NC Science Teachers Conference and National Science Teachers Association conference (see previous discussion of conference presentations).

One way that dissemination data are collected is by tracking “hits” to Fellows web sites. Curriculum projects developed by Kenan Fellows have drawn more than 120,000 “hits” and have been viewed by more than 50,000 visitors thus far.20 An inspection of the source of recent website hits21 reveals that the curriculum project sites are attracting viewers from all across the country as well as internationally. Slightly less than one-third were from within NC, while slightly more than one-half originated from other US states.

19 See http://www.ncsu.edu/kenan/fellows/curriculum.html for links to Fellows' web sites.

20 Includes all classes of Kenan Fellows; the number of hits is actually higher as data tracking did not begin until September of 2005.

21 This involves analysis of the most recent 150 page loads from Fellows’ websites and may not be representative of the entire history of the project web site.


Figure 18. Origin of Recent Kenan Fellow Project Web Site Hits, July 2006

Within North Carolina31%

Other States within US52%

International17%

Dissemination of Fellows’ curricula is also assessed by presentation evaluation surveys that are given at conference presentations. In addition to asking questions about the quality of the session, the evaluation also asks for the attendee’s home county (if within North Carolina) or state of origin. The Evaluator compiles these data to provide a picture of how the curricula developed are reaching other teachers statewide and nationally.

Data from conference presentations given by Fellows in 2005-06 show that Kenan Fellows presented at state conferences attended by teachers from 50 counties in North Carolina, as well as to many teachers nationally. Many of these counties are considered “high need” with regard to educational resources for science education.

Participants, as shown by the following session evaluation data, received these presentations very favorably in terms of the overall quality and the likelihood that the content would both used in their own teaching and shared with other teachers.


Figure 19. Kenan Fellows’ Conference Evaluation Data22

4.674.54 4.40

1

2

3

4

5

Session Quality Will share information withcolleagues

Will implement ideas in my teaching

As part of their presentation, Fellows often describe their participation in the Kenan Fellows program. Comments from participants at Fellows’ presentations illustrate the usefulness of the curriculum resources and how teachers often express a desire to become involved with the Kenan Fellows Program:

“This is great! I have been struggling with body system resources. I am really excited to start my unit now! Thanks!’

(teacher from Madison County, NC)

“I’m saved! Now I know how I want to teach the human body after Christmas. Thanks!”

(teacher from Macon County, NC)

“I plan on using these ideas on teaching biology with proteins for instruction next week!”

(teacher from Cabarrus County, NC)

“I’m a 3rd year teacher and had no idea that the Kenan Fellows program existed. This session introduced me to it and I am excited!”

(teacher from Nash County, NC)

22 Sessions were rated on a scale from 1 (Low) to 5 (High); data are as of June 2006.


Establishing Strategic Partnerships

The Kenan Fellows program attempts to help Fellows build and maintain relationships and partnerships with others in the community, including university faculty, community agencies and businesses, and the NC Department of Public Instruction. These partnerships are essential to the success of the Kenan Fellows program and are designed to fulfill the mandate put forth by the State Board of Education which prioritizes community and business support for education. In order to encourage this partnership building, Fellows participated in a summer programming on forming and sustaining school/business partnerships, and participate in “Fireside Chats” throughout the year with local business and community leaders.

More than 90% of Fellows from both classes reported on the Impact Survey that the program has enabled them to build relationships/partnerships with the broader community. Comments from the Impact Survey illustrate how Fellows believe these partnerships have broadened their interests and concerns and enhanced their collegiality with other teaching professionals:

“Through the program, I have met teachers from around the Triangle area and learned about what they do in their classrooms. I have been able to discuss how I teach with them and bounce ideas off of these fellow teachers as well. This has helped me as I continue teaching.”

“The KF program has opened doors to me that I would not have looked for on my own. I have been exposed to relationships that I would not have known about otherwise.”

“Getting a chance to meet other people that share a vision and the future of education is a huge benefit. I love meeting and dealing with other professionals that may provide a resource or an encouraging word that could not be found inside the walls of the school. It is great to have guest speakers and assistance in the classroom.”

Relationship with University Mentor

A key partner to each Kenan Fellow is the University Mentor, who provides guidance as they develop their curriculum project and often helps enrich their content knowledge in the process. During the summer Fellows work extensively with their mentor on their curriculum project, and mentors frequently attend parts of the summer internship experience with their Fellow. Fellows continue to communicate with their mentors throughout the school year. Each Fellow/Mentor team develops a strategic plan that details their methods for continuing their work on the curriculum project. Some Fellows work with their mentor on Saturdays, attend university classes taught by Mentors to enrich their content knowledge, and make joint presentations at conferences.

Results from the Impact Survey show that Fellows were very positive regarding the impact the relationship with their University Mentor had on their content knowledge.


However, only approximately one-third of the 2004 Fellows believed that they had developed a close relationship with their mentor and that he/she had helped them enhance their leadership skills. This finding was surprising and contradictory to previous findings from exiting classes of Fellows.

Figure 20. Fellows’ Perceptions of Relationship with University Mentor

80%86%

38% 36%

88%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Perc

ent A

gree

/Str

ongl

y A

gree

The relationship I have developed withmy University Mentor has enhanced my

knowledge within my content area.

My University Mentor has helped meenhance my leadership skills through

co-presentations, publications, etc.

I have developed a close and lastingrelationship with my University Mentor.


While 2004 Fellows were somewhat less positive about the quality of their relationship with their Mentor, 2005 Fellows believed the relationship was off to a good start and that they already were reaping the benefits, but some also mentioned challenges of maintaining the relationship:

“My mentor and I communicate almost on a daily basis. We have presented at several conferences, and are pursuing some grant ideas together. My project is still in the building stages, but I am encouraged constantly by the cutting edge changes provided by my mentor. We are looking forward to taking the project further in the state and to the national level.”

“I have a strong relationship with my mentor and spend a great deal of time working with her to prepare leadership institutes, conference presentations, teacher workshops, and curriculum materials.”

We have had some contact since this summer, but I think both of us are busy with school and family. He has two small children and is out of the country trying to get our research project going. I have had… lots of new committees and paperwork with our new administration. Our project probably won't really start gathering steam until after Christmas.”


CONCLUSIONS Results from this evaluation of the Kenan Fellows Program suggest that it clearly continues to fulfill its goals and is laying a foundation to provide much-needed benefit to the state of North Carolina in the area of science and mathematics education. This is especially true, given that the program has continued to expand outside the immediate Triangle area, and has plans for further expansion to underserved areas within the state. Outstanding educators are given an opportunity to further hone their teaching and leadership skills, and through the curriculum dissemination process will ultimately help to improve science and mathematics instruction statewide. The types of professional development provided were consistent with National Staff Development Council standards and were perceived to be of high quality and applicable to the professional lives of Fellows. Many Fellows also commented that a major strength of the program is that it provides an arena for them to form and maintain ongoing supportive relationships with other colleagues who have similar professional development interests.

The Kenan Fellows program appears to foster teacher leadership, as evidenced by reports of actual leadership behaviors, Teacher Leadership Survey data, and Fellows' perceptions of program impact. While relatively fewer Fellows were pursuing National Board Certification compared with previous findings, nearly one-quarter had begun pursuing or achieved a graduate degree. Half of the Fellows received grant awards, and nearly three-quarters made at least one professional presentation. A substantial majority also engaged in a variety of other leadership activities, such as curriculum writing and committee work. Kenan Fellows additionally showed significant gains in their self-perceptions of themselves as teacher leaders, as well as in their ability to work with educational stakeholders and policymakers. They also believed themselves to be much more proficient at providing professional development to other teachers, a critical component of the Kenan Fellows model.

Fellows clearly recognize the contribution of the Kenan Fellows program to their enhanced leadership, as evidenced by high percentages reporting that the program allowed them both to be leaders within their school as well as demonstrate leadership through interaction with educational policymakers. The enhanced teacher leadership Fellows experience may also contribute to their likelihood of remaining in the teaching profession. Although it is difficult to link teacher retention directly to the Kenan Fellows program without a controlled research design, 93% of all Kenan Fellows to date have remained as classroom teachers, and many attribute their decision to remain to their participation in the program.

The Kenan Fellows program also was successful in advancing innovative teaching strategies and may have resulted in enhanced teaching efficacy. Professional Efficacy Survey results suggested that Fellows made significant gains in terms of both their efficacy regarding instructional strategies and classroom management, and their leadership roles. Fellows believed themselves more capable with issues such as student comprehension/memory for material, providing challenges for very capable students and controlling disruptive behavior. They also felt more confident regarding helping other


teachers enhance their skills, fostering collaboration with administration, and influencing policymakers. Fellows reported that the program has made them a better teacher and caused them to significantly change their instructional approach to more effectively incorporate inquiry-guided techniques and classroom technology. Most Fellows from the 2004 class believe that their experience has resulted in significant gains for their students in terms of achievement and appreciation for science, math and technology. Future data collection efforts in this area will include student achievement data within a “value-added” assessment model to address the relative contribution of Fellows to their students’ learning as they progress through the program.

Web site hits and conference presentation data suggest that curricula developed by Kenan Fellows are being disseminated widely to many of North Carolina’s counties, nationally and internationally, thus fulfilling a key program goal. Additionally, the Fellow-University Mentor relationship appears to be fulfilling its intended purpose of enhancing teachers’ content knowledge. However, there was less consensus on the part of 2004 Fellows that they had developed a close relationship with their mentor and that this relationship had contributed to enhanced leadership skills. This finding is in contrast to previous years where Fellows have consistently reported strong and positive relationships with their Mentor, and preliminary evidence from 2005 Fellows suggests that this may not be a continuing trend. However, future data will be tracked closely and assessment procedures will be expanded to more frequently monitor the status of the Fellow-Mentor relationship so that any potential problems can be addressed in a timely manner.


evaluation report year five: kenan fellows …self-efficacy in teaching changed from the beginning...

Documents