running head: bcps s.t.a.t. year one evaluation …...nov 17, 2015 · s.t.a.t. year one summative...

Running head: BCPS S.T.A.T. YEAR ONE EVALUATION REPORT 1

Report for Baltimore County Public Schools:

Students and Teachers Accessing Tomorrow – Year One Summative Evaluation Report

Jennifer R. Morrison, Ph.D. Steven M. Ross, Ph.D.

Gary R. Morrison, Ph.D. Alan C. K. Cheung, Ph.D.

With Rachel M. Arthur, Ph.D.

Center for Research and Reform in Education (CRRE) Johns Hopkins University

2015

BCPS S.T.A.T. YEAR ONE EVALUATION REPORT 2

CRRE – JHU November 17, 2015

Contents Executive Summary ........................................................................................................................ 3

S.T.A.T. Year One Summative Evaluation Report ......................................................................... 6

Method ........................................................................................................................................ 7

Results ......................................................................................................................................... 9

Professional Development .................................................................................................... 10

Measurable Outcomes ........................................................................................................... 21

Perceptions of the S.T.A.T. Initiative ................................................................................... 43

Conclusion ................................................................................................................................ 45

Professional Development .................................................................................................... 45

Measurable Outcomes ........................................................................................................... 46

S.T.A.T. Initiative ................................................................................................................. 48

Appendix A: S.T.A.T. Teacher Program Survey .......................................................................... 50

Appendix B: Lighthouse Classroom Teacher Survey ................................................................... 52

Appendix C: Principal Interview Protocol .................................................................................... 54

Appendix D: STAT Teacher Interview Protocol .......................................................................... 56

Appendix E: Teacher Focus Group Protocol ................................................................................ 58

Appendix F: OASIS-21 Instrument .............................................................................................. 60

Appendix G: OASIS-21 Reference Guide .................................................................................... 62

Appendix H: OASIS-21 Results ................................................................................................... 64

Appendix I: Lighthouse School Behavioral Data ......................................................................... 67

Appendix J: Descriptive Statistics and Frequencies of Responses to S.T.A.T. Teacher Program Survey (April 2015) ...................................................................................................................... 68

Appendix K: Descriptive Statistics and Frequencies of Responses to Lighthouse Teacher Survey....................................................................................................................................................... 72



Executive Summary: S.T.A.T. Year One Summative Evaluation Report

The purpose of the present study was to conduct an evaluation of the Students and

Teachers Accessing Tomorrow (S.T.A.T.) initiative in Baltimore County Public Schools during the 2014-15 school year. S.T.A.T. is a multi-year transformation of all district schools in order to ensure each school has an equitable, effective digital learning environment. S.T.A.T. places an emphasis on the transformation of teaching and learning including access to personalized, interactive digital curriculum and individual student and teacher devices. Key components of S.T.A.T., as reflected in the evaluation model (see Figure 1 presented in the main report), include professional development and the resulting impact on measurable outcomes that will then affect the goals of improving student achievement and preparing globally competitive students.

This summative report examined aspects of professional development and collected data

from Lighthouse schools pertaining to measurable outcomes. The research questions focused on the professional development offered by the S.T.A.T. teacher and the early impact of professional development on measurable outcomes (e.g., classroom environment, teacher practice, digital content, student engagement, and P21 skills). Professional Development

Lighthouse classroom teachers were less positive towards the district summer professional development as compared with that offered by their S.T.A.T. teacher. The interviews and focus groups suggest the teachers were not ready for the district summer professional development and some of the presenters lacked the knowledge or skill needed. In contrast, the teachers were supportive of the professional development provided by the S.T.A.T. teachers in their schools. This training was adapted to meet the specific needs of grades and disciplines as well as individual needs. Overall, classroom teachers within Lighthouse schools viewed the S.T.A.T. teacher as instrumental in transforming their classrooms in accordance with S.T.A.T. goals. However, classroom teachers within non-Lighthouse schools were mixed on their views of their S.T.A.T. teacher. Measurable Outcomes

• Classroom environment: Over 90% of the classrooms were arranged to support collaborative learning and a majority of the classrooms displayed materials to support independent thinking. The observations, however, suggest that students seldom move about the room to acquire different resources and less than a third of the students were observed moving to a different area of the classroom for collaboration, small-group work, or for direct instruction. The S.T.A.T. teacher interviews suggested that they perceived students as moving around the room and using different areas. There appears to be an impression that students have the freedom to move around the room and work where they are most comfortable.

• Teacher practice: The impact of S.T.A.T. on teacher practice was evident throughout data sources. Teachers were observed more frequently acting as a coach or facilitator and less presenting instruction during the spring observations as compared with the baseline



observations. There was a reduction, however, in the frequency that flexible groups of students were observed and also in the frequency that students were asked higher-level questions and provided with higher-order instructional feedback by the teacher. Interviews indicated that teachers were emphasizing a more student-centered learning environment, differentiating instruction to students, and relinquishing control to the students for the learning process.

• Digital content: Lighthouse classroom teachers created more file and repository content tiles during the second semester than the first and student tile views also increased within these two categories. This finding is consistent with the indication by classroom teachers within Lighthouse schools that they used BCPS One to differentiate instruction for students. Concerns were expressed regarding the challenges of adjusting the curriculum for consistency with student-centered instruction.

• Student engagement: While the use of digital tools for learning remained consistent between observation time points, there was a noted decline in the frequency of independent work and an increase in the frequency of collaborative learning. There was also an increase in the frequency by which student discussion occurred across these time points. Participants conveyed that students were much more interested in learning and expanding their knowledge to a much deeper level due to the technological affordances available within Lighthouse schools. A significant decrease in the percentage of Lighthouse school students meeting the 94% attendance cut off during semester one of 2013-2014 compared with 2014-2015 was observed; the differences in semester two were not significant. The occurrence of student suspensions and office referrals decreased between these two years.

• P21 skills: Observations indicated noticeable changes in P21 skills, such an increase in the frequency that students were observed engaging in problem solving as well as learning that incorporated authentic or real world contexts. S.T.A.T. teachers conveyed that students were much more willing to work with their peers and think critically when posed with a problem.

Conclusion

Overall, Lighthouse schools have demonstrated substantial growth since the midyear report in transitioning to technology enhanced, learner-centered environments. Findings of the present study indicate evidence of the effects of professional development on measurable outcomes (e.g., classroom environment, teacher practice, digital content, student engagement, and P21 skills). Based on evaluation findings, recommendations are as follows:

• Professional development. Offer more differentiation in professional development as it is rolled out to more grade levels, and tailor support and training to the groups who have or have not had certain training, meeting teachers at their existing levels of technology knowledge and acquisition. Create videotaped modeled teacher lessons so teachers can see what strategies and approaches should look like.

• S.T.A.T. teacher role. Clarify the roles and responsibilities of the S.T.A.T. teacher throughout the district.

• Curriculum. Update the curriculum to fit the requirements of the S.T.A.T. initiative including thoughtful and planned integration of technology into the lessons.



• Technology integration. Increase the focus on making sure technology tools are not just being used without clear curriculum and instructional purposes.

• S.T.A.T. message. Leadership must understand and communicate that S.T.A.T. is not just technology; teaching and learning has to be in place before devices are provided.



S.T.A.T. Year One Summative Evaluation Report The purpose of the present study was to conduct an evaluation of the Students and

Teachers Accessing Tomorrow (S.T.A.T.) initiative during its first year of implementation within Baltimore County Public Schools for the 2014-15 school year. The purpose of S.T.A.T. is to provide personalized learning for every student through a redesign of curriculum in the core content areas, key pedagogical shifts to both a blended-learning and a learner-centered environment, the use of BCPS One, individual student devices, wireless and broadband infrastructure, and ten Lighthouse schools to serve as models for later S.T.A.T. implementation.

The longitudinal evaluation of S.T.A.T. will focus on the aforementioned key S.T.A.T. components and will examine aspects of the S.T.A.T. evaluation model (see Figure 1). Figure 1. S.T.A.T. evaluation model.

Key inputs to the initiative are represented at the left by the professional development

provided to administrators, S.T.A.T. teachers, and classroom teachers. Key evaluation questions related to professional development include:

1. What are the impacts and best practices in site-based and district-wide professional development?

2. What are the roles, perception, and best practices of S.T.A.T. teachers? 3. What are the roles and best practices of other key participants (principals, curriculum

specialists, technology directors)?

Measureable intermediary outcomes, represented in the middle of the figure, are hypothesized to be reflected in positive changes in classroom environment, teacher practice, access and use of digital content, student engagement, and P21 skills. Evaluation questions addressing the intermediary outcomes include:



1. What is the impact of S.T.A.T. on the classroom environment? 2. To what degree and how do teacher practices change over time? 3. What is the level of access of digital content by principals, teachers, students, and

parents? 4. To what degree does student engagement in learning increase over time?

A final evaluation question explored the experiences and perceptions of key stakeholders

and participants in S.T.A.T.

Method Participants and Design

Participants included the ten Lighthouse schools (see Table 1). Three of the Lighthouse schools received Maryland innovation grants and were selected to participate. Six additional Lighthouse schools applied to participate. One school, Mays Chapel, was a new school that opened in fall 2014 as a Lighthouse school. Table 1. Characteristics of Lighthouse school enrollment for the 2014-2015 school year.

School name S.T.A.T. Grades

Total Enrollment

Race/Ethnicity Free and

Reduced Price Meals

(FARMS)

Limited English

Proficiency (LEP)

White %

Black %

Other1 %

Chase Elementary 1-3 399 56% 30% 15% 62% 0% Church Lane Elementary 1-3 499 1% 88% 11% 63% 3% Edmondson Heights Elementary 1-3 566 6% 79% 15% 68% 5% Fort Garrison Elementary 1-3 369 82% 9% 8% 8% 2% Halstead Academy 1-3 526 4% 88% 8% 76% 2% Hawthorne Elementary 1-3 605 31% 46% 23% 74% 2% Joppa View Elementary 1-3 731 55% 19% 26% 30% 5% Lansdowne Elementary 1-3 491 51% 23% 26% 74% 5% Mays Chapel Elementary2 K-5 590 52% 16% 33% 28% 7% Rodgers Forge Elementary 1-3 436 78% 3% 19% 5% 3% 1 “Other” includes the following race/ethnicity categories: American Indiana/Alaska Native, Asian, Hispanic/Latino, Native Hawaiian or Other Pacific Islander, and Two or More Races. 2School opened in fall 2014 It is important to note that overall, the Lighthouse schools have a greater percentage (50%) of FARMS eligible students as compared with the rest of the schools in the Baltimore



County Public Schools (BCPS) system (44%). Specifically, six of the ten Lighthouse schools have a greater percentage (>60.0%) of FARMS eligible students than the average of the remainder of the schools within BCPS. Although the race and ethnicity within Lighthouse schools overall (40% White, 41% Black) are comparable to BCPS as a whole (42% White, 39% Black), four of the ten Lighthouse schools have a greater percentage of Black students than the remaining schools in the district. Data Sources and Instruments

S.T.A.T. Teacher Program Survey. The S.T.A.T. Teacher Program Survey (see Appendix A), developed by Baltimore County Public Schools, consisted of ten closed-ended items focusing on the accessibility, support, and professional development opportunities provided by the S.T.A.T. teacher. In addition, two open-ended items solicited feedback on the perceived successes and opportunities of the S.T.A.T. Teacher Program. The survey was accessed by a total of 3,478 teachers in both Lighthouse and non-Lighthouse schools. The completion rate of the survey was 90.5% by Lighthouse teachers (n = 174) and non-Lighthouse teachers (n = 2,974) combined.

Lighthouse Classroom Teacher Survey. The Lighthouse Classroom Teacher Survey (see Appendix B) was co-developed by the Center for Research and Reform in Education (CRRE) and BCPS. The survey consisted of 21 closed-ended items focusing on professional development, teacher practices, perceived student impact, technology integration, and Lighthouse school preparation. In addition, two open-ended items asked participants’ successes and challenges with technology integration. The survey was administered to 116 classroom teachers in Lighthouse schools with a completion rate of 77.6%.

Interviews and Focus Groups. Phone interviews were conducted with all Lighthouse

school principals (n = 10) and S.T.A.T. teachers (n = 11) during spring 2015. In addition, between three and six classroom teachers from each Lighthouse school were randomly selected to participate in in-person individual teacher focus groups (n = 10). The protocols for the principal and S.T.A.T. teacher interviews and the classroom teacher focus groups (see Appendix C, Appendix D, and Appendix E, respectively) solicited perspectives on professional development, the perceived impact of S.T.A.T. on measurable outcomes and educational goals, and experiences and perceptions of the S.T.A.T. initiative.

Observation of Active Student Instruction in Schools of the 21st Century (OASIS-

21). The classroom observation instrument (see Appendix F) was co-developed by CRRE and BCPS. The instrument integrated district-wide professional development goals for classroom instruction with S.T.A.T.-specific interests and goals regarding technology applications of teaching and learning. The observations focused on (a) student engagement, (b) the type of instructional strategies employed, and (c) how and to what degree technology devices are employed. A reference guide for the OASIS-21 Instrument items is presented in Appendix G.

The procedure employed involved trained observers visiting four randomly selected

Lighthouse school classrooms for 20 minutes each. The observers completed individual ratings of the frequency/pervasiveness of particular practices, as well as classroom environment



indicators (e.g., room arrangement, information and resources available, etc.). With the exception of two classroom environment items, observation items were recorded via a five-point scale that ranged from (1) Not Observed to (5) Extensively Observed.

A total of 40 classrooms were observed in December 2014 and the same in April 2015, resulting in 1,600 minutes of direct classroom observations conducted in the ten Lighthouse schools. First grade (n = 14), second grade (n = 13), and third grade (n = 13) classrooms were observed in the spring, and the distribution of content areas observed are presented in Figure 2. The frequency of the extent each OASIS-21 item was observed is presented in Appendix H. Figure 2. Distribution of content areas observed during the 2014-2015 classroom observations.

Digital Content Usage. Content usage data were measured in several ways. Units defined as ‘tiles’ measured overall engagement. A tile delivers digital content to students. Tiles can contain both a) teacher created or identified electronic files, wikis, urls, repository content, assignments, and tests and quizzes, and b) district provided electronic curriculum materials, including files, URLs, and repository content. Teachers were able to create tiles for the whole class, small groups of students, or individuals. This approach allowed teachers to personalize instruction by creating different tiles with different content for different learner needs. The number of logins by students, teachers, and parents also was tracked. Taken together, these data describe the digital content usage. Lighthouse School Behavioral Data. Data consisting of attendance, office referrals, and suspensions form 2013-2014 and 2014-2015 were collected for Lighthouse Schools (see Appendix I).

Results



The following sections present results of surveys, interviews and focus groups, classroom

observations, and digital content usage as related to the evaluation model components. The results begin with perceptions and activities related to professional development. This section is followed by results of data collected for measurable outcomes. Professional Development

Lighthouse classroom teachers participated in professional development offered by the

district during the summer and by their school’s S.T.A.T. teacher during the school year. Classroom teachers within non-Lighthouse schools received professional development throughout the school year as provided by their school’s S.T.A.T. teacher.

District professional development. Focus group interviews portrayed different opinions

of the district’s summer professional development activities. For example, one Lighthouse teacher stated, “The county offered a ton of training over the summer. I went to almost all of them. Each summer they offer them. There were sessions on kid blog, Edmodo and lots of tools and device training.” Another teacher noted one specific session that was very helpful, “The most helpful was when the Intel people basically walked us through using the device, I think I went to that one twice!”

Other teachers described the training as too fast, too soon. For example, one teacher said,

“Because we did not have the devices with our kids yet, it was almost too early. Some of the stuff they were talking about didn’t make sense yet because we hadn’t attempted to apply it yet.” One complaint with the previous summer training was the feeling of being overwhelmed:

I do think there was a mistake at the very beginning. When we went to the summer sessions and we had to go to session upon session upon session for two days in a row it was like being bombarded. By the middle of the second day I thought my head was going to blow up. It was too much and too overwhelming and not the way to get introduced to it all.

Another teacher described the training like a job fair:

Last summer the county had two or three days for all of the Lighthouse schools. It sort of felt like a job fair where you walked around and choose sessions that you thought would interest you. Some things were helpful, others not so much.

Another teacher also expressed concern about being ready for the training:

We went to OneNote, Google Docs, and Wixie – there was some good information but we were the guinea pigs. They (the presenters) could not share what might go wrong. Most of what they were teaching us about was not live at that time. There were lots of glitches with their demonstrations if there were any. We couldn’t see much – our systems were not working. Some teachers who were not designated as classroom teachers could not even get into the sessions even though they needed the training.



Another concern was training for implementation, as stated by a Lighthouse classroom

teacher:

The summer sessions were built around telling us about all the cool tools that we could use, but did not teach us how to implement them in a classroom. We had to learn how to use everything and integrate it on our own.

Another Lighthouse teacher offered:

There was never a day where we also saw it implemented. We learned features (during the PD) but it was sometimes hard to see how that was beneficial. So it was kind of like guess and check. We just went into the classroom and (used it)…then we collaborated with each other. Because we are the first ones, nobody has ever used this [BCPS One] before.

The participant was speaking of the “tile” feature in BCPS One here, but seemed to suggest this was representative of the overall STAT training. Another teacher suggested:

Identify needs – talk to teachers and see what they need. Teachers are very open when we feel we need something. …. Do learning walks – visiting classrooms and get a sense of where we need to go. For next year, they should consider sending out a survey to teachers to get their input on what they would like or need for PD. All three participant groups within Lighthouse schools offered recommendations for

future professional development. Both S.T.A.T teachers and focus group teachers indicated a need for differentiation of teachers during professional development. One classroom teacher offered a suggestion:

If they could make the systems small, like small groups, small cohorts in order to form the bonds that have helped us so much as the group of ten. Maybe grouped by staff or student demographics. Those bonds really help us focus on our areas of need and potential.

Similarly, a S.T.A.T teacher stated, “A little more differentiation as they roll out to more grade levels and tailor support and training to the groups who have or have not had certain training, meeting teachers where they’re at with tech knowledge and acquisition.” There were also suggestions to differentiate professional development by teacher comfort level. Teachers requested that they be better prepared in advance for the training, that is, having basic prior knowledge of the technology so that they can take advantage of the training. One frequent suggestion from S.T.A.T teachers was to include either more visits to in-session classrooms and videos of teachers implementing the strategies. Teachers expressed a desire for a model showing proper implementation. Last, there were several suggestions that future professional development focus on the needs of the teachers implying that a needs assessment or evaluation be conducted prior to the professional development so that it is specifically focused on teachers’ needs.



S.T.A.T. teacher professional development. The following section includes results from the S.T.A.T. Teacher Program Survey administered to classroom teachers in both Lighthouse and non-Lighthouse schools. Statistical analyses of survey results differed due to the type of survey item administered. The two survey items consisting of multiple responses related to professional development participation were analyzed using a pseudo Chi-square test with first-order Rao-Scott correction. In order to assess differences between Lighthouse and non-Lighthouse classroom teacher responses on the remaining Likert-type items, the non-parametric Kruskal-Wallis test was used. The Wilcoxon Signed-Rank test was adopted to compare medians between the October and April survey administrations. Mean scores were aggregated to the school level in order to perform paired comparisons. The descriptive statistics and frequency of responses for the close-ended survey items are presented in Appendix J. In addition, principals and classroom teachers in Lighthouse schools provided further information on the S.T.A.T. teacher role through interviews and focus groups. Professional development mode. Classroom teachers indicated which, if any, modes of professional development they participated in as facilitated by their school’s S.T.A.T. teacher (see Figure 3). At both times of survey administration (October and April), all classroom teachers reported the greatest frequency of participation in large group professional development, such as faculty meetings (Lighthouse teachers: 92.9% (October), 94.8% (April) and non-Lighthouse teachers: 91.8% (October), 93.3% (April)). The second most frequent form of professional development was small group instruction, including grade level, team, or content area meetings (Lighthouse teachers: 84.6% (October), 91.4% (April) and non-Lighthouse teachers: 72.4% (October), 77.1% (April)). Significant differences were observed in participation in professional development modes between classroom teachers within Lighthouse schools as compared to those in non-Lighthouse schools, χ2

R-S (N = 8885) = 119.970, p < .001 during the April survey administration:

• Lighthouse school classroom teachers participated to a greater extent in small group professional development opportunities as compared with non-Lighthouse teachers in the October survey (Lighthouse teachers: 84.6%, non-Lighthouse teachers: 72.4%) and the April survey (Lighthouse teachers: 91.4%, non-Lighthouse teachers: 77.1%).

• Lighthouse school classroom teachers reported participating in more individual/one-on-one support opportunities than did their counterparts in non-Lighthouse schools in October (Lighthouse teachers: 78.0%, non-Lighthouse teachers: 60.4%) and April (Lighthouse teachers: 82.8%, non-Lighthouse teachers: 60.1%).

• Lighthouse school classroom teachers participated to a greater extent in independent learning opportunities than non-Lighthouse school teachers in October (Lighthouse teachers: 60.4%, non-Lighthouse teachers: 38.7%) and April (Lighthouse teachers: 75.9%, non-Lighthouse teachers: 46.1%).

The modes of professional development completed were significantly different between

the October and April survey administrations for Lighthouse classroom teachers, χ2R-S (N = 1179)

= 22.760, p < .001). A greater proportion of teachers participated in independent learning opportunities in the April survey (75.9%) as compared with the October survey (60.4%). There was also a significant difference for the non-Lighthouse classroom teachers χ2

R-S (N = 16039) =



52.184, p < .001, with a greater proportion of teachers reporting participating in independent learning opportunities in April (46.1%) than what was reported in October (38.7%).

In addition, a small proportion of non-Lighthouse classroom teachers indicated they had not participated in any professional development modes facilitated by their S.T.A.T. teacher (October: 2.1%, April: 2.2%). For Lighthouse classroom teachers, 2.2% indicated they had not participated in any professional development modes facilitated by their S.T.A.T. teacher in October, but all Lighthouse teachers indicated participating in at least one professional development mode in April.

Figure 3. Frequency of classroom teacher participation in professional development modes as reported in the April survey administration.

Learning opportunities. Classroom teachers also reported their participation in various learning opportunities supported by the S.T.A.T. teacher (see Figure 4). Overall, the most commonly reported learning opportunities included training workshops facilitated by the S.T.A.T. teacher, planning sessions with the S.T.A.T. teacher, and one-on-one professional discussions or consultations with their S.T.A.T. teacher. Significant differences were observed in teacher participation in learning opportunities between those in Lighthouse schools as compared to those in non-Lighthouse schools in October (χ2

R-S (N = 10,889) = 250.940, p < .001) and April (χ2

R-S (N = 13,657) = 245.795, p < .001):

• One-on-one professional discussions or consultations occurred more frequently by Lighthouse classroom teachers as compared with those in non-Lighthouse schools in April (Lighthouse teachers: 84.5%; non-Lighthouse teachers: 64.2%). The difference in participation in this learning opportunity was not as substantial in October (Lighthouse teachers: 73.1%; non-Lighthouse teachers: 62.6%).



• Participation in learning walk or instructional walk-throughs was greater by Lighthouse classroom teachers than non-Lighthouse classroom teachers in October (Lighthouse teachers: 42.3%; non-Lighthouse teachers: 12.9%) and April (Lighthouse teachers: 75.9%; non-Lighthouse teachers: 43.3%).

• During the April survey administration, analysis of data occurred more frequently between S.T.A.T. teachers and classroom teachers in Lighthouse schools (58.6%) as compared with those in non-Lighthouse schools (38.4%). The difference between groups was less substantial in the October survey administration (Lighthouse teachers: 32.4%; non-Lighthouse teachers: 28.4%).

• Classroom teachers indicated an increase in the frequency that they participated in observations of another teacher’s classroom from the October survey to the April survey. This learning opportunity occurred to a greater extent by Lighthouse teachers as compared with non-Lighthouse teachers in both October (Lighthouse teachers: 21.4%; non-Lighthouse teachers: 6.8%) and April (Lighthouse teachers: 50.6%; non-Lighthouse teachers: 22.3%).

Significant differences were observed in teacher participation in learning opportunities

over time for both Lighthouse classroom teachers, χ2R-S (N = 1789) = 121.28, p < .001 and non-

Lighthouse classroom teachers, χ2R-S (N = 22,757) = 1355.84, p < .001. For Lighthouse

classroom teachers, learning opportunities increased between the October and April survey administration for participation in learning walk or instructional walk-throughs (October: 42.3%; April: 75.9%), analysis of data (October: 32.4%; April: 58.6%) and observations of another teacher’s classroom (October: 21.4%; April: 50.6%). The same trends occurred for non-Lighthouse classroom teachers with learning opportunities increasing between October and April for participation in learning walks or instructional walk-throughs (October: 12.8%; April: 42.3%), analysis of data (October: 32.4%; April: 58.6%) and observations of another teacher’s classroom (October: 21.4%; April: 50.6%).

The professional development offered by one S.T.A.T. teacher was described as:

She meets with teachers as individuals or at grade level groups, depending on what teachers’ prefer. She does a lot of teacher mentoring on the topic of learner-centered environments. Her teachers did instructional walk-throughs that were very helpful, allowing teachers to see what other teachers were doing in their classrooms.

Another teacher stated, “The PD never stops!”

Interviews at Lighthouse schools suggested that professional development takes many different forms including informal meetings to demonstrate and share ideas. For example, one S.T.A.T teacher offers a Tech-Tip Tuesday every Tuesday morning and afternoon that is attended on a voluntary basis. During this informal meeting, the S.T.A.T teacher provides professional development and BCPS One information. Initially, the S.T.A.T teacher ran the sessions, but now teachers are presenting and students are sharing their tech tips. Similarly, a S.T.A.T teacher in another school offers a “Digital Diner” at lunch for teachers who want to participate. Not all professional development, however, focuses on technology. Some meetings are devoted to discussing pedagogy and what is working and not working in the classroom.



S.T.A.T teachers described a variety of professional development offerings in their

schools. In addition to the summer professional development workshops, there are more formal professional development sessions attended by a majority of the teachers in the school. Several S.T.A.T. teachers reported offering grade-specific training on a regular basis as well as regular professional development sessions for the school. These meetings ranged from once each month to once every six weeks. S.T.A.T teachers also scheduled walk-throughs at other schools. In addition to technology, S.T.A.T teachers also addressed the blended learning environment, the student-centered environment, and facilitation skills in these workshops. Figure 4. Frequency of participation by classroom teachers in various professional learning opportunities as facilitated by the S.T.A.T. teacher (April survey administration).

Instructional support. Classroom teachers responded to survey items assessing instructional support provided by S.T.A.T. teachers. A range of differences regarding perceived instructional support was observed for Lighthouse classroom teachers and non-Lighthouse classroom teachers during both the October and April survey administrations:



• Lighthouse classroom teachers were more likely to report that their S.T.A.T. teacher models effective instructional strategies than were non-Lighthouse classroom teachers in both October (H = 5.887, p < .05) and April (H = 18.849, p < .001).

• Lighthouse classroom teachers were more likely to report that their S.T.A.T. teacher provides or directs them to useful resources than were non-Lighthouse classroom teachers in both October (H = 5.341, p < .05) and April (H = 20.956, p < .001).

• Lighthouse classroom teachers were more likely to report that their S.T.A.T. teacher supports the use of data to meet individual student needs than were non-Lighthouse classroom teachers in April (H = 25.756, p < .001).

• Lighthouse classroom teachers were more likely to report that a learner-centered environment was more evident as a result of support from the S.T.A.T. teacher than were non-Lighthouse classroom teachers in both October (H = 11.462, p < .01) and April (H = 28.844, p < .001).

• Lighthouse classroom teachers were more likely to report that their S.T.A.T. teacher has provided coaching on technology integration than were non-Lighthouse classroom teachers in October (H = 25.688, p < .001) and April (H = 36.856, p < .001).

The only statistically significant result regarding change across time (October to April)

for instructional support was observed for S.T.A.T. teachers supporting the teacher use of data to meet individual student needs. Non-Lighthouse teachers were more likely to report S.T.A.T. teacher supporting the use of data in the October than the April survey, Z = -2.353, p < .05.

Classroom teachers viewed the S.T.A.T. teachers as providing useful resources to help them implement the program. One teacher commented, “She is a good in-house resource. She gets it.” Another stated, “She does a lot of research herself. It also helps that she was a teacher herself and she gets the classroom.” Principals also reported the S.T.A.T. teachers providing resources, “The benefits are that they are the experts in the building on S.T.A.T and instruction. They are crucial resources to staff & administration. They could not be a Lighthouse School without the S.T.A.T teacher role.”

The focus groups also mentioned the transition to student-centered environments as

described by one teacher,

It’s a release of us being the sage on the stage and in control. It’s more of letting students have freedom and choices - providing resources and support, but more a facilitator role. Instruction shifted to a lot of small group instruction since we’re not spending all that time lecturing and hand holding.

Not all teachers, however, felt prepared for implementing a student-centered learning environment. According to one teacher, “When do we have training on student-centered learning? I learned from Pinterest. There’s no PD on teaching practices to support student-centered learning. Just bits from others.” Overall, most teachers appear to embrace the concept of student-centered learning. One teacher describes her experience:



The mindset shift has been the most mind blowing for me. I was so rigid in my classroom function, a tight ship! Flipping the mindset into the student-centered…getting the whole thing to change was so eye opening that students were going to make such bigger, more productive gains. That’s the whole point of S.T.A.T – get the kids to have the 21st century skills. The teacher being a facilitator and letting the kids learn to meet their own needs.

One S.T.A.T. teacher captured the impact of a student-centered learning environment: “The shift to student-based/student-centered learning is a profound improvement.” Almost all the principals supported the implementation of a student-centered learning environment and recognized that it was a more difficult transition for some teachers, as offered by one principal:

This is a hard question. For teachers who were open minded and ready to work, there was enough. For those who were more reticent, less confident, they could have used more. The county worked really hard; there were Lighthouse institutes, lots of training on learner-centered environments, etcetera. For some teachers, though, either because of a lack of skills or confidence, they were lagging. My school has a challenging student population, years and years of missing leadership means that they may have been in a different place from other schools when they got started.

Most S.T.A.T teachers viewed their role as one of providing support and resources to

teachers. One S.T.A.T teacher stated, “I’m a S.T.A.T cheerleader: bring everything back to teaching and learning and how to transform the focus.” Teachers also confirmed this view in the focus groups. The majority saw the S.T.A.T teachers as a significant resource, but also one who provides support in creating lesson plans, refining pedagogy skills, and solving technical problems. The focus group participants view the S.T.A.T teachers as key to successful implementation, and principals also reflected this view. One principal described the S.T.A.T teacher as “Having a job-embedded staff development person.” As a result, the principal was freed to tell the story of what’s happening in the school. The S.T.A.T teachers often were described as an extension of the administration with the relationship being particularly important. As noted by one principal:

The relationship between admin and the S.T.A.T teacher is imperative. My assistant principal and I look at her as an extension of us. I can sit down with her, plan, talk about the needs of the school. I give her big ideas and she can run with them. There’s a balance between not an administrator and not a teacher – it’s a difficult line to straddle.

Another principal said,

My hope is that principals really understand the importance of their [S.T.A.T. teacher] role and what the role really is supposed to be, utilize them, select the right person. It needs to be the best teacher you’ve ever seen. Ours didn’t know that much about technology when she started. The person needs to be bright, smart, willing to learn, and push the envelope.

The Lighthouse teachers and principals viewed the S.T.A.T teachers as important for successful implementation of the project.



Accessibility, follow-through, and confidentiality. Classroom teachers responded to

survey items assessing accessibility, follow-though, and confidentiality provided by their S.T.A.T. teacher. A range of differences regarding perceived instructional support was observed for Lighthouse classroom teachers and non-Lighthouse classroom teachers during both the October and April survey administrations:

• Lighthouse classroom teachers were more likely to report that their S.T.A.T. teacher follows through on requests than non-Lighthouse classroom teachers in April (H = 11.214, p < .01).

• Lighthouse classroom teachers were more likely to report that their S.T.A.T. teacher was accessible than non-Lighthouse classroom teachers in April (H = 13.583, p < .001).

• Lighthouse classroom teachers were more likely to report that their S.T.A.T. teacher effectively plans and facilitates meetings than non-Lighthouse classroom teachers in April (H = 25.822, p < .001).

• Lighthouse classroom teachers were more likely to report that their S.T.A.T. teacher maintains confidentiality than non-Lighthouse classroom teachers in October (H = 6.707, p < .05) and April (H = 10.978, p < .001).

The only statistically significant results regarding change across time (October to April) was observed for non-Lighthouse classroom teachers. Non-Lighthouse classroom teachers indicated that S.T.A.T. teachers were more accessible in October than they were in April (Z = -2.472, p < .05) and that S.T.A.T. teachers more effectively plans/facilitates meetings in October as compared with April (Z = -2.203, p < .05).

In addition to the close-ended survey items, teachers were asked to provide comments

about what is working in the S.T.A.T. Teacher Program and how the program could be improved. Similar to the October survey administration, teachers overwhelmingly viewed the S.T.A.T. program favorably and the following themes emerged during the analysis of open-ended responses.

• S.T.A.T. teachers were frequently described as being supportive, knowledgeable, and accessible. Lighthouse classroom teachers particularly stressed the positive qualities of the S.T.A.T. teachers within their schools, though these characteristics were also present in the comments provided by non-Lighthouse classroom teachers. They commented that S.T.A.T. teachers were very responsive to their needs and were knowledgeable on strategies and resources to create a learner-centered environment:

In my opinion, our S.T.A.T. teacher is knowledgeable, accessible, and a strong leader. Most importantly, as with our administration team, our S.T.A.T. teacher introduces new ideas and technology opportunities in a manageable way, and allows us to feel comfortable asking questions about how to implement new technology into our classroom routines. Without her support, I doubt very much that I'd have tried as many new technology programs in my classroom this year.



I feel that this year it has been nice to have someone to go to in that "mentor teacher" role even though I have been teaching for several years. Our School’s [S.T.A.T.] teacher has a wide depth of knowledge about the curriculum across grade levels, helps with technology, and makes our faculty meetings interesting and meaningful. [My S.T.A.T. teacher] is awesome! She is always available to help me learn/try new methods of teaching/technology. I think she has really helped me grow professionally this year. She is the best [S.T.A.T.] teacher/mentor I have ever worked with. Our S.T.A.T. [teacher] has been pivotal in creating a positive climate in our school where we are encouraged to learn, try and explore new strategies, technology and methods of teaching. We are grateful to have sensible direction and articulate, coherent and effective staff developments. Our teachers are inspired and motivated by this intelligent, dedicated and supportive professional.

• Professional development opportunities offered by the S.T.A.T. teachers were

viewed as highly beneficial. In addition to describing knowledgeable and accessible S.T.A.T. teachers, both Lighthouse and non-Lighthouse classroom teachers conveyed an appreciation for the professional development offered by those in this position. Lighthouse classroom teachers also described the S.T.A.T. teacher as being able to translate county and administration goals to the teachers. Some participant comments follow:

[Our S.T.A.T. teacher] does a wonderful job providing professional development to all the teachers in our school. She does this through grade level planning, small group planning, and individual planning sessions. I have been in my school for over 10 years and the professional development provided by our S.T.A.T. teacher is remarkable. She consistently tries to engage teachers and provides a model for what a learner-centered classroom should be. Teachers are able to walk away from her PD with strategies and ideas that can be implemented in the classroom to motivate students.

• Respondents suggested that S.T.A.T. teachers had helped to reform classroom

and school practice. It was evident from the spring 2015 survey responses that many teachers felt that the S.T.A.T. teachers had played an important role in reforming classroom practices to reflect digital transformation in schools and child-centered learning environments. Many teachers reported that S.T.A.T. teachers have played an active role in integrating technology-based learning in the classroom. Some comments made by respondents are reported below.

The S.T.A.T. Teacher Program does a wonderful job of supporting teachers as we shift to student-centered learning environments. We are given the tools



(programs, websites, technology ideas) necessary to infuse 21st century digital technology into our instruction. Our kids have also grown to see our S.T.A.T. teacher as a resource for them, not just the teachers which has helped to develop skills far beyond the limited time a classroom teacher has to introduce advanced skills in programs or researching skills. She has come into classrooms and taken small groups to work on technology based projects! I love the integration she has done with our classes. [Our S.T.A.T.] teacher has been the driving force in helping my school move forward this year. Her help and enthusiasm have been contagious. She has helped the staff implement changes and grow in ways I did not think was possible.

The respondents also offered several valuable suggestions for improvement:

• S.T.A.T. teachers should share best practices from other Lighthouse schools. Classroom teachers within Lighthouse schools indicated a desire to learn the practices within other Lighthouse schools and suggested the S.T.A.T. teacher could share this information.

• S.T.A.T. teacher roles and responsibilities to be more clearly articulated to classroom teachers. Lack of understanding regarding the goals and purpose of S.T.A.T. teachers was a more common concern in the April survey results, especially among non-Lighthouse classroom teachers. There was confusion about what S.T.A.T. teachers should be doing within BCPS (e.g., integrating technology in classrooms, addressing technical problems, providing professional development, mentoring teachers and providing administrative help to the school) and how to utilize S.T.A.T teachers. As one teacher noted,

I think teachers need to know [S.T.A.T. teachers’] specific job role. I am not sure I need help from her, but I am not really sure what she is supposed to be helping me with. She is available to us and asks if we need things, but I am not sure what "things" she is really referring to.

Perceived lack of clarity regarding the S.T.A.T. teacher position sometimes negatively impacted classroom teacher buy-in. Teachers expressed concern about “overlap” between the role of the S.T.A.T. teacher and other school professionals and a selection of respondents suggested that school resources could be put to better use (e.g., “increasing the number of teachers in schools so class sizes can be greatly reduced”). According to one respondent, it is vital that teachers across the school district be made fully aware of the S.T.A.T. teacher role:

Teachers need to be informed with a clear purpose of the position and hear how other schools have utilized the position in order to get more on board with using



the S.T.A.T. teacher to help them move forward and develop a 21st century classroom.

• The S.T.A.T. teacher should have dedicated time within his or her school. Similar

to the October survey administration, it was suggested that the S.T.A.T. teacher was being pulled in too many directions and this impacted his or her effectiveness overall. Non-Lighthouse classroom teachers commented that often their S.T.A.T. teacher was either shared with another school, away from the location due to PD, or was completing tasks they did not perceive as fulfilling the S.T.A.T. teacher role. Many classroom teachers, though, acknowledged the move to a full-time S.T.A.T. teacher the following year. Some comments included:

She is doing the work of two people. S.T.A.T teachers need to be at a school full time. I definitely think that all S.T.A.T. teachers should be full time. They have myriad responsibilities.

Lighthouse school preparation. Classroom teachers within Lighthouse schools were asked to respond to survey items soliciting their perceptions of the preparation to serve as a Lighthouse school. The majority of teachers agreed (27.8% agreed, 50.0% strongly agreed) that they were adequately informed of the expected role of their school as a Lighthouse school. The teachers also indicated agreement (30.0% agreed, 51.1% strongly agreed) that they were adequately prepared to support their school in fulfilling its role as a Lighthouse school. Nearly all (83.3%) of teachers strongly agreed that they felt their school was successful in fulfilling its roles as a Lighthouse school. Measurable Outcomes This section presents results from data sources that related to measurable outcomes including the impact on the classroom environment, teacher practice, digital content, student engagement, and P21 skills. Readers should be cautious in making any conclusions based on the results of the observations as only four classrooms within the schools were observed during the two time points, and the observations served as only a “snapshot” of classroom practices for a brief amount of time. Observation data were analyzed using independent samples t tests to compare differences between observation findings in the fall of 2014 and spring of 2015. Significant differences were found on seven of the observation items and are reported below. Frequencies of responses and descriptive statistics for all observation items may be found in Appendix H. While observation data were collected regarding the impact on student engagement, data for attendance, office referrals, and suspensions within Lighthouse schools also were examined. Wilcoxon signed ranks tests were conducted to identify differences during the 2013-14 school



year as compared with the 2014-15 school year. Data for these three areas are presented in Appendix I. Classroom environment. Five of the observation items pertained to the classroom environment, including information displayed in the classroom and student activities. Observers documented the room arrangement in the classrooms. Nearly all (92.5%) of classrooms consisted of desks arranged in groups. The remainder of classrooms had desks arranged in either rows or a combination of groups and rows. A minority of classrooms (10.0%) were found to have an extensive amount of information and resources to support independent thinking displayed in the classroom, whereas many (45.0%) had a moderate amount of this information displayed. Nearly half (42.5%) of classrooms had lesson-specific content displayed and roughly half (52.5%) had information related to the content area displayed. However, there were no statistically significant effects pertaining to variables associated with the classroom environment between fall 2014 and spring 2015 points of observation. One of the classroom environment items assessed student utilization of different workspaces, that is, whether students used different areas for such activities as collaboration, receiving direct instruction, or engaging in independent work. Students were utilizing different work spaces at various levels to a lesser extent in the spring observations (27.5% at least occasionally) as compared with fall observations (55% at least occasionally). This difference approached statistical significance, t(58) = -1.813, p = .074. Students remained in a single location for 42.5% of the spring classroom observations. For clarification purposes, “occasionally observed” indicates that the approach received a moderate emphasis in class or was observed for a minimal amount of time. A second observation item pertained to students’ movement within the classroom (see



Figure 5). Specifically, the item assessed whether students acquired needed materials for a task or project without teacher direction. As displayed below, similar to fall observations, spring observations revealed that most students (82.5%) were not observed or rarely observed independently acquiring materials and resources and thus there was no significant change for student movement to acquire resources between times of assessment.



Figure 5. Frequency of extensiveness observed on OASIS-21 classroom environment items.

Interviews with the S.T.A.T. teachers suggested Lighthouse classrooms provided various

areas or stations for students to use. One S.T.A.T. teacher stated, “More small groups, more rotating stations, more meaningful work.” One focus group teacher stated that students can move about the room to complete their work, “The room is set up and students are having more choice. Not just all in rows and desks. They’re moving around, able to sit where they’re most productive.” Another focus group teacher described her class and how the student-centered environment changed the classroom:

The student-centered learning environments have helped, too. It used to be you sat at your desk, or you sat where I put you, but now they have 100% choice. They move around the room whenever, wherever, on almost whatever it seems and for most of the students that really works for them. When it doesn’t work and there is some redirection that also is helping them figure out the most conducive learning environments for them.

Similarly, a principal described a classroom that allowed students to work in different areas:

Oh my gosh. It’s been dramatic in so many different ways. Students that – they’re always engaged. Every child is always working all the time, independently, with partners, in a small group. You don’t see kids just sitting there. I noticed… we have several fully included high functioning autistic children – last year there were lots of referrals to the office. This year not so much. It’s not just about devices, it’s about the environment – freedom to sit where they want, learn how they want. Choice: who they work with and how they work with that person. Often it’s difficult for kids to keep focused – that’s not the case anymore.

Overall, the interviews and focused groups reported relatively few descriptions of how students could use different areas of the classroom.



Teacher practice. Observers also documented the practices exhibited by teachers (see

Figure 6). Teachers were viewed acting as a coach or facilitator at a more frequent extent (75.0% at least occasionally) as compared with fall observations (62.5% at least occasionally). This finding approached statistical significance, t(78) = 1.935, p = .057. A reduction in the amount of teacher presentation was also observed in the spring (35.0% at least occasionally) as compared with the fall observations (57.5% at least occasionally), although the finding was not statistically significant, t(78) = -1.578, p = .119. Flexible grouping of students based on student ability or task needs, though, was observed to a lesser extent in the spring (10.0% at least occasionally) as compared with the fall (27.5% at least occasionally). Again, this finding was not statistically significant, t(78) = -1.776, p = .080. Figure 6. Frequency of extensiveness observed for OASIS-21 items related to Teacher Practice items.

The S.T.A.T teachers also described how teachers have translated to facilitator in their interviews. One S.T.A.T. teacher described the process as, “It is us being able to relinquish the control. We had to learn how to hand that over to the students.” Another stated, “The facilitation has taken over our day – in a good way.” The result of this teacher transition to facilitator has had an impact on the classroom as explained by a S.T.A.T. teacher:



Teachers are no longer isolated in the front of the room. Today, I walked by and could hear a teacher but couldn’t find her. Now, they are really integrated into the classroom and that’s a big shift in the culture. Most teachers have stopped standing in front of a room; we’ve become more aware of how to make students their own learning facilitators rather than teachers. I have seen students facilitate discussions within small groups amongst themselves alongside the teachers working with other small groups.

Similarly, another S.T.A.T. teacher described the transformation in her school:

Again teachers are acting as a facilitator more so than the person spoon feeding information. There’s more exploration and time for students to work collaboratively to develop a concept or figure out a strategy. There’s time to problem solve together without the teacher telling them how to do it. They determine the best way to solve a problem. Teachers are collaborating a lot more, a lot more with each other. The way the school is doing PD teachers are getting personalized PD during grade level planning day – they plan together, unpack standards, develop ways to incorporate technology meaningfully – they’re working together. They’re doing the same things across grade levels - tools might be the same but how they’re delivering to different students is different. There’s more collaborative planning going on this year. It’s definitely changed the culture of the school – they’re more willing to take risks because there’s communication with each other about what works and what doesn’t work. They feel safe because of a stronger relationships with their peers. In the past there wasn’t time for that much collaboration. They’re planning as a grade level for the upcoming unit and there is time built in.

Additional teacher practice items examined the questions and feedback posed to students,

as well as student-initiated communication (see

Figure 7). Teachers were not observed asking students higher-level questions in nearly half of the spring observations (47.5%) as compared with only 12.5% of teachers not asking these questions in the fall. The difference in the use of higher-level questions significantly changed between the two times of assessment, t(78) = -2.176, p = .03. Higher-order instructional feedback to students, though, was exhibited to a comparable extent (45.0% at least occasionally) during both the fall and spring observations and thus no significant difference occurred between times of assessment. There was a decline, though, in the degree to which students initiated communication with peers or the teacher in the spring (12.5% at least occasionally) as compared to the fall (40.0% at least occasionally). This finding was significant, t(78) = 4.609, p < .001.



Figure 7. Frequency of extensiveness observed on OASIS-21 teacher practice items.

Digital content usage. Usage data for BCPS One were provided by Engrade and analyzed to determine use by teachers, students, and parents. Data consisted of access information, such as the number of user accounts and user logins by teachers, students, and parents, as well as tile engagement, which related to the specific types of content created by teachers and viewed by students. Although teacher and student access remained fairly consistent with the midyear evaluation findings, there was an increase observed in tile engagement. Specifically, Lighthouse classroom teachers created more file and repository content tiles during the second semester as compared to the first. Student tile views also increased within these same categories. More detailed comparisons on access and tile engagement follow.

Access. In terms of district-wide users, 12,125 teachers, 115,486 students, and 115,486 parents had user accounts created to access BCPS One from December 20, 2014 through May 15, 2015. Teacher, student, and parent account quantities increased (6.8%, 1.6%, and 1.6%



respectively) from the number of users reported in the midyear report. The quantity of student and parent accounts created is identical due to the default parent account created by Engrade for each student and may not reflect actual parent access of BCPS One. In addition, the quantity of student accounts created is greater than the quantity of students enrolled in the district; the value reflects all student accounts and includes students that may not currently be enrolled.

Consistent with the midyear evaluation, teachers had the greatest percentage of users who

accessed their accounts during this time period, followed by students, and then parents (see Figure 8). Overall, Lighthouse schools had a greater percentage of teacher and student logins to user accounts than did non-Lighthouse schools.

Figure 8. Percentage of available users that logged into BCPS One from 12/20/2014 to 5/15/2015.

Teacher tile engagement. Data also were provided in order to examine how BCPS One was used by teachers and students. Teachers used BCPS One in order to create categories of tiles such as:

• Instruction tiles containing teacher-created or identified digital content • Repository tiles incorporating items from the BCPS One repository including

curriculum and instruction content • Link tiles consisting of teacher- or district-provided URLs • File tiles containing teacher-owned or district-provided instructional content files • Assignment tiles (e.g., homework) • Assessment tiles consisting of teacher-created tests and quizzes



Tile engagement frequency data were derived by dividing the total number of tiles created by the number of tiles created within each category. The resulting values provide an indication of how teachers, on average, were using BCPS One. Figure 9 illustrates the comparison between Lighthouse and non-Lighthouse school usage during the second semester.

• Similar to the midyear report, assignment tiles accounted for the greatest number of

tiles created within BCPS One for all teachers of all grades within the district. Creation of tiles within this category accounted for a substantially greater percentage of all tiles created in non-Lighthouse schools, Grades 1 through 3 (86.6%) as compared with Lighthouse schools, Grades 1 through 3 (40.9%).

• Conversely, teachers of Grades 1 through 3 within Lighthouse schools created tiles related to instructional content including repository tiles (35.5%) and link tiles (15.2%) more so than those in non-Lighthouse schools. Lighthouse classroom teachers increased the concentration of repository content tiles from the first semester (28.2%) to the second semester (35.5%).

Figure 9. Frequencies of tiles created by category.

Data also were examined to determine the average number of tiles created by grade level within the district as a whole, as well as by Grades 1 through 3 within Lighthouse schools and non-Lighthouse schools. Overall, 1,244 tiles were created per grade within all district schools



during the second semester, fairly consistent with the first semester (1,215). Approximately 933 tiles were created per grade in Grades 1 through 3 of Lighthouse schools (798 in the second semester), and 564 for the corresponding grade span in non-Lighthouse schools (460 in the second semester). A breakdown of number of tiles created per grade is presented in Table 2. Teachers of Grades 1 through 3 in Lighthouse schools utilized BCPS One tiles more extensively for content than did their counterparts in non-Lighthouse schools:

• Lighthouse schools averaged 331 repository content tiles created per each grade in Grades 1 through 3, and approximately 43.8 repository content tiles were created per each of the same grade levels within non-Lighthouse schools.

• Lighthouse schools Grades 1 through 3 averaged 142 link tiles per grade as compared with three per grade in the same for the remainder of the district.

• Teachers of Grades 1 through 3 in Lighthouse schools created fewer assignment tiles (381 per grade) than did those in non-Lighthouse schools (488 per grade) Grades 1 through 3.

Table 2. Average tiles created within BCPS One per grade.

Lighthouse Non-

Lighthouse Lighthouse Non-

Lighthouse All BCPS Schools

Grades 1-3a Grades 1-3 All Grades All Grades All Grades Tile Type n = 30b n = 299 n = 33 n = 986 n = 1,019 Instruction 4.8 0.3 2.6 4.9 4.8 Repository content 331.4 43.8 383.3 20.6 32.4 Link 142.0 3.0 134.2 13.2 17.1 File 53.8 16.7 50.1 34.6 35.1 Assignment 381.2 488.3 390.2 1169.9 1144.7 Assessment 9.5 0.5 15.8 4.24 4.6 a Only classrooms of Grades 1 through 3 of Mays Chapel were included in the calculation for ease of comparison. b Sample size refers to the number of schools times the number of grades within the schools.

According to results from the Lighthouse Classroom Teacher Survey, the majority (63.3%) of Lighthouse classroom teachers indicated using BCPS One to deliver instruction customized to students’ needs at least once a week. In terms of using BCPS One to develop formative assessments, 27.8% of Lighthouse classroom teachers used this feature at least once per month and 22.22% never used this feature. Nearly half (43.3%) of Lighthouse classroom teachers used BCPS One to develop assignments at least once per week and the majority (82.2%) indicated never using BCPS One to post homework assignments.

Technology integration. The majority of Lighthouse classroom teachers indicated that

the use of technology was an integral part of their teaching practices this year (75.6% very strong). In addition, the majority (77.8%) indicated that the use of technology was a very strong part of their instructional planning and administration activities.



In the Lighthouse Classroom Teacher Survey, teachers were asked to respond to two open-ended questions regarding successes and challenges with technology integration in their classrooms, respectively. In terms of successes, many teachers reported that technology integration has helped to create a 21st Century, student-centered learning environment that has strong potential to enhance classroom performance. Teachers highlighted several benefits for students. They stated that students play an active role in the learning process and are exposed to real-world learning paradigms. Technology-based learning also was said to be effective in engaging student interest and fostering motivation to learn. Teachers commented that offering students choices (e.g., selection of projects, learning resources and modalities) not only enhances learning, but also fills gaps in knowledge and increases feelings of ownership associated with schoolwork. A number of teachers also thought that digital learning experiences enabled students to demonstrate knowledge in different contexts, in a range of academic domains (e.g., mathematics, reading, and science).

Successes associated with technology integration were also experienced by teachers. In

addition to learning about technology, teachers felt they had greater flexibility regarding their choice of teaching methods. Technology-based learning was reported to increase opportunities for small-group tasks, collaborative learning (e.g., students discussing technical problems), and differentiated instruction techniques. Furthermore, one respondent commented that because technology allows students to become more self-starting, independent learners, less input is required on the part of teachers.

Examples of teacher testimonies regarding the successes of technology integration are

reported below. The technology created and supported student centered-learning and allowed students to take an active role in their learning. Creating a student-centered learning environment, creating a blended learning experience, planning lessons that allowed for creativity and collaboration, giving students choice and individualized tasks. I feel that my biggest success was being able to differentiate instruction easily. The confidence and independence that I've seen in my students has been wonderful. Students who were so reliant on me at the beginning of the year are much more independent; taking initiative for their own learning.

In terms of challenges with technology integration, teachers most commonly reported

concerns associated with faulty devices (e.g., broken or malfunctioning hardware or equipment, such as headphones), problems with Internet connections (e.g., programs blocked due to server malfunctions) and incompatibility between school systems and learning programs. A number of teachers indicated they would like increased technical support from school staff to troubleshoot and fix faulty equipment. Several teachers also reported the need for additional equipment and access to backup equipment when technical difficulties arise.



Technology integration varies between schools and between S.T.A.T. teachers. How technology is integrated appears dependent on the philosophies of the S.T.A.T. teacher and the principal. One S.T.A.T. teacher described the importance of understanding technology integration:

When this started, my position was perceived to be a coach and mentor in the integration of technology, but it’s not about technology it’s about instruction and choosing the right tool at the right time, and because I knew instruction and that was one of my strong skills, that’s why I was selected to be a S.T.A.T teacher and it was an advantage. The county did a good job with attending to needs that came up along the way.

Similarly, another S.T.A.T. teacher explained technology integration in her school:

Teachers are really matching learning goals to device or tech-based options and looking for multiple platform options on the topic in order to engage the students. So maybe some teachers want to use Wixi, but teachers are allowing students to do things traditionally if they want to. They’re really looking for interactivity in the digital content rather than the device replacing a lecturer or teacher. We work to plan the right tool for the right moment for the lesson.

One principal provided a similar view:

Looking at technology not just for its own sake, not for substitution, not making things unnecessarily complicated, but using it efficiently to find something more authentic or capitalize on multiple modalities… it’s really helped to find things more authentic and to deepen understanding. The S.T.A.T teacher has really helped us make it meaningful.

However, not all teachers shared a similar view of technology integration as expressed by one principal: “They are still using technology as a substitution for worksheets. They need to create more meaningful and rigorous tasks that integrate the technology. I want to see [the students] moving away from playing math games to doing more rigorous work.”

A sizeable proportion of respondents also talked about the time required to implement

technology in their classrooms (e.g., learning how to use the technology and which programs to include in their classes). Concerns associated with time investment were particularly prevalent among teachers with limited technology experience. While many respondents commented that they had learned valuable lessons for next year, they felt that more professional development is needed in terms of helping teachers use technology and select educational software appropriate for their subject domain and grade level. Some teachers believed that foundational technological training (e.g., how to log on) is also needed for students to avoid large segments of class time being devoted to helping students operate devices and software.

Some teachers mentioned there is a lack of digital resources to support areas of the

curriculum they teach. Several stated that they could not find (free of charge) programs for their subject area or grade level. Respondents indicated they would find it helpful if BCPS put together a list of programs for all subject domains and grades that are accessible and free of



charge. Whether the result of technical difficulties or lack of relevant material, respondents suggested “having a Plan B,” whereby classes can be conducted without relying on technology. Finally, teachers expressed concern about students’ inappropriate use of technology. Students reportedly used the Internet for non-work purposes, watched videos, or engaged in conversations unrelated to schoolwork. Teachers requested that there be a way of seeing what students are doing on their machines and a protocol for inappropriate use of devices, which outlines consequences for students.

Examples of teacher testimonies regarding the challenges of technology integration

together with suggested remedies are reported below. Greatest challenges are the devices breaking and the internet going out in the middle of a test or lesson. We really had no PD or preparation as to what was expected. We had to pave the way, and come up with everything on our own. Finding activities online for students to use that were appropriate, not blocked by BCPS, and free. BCPS needs to put together a list of online activities that fit each unit at each grade level. It would be great if there was a program or way for us to see their screens at any given time to monitor their appropriate usage with our devices.

Curriculum. Teacher perceptions of the curriculum materials are mixed with most

teachers expressing serious concerns. One S.T.A.T. teacher described the implementation of the curriculum in her school:

Given the nature of S.T.A.T., increased engagement, we’re on the right track for allowing students an easier opportunity to master standards. Our school had focused on reading standards this year – looked at reading standards, applied to lesson study with peers, that would help mastery of Common Core State Standards for reading. We’re focused on math next year. S.T.A.T. has helped us and our students are working towards mastery. I’m not sure if other Lighthouse schools have done that. Specific things we’ve done that have allowed us to… it ties into what STAT is all about – personalized PD for staff, too. Teachers understand the standards better and are able to teach the standards better. Yes, it has had an impact.

The Lighthouse teachers, however, expressed a different view of the curriculum that included concerns on the materials being teacher-centered and not understanding the use of technology. One teacher said, “The people who wrote our curriculum didn’t seem to have real-life classroom experience. They seemed to be very out of touch.” Several other teachers indicated the curriculum had not changed with the implementation of the S.T.A.T. initiative. One offered the following:



Because everything is so project-based, under the guise of language arts, wouldn’t it be delightful since those curriculum haven’t changed, and as they change this summer that it became one curriculum. I want it to all be one – science and social students merged activities.

Another teacher expressed a similar concern:

Curriculum – Language Arts – it’s all teacher-centered. We have had to say, here are the standards, end point, resources, but how am I going to make this lesson student-centered? The county curriculum doesn’t in any way match that. Science is all whole-group, not differentiated. The curriculum doesn’t match the S.T.A.T. initiative – it’s all on us to figure out how to make project-based and inquiry-based instruction happen and what that even means and make the lessons to do it.

Other teachers expressed concern about the need to modify the lesson plans: “The curriculum we are given doesn’t necessarily come ready-made and we are doing a lot of the work to get it to a good place.” Another Lighthouse teacher also noted a mismatch between the lesson plans and curriculum: “The curriculum and learning plans need to lend themselves to the technology programs; nothing matched.” A final concern was the lack of planning for integrating the technology into the curriculum, as one teacher described:

We had to re-write every lesson that we use. We tried to just revise or use materials from the county but that did not work. We had to truly integrate the technology into the lessons which meant that we had to make up entirely new lessons. We also did not receive any lessons for teaching students how to use the computers or the application. This learning took twice as long as we anticipated which put us behind in reading instruction. For example, just teaching students how to create and use user names and passwords took time. Students do not know how to type, and the device chosen is not user friendly for anyone, especially young children.

Last, a principal provided his perspective on concerns with the curriculum:

We look at it a little differently than some schools. We look at unit by unit holistically. Teachers are good at looking at it, the objectives, standards, and end goal. They go through and create their own units using the curriculum as a guide. They’re doing a nice job of adapting to be much more personalized than it is. We recognize central office is writing to the masses. My teachers are adjusting based on the needs of our kids. I think people (other schools) are getting stuck on curriculum being the bible and it’s not meant to be the bible. Technology isn’t built in but central office has been open and respectful of suggestions of what could be put in.

Student tile engagement. The average tile views were calculated in order to assess

student use of BCPS One. This calculation involved dividing the number of views per tile category by the number of tiles created within the specific category. The resulting values indicate how students were using BCPS One in terms of which categories of tiles they were



predominantly accessing. An assumption was made that the data pertaining to tile views consisted of only student views.

Tile views during the second semester were consistent with results presented in the midyear report. Although assignment tiles accounted for the greatest number of tiles created by teachers across the district, link tiles had the greatest average student views at five views per tile created. Instruction tiles were viewed on average 2.9 times per tile created, followed by repository content tiles at 2.3 views per tile created for all schools in the district. As presented in Table 3, classrooms of Grades 1 through 3 in Lighthouse schools had the greatest number of link tile views per tile created, followed by repository content tiles and instruction tiles. These same grade-level classrooms in Lighthouse schools had the greatest ratio of views to tiles created for link tiles, followed by instruction tiles. Table 3. Average tile view by students per tile created within BCPS One.

Lighthouse Non-

Lighthouse Lighthouse Non-

Lighthouse All BCPS Schools

Grades 1 -3* Grades 1-3 All Grades All Grades All Grades Tile Type n = 30 n = 300 n = 33 n = 975 n = 1,008 Instruction tiles 5.7 3.5 5.7 2.8 2.9 Repository content tiles 5.9 1.1 4.6 0.9 2.3 Link tiles 10.0 4.3 10.1 3.4 5.1 File tiles 4.6 0.1 4.6 0.6 0.8 Assignment tiles 0.3 0.0 0.3 0.0 0.0 Assessment tiles 0.0 0.0 0.0 0.0 0.0 The ratio of student logins to tile views also was calculated in order to provide information on depth of student usage of BCPS One (see Figure 10). As a whole, tiles views increased within each category of tiles from the midyear evaluation report. The increases in tile views were fairly minor (less than one view per tile created) for the district as a whole. Consistent with the midyear results, average student views in Lighthouse schools were greater than those in Non-Lighthouse schools across all tile categories with the exception of test and quizzes (0 for both groups). Although assignment tiles accounted



for the greatest concentration of tiles created within Lighthouse classrooms, repository content tiles were viewed the greatest frequency of 12.9 views per student (increased from 8.3 in the first semester), followed by link tiles 8.26 views per student (increased from 8.9 in the first semester). These average views demonstrate that students in Lighthouse schools were repeatedly accessing the tiles used for instructional purposes. In contrast, all tile views occurred at a frequency lower than one view per student in non-Lighthouse schools. These findings are not surprising as students within Lighthouse schools have greater access to BCPS One on their individual devices than those in non-Lighthouse schools. Figure 10. Average tile views per student based on student login counts.

Student engagement. Observers rated five OASIS-21 items related to student

engagement during classroom observations. As with the fall observations, in nearly half of the



classrooms observed during the spring, students were using digital tools for learning to a frequent or extensive degree (see

Figure 11) and there was not a significant difference in the use of digital tools over time. In contrast with fall observations, independent work was observed in the spring to a lesser extent (65.0% at least occasionally) and collaborative learning more frequently (32.5% at least occasionally). However, within-groups t-tests indicated that differences observed for independent and collaborative work between fall 2014 and spring 2015 were not statistically significant (Independent work, t(78) = 1.728, p = .088; Collaborative work, t(78) = 1.094, p = .278). Figure 11. Frequency of extensiveness observed on OASIS-21 student engagement items.



An additional student engagement item (see

Figure 12) pertained to student discussion, specifically whether students discussed a prompted or higher-level topic in pairs, groups, or within the whole class. Student discussion was observed more frequently in the spring (22.5% at least occasionally) as compared with the fall observations (10.0% at least occasionally), but the difference was not statistically significant, t(78) = 1.719, p = .090. A decline in the extent to which multiple modes of student responses, such as verbal, written, physical, or through technology were observed was found during the spring observations (45.0% at least occasionally) as compared with the fall observations (65.0% at least occasionally). This difference was at the threshold of statistical significance, t(78) = -1.988, p = .05.



Figure 12. Frequency of extensiveness observed on OASIS-21student engagement items.

Student engagement was one of the most frequently mentioned themes in the interviews and focus groups. Many S.T.A.T. teachers attribute the increased engagement to the use of technology: “It has completely changed engagement. It is a lot easier because naturally students are drawn toward technology.” Others attributed the increase in engagement to the choices students have: “Students are more engaged because of the push for the student choice in the classroom, not necessarily devices. They make choices about what meaningful independent work they want to do.” One S.T.A.T. teacher described the difference in engagement this year: “It’s taken it to a completely different level; before students were well-behaved but compliant, not necessarily engaged; they were quiet but how do you know they were learning? Now they are highly actively engaged and participating in their learning.”

A S.T.A.T. teacher described how the student engagement has resulted in more rigorous

discussions: I think there’s been a huge impact on student engagement. There’s more of a choice in the product they’re producing and how they’re learning, more eager to participate in learning at a pace more appropriate for them, retaining information more, engaging in more rich conversations. They communicate with each other about their learning. There’s more rigorous conversations and discussion about what they’re learning. In most classrooms there’s not necessarily a kid raising a hand to participate, it’s kind of an ongoing conversation. Kids are learning how to interact in a respectful manner to keep the conversation going about what they’re learning.



Engagement also changes the type of learning environment, according to one Lighthouse teacher: “Students are much more engaged and able to work independently for longer periods of time. Students are working more actively; not passively doing worksheets.” Increased motivation from the engagement also allows teachers to individualize the instruction giving more attention to those in need. A classroom teacher commented:

I have a very diverse group with a big spread between the top and lowest performance. There is so much for them to do – this has taken learning to the next level for the top kids. I can let them use PebbleGo while I teach phonics to other students. In the past, they all had to sit through phonics whether they needed it or not. Now they can get enrichment.

Similarly, another teacher stated, “Bottom kids – I can spend more time with them since I’m not spending time differentiating and challenging them as they have sites to go to and get those activities.”

Several teachers described the increased choices that have influenced engagement. For example one Lighthouse teacher explained:

The initiative being the computers as well as the overall environment has completely changed that student engagement piece because students are choosing what they are learning in a way. They can reach out and access things they have never been able to access before in order to create or take in content that is more interesting to them.

Another teacher described her observations:

Students have evolved over the year about how they engage with the technology. There are kids that were particularly excited about using the device in the beginning but have now figured out that they don’t want technology for everything. There were other kids that are just now finding their comfort zone with certain pieces. The choice is really nice and leads to greater engagement.

A few teachers described a level of engagement that did not occur in their traditional

classrooms. One teacher said, “When the visitors come in, the students are very engaged, but if I as the teacher want to stop them, they won’t give me their attention. I’ve had to pull them to the carpet to get their attention.” Similarly, another teacher told us, “I’ve had to say ‘half screen’ which means they should not be looking at their device. A lot of times we’ll give them time to work, but then when you want to pull them together and talk, they’re still on the devices.”

The principals also frequently described the increased student engagement. One principal stated:

Oh, it is so vast and so deep. I would say there has been a 100% increase in student engagement – for a lot of different reasons. The one-to-one device made so much of what engages students truly available to them. It’s also moved into actually pushing students into engaging themselves in continued learning.



Another principal noted the impact of engagement in his school to address individual needs:

There’s a significant impact on engagement because teachers are now meeting the needs of students versus before, they were more consistently meeting needs through access to leveled text, for example, a more advanced reader can be given a more challenging assignment whereas a student reading on a lower level is given an assignment more suitable to their ability. Student attendance. Data from BCPS indicated that the percentage of students at

Lighthouse Schools (Grades 1-3) (n = 8) meeting the 94% attendance cutoff decreased in semester one between 2013-14 (75.1%) and 2014-15 (66.4%) (see Figure 13). This difference was statistically significant (Z = -2.521, p = 0.012). Though not significant, the cutoff increased between semester two between 2013-14 (67.2%) and 2014-15 (67.7%). Figure 13. Attendance data for Lighthouse schools (n = 8) during the 2013-14 and 2014-15 school years.

Student suspensions. Records of BCPS student suspensions in Lighthouse Schools

(Grades 1-3) (n = 8) indicated that there were 21 occurrences in academic year 2013-14 and 16 suspensions in academic year 2014-15. However, there was no statistically significant difference in the decline of suspensions (Z = - 0.647, p = 0.518).

Student office referrals. The number of office referrals in Lighthouse Schools (Grades 1-

3) (n = 8) declined between academic year 2013-14 (n = 95) and academic year 2014-15 (n = 45) (see



Figure 14). However, the decline was not statistically significant (Z = - 1.402, p = 0.161). Figure 14. Office referrals in Lighthouse schools (n = 8) during the 2013-14 and 2014-15 school near.

Both teachers and principals reported a decrease in referrals this year. One S.T.A.T.

teacher attributed this reduction to student engagement: “I think because the engagement is up the bad behaviors are down.” A teacher in a focus group offered another explanation:

Outside of the box thinkers are even more engaged and have the resources to do it. It helps behavior because they can meet their own needs. They can go on and do it themselves, independent learning, not spoon feeding. They’re more independent.

Another S.T.A.T teacher described positive and negative changes in behavior:

I don’t think we realized how behavior would change, but it really has in both positive and negative, or rather new ways and so there has been a lot more learning along the way about how to deal with the new behaviors that crop up because of the digital side of things in particular.



One principal noted the drop in referrals this year:

There has been tremendous change. My office referrals are down a least 80%. I have a skinny folder with referrals. It would have been bulging before. The most impactful outcome of that is that children aren’t interrupting their own learning. It’s been incredible – and noticeable to everyone. More kids are getting the Friday Award because of it.

Another principal suggested why referrals are down: “… not a lot of office referrals. Even that has lessened. Kids would way more rather be in classrooms.” Similarly, another principal suggested the drop in behavior problems is a result of increased student engagement:

This is a Title I school, and it has also seen a reduction in behavior referrals throughout the year; 89% decrease in behavioral referrals in grades 1-3 is absolutely directly related to student engagement through the S.T.A.T. initiative….Students are more engaged, spending more time in the classroom versus being pulled out for behavioral issues.

Last, a principal provided insight into the drop in referrals:

The confidence level of struggling learners has really shot through the roof and the behaviors that used to be paired with their low confidence like acting out and being disruptive have gone down because we can now meet them at their level.… Similarly, the students who need more rigorous tasks aren’t winding up bored and off task; they can push forward and continue to research in a more personalized way. P21 skills. Four of the observation instrument items assessed P21 skills. The first item

related to problem solving, where students used multiple resources, used resources effectively, and engaged in critical thinking in order to solve a problem. There was an increase in the extent to which students engaged in problem solving between the fall and spring observations, t(78) = 2.540, p = 0.01 (see Figure 15), though in the majority of classrooms (75.0%) observed in the spring, students were not perceived as engaging in problem solving. Further, learning that incorporated authentic or real world contexts was observed to a significantly greater extent (37.5% at least occasionally) during the spring as compared with the fall, t(78) = 2.002, p < .05. Figure 15. Frequency of extensiveness observed for OASIS-21 items related to P21 Skills.



A third item (see Figure 16) pertained to inquiry-based approaches to instruction, which

involved student exploration of a question or topic in-depth, development and asking further questions, and conducting research and problem-solving in order to answer the questions. A related item assessed project-based approaches to instruction, where the instructional focus is centered on an inquiry or question and students may produce a tangible product as a result of the approach, such as a research report or question. During the spring observations, inquiry-based and project-based approaches to instruction were observed to a similar extent (20.0% at least occasionally observed). Both of these approaches were observed to a significantly greater extent during the spring as compared to the fall (7.5% at least occasionally observed). Within-groups t-tests indicated a near significant increase in the use of inquiry-based approaches (t(78) = 1.995, p = .054) and a significant increase in project-based approaches (t(78) = 2.062, p < .05.) between fall 2014 and spring 2015. Figure 16. Frequency of extensiveness observed for OASIS-21 items related to P21 Skills.



Teachers and principals expressed positive views on problem solving and critical thinking that are examples of P21 skills. The S.T.A.T. teachers were the most positive. One S.T.A.T. teacher said, “We are seeing an improvement on daily assignments but haven’t seen as much on the benchmarks as we would have liked.” Another S.T.A.T. teacher described her observations:

This is clearer - we can definitely say that problem solving and critical thinking skills have improved. The first graders are entering second grade more prepared for sure and just the whole learning in their hands has upped the problem solving/critical thinking – they are finding answers themselves, looking to programs, to friends – they know how to get info, they understand better how they process info.

Similarly, a Lighthouse teacher described problem solving in her classroom:

Problem solving is huge. Even things like ‘my laptop won’t turn on what should I do?’ have helped their problem solving skills. Just the idea of the teacher isn’t the only one with the answer has empowered them to become better problem solvers, for themselves and their peers.

One principal stated, “Positive impact. As curriculum changes and they have access to more, students have more opportunities to work on problem-solving and critical thinking.” Another principal was less optimistic, commenting:

I don’t know that you really master those skills; those are goals infused in the curriculum. How to measure those? We see them being utilized more than before, but can’t say you can master them; you can get good at them, use them effectively and efficiently. How do you measure creativity? Is mastery the goal? Kids are actively acquiring this knowledge; they have increased collaboration and communication with classmates; they are using critical thinking skills.



Perceptions of the S.T.A.T. Initiative This section summarizes S.T.A.T. teacher, focus group, and principal interview comments on the impact of the S.T.A.T. initiative. The interviews and focus groups revealed that teachers and principals see the S.T.A.T. initiative as both positive and transformational. The impact is best summarized by quotes from three principals:

The strength is in the idea itself, of transforming teaching and learning. It’s hard to replicate, but the vision... There’s a clear vision of what we want this to look like. There was a methodical implementation plan and it was strategically thought out. Every step along the way has been methodical in planning and implementation. Empowering the student learner.

Another principal described the impact on his school:

It has definitely accomplished what the idea of it was – leveling the playing field for kids. My school is a high poverty school. What’s been so amazing to me, when I go over to another Lighthouse school, the instruction is comparable, the level of rigor is comparable, the access is comparable. If we swapped kids between schools, they would be receiving the exact same thing just a difference in their skin color. That’s had the most impact and impression on me. We’re doing what we’re supposed to do – equal access and opportunity.

Similarly, a principal at a different school described the impact on the teachers in the school:

I’m really impressed that teachers are very passionate, and the level of passion increased. They’re more excited about what they’re doing, pushing limits. Not necessarily the technology piece, it’s the instruction piece. They’re pushing boundaries, thinking outside the box and their individual kids. There’s a shift in mindset in the building. This year I feel like I hear teachers are saying much more often, “Could we try this or do that?”

One of the significant changes has been the impact on teacher practices as described by a principal:

Teacher practices have evolved, but they used to be “stand and deliver” where the teacher stands in front of the classroom, gives work, and students work on their own. Our school took a bigger leap this year with more teachers willing to stop talking, teach what they needed to teach and push learning to the kids. They’re willing to get that catch/release concept: here’s the standard, go do it, and provide support in small groups. Everybody is now doing small groups. Everybody significantly stopped doing long stand and deliver sessions and are pushing the individualized level of learning to the kids.

This transformation was confirmed by several teachers. One teacher said, “This year has been completely different. The facilitation has taken over our day – in a good way.” Another



teacher stated, “It is us being able to relinquish the control; we had to learn how to hand that over to the students.” The change to a student-center learning environment has impacted the way teachers work, as a teacher commented:

I am comparing only to my student teaching experience since I am a first year teacher, but I feel that I have more time to plan and come up with ideas for instruction – I don’t have to spend time designing hand-outs, making copies, and then have students crumple them up. Instead, I can review what has been developed and decide what is best for my students. I can alter it and right away I have a complete lesson with materials that the students enjoy using.

Similarly, teachers recognize that they need a different approach, as one teacher stated:

Anytime you dive into a new mode, you are forced to think much more about how best to use the various tools and so this has been a very aware year for me. I’m really focused on how to blend the classroom in the most effective way possible.

A S.T.A.T. teacher noted, “I think teachers have changed practices and tried new things because of them, or are open to it, in student-centered learning, adopting technology and responsiveness to student needs in the classroom.” Another outcome of the program is increased collaboration amongst students, as stated by a S.T.A.T. teacher:

Classroom participation has increased via student response formats so that all students have a voice, versus one or two raising their hands to answer a question; now they are all sharing answers to a question at once. Before S.T.A.T., one or two students would answer. Now they can all share online; students are really writing more; not just one word responses, but really reflecting on essential questions.

Another S.T.A.T. teacher observed:

S.T.A.T. has had a huge impact on students. They are so excited coming into the classroom, they get started right away, and they are much better at maintaining their focus through to the end of an assignment. Paper and pencil doesn’t sustain their attention as well. They can do research immediately, can create Power Points immediately, etcetera.

While the transformation has increased the focus and collaboration with students, it also has created a new working environment for teachers as described by a S.T.A.T. teacher:

The collaborative piece has created a more cooperative culture in the staff, a culture more willing to take risks and try things out. They know that if they want to try a tool for the first time there’s someone there to help figure it out together. If it doesn’t work, find something else.



Lighthouse teachers also observed the impact on the initiative on their students:

It is so exciting to hear a kid that is usually checked out or quiet and all of the sudden because of all of these rich units and resources, they are making connections which you generally don’t see in a 6 year old. Phenomenal.

Another teacher observed the increase in student ownership of their learning, “The shift to a student-centered mode of operating has been wonderful and the technology has contributed to the ability for the kids to take ownership of how they learn.” While anecdotal, the interviews showed a positive impact of the program on the schools, classrooms, teachers, and students. As one S.T.A.T. teacher stated, “We have to continue to change the mindset and can’t go back to the way we used to teach.”

Conclusion

The purpose of the present study was to gather formative information on the S.T.A.T. initiative as implemented during the 2014-2015 school year in Baltimore County Public Schools. Throughout this summative report, we have presented results related to professional development and measureable outcomes represented in the temporal logic model originally displayed in Figure 1. The research questions examined in this study, therefore, pertained to the impact of professional development on measurable outcomes including the classroom environment, teacher practice, access and use of digital content, student engagement, and P21 skills. In addition, a research question explored stakeholders’ perceptions of the S.T.A.T. initiative. Measures included surveys, classroom observations, interviews and focus groups, BCPS One data, and student behavioral data.

Professional Development The interviews and focus groups suggest the teachers were not ready for the district summer professional development and some of the presenters lacked the knowledge or skill needed. Similarly, teachers reported the training was too much, too fast, and they could not grasp all the information. In contrast, the teachers were supportive of the professional development provided by the S.T.A.T. teachers in their schools. This training was adapted to meet the specific needs of grades and disciplines as well as individual needs.

During the past year, a variety of professional development activities were provided for both Lighthouse non-Lighthouse classroom teachers. The most frequently attended form of professional development was large-group instruction followed by small-group instruction. The increased attendance at the large group sessions might be the result of required attendance or a pay incentive to attend. In contrast, the small group sessions often were voluntary and often informal such as Tech Tip Tuesday. One might expect a lower attendance rate to small group sessions than the large-group sessions, especially if the larger-group sessions were required or involved a financial incentive to attend.



Lighthouse teachers were much more likely to take advantage of 1:1 support and independent learning as compared with those in non-Lighthouse schools. Of particular interest is the approximately 20% increase in the use of independent learning between the October and April surveys by both Lighthouse and non-Lighthouse teachers. As teachers become more familiar and comfortable with the technology and instructional strategies, we might expect them to make greater use of independent study to increase their knowledge and skills. Overall, classroom teachers within Lighthouse schools viewed the S.T.A.T. teacher as instrumental in transforming their classrooms in accordance with S.T.A.T. goals. The S.T.A.T. teacher frequently was noted as being a supportive, knowledgeable, and critical component to their success. However, classroom teachers within non-Lighthouse schools were mixed on their views of their S.T.A.T. teacher. While some teachers reflected the positive view held by those within Lighthouse schools, others conveyed concerns as to the role and responsibilities of the S.T.A.T. teacher and questioned the need for the position within their school relative to other priorities (such as reduced class size). At this early phase of the S.T.A.T. initiative, the more reserved attitudes of teachers in non-Lighthouse schools seems understandable. Because their schools have not begun formal implementation of the program, they are yet to directly experience the impacts of the technology integration and the concomitant supports offered by the S.T.A.T. teachers. Measurable Outcomes Classroom environment. Over 90% of the classrooms were arranged to support collaborative learning, and a majority of the classrooms displayed materials to support independent thinking. The observations, however, suggest that students seldom move about the room to acquire different resources. Similarly, less than a third of the students were observed moving to a different area of the classroom for collaboration, small-group work, or for direct instruction. The S.T.A.T. teacher interviews, however, suggested that they perceived students as moving around the room and using different areas. There appears to be an impression that students have the freedom to move around the room and work where they are most comfortable. Data from some of the interviews suggest that students can access the majority of the information they need on their devices. Thus, they may be comfortable sitting in their assigned group and completing both individual and group work. Unlike the past, where students would need to use a dictionary, encyclopedia, or even a worksheet that was placed in the back of the room, students in these connected rooms can access all their materials via the Internet and network may have little need to move about the room.

Teacher practice. The impact of S.T.A.T. on teacher practice was evident throughout

data sources. Though findings between baseline and spring observations were not statistically significant, teachers were observed more frequently acting as a coach or facilitator and less presenting instruction during the spring observations as compared with the baseline observations. There was a reduction, however, in the frequency that flexible groups of students were observed and also in the frequency that students were asked higher-level questions and provided with higher-order instructional feedback by the teacher. However, these types of supports would tend to become less applicable and visible as instructional roles shift from teacher presentations to student-centered activities such as increased collaborative learning opportunities. Consistent with



this interpretation and the classroom observations, interviews with S.T.A.T. teachers and with principals indicated that teachers were emphasizing a more student-centered learning environment, differentiating instruction to students, and relinquishing control to the students for the learning process.

Digital content. Consistent with results presented in the mid-year evaluation report, the

analysis of BCPS One usage by schools indicated a greater use of BCPS One within Lighthouse schools as compared to non-Lighthouse schools. Lighthouse classroom teachers most frequently utilized BCPS One for the creation of assignment tiles, followed by repository content tiles. In contrast, teachers within non-Lighthouse schools predominantly created assignment tiles with very minimal creation of tiles within other categories. It is to be expected, though, that Lighthouse classroom teachers would exhibit a greater usage of BCPS One due to the implementation of devices for both students and teachers within these schools. Classroom teachers within Lighthouse schools utilized BCPS One to a greater extent during the second semester as compared with the first, as reflected in an increase in tiles created most notably in the file and repository content tile categories. Accordingly, student tile views increased within these two categories during the second semester. In addition to BCPS One usage data, classroom teachers within Lighthouse schools indicated using BCPS One to differentiate instruction for students. Further, teacher survey responses indicated a substantial use of technology within the classrooms. Specifically, teachers indicated using BCPS One to deliver customized student instruction at least once a week. Lighthouse teachers did express concerns about the mismatch between the S.T.A.T. initiative and the current curriculum. One often mentioned concern was that the curriculum materials were teacher-centered rather than student-centered. As a result, the lesson plans needed to be revised or new ones developed to create student-centered instruction.

Student engagement. While there were not statistically significant differences in the use

of digital tools for learning between baseline and spring observations, there was a noted decline in the frequency of independent work and an increase in the frequency of collaborative learning. There was also an increase in the frequency by which student discussion occurred across these time points. In addition to observation data, student engagement also was examined broadly and indirectly through attendance, suspensions, and office referrals. Though there was a slight decline in attendance between the 2013-2014 and 2014-2015 school year, the number of student suspensions and student office referrals decreased between these two years.

The improvement in student engagement was frequently noted by all Lighthouse school

participants, attributed most often to the increase in student choice and availability of options for the medium and format (e.g., independent, in pairs, in a group) the students desired to learn. Participants conveyed students were much more interested in learning and expanding their knowledge to a much deeper level due to the technological affordances available within Lighthouse schools.

P21 skills. There were also noticeable changes in P21 skills indicated in observation results. Specifically, there was an increase in the frequency that students were observed engaging in problem solving as well as learning that incorporated authentic or real world contexts. S.T.A.T. teachers conveyed that students were much more willing to work with their peers and think critically when posed with a problem. Though instances of inquiry-based and project-based



approaches to instruction remained few, there was an increase observed in the two between the baseline and spring observations. It is important to note that classroom teachers conveyed little professional development offered in order to begin designing lessons that are more project-based and inquiry-based.

S.T.A.T. Initiative Overall, Lighthouse schools have demonstrated substantial growth since the midyear report in transitioning to technology enhanced, learner-centered environments. As reflected in the S.T.A.T. evaluation model (Figure 1), findings of the present study indicate evidence of the effects of professional development on measurable outcomes (e.g., classroom environment, teacher practice, digital content, student engagement, and P21 skills). It is important to note that it would likely be unreasonable for teachers to employ all of the strategies contained in the observation instrument during this first year of implantation, though it is promising that teachers exhibited growth in addressing higher-order and P21 instructional objectives. The S.T.A.T. Teacher Program is consistently perceived by classroom teachers within Lighthouse schools as a valuable asset to assist in the transformation of BCPS schools.

Based on the evaluation study findings, the following recommendations are offered for future S.T.A.T. implementation:

• Professional development. Provide increased time for professional development including visiting classrooms. Offer more differentiation in professional development as it is rolled out to more grade levels and tailor support and training to the groups who have or have not had certain training, meeting teachers at their existing levels of technology knowledge and acquisition. Provide modeled teacher lessons videotaped so teachers can see what strategies and approaches should look like. Increase collaboration on the teacher level so that everyone can see and determine best practices and possibilities. Invite teachers who have already made the digital transformation with their students to provide professional development to other teachers.

• Curriculum. Update the curriculum to fit the requirements of the S.T.A.T. initiative including thoughtful and planned integration of technology into the lessons.

• Technology integration. Increase the focus on ensuring technology tools are not just

being used without clear curriculum and instructional purposes. All schools should start with the language now so that if a school does not receive devices until December, students won’t be behind. Make sure there are extra devices to accommodate all students in a classroom at one time.

• S.T.A.T. message. Leadership must understand and communicate that S.T.A.T. is not

just technology; it is using technology to support the goal of increasing higher-order, student-centered learning to improve academic achievement and mastery of P21 skills.



Appendix A: S.T.A.T. Teacher Program Survey I have participated in the following mode(s) of professional learning facilitated by our S.T.A.T. Teacher (check all that apply):

a. Large Group (e.g. faculty meeting) b. Small Group (e.g. grade level, team, or content area meeting or PLC) c. Individual/1:1 Support d. Independent Learning (e.g. accessing resources on my own provided by the S.T.A.T.

Teacher) e. None

No Basis to Assess

Strongly Disagree Disagree Agree

Strongly Agree

The S.T.A.T. Teacher in my school is accessible to me. □ □ □ □ □ The S.T.A.T. Teacher in my school follows through on requests. □ □ □ □ □ The S.T.A.T. Teacher in my school models effective instructional strategies (e.g. during team or staff meetings, trainings, working with teachers in the classroom, workshops).

□ □ □ □ □

The S.T.A.T. Teacher in my school effectively plans and/or facilitates meetings (e.g. staff, team, department, committee).

□ □ □ □ □

The S.T.A.T. Teacher in my school provides or directs me to useful resources (e.g. student data, CPD courses, certification, professional reading, current research).

□ □ □ □ □

The S.T.A.T. Teacher in my school supports the use of data to inform instruction to meet students’ needs.

□ □ □ □ □

The S.T.A.T. Teacher in my school has helped me create a more learner centered environment in my classroom.

□ □ □ □ □

The S.T.A.T. Teacher in my school provides coaching on how to integrate technology into instruction.

□ □ □ □ □

I trust my S.T.A.T. Teacher to maintain confidentiality. □ □ □ □ □



As part of my professional development this year, I have participated in the following learning opportunities supported by the S.T.A.T. Teacher (check all that apply):

o One-on-one professional discussions/consultations with the S.T.A.T. Teacher o Individual, team, or departmental planning sessions with the S.T.A.T. Teacher o Training or workshop(s) facilitated by the S.T.A.T. Teacher o Developed my teacher development plan with assistance from the S.T.A.T. Teacher o Learning walk or instructional walk-through facilitated by the S.T.A.T. Teacher o Analysis of data with the S.T.A.T. Teacher o Observed another teacher’s classroom facilitated by the S.T.A.T. Teacher o Study group or lesson study with the S.T.A.T. Teacher o Observed the S.T.A.T. Teacher model teaching or conduct a demonstration lesson o Developed an SLO with assistance from the S.T.A.T. Teacher

Please provide comments about the S.T.A.T. Teacher Program to help us understand what is working:

____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ Please provide comments about the S.T.A.T. Teacher Program to help us understand what needs improvement:

____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________



Appendix B: Lighthouse Classroom Teacher Survey Professional development items:

1. I was able to apply what I learned during professional development in my classrooms. 2. I received sufficient professional development on the use of technology in my classroom. 3. I received sufficient professional development on the creation of a learner-centered

environment. Strongly disagree (1) Somewhat disagree (2) Somewhat agree (4) Strongly agree (5)

Teacher practice items:

4. I am skilled at engaging my students in collaborative learning activities using technology as a resource or tool.

Strongly disagree (1) Somewhat disagree (2) Somewhat agree (4) Strongly agree (5)

5. How often did your students collaborate with other students on an in-class activity or investigation?

Never (1) At least once per month (2) At least once a week (3) More than once a week (4)

To what extent did you use the following types of teaching practices this year?

6. Direct instruction/lecture 7. Cooperative learning 8. Student projects 9. Individualized learning (at desk or computer)

Never (1) Rarely (2) Moderately (3) Frequently (4)

Student impact:

10. My students have improved in their mastery of CCSS this year. 11. My students have improved in their mastery of P21 skills this year. 12. The student behavior in my classroom has improved this year.

Technology integration: How often did you use BCPS One to…

13. deliver instruction customized to students’ needs?



14. develop formative assessments? 15. develop assignments? 16. post homework assignments?

Never (1) At least once per month (2) At least once a week (3) At least once a day (4)

17. To what degree is the use of technology an integral part of your teaching practices this year?

Not at all Minimal Moderate Fairly strong Very strong

18. To what degree is the use of technology part of your instructional planning and administration (preparing lessons, grading, data management, etc.)?

Not at all Minimal Moderate Fairly strong Very strong

Lighthouse preparation: 1. I was adequately informed of the expected role of my school as a Lighthouse School. 2. I was adequately prepared to support my school in fulfilling its role as a Lighthouse

School. 3. I feel my school was successful this year in fulfilling its role as a Lighthouse School.

Strongly disagree (1) Somewhat disagree (2) Somewhat agree (4) Strongly agree (5)

Open-ended items:

1. What do you feel were your greatest successes this year in integrating technology into instruction?

2. What were the greatest challenges and how could they be overcome in the future?



Appendix C: Principal Interview Protocol Classroom impact

1. What impact, if any, has the STAT initiative had on student engagement? (prompt for examples)

2. What impact, if any, has STAT had on student behavior in your school’s classrooms?

3. What sort of changes have you observed in teacher practices this year as compared with last year?

4. Please describe how your school has integrated technology into your classrooms this year. What have been your greatest successes with technology integration?

5. What do you believe are your areas of opportunity with technology integration?

6. How have you used BCPS One this year within your school? Student impact

7. What impact, if any, has the STAT initiative had on student mastery of Common Core State Standards?

8. What impact, if any, has the STAT initiative had on student mastery of P21 skills such as problem solving and critical thinking?

Professional development

9. Were your teachers adequately prepared to implement STAT this year? Why or why not?

10. What professional development do you feel your teachers are still in need of in order to implement STAT?

Lighthouse School preparation

11. How, if at all, have you been prepared to support your school in its designation as a Lighthouse school?

12. What additional preparation, if any, do you recommend for future Lighthouse schools? STAT Teacher role

13. How, if at all, has your STAT teacher supported your implementation of STAT (beyond PD sessions)?

14. What are the benefits of the STAT teacher role?

15. What areas of opportunity, if any, are there with the STAT teacher program?



Principal role 16. How have you supported your teacher’s implementation of STAT?

17. How have others within the district (peers, curriculum specialists, technology directors)

supported your school’s implementation of STAT? STAT perceptions

18. What are the overall strengths of the STAT initiative?

19. What changes, if any, do you recommend in the STAT initiative?



Appendix D: STAT Teacher Interview Protocol Classroom impact


2. What sort of changes have you observed in teacher practices this year as compared with last year?

3. Please describe how your school has integrated technology into your classrooms this year. What have been your greatest successes with technology integration?


Student impact




7. What sort of professional development have you received in terms of the STAT initiative?

8. Do you feel adequately prepared to serve as a STAT teacher? Why or why not?

9. What professional development do you feel you are still in need of in order to better serve

as a STAT teacher? Lighthouse School preparation


11. What additional preparation, if any, do you recommend for future Lighthouse schools? STAT Teacher role

12. Describe your role as a STAT teacher in terms of supporting the STAT initiative within your school.

13. What sort of professional development have you offered to the teachers within your

school?

14. What has been the impact of the professional development you’ve offered?



Principal role

15. How, if at all, has your principal supported your implementation of STAT?

16. How, if at all, have others within the district supported your implementation of STAT? STAT perceptions





Appendix E: Teacher Focus Group Protocol Classroom impact


2. What sort of changes have you made in your teaching this year as compared with last year?

3. Please describe how you have integrated technology into your classrooms this year. What have been your greatest successes with technology integration?


5. Describe your use of BCPS One with your students. What about for administrative or planning activities?

Student impact




8. What sort of professional development have you received in terms of the STAT initiative?

9. Were you adequately prepared to implement STAT this year? Why or why not?

10. What professional development do you feel you are still in need of in order to implement

STAT? Lighthouse School preparation


12. What additional preparation, if any, do you recommend for future Lighthouse schools? STAT Teacher perceptions

13. How, if at all, has your STAT teacher supported your implementation of STAT (beyond PD sessions)?

14. What are the benefits of the STAT teacher role?



15. What areas of opportunity, if any, are there with the STAT teacher program? Principal role

16. How, if at all, has your principal supported your implementation of STAT? STAT perceptions





Appendix F: OASIS-21 Instrument Classroom Environment

• Information and communications that support independent thinking are highly visible in the classroom. □ Not observed □ Somewhat □ Extensive

• Information and resources that reflect content being taught is visibly displayed in classroom.

□ Not observed □ General Subject □ Lesson-specific

• Students move around the room independently acquiring material and resources. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Students utilize different work spaces for different learning environments (e.g. collaborative,

independent, receiving direct instruction). □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

Student Engagement

• Students using digital tools for learning. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Multiple modes of student responses (e.g. verbal, written, through technology, active votes, texting, physical response.) □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Independent work. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Collaborative learning.

□ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Student discussion. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

P21 Skills

• Problem solving. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Project-based approaches to instruction.


• Inquiry-based approaches to instruction. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Learning incorporates authentic/real world contexts.




Teacher Practice • Teachers acting as coach/facilitator. (Teacher facilitates the efficient and effective use of

digital tools and content.) □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Teacher presentation. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Higher-order instructional feedback given.


• Communication is initiated by students. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Higher-level questioning. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

• Flexible grouping based on student and task needs. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively



Appendix G: OASIS-21 Reference Guide Classroom Environment Student Engagement

Information supporting independent thinking • Quotes, slogans conveying that inquiry is valued

Multiple modes of student responses • Verbal, written, through technology, active votes,

texting, physical response • More than one mode used when responding to

other students or to teacher.

Information reflecting content being taught • Dependent on subject matter of lesson Ex: Lesson is on multiplication and a poster conveying steps for multiplication is displayed

Students using digital tools • Using devices independently or in group • Watching a video, reading, writing Non-ex: Teacher using of digital tools

Students move around the room independently • Students acquire materials needed for a task or

project they’re working on. Non-ex: Teacher directs students to obtain notebooks from the bookshelf.

Independent work • Students working alone on an assignment or

practicing content Non-ex: Students working on non-instructional task should not be coded

Students utilize different work spaces • Spaces for collaboration, independent work, etc.

are utilized by students • At least two different workspaces are being used Ex: Students working in a group at a cluster of desks while another group is seated on a reading mat doing independent work. Non-ex: All students seated in front of white board for teacher presentation, though other areas are present.

Collaborative learning • Students working in pairs or small groups to

complete a task or project. • Involves collaboration, helping each other. Non-ex: Students talking to each other on topic not related to the lesson.

Student discussion • Discussion amongst students (pairs, groups, class)

on a prompted topic or higher-level question Non-ex: collaborating to complete a task.

Ratings: Not observed (NO): Strategy never observed. Rarely (R): Received little emphasis, not a dominant instructional or learning component Somewhat/Occasionally (S/O): Receives modest emphasis or time in class Frequently (F): Receives substantial emphasis or time in class, dominant component Extensive(ly) (E): Highly prevalent in class, strongly emphasized



P21 Skills Teacher Practice

Problem solving • Students work together to solve problems • May be prompted by teacher, but teacher is not

directly involved. • Higher standard than problems involving recall. • Multiple resources used, using resources

effectively, critical thinking involved Non-ex: Mathematics problems.

Teacher as coach/facilitator. • Teacher facilitates the efficient and effective use

of digital tools and content. • Teacher is supportive Non-ex: Teacher disciplining students.

Project-based approaches • Instructional focus is centered on an inquiry or

question • Projects may result in tangible product (research

report, presentation, etc.) • Students are seen working on the project Non-ex: Project as part of the day’s lesson.

Teacher presentation • Teacher lecture, teacher offering direct instruction • Do not code classroom management.

Inquiry-based approaches • Students explore a question/topic/theme in-depth,

develop and ask further questions, and conduct research and problem-solve to answer the questions

Ex: Students given a topic to explore, students develop questions, use the Internet to research the topic.

Higher-order instructional feedback • Feedback related to learning process • Provides elaborative feedback • Offers an explanation, provides new information Ex: Teacher agrees that student response is correct, then extends student response by adding new information. Non-ex: Only stating correctness of response and moving on. Motivational/encouraging phrases.

Authentic/real world contexts • Problems that students investigate may relate (or

stem from) problems students can relate to in their own world

• Lesson or problems are specifically tailored to students’ world.

Communication is initiated by students • Asking questions of peers or teacher • Communicate beyond what is asked

Higher level questioning • Questions beyond factual recall • Questions that stimulate discussion

Ex: Questions that involve producing an explanation, providing an example, making a prediction, compare/contrast. Non-ex: Questions that involve memorization to produce a correct answer.

Flexible grouping of students • Grouping based on ability level • Grouping based on tasks • Differentiated instruction

Will likely need to ask teacher how student groups were formed.



Appendix H: OASIS-21 Results Classroom Environment

Not Observed Somewhat Extensive M SD

% % % Information and communications that support independent thinking are highly visible in the classroom.

Fall 2014 50 37.5 12.5 1.63 0.71 Spring 2015 45 45 10 1.65 0.66

Not Observed

General Subject

Lesson Specific M SD

% % % Information and resources that reflect content being taught is visibly displayed in classroom.

Fall 2014 50 37.5 12.5 1.63 0.71 Spring 2015 5 52.5 42.5 2.38 0.59

Not Observed Rarely Occasionally Frequently Extensively M SD

% % % % % Students move around the room independently acquiring materials and resources.

Fall 2014 52.5 22.5 12.5 7.5 5 1.90 1.19 Spring 2015 62.5 20.0 15 2.5 0 1.58 0.84

Students utilize different work spaces for different learning environments Fall 2014 40 5 22.5 25 7.5 2.55 1.43

Spring 2015 42.5 30 12.5 12.5 2.5 2.03 1.14



Student Engagement


% % % % % Students using digital tools for learning.

Fall 2014 22.5 12.5 17.5 32.5 15 3.05 1.41 Spring 2015 20 25 12.5 32.5 10 2.88 1.34

Multiple modes of student responses. Fall 2014 22.5 12.5 35 30 0 2.73a 1.13

Spring 2015 22.5 32.5 40 5.0 0 2.28 0.88 Independent work

Fall 2014 15 0 32.5 42.5 10 3.33 1.16 Spring 2015 22.5 12.5 32.5 22.5 10 2.85 1.29

Collaborative learning.

Fall 2014 57.5 22.5 7.5 12.5 0 1.75 1.06 Spring 2015 55 12.5 12.5 12.5 7.5 2.05 1.38

Student discussion.

Fall 2014 82.5 7.5 7.5 2.5 0 1.3 0.72 Spring 2015 65 12.5 17.5 5 0 1.63 0.95

a p = .05 P21 Skills


% % % % % Problem solving.

Fall 2014 90 7.5 2.5 0 0 1.13 0.4 Spring 2015 75 5 5 7.5 7.5 1.68a 1.31

Project-based approaches to instruction. Fall 2014 92.5 0 2.5 0 5 1.25 0.93

Spring 2015 75 0 5 7.5 12.5 1.83a 1.50 Inquiry-based approaches to instruction.

Fall 2014 90 2.5 2.5 2.5 2.5 1.25 0.84 Spring 2015 75 5 0 7.5 12.5 1.78 1.48

Learning incorporates authentic/real world contexts. Fall 2014 50 27.5 20 0 2.5 1.78 0.95

Spring 2015 42.5 20 12.5 12.5 12.5 2.33a 1.46 a p < .05



Teacher Practice


% % % % % Teacher acting as coach/facilitator.

Fall 2014 20 17.5 30 25 7.5 2.83 1.24 Spring 2015 7.5 17.5 25 32.5 17.5 3.35 1.19

Teacher presentation. Fall 2014 27.5 15 27.5 20 10 2.7 1.34

Spring 2015 42.5 22.5 12.5 15 7.5 2.23 1.35 Higher-order instructional feedback given.

Fall 2014 37.5 17.5 32.5 10 2.5 2.23 1.14 Spring 2015 37.5 17.5 20 15 10 2.43 1.39

Communication is initiated by students. Fall 2014 12.5 47.5 35 2.5 2.5 2.35b 0.83

Spring 2015 57.5 30 12.5 0 0 1.55 0.71 Higher-level questioning.

Fall 2014 12.5 17.5 35 32.5 2.5 2.95a 1.06 Spring 2015 47.5 5 30 2.5 15 2.33 1.47

Flexible grouping based on student and task needs. Fall 2014 62.5 10 12.5 7.5 7.5 1.88 1.32

Spring 2015 75 15 5 2.5 2.5 1.43 0.90 a p < .05, b p < .01



Appendix I: Lighthouse School Behavioral Data Office Referrals 2013-2014 2014-2015 Lighthouse school Semester 1 Semester 2 Total Semester 1 Semester 2 Total Chase Elementary 2 4 6 1 7 8 Church Lane 0 1 1 0 0 0 Edmondson Heights 3 1 4 2 8 10 Halstead Academy 3 7 10 1 0 1 Hawthorne Elementary 28 20 48 1 11 12 Joppa View Elementary 9 9 18 5 5 10 Lansdowne Elementary 2 4 6 1 2 3 Rodgers Forge 1 1 2 0 1 1 Total 48 47 95 11 34 45 Note: No data was available for Fort Garrison the data warehouse for the selected grades, possibly due to the school not tracking the data in the system. Mays Chapel opened in the 2014-2015 school year, therefore it is not included in the comparison. Suspensions 2013-2014 2014-2015 Lighthouse school Semester 1 Semester 2 Total Semester 1 Semester 2 Total Chase Elementary 1 1 2 1 2 3 Church Lane 0 1 1 0 0 0 Edmondson Heights 3 1 4 2 5 7 Halstead Academy 3 4 7 1 0 1 Hawthorne Elementary 0 1 1 1 0 1 Joppa View Elementary 0 1 1 0 0 0 Lansdowne Elementary 0 3 3 1 2 3 Rodgers Forge 1 1 2 0 1 1 Total 8 13 21 6 10 16 Note: No data was available for Fort Garrison the data warehouse for the selected grades, possibly due to the school not tracking the data in the system. Mays Chapel opened in the 2014-2015 school year, therefore it is not included in the comparison. Attendance 2013-2014 2014-2015 Lighthouse school Semester 1 Semester 2 Semester 1 Semester 2 Chase Elementary 66.0% 63.4% 62.5% 67.1% Church Lane 80.2% 65.1% 60.2% 77.4% Edmondson Heights 71.9% 68.5% 59.3% 63.8% Fort Garrison 79.8% 69.8% 73.5% 74.0% Halstead Academy 64.7% 55.7% 62.0% 59.5% Hawthorne Elementary 60.5% 56.7% 56.5% 63.3% Joppa View Elementary 86.5% 74.9% 77.0% 76.0% Lansdowne Elementary 68.1% 59.6% 57.4% 43.1% Rodgers Forge 89.4% 86.1% 85.0% 83.6% Total 75.1% 67.2% 66.4% 67.7% Note: Attendance was calculated by dividing the number of days a student was recorded was present by the number of days the student was enrolled. If attendance was not taken on a given day, the students are recorded as being present. Mays Chapel did not open until school year 2014-2015, therefore it is not included in the comparison.



Appendix J: Descriptive Statistics and Frequencies of Responses to S.T.A.T. Teacher

Program Survey (April 2015) I have participated in the following mode(s) of professional learning facilitated by our S.T.A.T. Teacher.

Participant None Large Group Small Group Individual/ 1:1

Support Independent

Learning % % % % % All teachers 1.94 93.36 77.86 61.37 47.71

LH teachers* 0 94.83 91.38 82.76 75.86 Non-LH teachers 2.05 93.28 77.07 60.12 46.07

*p < .001 Indicate your level of agreement or disagreement with each of the following statements. 1. The S.T.A.T. Teacher in my school is accessible to me.

Participant No basis to

assess Strongly Disagree Disagree Agree

Strongly Agree M SD

% % % % % All teachers 1.37 3.43 4.83 27.32 63.06 3.52 0.75

LH teachers 0.0 4.6 1.1 16.7 77.6 3.67* 0.72 Non-LH teachers 1.4 3.4 5.0 27.9 62.2 3.51 0.75

* p < .001 2. The S.T.A.T. Teacher in my school follows through on requests.



Strongly Agree M SD

% % % % % All teachers 4.1 3.21 3.97 22.71 66.01 3.58 0.73


* p < .01 3. The S.T.A.T. Teacher in my school models effective instructional strategies.



Strongly Agree M SD

% % % % % All teachers 3.84 4.22 6.16 28.08 57.69 3.45 0.80


* p < .001



4. The S.T.A.T. Teacher in my school plans and/or facilitates meetings effectively.



Strongly Agree M SD

% % % % % All teachers 2.92 3.94 6.19 27.13 59.82 3.47 0.79


* p < .001 5. The S.T.A.T. Teacher in my school provides/directs me to useful resources.



Strongly Agree M SD

% % % % % All teachers 4.35 4.0 6.67 27.19 57.78 3.45 0.80


* p < .001 6. The S.T.A.T. Teacher in my school supports the use of data to inform instruction to meet students’ needs.



Strongly Agree M SD

% % % % % All teachers 10.67 3.3 4.89 30.5 50.63 3.44 0.76


* p < .001 7. The S.T.A.T. Teacher in my school has helped me create a more learner centered environment in my classroom.



Strongly Agree M SD

% % % % % All teachers 12.55 4.54 10.71 27.67 44.54 3.28 0.87


* p < .001 8. The S.T.A.T. Teacher in my school provides coaching on how to integrate technology into instruction.



Strongly Agree M SD

% % % % % All teachers 7.56 5.11 9.56 30.37 47.4 3.30 0.87


* p < .001



9. I trust my S.T.A.T. Teacher to maintain confidentiality.



Strongly Agree M SD

% % % % % All teachers 5.65 5.08 5.85 24.65 58.77 3.45 0.83


* p < .001 As part of my professional development this year, I have participated in the following learning opportunities supported by the S.T.A.T. Teacher One-on-one professional discussions/consultations with the S.T.A.T. Teacher Participant % All teachers 65.28

LH teachers 84.48 Non-LH teachers 64.16

Individual, team, or departmental planning sessions with the S.T.A.T. Teacher % All teachers 72.55


Training or workshop(s) facilitated by the S.T.A.T. Teacher % All teachers 76.46


Developed my teacher development plan with assistance from the S.T.A.T.

Teacher % All teachers 29.51


Learning walk or instructional walk-through facilitated by the S.T.A.T.

Teacher % All teachers 45.08


Analysis of data with the S.T.A.T. Teacher % All teachers 39.49


Observed another teacher’s classroom facilitated by the S.T.A.T. Teacher % All teachers 23.82




Study group or lesson study with the S.T.A.T. Teacher Participant % All teachers 20.08


Observed the S.T.A.T. Teacher model teaching or conduct a demonstration

lesson % All teachers 20.04


Developed an SLO with assistance from the S.T.A.T. Teacher % All teachers 36.88


None of the above % All teachers 4.84




Appendix K: Descriptive Statistics and Frequencies of Responses to Lighthouse Teacher Survey

Strongly disagree

Somewhat disagree

Somewhat agree

Strongly agree

M SD % % % % I was able to apply what I learned during professional development in my classrooms.

6.67 4.44 31.11 57.78 4.29 1.13

I received sufficient professional development on the use of technology in my classroom.

13.33 16.67 28.89 41.11 3.68 1.48

I received sufficient professional development on the creation of a learner-centered environment.

12.22 11.11 28.89 47.78 3.89 1.43

I am skilled at engaging my students in collaborative learning activities using technology as a resource or tool.

3.33 4.44 31.11 61.11 4.42 0.96

My students have improved in their mastery of CCSS this year.

2.22 3.33 44.44 50.00 4.37 0.84

My students have improved in their mastery of P21 skills this year.

2.25 2.25 34.83 60.67 4.49 0.81

The student behavior in my classroom has improved this year.

3.33 14.44 44.44 37.78 3.99 1.13

I was adequately informed of the expected role of my school as a Lighthouse School.

11.11 11.11 57.78 50.00 3.94 1.40

I was adequately prepared to support my school in fulfilling its role as a Lighthouse School.

6.67 12.22 30.00 51.11 4.07 1.27

I feel my school was successful this year in fulfilling its role as a Lighthouse School.

4.44 3.33 8.89 83.33 4.63 0.99

Never

At least once per month

At least once a week

More than

once a week M SD

% % % % How often did your students collaborate with

other students on an in-class activity or investigation? 0.00 3.33 75.56 0.00 3.72 0.52 How often did you use BCPS one to: deliver instruction customized to students’ needs? 2.22 6.67 63.33 0.00 3.52 0.72 develop formative assessments? 22.22 27.78 16.67 0.00 2.44 1.02 develop assignments? 10.00 11.11 43.33 0.00 3.12 0.97 post homework assignments? 82.22 8.89 5.56 0.00 1.32 0.79



Never Rarely Moderately Frequently M SD % % % %

To what extend did you use the following types of teaching practices this year?

Direct instruction/lecture. 0.00 17.78 33.33 0.00 3.16 0.70 Cooperative learning. 0.00 0.00 84.44 0.00 3.84 0.36

Student projects. 0.00 5.56 58.89 0.00 3.53 0.60 Individualized learning (at desk or

computer). 0.00 3.33 71.11 0.00 3.68 0.54

Not at all Minimal Moderate

Fairly strong

Very strong M SD

% % % % % To what degree is the use of

technology an integral part of your teaching practices this year?

0.00 0.00 5.56 18.89 75.56 4.70 0.57

To what degree is the use of technology part of your instructional planning and administration (preparing lessons, grading, data management, etc.)?

0.00 0.00 4.44 17.78 77.78 4.73 0.54

running head: bcps s.t.a.t. year one evaluation …...nov 17, 2015 · s.t.a.t. year one summative...

Documents