running head: bcps s.t.a.t. year three evaluation …...key inputs to the initiative are represented...

Running head: BCPS S.T.A.T. YEAR THREE EVALUATION REPORT 1

Report for Baltimore County Public Schools:

Students and Teachers Accessing Tomorrow – Year Three Evaluation Report

Jennifer R. Morrison, Ph.D.

Steven M. Ross, Ph.D.

Catherine T. Wilson, Ed.D.

Joseph M. Reilly, M.A.

Jane M. Eisinger, M.S.

Gavin C. Latham, B.A.

Winnie Tam, M.S.

Alan C. K. Cheung, Ph.D.

Center for Research and Reform in Education (CRRE)

Johns Hopkins University

July 2017

BCPS S.T.A.T. YEAR THREE EVALUATION REPORT 2

CRRE – JHU July 10, 2017

Contents Executive Summary .............................................................................................................................. 3

S.T.A.T. Year Three Evaluation Report.............................................................................................. 6

Method................................................................................................................................................... 7

Participants and Design .................................................................................................................... 7

Data Sources and Instruments........................................................................................................ 10

Results ................................................................................................................................................. 14

Professional Development ............................................................................................................. 14

Measurable Outcomes: Classroom Environment ......................................................................... 30

Measurable Outcomes: Teacher Practice ...................................................................................... 33

Measurable Outcomes: Digital Content ........................................................................................ 42

Measurable Outcomes: Student Engagement ............................................................................... 53

Measurable Outcomes: P21 Skills ................................................................................................. 66

Goals: Student Achievement.......................................................................................................... 71

Perceptions of the S.T.A.T. Initiative ............................................................................................ 83

Conclusion........................................................................................................................................... 93

Appendix A: Classroom Teacher Survey .......................................................................................... 98

Appendix B: Descriptive Statistics and Frequencies of Responses to Classroom Teacher Survey

(2017) ................................................................................................................................................ 101

Appendix C: Principal Interview Protocol ...................................................................................... 108

Appendix D: S.T.A.T. Teacher Interview Protocol ........................................................................ 110

Appendix E: Teacher Focus Group Protocol .................................................................................. 112

Appendix F: OASIS-21 .................................................................................................................... 114

Appendix G: OASIS-21 Reference Guide ...................................................................................... 116

Appendix H: OASIS-21 Results ...................................................................................................... 118

Appendix I: Lighthouse School Behavioral Data ........................................................................... 137

Appendix J: Descriptive Statistics and Frequencies of Responses to BCPS Climate Survey ..... 138



Executive Summary:

S.T.A.T. Year Three Evaluation Report

The purpose of the present study is to evaluate the ongoing implementation and outcomes

of the Students and Teachers Accessing Tomorrow (S.T.A.T.) initiative in Baltimore County

Public Schools that began in the 2014-15 school year. Key components of S.T.A.T., as reflected

in the evaluation model (see Figure 1 presented in the main report), include professional

development and the resulting impact on measurable outcomes relating to the goals of improving

student achievement and preparing globally competitive students.

Roles and Practices of S.T.A.T. Teachers and Classroom Teachers

Classroom teachers continue to view S.T.A.T. teachers as highly valuable, critical to the

initiative’s success, and an important resource as they (the teachers) gain further experience with

technology integration and changing instructional practices. Continuing improvement needs are

engaging the S.T.A.T. teachers according to program expectations (i.e., not for incidental or

unrelated school assignments) and increasing the consistency of S.T.A.T. teacher roles and

responsibilities across schools. In addition, S.T.A.T. teachers expressed interests in additional PD

pertained to incorporating technology in the curriculum, conducting small-group instruction,

managing behavior, and engaging all teachers in their building.

Principals and teachers described a wealth of support and preparation for S.T.A.T.

implementation. The vast majority of teachers positively viewed their principal’s role and support.

Nearly all principals were satisfied with how their teachers were implementing S.T.A.T. As would

be expected, isolated concerns were raised by some participants, especially with the amount of

time needed to plan for technology integration in the midst of many other district and school

programs. Teachers expressed the need for future PD on using digital tools and programs,

including BCPSOne, and P21 skills. A positive development is the increased sharing of model

lessons and common lesson planning sessions at many of the S.T.A.T. schools.

Impact of S.T.A.T. on Measurable Outcomes

Classroom environment. Across the eight subgroups that made up the cohorts, only

isolated differences between the baseline and most recent observations were evidenced and none

(e.g., students using workspaces) that appear central to S.T.A.T. implementation. Seemingly, a

usage plateau for some components is reached early in implementation (Year 1 or 2), with a

minority of teachers remaining holdouts over time. This finding would be consistent with the logic

model with the majority of changes to the classroom environment exhibited during Year 1.

Teacher practice. As conveyed through multiple data sources, teacher practices have

changed substantively as a result of S.T.A.T. Principals in all groups described “mindset changes”

in their teachers, referencing increased collaboration with peers and more time spent in individual

and group planning. Interviews and focus groups with all participant groups conveyed that teachers

were increasingly engaging in meaningful integration of technology, exploring new strategies and

programs, and in general personalizing instruction and creating interactive learning for their

students. In addition, the introduction of digital tools (i.e., initial implementation of S.T.A.T.) has

had a substantial early impact on shifting pedagogy from teacher-centered approaches



(presentations, recitation, prompted discussion) to student-centered approaches

(coaching/facilitation and personalized/individualized work). Increases in the quality rather than

quantity of priority strategy usage will become the important goal as the initiative continues.

Digital content. Results from surveys, interviews, and focus groups indicated that all

participant groups perceived an improvement in the access to and use of digital content and

resources. Technology integration was described by all participant groups as progressing each year

and they also noted the positive impacts on students, such as taking ownership of their learning,

improving technology skills, and increasing engagement. Some teachers, though, are experiencing

challenges with designing and delivering lessons that promote P21 skills and higher-order learning

in general. Additional challenges include frequent reports of students’ off-task or inappropriate

uses of devices, technical issues and having inadequate planning time or PD to accomplish

instructional goals. These issues seem typical but also resolvable as a systemic technology

integration initiative expands and matures.

Student engagement. There were mixed findings regarding an impact on student

engagement. The observation data reflected high student engagement in using digital tools for

learning, particularly for independent work, and frequent informal interactions between students

as they completed independent work and projects. In addition, principals and S.T.A.T. teachers

perceived increased student engagement, bolstered by student-centered and personalized learning,

as a beneficial impact of S.T.A.T. Classroom teachers, although mostly in agreement, conveyed

more mixed reactions. Some believed that engagement had decreased somewhat over time.

Possible reasons could be some eroding of the novelty of personalized learning and technology.

Other teachers noted the need to design technology-enhanced lessons that would be more

challenging and interesting for students. Reactions to behavior were also mixed. Participant

reactions did not support an overall conclusion that behavior in the S.T.A.T. era has changed for

the better or worse. Rather, experiences seem teacher specific, and could largely depend on

students’ level of engagement with lessons. Although the behavioral data (suspensions and

referrals) have shown slight upticks over time, it is not clear to what degree, if at all, S.T.A.T.

contributed relative to other factors (e.g., changes in student demographics or discipline policies

P21 skills. One of S.T.A.T.’s longer-term goals is to promote student mastery of P21 skills.

Logically, technology integration can be useful for this purpose only to the degree that associated

instructional activities and lesson content address those competencies. Here, our observation data

and reactions by teachers and principals evidenced slower progress than for other implementation

components. These types of practices are time-consuming and procedurally difficult to implement,

so expected uses relative to traditional practices would naturally be low. In accord with S.T.A.T.

teachers’ perceptions, despite some perceived improvements this year, there is more work to be

done in preparing teachers to develop students’ P21 skills.

Goals: Student Achievement

MAP scores in Lighthouse and non-Lighthouse schools showed positive trends in student

achievement. Lighthouse students in Grades 1-2 improved in reading and mathematics across all

three years of implementation and Grade 3 exhibited improvements with the exception of the

present year. Notably, following the initiation of S.T.A.T., all grades exceeded the national average



in mathematics and reading scores. Non-Lighthouse Grades 1-3 demonstrated similar positive

trends, although only Grade 2 exceeded the national average mathematics scores.

Principals and S.T.A.T. teachers perceived that enhanced teaching practices and stronger

curricula were increasing mastery of CCSS. However, they were generally hesitant to attribute the

MAP gains directly or solely to S.T.A.T. We agree with this assessment for several reasons. First,

gains in achievement were not projected by the Logic Model this early in the implementation,

although we cannot rule out more rapidly occurring impacts. Second, there are numerous programs

and initiatives in BCPS, which could contribute to improved student achievement independently

of S.T.A.T. Regardless of the degree of S.T.A.T.’s contribution, the MAP outcomes importantly

reflect positive achievement trends for the district’s schools. The PARCC outcomes for Grade 3,

to be available in the fall of 2017, will provide further evidence for evaluating district performance.

Concluding Perspectives

In Year 3 of S.T.A.T., implementation progress clearly seems to be meeting and often

exceeding expectations. Prominent strengths of the initiative include increases in student

engagement, a focus on instruction and student-centered learning, and a variety of options and

resources available for teachers to support learning. Support and reactions by all stakeholder

groups, including teachers, principals, S.T.A.T. teachers, students, and parents, are remarkably

positive, especially for such a highly comprehensive initiative in its third year. Both personalized

learning and technology usage in instruction have gained broad acceptance as effective

orientations for teaching and learning.

Areas for improvement also emerged from our findings. First, the roles and responsibilities

of S.T.A.T. teachers still require clearer definitions. Second, many teachers feel that too much is

happening too quickly, and consequently, they need more time to digest, plan, and perfect what is

already on their plates. In particular, they desire additional support in designing meaningful lessons

and in integrating technology to teach higher-order skills. Third, strengthening and refining

technology policies and operations are suggested to address student off-task or inappropriate

behaviors, whether devices can be taken home by students at various grade levels, and technical

support needs for different programs and applications. Given the clear implementation progress,

the positive achievement trends on the MAP, and the strong buy-in to the initiative by most

teachers and principals, addressing these weaker areas can only work to make the present

accomplishments even stronger in Year 4.



S.T.A.T. Year Three Evaluation Report

The present study reports on the continued implementation of the Students and Teachers

Accessing Tomorrow (S.T.A.T.) initiative in Baltimore County Public Schools that began in the

fall of the 2014-15 school year. The S.T.A.T. initiative has now concluded its third year of

implementation with a focus on personalized learning for every student. The longitudinal

evaluation of S.T.A.T. focuses on key S.T.A.T. components and examines aspects of the S.T.A.T.

evaluation model (see Figure 1).

Figure 1. S.T.A.T. evaluation model.

Key inputs to the initiative are represented at the left by the professional development

provided to administrators, S.T.A.T. teachers, and classroom teachers. Key evaluation questions

related to professional development include:

1. What are the impacts and best practices in site-based and district-wide professional

development?

2. What are the roles, perceptions, and best practices of S.T.A.T. teachers?

3. What are the roles and best practices of other key participants (principals, curriculum

specialists, technology directors)?

Measurable intermediary outcomes, represented in the middle of the figure, are

hypothesized to be reflected in positive changes in classroom environment, teacher practice, access

and use of digital content, student engagement, and P21 skills. Culminating goals are increases in

student achievement, as measured by the Northwest Evaluation Association (NWEA) Measure of

Academic Progress (MAP) and Partnership for Assessment of Readiness for College and Careers



(PARCC) assessments and in students who graduate high school prepared to be globally

competitive in academic and P21 skills. Evaluation questions addressing the intermediary and

educational outcomes include:

1. What is the impact of S.T.A.T. on the classroom environment?

2. To what degree and how do teacher practices change over time?

3. What is the level of access of digital content by principals, teachers, students, and

parents?

4. To what degree does student engagement in learning increase over time?

5. To what degree do students demonstrate over time mastery of P21 skills?

6. To what degree do students grow over time in mastering CCSS?

Additional evaluation questions explored the experiences and perceptions of key

stakeholders and participants in S.T.A.T.

Method

Participants and Design

Participants included both Lighthouse and non-Lighthouse elementary and middle schools,

in addition to Lighthouse high schools. Grades within these schools are in various years of S.T.A.T.

implementation (see Table 1) and these groups are further discussed below.

Table 1. First year of S.T.A.T. implementation within Lighthouse and non-Lighthouse schools. Cohort 1: 2014-15 Cohort 2: 2015-16 Cohort 3: 2016-17

Lighthouse Grades 1 – 3 Lighthouse Grades K, 4, and 5 Non-Lighthouse Grades K, 4, and 5

Non-Lighthouse Grades 1-3 Lighthouse Grade 7

Lighthouse Grade 6 Non-Lighthouse Grade 6

Lighthouse Grades 9 – 12

Lighthouse elementary. The study included the ten Lighthouse elementary schools (see

Table 2) that began implementation of S.T.A.T. in the fall of 2014. Nine of the ten Lighthouse

elementary schools began implementing S.T.A.T. in the fall of 2014 in Grades 1-3, then in Grades

K, 4, and 5 in the fall of 2015. Mays Chapel, however, began implementing S.T.A.T. in all grades

during the fall of 2014.



Table 2. Characteristics of Lighthouse elementary school enrollment for the 2016-2017 school

year.

School name

S.T.A.T.

Grades

Total

Enrollment

Race/Ethnicity Free and

Reduced

Price Meals

(FARMS)

White

%

Black

%

Other1 %

Chase Elementary K - 5 348 61.8% 22.4% 15.8% 55.5%

Church Lane Elementary K - 5 454 2.0% 87.9% 10.1% 62.3%

Edmondson Heights Elementary K - 5 585 6.2% 78.5% 15.4% 67.0%

Fort Garrison Elementary K - 5 319 81.5% 8.8% 9.8% 6.0%

Halstead Academy K - 5 521 3.3% 88.7% 8.1% 79.7%

Hawthorne Elementary K - 5 515 31.7% 48.3% 20.1% 60.4%

Joppa View Elementary K - 5 792 49.4% 23.7% 26.9% 29.9%

Lansdowne Elementary K - 5 521 48.6% 23.8% 27.7% 72.6%

Mays Chapel Elementary2 K - 5 744 47.7% 16.9% 35.4% 21.9%

Rodgers Forge Elementary K - 5 428 81.8% 1.6% 16.6% 3.0%

All Lighthouse elementary

schools 5227 39.2% 40.6% 20.3% 46.0%

All BCPS elementary schools 55553 40.3% 36.8% 22.9% 47.7% 1 “Other” includes the following race/ethnicity categories: American Indian/Alaska Native, Asian, Hispanic/Latino,

Native Hawaiian or Other Pacific Islander, and Two or More Races. 2School opened in fall 2014

The 10 Lighthouse schools on average exceed the proportion of Black students in all BCPS

elementary schools. The Lighthouse schools also have a slightly lower percentage of FARMS

eligible students, though four have greater than 60% of FARMS eligible students.

Non-Lighthouse elementary schools. Non-Lighthouse elementary schools began

implementing S.T.A.T. in Grades 1-3 during the fall of the 2015-16 school year, then Grades K, 4,

and 5 in the fall of the 2016-17 school year. Participants also included 10 Phase 2 elementary

schools, a subsample of non-Lighthouse elementary schools. These schools were selected by

BCPS for observation primarily based on geographic location within the district to ensure schools

were selected across different areas. In comparison with the Lighthouse elementary schools, these

schools have a higher average percentage of students eligible for FARMS (53.1% as compared

with Lighthouse elementary 46.0%) and a lower proportion of Black students (36.4% compared

with Lighthouse elementary 40.6%). Demographics for these schools are presented in Table 3.



Table 3. Characteristics of Phase 2 elementary school enrollment for the 2016-17 school year.

School name

S.T.A.T.

Grades

Total

Enrollment


Reduced

Price Meals

(FARMS)

White

%

Black

%

Other1 %

Baltimore Highlands Elementary K - 5 712 27.0% 22.9% 50.0% 76.8%

Colgate Elementary K - 5 432 43.3% 8.1% 48.6% 77.8%

Hampton Elementary K - 5 585 62.2% 13.7% 24.1% 18.3%

Harford Hills Elementary K - 5 431 53.1% 29.5% 17.3% 41.8%

Hernwood Elementary K - 5 451 1.8% 89.6% 8.5% 60.3%

Johnnycake Elementary K - 5 699 7.6% 60.1% 32.3% 70.1%

Prettyboy Elementary K - 5 423 91.5% 2.1% 6.4% 6.4%

Reisterstown Elementary K - 5 601 26.6% 37.4% 36.0% 60.6%

Sandalwood Elementary K - 5 563 8.7% 70.0% 21.3% 84.4%

Vincent Farm Elementary K - 5 903 50.7% 27.9% 21.4% 31.0%

All Phase 2 elementary schools 5800 36.0% 36.4% 27.6% 53.1%

All BCPS elementary schools 55553 40.3% 36.8% 22.9% 47.7% 1 “Other” includes the following race/ethnicity categories: American Indian/Alaska Native, Asian, Hispanic/Latino,

Native Hawaiian or Other Pacific Islander, and Two or More Races.

Lighthouse middle schools. S.T.A.T. was also implemented in seven Lighthouse middle

schools in grade 6 during the fall of the 2015-16 school year, then grade 7 in the fall of the 2016-

17 school year. Demographics for these schools are presented in Table 4.

Table 4. Characteristics of Lighthouse middle school enrollment for the 2016-2017 school year.

School name

S.T.A.T.

Grades

Total

Enrollment


Reduced Price Meals

(FARMS)

White

%

Black

%

Other1 %

Cockeysville Middle 6 - 7 802 46.5% 25.2% 28.2% 29.9%

Dumbarton Middle 6 - 7 1145 64.6% 10.7% 24.6% 18.6%

Pikesville Middle 6 - 7 965 25.2% 56.1% 18.7% 44.4%

Ridgley Middle 6 - 7 1233 67.6% 8.7% 23.8% 9.2%

Sparrows Point Middle 6 - 7 548 83.0% 7.8% 9.1% 44.9% Stemmers Run Middle 6 - 7 740 55.7% 27.0% 17.2% 63.9%

Windsor Mill Middle 6 - 7 495 2.2% 91.3% 6.4% 61.6%

All Lighthouse middle schools 5928 51.7% 28.1% 20.1% 34.1%

All BCPS middle schools 24550 40.6% 39.1% 20.3% 44.7% 1 “Other” includes the following race/ethnicity categories: American Indian/Alaska Native, Asian, Hispanic/Latino,


The Lighthouse middle schools appear to have a lower proportion (34.1%) of FARMS eligible

students as compared with all middle schools in the BCPS system (44.7%). Two of the seven

Lighthouse schools, though, have over 60% FARMS-eligible students. In addition, the Lighthouse

middle schools have a lower proportion of Black students (28.1%) students as compared with all

middle schools in the district (39.1%).

Non-Lighthouse middle schools. Non-Lighthouse middle schools began implementing

S.T.A.T. in Grade 6 during the fall of the 2016-17 school year. Participants also included seven

Phase 2 middle schools, a subsample of non-Lighthouse middle schools. These schools were



selected by BCPS for observation primarily based on geographic location within the district to

ensure schools were selected across different areas. In comparison with the Lighthouse middle

schools, these schools have a higher average percentage of students eligible for FARMS (48.8%

as compared with Lighthouse middle 34.1%) and a higher proportion of Black students (41.1%

compared with Lighthouse middle 28.1%). Demographics for these schools are presented in Table

5.

Table 5. Characteristics of Phase 2 middle school enrollment for the 2016-17 school year.

School name

S.T.A.T.

Grades

Total

Enrollment


Reduced

Price Meals

(FARMS)

White

%

Black

%

Other1 %

Catonsville Middle 6 841 58.4% 19.4% 22.2% 25.6%

Deep Creek Middle 6 855 26.4% 55.6% 18.0% 71.7%

Franklin Middle 6 1316 34.3% 42.5% 23.3% 42.6%

Lansdowne Middle 6 755 31.7% 29.4% 38.9% 74.0%

Parkville Middle 6 1122 48.8% 31.6% 19.7% 40.9%

Pine Grove Middle 6 892 51.0% 31.7% 17.3% 35.1%

Southwest Academy 6 809 2.8% 73.2% 24.0% 61.2%

All Phase 2 middle schools 4894 35.0% 41.1% 23.9% 48.8%

All BCPS middle schools 24550 40.6% 39.1% 20.3% 44.7% 1 “Other” includes the following race/ethnicity categories: American Indian/Alaska Native, Asian, Hispanic/Latino,


Lighthouse high schools. S.T.A.T. was also implemented in three Lighthouse high schools

in Grades 9 - 12 during the fall of the 2016-17 school year. Demographics for these schools are

presented in Table 6.

Table 6. Characteristics of Lighthouse high school enrollment for the 2016-2017 school year.

School name S.T.A.T. Grades

Total Enrollment


Reduced

Price Meals (FARMS)

Limited

English

Proficiency (LEP)

White

%

Black

%

Other1 %

Chesapeake High 9 - 12 965 31.2% 53.7% 15.1% 63.0% 0.4%

Owing Mills High 9 - 12 1013 10.9% 54.4% 34.7% 48.2% 27.0%

Pikesville High 9 - 12 837 36.0% 49.3% 14.7% 33.2% 0.5%

All Lighthouse high schools 2815 25.3% 52.6% 22.1% 48.8% 10.0%

All BCPS high schools 30736 39.8% 41.5% 18.5% 38.1% 3.5% 1 “Other” includes the following race/ethnicity categories: American Indian/Alaska Native, Asian, Hispanic/Latino,


As shown above, Lighthouse high schools differed from high schools in the BCPS system.

Specifically, these schools appear to have a higher proportion of Black students (52.6%) as

compared to BCPS overall (41.5%), a higher proportion of FARMS eligible students (48.8% vs.

38.1%), and a higher proportion of English language learners (10.0% vs. 3.5%).

Data Sources and Instruments

The mixed-method design included classroom observations, surveys, interviews, focus

groups, school behavior and attendance data, and student achievement data in order to evaluate



logic model components and perceptions of S.T.A.T. Measures, data sources, and timing for each

are displayed in Table 7.

Table 7. Data source, instrument, and timing.

Logic Model Component

Data Source Instrument Timing

Professional Development

Survey Classroom Teacher

Survey Spring 2016-17

Interviews and focus

groups

Interview and focus group

protocol Spring 2016-17

Classroom Environment

Classroom observations

Observation of Active

Student Instruction in

Schools of the 21st

Century (OASIS-21)

Fall and spring 2014-15*,

2015-16, 2016-17


groups



Teacher Practice

Classroom observations OASIS-21 Fall and spring 2014-15*,

2015-16, 2016-17


groups





Digital Content


groups





Student Engagement

Classroom observations OASIS-21 Fall and spring 2014-15*,

2015-16, 2016-17


groups





Suspensions, referrals,

attendance data BCPS data collection Spring 2016-17

P21 Skills

Classroom observations

OASIS-21

Fall and spring 2014-15*,

2015-16, 2016-17


groups





Student Achievement MAP BCPS data collection Spring 2016-17

*Note: time points of classroom observations varied by group.

Classroom Teacher Survey. The Classroom Teacher Survey (see Appendix A) was co-

developed by the Center for Research and Reform in Education (CRRE) and BCPS. The survey

consisted of 18 closed-ended items focusing on professional development, teacher practices,

perceived student impact, and technology integration. In addition, two open-ended items asked

participants’ successes and challenges with technology integration, and one question asked

participants’ experiences with students misusing technology. For teachers in Lighthouse high

schools only, three additional questions were asked concerning Lighthouse school preparation

practices. In total, the survey was administered to 777 content-area classroom teachers across the



participating schools with a completion rate of 84.4%. Frequencies and descriptive statistics for

the spring 2017 administration of the Classroom Teacher Survey are presented in Appendix B.

Interviews and focus groups. Phone interviews were conducted with a randomly selected

subsample of principals (n = 22) and S.T.A.T. teachers (n = 21) during spring 2017 from across

the BCPS S.T.A.T. schools. Additionally, in-person focus groups (n = 23) were conducted with

91 classroom teachers from a randomly selected subsample of BCPS S.T.A.T. schools. The

protocols for the principal and S.T.A.T. teacher interviews and the classroom teacher focus groups

(see Appendix C, Appendix D, and Appendix E, respectively) solicited perspectives on

professional development, the perceived impact of S.T.A.T. on measurable outcomes and

educational goals, and experiences and perceptions of the S.T.A.T. initiative.

Observation of Active Student Instruction in Schools of the 21st Century (OASIS-21).

The classroom observation instrument (see Appendix F) was co-developed by CRRE and BCPS.

The instrument integrated district-wide professional development goals for classroom instruction

with S.T.A.T.-specific interests and goals regarding technology applications of teaching and

learning. The observations focused on (a) student engagement, (b) the type of instructional

strategies employed, and (c) how and to what degree technology devices are employed. A reference

guide for the OASIS-21 instrument items is presented in Appendix G.

The procedure employed involved trained observers visiting 20 elementary schools, 14

middle schools, and 3 high schools. Observers randomly selected four to six classrooms to observe

for 20 minutes each. The observers completed individual ratings of the frequency/pervasiveness

of particular practices, as well as classroom environment indicators (e.g., room arrangement,

information and resources available, etc.). With the exception of two classroom environment

items, observation items were recorded via a five-point scale that ranged from (1) Not Observed

to (5) Extensively Observed. A reliability study was conducted in the spring of the 2014-15 school

year. Two observers independently observed the same 380 classrooms for 20 minutes each using

the OASIS-21 instrument. Overall, there was 88.95% agreement in the independent observation

ratings. For those items where ratings differed, 100% of differences were by one scale point. The

overall inter-rater reliability consistency, as measured through Cronbach’s alpha, was α = 0.972.

In consistency estimates, values above .70 are deemed acceptable (Brown et al., 2004; Stemler,

2004).

A total of 177 classrooms were observed in the spring of 2017, resulting in 3,540 minutes

of direct classroom observations conducted in the Lighthouse elementary schools (n = 10), Phase

2 elementary schools (n = 10), and Lighthouse middle schools (n = 7), Phase 2 middle schools (n

= 7), and Lighthouse high schools (n = 3). Table 8 presents the distribution of classrooms observed.



Table 8. Summary of observations conducted. Time Point

Group

Fall

2014

Spring

2015

Fall

2015

Spring

2016

Fall

2016

Spring

2017 Total

Lighthouse Grades 1-3 40* 40 32 29 32 32 205

Lighthouse Grades K, 4, and 5

27*

26 25 25 103

Phase 2 Grades 1-3 40* 40 40 30 28 178

Phase 2 Grades K, 4, and 5 28* 24 52

Lighthouse Grade 6 28* 28 16 15 87

Lighthouse Grade 7 12* 13 25

Phase 2 Grade 6 28* 28 28 84

Lighthouse high school 12* 12 24

Total 40 80 127 151 183 177 758

* Denotes baseline observations

In both Lighthouse and Phase 2 elementary schools, classroom instruction of ELA was observed

most frequently during spring 2017 observations. For Lighthouse and Phase 2 middle schools as

well as Lighthouse high schools, observations were evenly distributed across all subject areas.

Results focus on presenting spring 2017 observations as compared with baseline observations

where applicable. Frequencies and descriptive statistics for classroom observation results are

presented in Appendix H.

School behavioral data. Data consisting of attendance, office referrals, and suspensions

were collected for Lighthouse and non-Lighthouse elementary and middle schools, as well as

Lighthouse high schools. Data were collected for the pre-program year for each group through the

present (2016-17) school year. School behavioral data are presented in Appendix I.

Achievement data. Student achievement data on the Northwest Evaluation Association

(NWEA) Measure of Academic Progress (MAP) assessments for reading and mathematics were

collected for Lighthouse and non-Lighthouse Grades 1-3. The MAP assessments are standardized

computer-based exams administered to BCPS students to benchmark progress in reading and

mathematics in preparation for the state’s annual summative achievement assessment (PARCC).

MAP data for the 2013-14 school year through the current 2016-17 school year was collected to

examine the achievement of BCPS S.T.A.T. students from the initiative’s inception through the

2016-17 school year. Where applicable, student performance is compared to national averages on

these assessments.

BCPS Climate Survey. The BCPS Climate Survey was administered to all students in

Grades 4 – 12 and all parents within BCPS. Three of the survey items were specific to the S.T.A.T.

initiative. Student survey respondents were analyzed by cohort as students indicated their grade

level. Parent survey respondents were analyzed by school-level grouping (elementary, middle, and

high school), resulting in inclusion of all middle school and high school grades, though not all

grades may be currently implementing S.T.A.T. (see Table 10). Descriptive statistics and

frequencies are presented in Appendix J.



Table 10. Parent and student responses to S.T.A.T.-specific climate survey items. Parents

n

Students

n

Cohort 2 Lighthouse elementary Grades 4-5 1,444

Lighthouse middle Grade 6 1,758

Cohort 3

Non-Lighthouse elementary Grades 4-5 15,589

Non-Lighthouse middle Grade 6 5,708

Lighthouse middle Grade 7 1,927

Lighthouse high school 1,908

Elementary Lighthouse 606

Non-Lighthouse 5,191

Middle Lighthouse 670

Non-Lighthouse 1,575

High Lighthouse high school 135

Total 14,658 28,334

Results

The following sections present available results of as all measures related to the evaluation

model components. The results begin with perceptions and activities related to professional

development. This section is followed by results of data collected for measurable outcomes and

goals. The reader should note that results for all grades within the Lighthouse elementary school,

Mays Chapel, are reported within the Lighthouse Grades 1-3 group due to their implementation of

S.T.A.T. during the 2014-15 school year. Whenever possible, results are reported by Cohort (see

Table 1).

Professional Development

Preparation and support, including professional development (PD), was offered in a variety

of ways to schools implementing the S.T.A.T. initiative. This section first reviews how principals

and district administrators offer support to schools implementing S.T.A.T. Second, perceptions of

the PD offered to S.T.A.T. teachers is presented, followed by perceptions of the S.T.A.T. teacher

program. Third, perceptions regarding whether classroom teachers have been prepared to

implement S.T.A.T. is presented. This section concludes findings related to Lighthouse school

preparation and support.

Administrative support. Administrative support for the implementation of S.T.A.T. was

offered both by principals and by the district. Participants were asked to comment on how these

stakeholders have offered support to schools.

Principals. Principals cited support through professional development most frequently in

all groups. This support occurred in multiple ways, including setting or coordinating professional

development schedules, either for the school or for personalized learning; creating book and

standards studies; soliciting teacher input regarding professional needs; allowing teachers to

participate in “learning walks” around the school; as well as visiting other schools and conferences

to observe or learn about best practices. As one principal commented, “When we’re not seeing

innovative programming, we speak to it, give help and suggestions.”



Principals also frequently noted that providing “common planning time” allowed teachers

the “time necessary to dive into standards and figure out how to implement technology in the best

way possible to support units and deepen critical thinking” as well as to “collaborate.” One

principal observed:

We have been getting creative with using our free time to talk and collaborate about what

we want to see in the classroom. We’re allowing teachers to use this as evidence of

professional growth rather than formal evaluation and that creates buy in from many staff.

Principals also noted they were “collaborating more” with their staff and S.T.A.T. teachers.

Principals at all levels also reported supporting teacher’s needs by creating a “risk-free

culture” within their schools to allow teachers opportunities to experiment, try new ideas, reflect

on successes and challenges, and thoughtfully consider next steps. A Lighthouse elementary

principal summed up comments from a variety of groups, stating, “It took me understanding that

it is not my class but our class and our children that we want to succeed. I must give teachers

autonomy to transform their classrooms to what they think will work best.”

Finally, principals also noted that providing teachers the “resources needed” was another

way to effectively support teachers. One principal from a Lighthouse middle school summed up

principal support by stating: “I support them infra-structurally, support their time, and support their

growth.”

S.T.A.T. teachers. During interviews, S.T.A.T. teachers described their principals as having

provided support, focus, understanding, and feedback throughout the year. One Phase 2 middle

school teacher commented that the principal is “very supportive in that if there’s something that I

need, he’s going to try to get an answer for me, which is very good.” Another conveyed a level of

partnership with the principal:

We are like two peas in a pod. We talk every day and plan together. She is my greatest

cheerleader and I could never imagine the initiative being implemented well without that

relationship. We have the same vision and goals for the school. It has been a relationship

fostered with respect.

A third S.T.A.T. teacher echoed a similar sentiment of feeling strong support from the principal:

She’s instrumental. She’s our biggest cheerleader, biggest supporter, she bends over

backwards, and she’s always willing to come into the classroom and help. She’s always

there, giving timely feedback. She takes what she’s learned at her PD and brings it back to

school just like I do. She’s a problem solver and helps teachers talk things through to find

solutions.

A small number of these teachers indicated that while their principal’s time had been directed at

other issues, their support had typically been consistent and strong.



Classroom teachers. Teachers in all focus groups indicated their principals were highly

supportive of the implementation of the S.T.A.T. initiative. Principals were described as “student-

centered,” “balanced,” “helpful,” “encouraging,” “visionary,” “cheerleader,” and “enthusiastic.”

One teacher summed up a variety of other comments, stating, “I never feel like I’m being watched

for something that’s going to go wrong; I don’t mind taking a risk with technology, and I would

have before.”

Classroom teachers conveyed that principals frequently provided non-judgmental feedback

and had realistic expectations for their teachers. Such expectations included trying one new

practice at a time and being open to suggestions. In addition, many teachers conveyed that their

principal does not expect everything to be perfect in these early years.

However, a few teachers felt their principal was not as supportive as would be liked.

Several teachers across groups commented the principal did not provide enough planning time,

was not familiar with the program, and did not support teachers for technology-related discipline

issues.

Overall, though, the majority of teachers across all groups felt their principal was

instrumental in the implementation of the S.T.A.T. initiative, as typified in this classroom teacher’s

response:

The principal moves you forward; always has something to say that takes you to the next

step. Even if you step back a couple of times, she’s going to find something that gets you

moving forward again. She finds that way . . . something that makes sense to you.

District. Principals were asked to comment during interviews how others within the district

have supported the implementation of S.T.A.T. The most commonly reported themes centered on

technical support received, visits and meaningful feedback from central office, and professional

development support.

The majority of principals interviewed across groups felt that the technical support they

received to deal with issues was prompt and professional. One Phase 2 middle school summed this

up by stating, “All of our tech issues have been resolved very quickly. I’ve received answers within

24 hours, sometimes even less and it’s been wonderful to feel like I have that information at my

fingertips.”

However, three principals voiced some dissatisfaction in this area. One Phase 2 elementary

principal commented: “There really hasn’t been anyone who’s come along who has helped us

beyond the S.T.A.T. team and the leadership here.” Another Lighthouse middle school principal

noted that it “has been challenging when we ask for support and help and a person can’t always

come on site.” Another middle school principal from a Phase 2 school commented that they had,

until the spring, an IT person in the school daily and not having this person present on a regular

basis had been “a loss.”

Principals in all groups at all levels also commented upon the value of “outside visitors”

coming to the school and providing feedback, resources, or assistance. However, there were a few



principals who commented these visits had not been helpful or the frequency had declined to be

negligible. One Lighthouse middle school principal observed that because some visitors had never

worked in a school or served as an administrator, they “did not know what it’s like to run a school”

and had “unrealistic expectations.”

A final area that principals observed where others outside their schools provided support

was in supplying professional development. All groups noted that this was provided for teachers,

S.T.A.T. teachers, and principals either collectively or separately.

Lighthouse Elementary and Phase 2 Elementary S.T.A.T. teachers were overwhelmingly

positive in their responses regarding district-wide support that they have received. S.T.A.T.

teachers spoke of a community of individuals that was supportive and enabling. One Phase 2

elementary S.T.A.T. teacher remarked, “I have never felt more supported since this init iative

started.” Similarly, another noted,

There is a huge system of support around us, specifically within the Lighthouse School,

because they’re looking to us to provide that opportunity to show how things can be for

other schools that have not embarked on the journey with us yet.

Another commented,

That sense of community makes it feel like we can reach out to anybody, be it a principal,

or community superintendent. The culture of growth contributes to a feeling of success that

everyone is there to succeed and support one another.

S.T.A.T. teachers named several areas of district support that they valued, including professional

development, monthly S.T.A.T. meetings, and program feedback. They mentioned the Curriculum

Department, Organizational Development, Digital Learning, and Innovative Learning as valuable

resources for supporting the initiative. As one S.T.A.T. teacher commented, “The district support

is amazing, from the curriculum offices all of the way down to our tech offices. They’ve been

really good about giving us the level of support we need.”

Another S.T.A.T. teacher noted, “The Office of Digital Learning is incredible. The staff

there and the resource teachers there are absolutely incredible.”

S.T.A.T. Teachers. BCPS has provided training and support to S.T.A.T. teachers in order

for those in this position to support classroom teachers’ implementation of S.T.A.T. During

interviews, S.T.A.T. teachers were asked to describe the PD they have received specific to

S.T.A.T. Both Lighthouse and Phase 2 S.T.A.T. teachers indicated there were many avenues of

professional development available to them. Monthly meetings, school visits, conferences, online

resources, Lighthouse PD, summer workshops, and resource teachers were just some of the options

named. S.T.A.T. teachers clearly appreciated this access to training and support. One Phase 2

middle S.T.A.T. teacher commented,

Resource teachers have been amazing in what they’ve offered. My strength wasn’t

necessarily in technology initially. I had been a math teacher but I’ve seen how you can



collect data easily using the technology and how it can revolutionize our teaching. We also

meet monthly as S.T.A.T. teachers and that professional development has been really

helpful in connecting all the S.T.A.T. teachers so we have the opportunity to learn from one

another and hear what is going on at other schools, what is working and what we could

perhaps incorporate that we haven’t yet.

Many stressed the value of being able to select the sessions they attended based on content. Also

important was their ability to quickly share information they learned with the teachers in their

schools:

This year we’ve been able to choose based on what is in the school progress plan or what

our principal wants us to attend and that has been a huge game changer. I can turnkey that

PD and give to teachers to implement right away. It’s a shift about how we’re able to

personalize it for our needs and get it out quickly, which is huge.

The vast majority of S.T.A.T. teachers indicated feeling prepared to serve in their role. A

small number of S.T.A.T. teachers conveyed that while they may not have felt prepared initially,

having experience in their role for a year or more has helped them. Many described the strong PD

and support from their school community as being factors in their response. As one S.T.A.T.

teacher replied,

I do, I definitely do feel prepared, but I think it really helps that I have a supportive

administration. Everything is focused on our school progress plan, which is why I think

we’ve been so successful with our achievement. Teachers know my position is not

evaluative and so they feel comfortable coming to me and they appreciate being able to

have a say in the professional development that they receive. I also think the county does

a good job providing ongoing PD. They give opportunities to seek out help in whatever

area is needed.

Another commented,

Every time I have a question or my teachers have a question, I feel I have a resource to

find the answer whether that entails looking back to professional development resources I

have received to find the answer or whether it means I need to go and ask a peer or my

S.T.A.T. resource teacher, I know that if I don’t have the answer at hand, I will be able to

find it.

A third S.T.A.T. teacher replied, “I don’t have all of the answers, but I have a million and one

places, people, resources to go to, to seek out information and ideas. In that way, I feel incredibly

prepared.”

One of the Lighthouse high school S.T.A.T. teachers noted that they had received great

information about the initiative but sometimes did not feel that it was well-targeted for their school,

saying, “The initiative is modeled that it can work at all educational levels, when it really needs to

be more education-level targeted.” This S.T.A.T. teacher also observed that the initiative was



working very well in elementary and middle schools but implied that scheduling in high schools

made it more of a challenge.

S.T.A.T. teachers were also asked to describe PD they still needed to better serve their

schools. The majority of S.T.A.T. teachers across elementary and middle schools felt that they

would benefit from additional PD, although some felt that they were already adequately trained or

felt overwhelmed with resources and training and wanted time to catch up. One Phase 2 elementary

school S.T.A.T. teacher said, “I truly believe they have offered options and opportunities for every

area I could possibly want or need to develop my own practice as a S.T.A.T. Teacher,” while a

Lighthouse elementary S.T.A.T. teacher commented, “We need time to just perfect our practice.

With an overabundance of professional development, you can’t keep up, and there’s always

something new.”

Lighthouse and Phase 2 elementary S.T.A.T. teachers requested additional PD in areas that

included assisting below grade-level students, integrating technology in the curriculum, content

for kindergarten, and how to use new technology effectively. Middle school S.T.A.T. teachers also

provided a varied list of PD they would like to receive, which included training on ClassFlow,

small-group instruction, behavior management, and, like their elementary peers, the use of new

technology. In addition, several S.T.A.T. teachers from both elementary and middle schools

described a desire for PD to help them bring some teachers on board with the initiative and its

goals. As one volunteered, “There is always room to learn more about how to meet everyone’s

needs. There are still teachers who are not the risk takers and are harder to reach out to, so PD for

that would always be great.”

Several S.T.A.T. teachers also commented on their desire to learn new ways to deliver PD.

They were concerned about maintaining momentum and wanted to be able to supply their teachers

with information that remains fresh, interesting, and relevant each year.

Lighthouse high school S.T.A.T. teachers were focused on receiving additional direction

from their own school leadership and on learning how to be better leaders themselves. One

S.T.A.T. teacher from this group commented:

I would like to address the part where I am seen as a teacher leader. I have never been in

a leadership position and I would like to have more training on this so I know that I am

leading my teachers in the right direction.

S.T.A.T. teachers across all school levels indicated that their primary role in their school

was serving as an instructional coach, including providing professional development, lesson

planning, modeling, analyzing data, assisting teachers with technology, and offering general

support to teachers both instructionally and with morale.

S.T.A.T. teacher role. Many S.T.A.T. teachers described their role as that of a ‘Jack of all

trades,’ with a Phase 2 middle S.T.A.T. teacher stating, “I am here to support teachers really with

anything. Anything you need, come to me first and we’ll figure it out.” A Lighthouse elementary

S.T.A.T. teacher added, “I model, I co-teach, I plan weekly with each grade level just to make sure



they’re staying on track with pacing, but also making sure they’re integrating 21st Century skills,

technology and things of that nature.”

Some S.T.A.T. teachers described the importance of having built strong ties with their school ’s

teachers as a result of their role within the initiative. One Lighthouse elementary S.T.A.T. teacher

stated, “From morning until bedtime, I am a S.T.A.T. teacher. I’ve built great positive relationships

and they know I’m available to them all day.” A Lighthouse high school S.T.A.T. teacher

remarked,

One of my favorite parts about the S.T.A.T. role is working with eager teachers. From the

beginning, I had an idea of what teachers would have buy in on the initiative and take off

running with it. Those teachers are so willing to meet, lesson plan, and try new and

inventive ways to target instruction. Seeing their success has been the most rewarding part

of my role.

Lighthouse and Phase 2 S.T.A.T. teachers have offered a very broad array of activities

related to professional development, offering support both during the school day and sometimes

beyond. In describing the PD they have offered, one Lighthouse Middle school teacher

commented, “I feel if you can name it, we’ve done it,” and one of their peers added:

I try to offer as much PD to my teachers as possible. I’ll set up school wide PD sessions

based on what teachers ask for the most. I offer one on one lesson planning and small

group sessions. I also visit classrooms and observe where improvements could be made.

Tech-Tip Tuesdays, nighttime Twitter chats, newsletters, and full-day planning sessions were just

a few of the PD opportunities named. PD formats ranged from personalized training sessions to

grade-level meetings or larger. Topics covered during the past year included lesson planning,

technology best practices, BCPSOne tutorials, and information on grading and assessment. One

S.T.A.T. teacher reported inviting outside speakers and consultants to visit their school and another

spoke of holding a session where middle school and high school participants learned what was

being done at the elementary level.

S.T.A.T. teachers reported that their professional development has typically been very

effective in their schools and that the teachers in their schools value their presence there. One

commented:

I’ve had teachers that have thanked me for coming in and they appreciate seeing the

lessons modeled. I think the way we’re doing PD now is similar to how we teach kids—

we’re meeting the different learning styles of teachers. Sounds cliché, but we really are

trying to customize the professional development to be tailored to the specific needs of

individual teachers.

They have seen increases in teacher confidence and have more teachers seeking them out for

answers to initiative-related questions. They indicated that more of their teachers are seeing

content, data, and the arrangement of their classrooms in a different way, with one teacher stating,

“I’m seeing teachers really stretch and focus on transformative lesson planning. I link it to the



professional development done at the school.” Teachers who initially were required to attend PD

sessions now want to attend sessions, especially as they can now select PD based on topics they

feel will be most impactful for their students. In support of this, one S.T.A.T. teacher observed,

“There’s definitely more buy-in for teachers. They’re definitely more excited and engaged to

participate with students in these new initiatives or special development opportunities,” and

another commented:

I think the impact is really big. I think we’ve done a good job. We design our PD in the

same way we want teachers to plan or design their instruction. I use our school progress

plan to align everything that I do to that and with teachers it’s like the curricular guide,

because we model to teachers the way we want them to teach students. I think it’s had a

big impact for that reason. That, and, because teachers not only pick their own PD, they

also lead it so they feel a part of the development and achievement. It’s not two teams of

leadership and faculty, it’s all of us together.

Several S.T.A.T. teachers made note of their teachers’ current excitement and engagement

with the PD and the initiative, and the impact of that on student engagement, with one respondent

noting, “I do think it has made a big impact. It has gotten teachers to think more flexibly and helped

them to see connections between their own teaching and what happens throughout the building.”

Perceptions of S.T.A.T. teachers. Principals and classroom teachers in schools

implementing S.T.A.T. conveyed their perceptions of S.T.A.T. teachers through interviews and

focus groups.

Principals. Principals were overwhelmingly positive towards how their S.T.A.T. teacher

has supported the implementation of the initiative. Principals commented that their S.T.A.T.

teacher supported the program as a “cheerleader,” “PR person,” “resource provider,” “contact with

key district officials,” “coach,” “mentor,” as well as a “model.” The word most commonly used to

describe the S.T.A.T. teacher was “instrumental.” At all levels, principals noted that these teachers

were readily available, had their finger on the pulse of the staff, were collaborative, good listeners,

nonjudgmental, and focused on “changing instruction, not just the use of the device.”

Principals also listed a variety of other roles where S.T.A.T. teachers provided support. For

example, some of these teachers serve on school leadership teams and PD committees. In addition,

they co-plan with teachers, communicate with parents and assist them in learning about their

child’s device, create presentations, conduct learning walks, set up times for teachers to visit other

classrooms, and work constantly with teachers. Two principals summed up a number of principal

statements regarding their S.T.A.T. teacher:

The S.T.A.T teacher isn’t just there as the person teaching us about the technology, or

fixing things or answering questions. Her role is that of a mentor and to give us the ability

to be more dynamic in how we use her strategically, whether in the classroom or at faculty

meetings when we have time for planning.

And:



This initiative wouldn’t be what it is without our S.T.A.T. teacher. She connects with

teachers in different ways. She’s been a great resource and cheerleader and advocator for

this initiative. This role is critical. A principal without her couldn’t make it happen.

When asked to comment on the benefits of the S.T.A.T. teacher role, principals at all levels

cited numerous advantages. One elementary school principal observed, “She’s highly beneficial

because not only does she help with the tech piece; she’s our teacher mentor. She’s helping them

beyond just a tech liaison, she’s helping teachers with supporting them in any way they need.” A

middle school principal commented that, in the past, schools had mentors for new teachers, but the

S.T.A.T. teacher “focuses on everybody.”

Several principals commented on the fact that the S.T.A.T. teacher made their job easier.

It was observed that these teachers “can pinpoint PD needs the principal may not have time to do,”

and they can “fill gaps” and “address weaknesses.” Faculty view S.T.A.T. teachers as “experts”

they can readily approach for assistance. In addition, the S.T.A.T. teacher helps create a “culture

of forward momentum, creativity, learning, and coaching across the staff,” as well as “easing the

burden of technology integration.” Finally, one elementary principal stated emphatically that the

S.T.A.T. teacher “is a critical component to the success of S.T.A.T. With one hundred exclamation

points behind it, I hope the position never goes away!”

In addition to benefits of this role, principals were asked to comment on what

improvements they might offer for the S.T.A.T. teacher program. Three Lighthouse principals

(two from elementary and one from middle school) said they had no suggestions for improvement.

Three principals (one each from a Lighthouse middle, Phase 2 middle, and high school) felt the

S.T.A.T. teacher’s job should be made a 12-month position. Three principals (one each from a

Phase 2 elementary, Lighthouse middle, and Phase 2 middle) commented that their S.T.A.T.

teacher often had to be out of the building, and this was “a double-edged sword” as it “interrupts

scheduling.” Two additional principals felt every school should have a S.T.A.T. teacher “no matter

what their test scores” and “based upon school ratio of tenured versus non-tenured teachers, in

addition to supplemental funding.” Beyond these themes across groups and grade levels, principals

had additional suggestions relative to their specific grade levels.

At the elementary level, three Lighthouse principals commented they hoped the “status

quo” would remain in place and that “they [BCPS] keep the program, if removed it would be

detrimental.” Phase 2 elementary school principals expressed the wish that more teachers could

attend S.T.A.T. institutes. Two Phase 2 principals commented they hoped to see the S.T.A.T.

program “continue to push on initiatives that have started with the understanding that not everyone

has arrived with the same level of understanding and implementation.” Finally, one Phase 2

principal commented on the need for S.T.A.T. teachers to have more training in project-based

learning, stating: “I think that having more PD around ways we can develop more project-based

activities around the standards will help us tap into 21st century learning opportunities.”

Middle school principals had slightly different views. Several across groups (Lighthouse

and Phase 2) expressed the desire to have more autonomy in choosing the roles the S.T.A.T.

teacher took on within the school. Three principals noted that the “right” person for the job, with

skill sets in solid instructional practices, was critical.



High school principals also expressed differing views. One commented that S.T.A.T.

teachers “needed more help or access with data analysis in compiling reports related to

effectiveness rather than going to the APs and Principal for data.” Another noted, “If there were

an overview of what PD they [S.T.A.T. teachers] are receiving, it would be helpful for that to be

communicated to principals—it would help me plan better.” Finally, a third high school principal

observed, “BCPS used to have ‘mentor teachers’ but the school does not have these any more. It

would be good if the S.T.A.T. teachers could play a bigger role in helping specifically the new

teachers.”

Classroom teachers. Emphatically, classroom teachers reported that S.T.A.T. teachers

have supported the initiative in ways beyond offering PD sessions. The majority of responses

described S.T.A.T. teachers as “the best of the best,” “enthusiastic,” “wonderful,” and

“responsive.” Numerous teachers credited the S.T.A.T. teacher for ongoing support, advice,

motivation, a “positive attitude,” and making the initiative more “doable.”

However, there were some negative comments. One teacher from a Lighthouse elementary

school felt that the “perception over the County is more of a negative regarding that teacher slot,”

indicating that some would prefer for additional classroom teachers, while another remarked “there

is no consistency between buildings.” Several teachers across all groups felt the S.T.A.T. teacher,

often through no personal fault, simply did not have enough time to meet all needs, as they were

often pulled in multiple directions or served too large a population, and consequently had to be

more “reactive than proactive.” A Lighthouse elementary classroom teacher commented there was

a “tremendous amount of confusion as to the role of the S.T.A.T. teacher within the school.”

Teachers across groups echoed this remark.

The majority of teachers, though, described a variety of ways in which the S.T.A.T. teacher

offered support beyond PD, including co-planning, modeling, coordinating visits from specialists,

working with students, sharing resources, visiting classrooms, and facilitating cross-school teacher

collaboration. Classroom teachers also noted the availability and dedication of their S.T.A.T.

teachers. As one Lighthouse middle school teacher stated, “She is the fearless leader, she will sit

by my side and help work out kinks or she will present to me some new way of doing something.”

One Phase 2 middle school teacher commented lightheartedly, “What she doesn’t do! That would

be a lot easier question!”

In terms of benefits of S.T.A.T. teachers, classroom teachers in Lighthouse and Phase 2

schools shared many of the same answers. S.T.A.T. teachers are viewed as “essential” to the entire

school, “resourceful,” “enthusiastic,” “knowledgeable,” “highly supportive,” and serve as a

“navigator,” in a “nonjudgmental” capacity. As one classroom teacher noted, the S.T.A.T. teacher

“helps us get more time back, because I’m not spending time solving problems, she’s solving

problems for me,” and another noted,

Whenever we have a question, even about something she’s already taught in a PD session

but we don’t remember, she’s always there, always willing to come in and do a team teach

or even just meet with you individually go over what you have a question about.



Teachers also appreciated that the S.T.A.T. teacher often had a strong instructional background.

As one teacher commented,

She’s helpful because she works with all grade levels, she knows the curriculum inside out.

She knows what students have already had; she has omniscience and sees the whole

picture, where we only see our own grade level so it’s helpful to give us information.

Classroom teachers’ one concern with the S.T.A.T. teacher role was that S.T.A.T. teachers

were often pulled to fulfill other roles (substitute, manage discipline problems, coordinating

Lighthouse school visits, testing, etc.), stating that it is “difficult for them to do their job effectively

at times.” One teacher wryly noted, “Her position feels like it gets abused for all the wrong

reasons.”

Classroom teachers did offer suggestions for the S.T.A.T. teacher program. Some

expressed the desire for BCPS to hire more S.T.A.T. teachers permanently, particularly for schools

with large populations. A second theme that was noted was the need for S.T.A.T. teachers to

“implement lessons in classrooms,” “make regular classroom visits across grade levels,” and

“model teaching on S.T.A.T. initiatives” on a rotating basis. In addition, teachers would like “more

granular feedback for specific classroom practices,” as they feel the feedback is too general, such

as, “That’s great.” In short, they want more individualized feedback on common mistakes, as well

as ways to become more successful. They view the S.T.A.T. teacher as a mentor, not an evaluator,

and therefore value feedback that will help them improve practice.

A final theme that teachers across all groups noted was the need for clear roles and

responsibilities for S.T.A.T. teachers. As one teacher commented,

There needs to be clear guidelines from the top down about what the S.T.A.T. teacher is

supposed to be doing in each building; how are they supposed to be helping teachers and

students; what are they supposed to be doing for administrators. I just don’t think tha t

message has been made clear.

The vast majority of teachers interviewed agreed that the S.T.A.T. teacher made their work

and involvement with this initiative much easier. They felt strongly that the successes they had

achieved were in large part due to the work that the S.T.A.T. teacher did in their building. Many

of the teachers interviewed were hesitant to make suggestions as they felt their S.T.A.T. teacher

“did the work of five.” One teacher commented she “couldn’t think of improvements but just

wanted to emphasize that we can’t live without her!” However, one Lighthouse high school teacher

viewed the S.T.A.T. position through a different lens, stating “It would be nice if it didn’t take a

teacher position away from the building. Because we added that position it took away a teacher

position.”

Classroom teacher preparation to implement. Classroom teachers implementing

S.T.A.T. participated in professional development (PD) offered by the district during the summer

and by their school’s S.T.A.T. teacher throughout the school year. The following sections provide

results from interviews, focus groups, and the teacher survey regarding preparation and support

for S.T.A.T. implementation.



Principals. While the majority of principals in all groups affirmed during interviews that

teachers are now prepared to implement S.T.A.T., a number felt teachers were either not initially

prepared or did not have a full understanding of S.T.A.T. entailed. One Phase 2 middle school

principal said, “Some teachers were prepared and some were not,” while a Lighthouse high school

principal stated, “As a group, there was a readiness in terms of willingness. Teachers didn’t always

have a clear picture of what it would entail.” Three principals (two Lighthouse elementary and one

Phase 2 elementary) felt that due to high teacher turnover and the need for additional professional

development, “not all teachers were prepared.” Three principals (one Lighthouse elementary and

two Phase 2 middle) reported their teachers “were not prepared.” One Lighthouse elementary

principal made a statement that seemed to sum up all comments by stating:

Over the course of the year we’ve learned and grown. This is the first year that everyone

has had devices from the beginning of the school year. We still have to learn and grow on

how to use what’s available to us. No one can capture the possibility on how to use devices

in a year or two. It will take time to build on what we know and become better on how we

use them.

Teachers. Classroom teachers in Cohort 1, 2, and 3 responded to survey items regarding

professional development they received (see Figure 2).

Figure 2. Teachers’ frequency of responses to survey items regarding professional development.



The majority of teachers across all groups agreed (41.0% strongly agreed) that they had been able

to apply what they learned through professional development in their classrooms. Classroom

teachers also agreed (82.2% somewhat or strongly agreed) they had received sufficient

professional development on the use of technology in the classroom. Cohort 1 (86.0% at least

somewhat agreed) and Cohort 3 (84.5%) teachers were slightly more in agreement than Cohort 2

teachers (77.2%). Similar to professional development on technology integration, classroom

teachers agreed (87.0% somewhat or strongly agreed) they had received sufficient professional

development on the creation of a learner-centered environment. Notably, as might be expected,

Cohort 1 teachers were more likely to agree (92.0% somewhat or strongly agreed) than Cohort 2

(84.2%) and Cohort 3 teachers (87.4%). There were not significant differences across time points

for any grade group participants that responded to both the spring 2016 and spring 2017 teacher

survey items pertaining to professional development.

Teachers in Lighthouse and Phase 2 schools also discussed in focus groups their overall

preparation and support to implement S.T.A.T. They felt that although there were numerous PD

opportunities made available to them, the sheer number of programs was “overwhelming” and not

enough time was provided to become adequately prepared to implement newfound knowledge.

Many teachers felt they “were learning on the fly.” For example, one teacher commented, “I just

feel like there’s so much happening…I think they should have had more in place before they tried

to deliver it to us. I feel like it was pieced together.” Similarly, another teacher noted, “Baltimore

County is great at introducing things and exposing us to different opportunities but in terms of

giving time to truly learn, understand, practice to the point of being comfortable…we’re not there.”

Teachers often described the time needed to better grasp how to integrate technology. For example,

one teacher stated, “I need more time to plan it. I need to figure out how it works before I present

it to a group of kids.” Another teacher conveyed a similar perception:

PD means what is needed to be an expert. To have a comfort level to incorporate with

students, discuss with parents, we were introduced to lots of different things. But it’s like

walking into a car showroom and being able to look at all the models on the floor, you can

see this one is red, this one is white, and this one is blue. And you can kind of peek in the

window. With the S.T.A.T. initiative, that’s what the County did for us. They walked us all

into the showroom, they said ‘here’s all these pretty vehicles and things to look at, you can

touch the outside, but nobody open the door and take it for a test drive.

Several teachers also pointed out that “tech-savvy” classroom teachers were naturally better suited

than those who were novices in learning new technologies. One teacher commented:

I learned in school and college. It [S.T.A.T.] felt not as overwhelming for me because I was

only in the classroom for two years when S.T.A.T. came out. It was a huge switch for

teachers teaching a long time, and was overwhelming because there’s a lot to do.

Various groups commented that having a choice in the PD training they attend would also be

beneficial, as some training either did not “always honor the limited the time [teachers] do have,”

or was “redundant,” “not grade appropriate,” and often at times when teachers could not attend.



Needed professional development. During interviews, principals were asked to comment

on what professional development their teachers might need to fully implement the goals of

S.T.A.T. Across all schools and principals, it is evident that more professional development is

needed, or at the very least, maintained. The most commonly cited need at all levels was

professional development in P21 skills. As one Lighthouse elementary principal noted, this takes

“creativity and some out-of-the box thinking.” Principals also commented on a variety of other

professional development needs, but noted that as important as specific types of professional

development were to meet teacher needs, it was equally critical to give teachers the time to process,

collaborate, build confidence in strategies, and plan what they have learned effectively. One

Lighthouse elementary principal commented:

What makes the greatest impact in terms of professional development is collaborative

planning which is led by our S.T.A.T. teacher or another content expert, to help them really

show understanding of those standards and expectations; then being able to design

instruction which is very focused on their grade level, content and students.

This was further supported by a number of middle and high school principals. A Lighthouse middle

school principal stated:

I can’t think of one particular PD that teachers are lacking, but I think the biggest

overarching piece is providing teachers with the time to plan, because that allows them to

be able to look at the learning process differently.

A Lighthouse high school principal went further by stating that teachers needed time “to build on

the momentum that they started,” while a second principal noted, “Teachers are not at the point of

expertise, so allowing time to learn, and gain expertise is important.”

Classroom teachers. Classroom teachers also specified areas for needed professional

development. Lighthouse elementary teachers mentioned the need for PD in several programs,

including Class Flow, Wixie, and PowerPoint. Another teacher noted the need for “training in

Lesson Tiles.” Teachers also mentioned needing training in “skills and usage.” Lighthouse

Middle school teachers requested PD that was “practical” and “specific” with more frequency (at

least two times per month). One teacher commented:

Last year we had more PD opportunities where we had to sign up throughout the day, pick

a session we wanted to attend and learn about different things. It was different last year.

All PD is now after school at faculty meetings, just once a month. Last year we had it

almost twice a month, in school, during planning time.

Some specific suggestions for PD from Lighthouse middle school teachers included “more

training on BCPSOne.” As one teacher noted, “There is a lot about BCPSOne that I have no idea,

honestly. I don’t have the time to sit down and do it.” They also wanted more training on not only

“using the devices, but also on all the programs associated with the device,” and strategies for

dealing with “discipline for off-task device usage.”



Lighthouse high school teachers also had several suggestions. One teacher mentioned the

need for training on “the wiki on PDWorks.” The teacher explained,

It doesn’t tell us what is broken, or what grade levels it is meant for, and what doesn’t work

with our firewall... If they could shorten the list to just what works, and their appropriate

context, that would save an enormous amount of time.

Other teachers stated they needed more training in Edmodo, OneNote, and echoed Lighthouse

elementary teachers in the need for a better understanding of ClassFlow. In addition, teachers

requested training with differentiated levels of instruction as well as question and answer PD

sessions where teachers “could get clear advice and guidance on how to troubleshoot issues on the

devices, as well as logistical issues.”

Phase 2 elementary teachers felt they needed more training in “problem-based and inquiry

based learning approaches” and “how to better integrate technology into classroom use.” Phase 2

middle school teachers also echoed Lighthouse middle teachers in the need for more training on

BCPSOne and a “better understanding of the grade book.”

Lighthouse school preparation. Participants in Lighthouse high schools, new to S.T.A.T.

as of the 2016-17 school year, were asked to comment through interviews, surveys, and focus

groups regarding their preparation to serve as a Lighthouse school.

Principals. High school principals described in interviews numerous examples of support

they received in order to serve as a Lighthouse school. For instance, one noted the value of having

a “pedagogy shift first and technology implementation second.” This principal also remarked that

it “helped that teachers were ready and willing to engage with new technology,” and that being

“exposed to the global awareness of the county” and PD had “jump started some ideas.” In

addition, this principal specifically noted the value of S.T.A.T. summer training, stating, “We

participated in the S.T.A.T. institute at same time as teachers, and I was learning and planning

what teachers needed at the same time as them, and that was helpful.”

Another principal also mentioned the importance of PD and discussed “end of school... and

summer workshops,” as well as “training during the school year,” such as “after school workshops

that provided training on apps and programs associated with the devices” and “monthly

workshops.” Strategy and session planning during this time was also noted by two principals as

being of particular value. One principal stated, “PD allowed us access to strategy planning sessions

from other Lighthouse schools, grow, network and get ideas from other department chairs.”

Another point made by a principal was how the S.T.A.T. teacher was a “helpful and a ready

resource when busy managing the school and development in the area of digital conversion.”

These principals noted “each school is different because of demographics and there is no

cookie cutter training.” Other high school principal responses reflected this comment. One

principal stated they would like to see more professional development dealing with formative

assessment, as “foundational work in formative assessment training helps shift mindsets around

small group instruction,” and they wished “that staff had engaged in that prior.” A second principal



noted that finding times for teachers to have common planning was critical, so that teachers “don’t

just cooperate, but also collaborate.” Another principal suggestion for teacher preparation was to

make sure “correct instructional practices and digital frameworks are utilized.” This principal

pointed out that although professional development should be about improving student success,

teachers need affirmation they are moving in the right direction:

Teachers, just like students, want to know how they are going to be graded and what are

they looking for and some teachers needed more assurance and feedback from a

Lighthouse perspective that they are on the right track.

The principal explained this in more detail, stating,

I think, for me, our teachers needed some more time with just knowing what the rubric was

that we were going to be judged against. I don’t know if that was crystal clear and if that

was on purpose.

S.T.A.T. teachers. Lighthouse high school S.T.A.T. teachers reported attending monthly

meetings on how to implement the initiative. They described meetings with principals and

department chairs, and participating in learning walks. One of the S.T.A.T. teachers commented

that she appreciated the opportunity to meet with S.T.A.T. teachers from other schools and to hear

about the “growing pains” that they are going through. Another commented on a challenge they

had faced: “This year was difficult because the person over all of the S.T.A.T. teachers changed

to a new person. The new supervisor has been trying to align her vision with the last supervisor’s.”

One Lighthouse high school S.T.A.T. teacher reported that delivering PD for high school

teachers was difficult due to scheduling conflicts, but did indicate that future Lighthouse schools

receive additional professional development in the area of small group instruction. They also

thought that for the initiative to be successful, new Lighthouse school teachers should be allocated

time to develop materials. A second Lighthouse high school S.T.A.T. teacher focused their

response on the involvement of Assistant Principals, saying that they are often “left out.” This

S.T.A.T. teacher noted that Assistant Principals “are often on board with the initiative, but the

central office needs to provide support so they can be another helper in the school.”

Classroom teachers. Lighthouse high school classroom teachers indicated on survey

responses that they were adequately informed of the expected role of their school as a Lighthouse

school (92.0% at least somewhat agreed). Teachers indicated they were prepared to support their

school in fulfilling its role as a Lighthouse school (83.0% at least somewhat agreed) and that their

school was successful this year in fulfilling its role as a Lighthouse school (88.6%).

These teachers elaborated on survey responses during their focus groups. Teachers noted

that the PD, S.T.A.T. teacher, “learning walks,” and visits to other schools had been helpful.

However, across all schools, it is evident that more PD is needed and that teachers need more time

to experiment, process, and implement PD they had received. In addition, teachers mentioned

wanting to be provided “with unstructured time and PD sessions where we could simply focus on

getting all our questions answered.”



Summary. Participants discussed a wealth of support and preparation for S.T.A.T.

implementation. Principals conveyed their role to support and lead the initiative within their school

and most classroom teachers indicated the principal was effective in their role as it related to

S.T.A.T. implementation. Teachers described encouraging and supportive principals, but some

potentially isolated concerns were expressed by teachers who indicated they needed more planning

time and assistance with discipline issues.

S.T.A.T. teachers conveyed feeling prepared to serve in their role and supporting classroom

teachers. They did request PD pertaining to incorporating technology in the curriculum, small

group instruction, behavior management, and guidance on engaging all teachers in their building.

As with prior reports, S.T.A.T. teachers are viewed by classroom teachers as very valuable and

critical to the initiative’s success. They are an important resource for classroom teachers as they

gain further experience with technology integration and changing teaching practices. Also

consistent with prior reports was the concern that S.T.A.T. teachers may not be utilized in the

manner by which BCPS intends, in addition to inconsistencies with the S.T.A.T. teacher roles and

responsibilities across schools.

Overall, classroom teachers conveyed generally feeling prepared and supported for

S.T.A.T. implementation. They did, though, often reference the amount of time needed to plan for

technology integration. In terms of future PD, teachers expressed the need for PD on tools and

programs, including BCPSOne, and some expressed the need for PD on P21 skills.

Measurable Outcomes: Classroom Environment

The impact of S.T.A.T. on the classroom environment was assessed through observations

in Cohort 1, 2, and 3 classrooms. The classroom environment was examined in terms of the

classroom layout, information displayed in the classroom, and student activities. Results are

described by group and comparisons across time points are reported where applicable. Readers

should be reminded that only four to six classrooms within the schools were observed and the

observations served as only a “snapshot” of classroom practices for a brief amount of time. Data

from the spring 2017 classroom observations for each of the cohort groups are presented in Figure

3 and further discussed below.



Figure 3. Observation results on OASIS-21 classroom environment items for spring 2017.

Cohort 1. Classroom environment observations in spring 2017 indicated that there were

not significant differences between the most recent observations and baseline observations from

fall 2014. Almost all classrooms were arranged with desks in groups (29 of 32) and over half of

classrooms observed (62.6%) had at least some information and resources displayed that supported

independent thinking. The vast majority of classes had information and resources that reflected the

content being taught posted around the classroom (90.6%). Moreover, the majority of classrooms

featured posters and displays in this area that were lesson specific, as opposed to simply reflecting

a general subject area. Students were seldom observed moving around the classroom

independently to acquire materials and resources. In slightly more than half of the classrooms

however (59.4%), students were observed making at least some use of different workspaces for

different tasks.

Cohort 2. Classroom environment observations in spring 2017 indicated slight,

statistically nonsignificant changes within Cohort 2 classrooms. Most classrooms were arranged

with desks in groups (56 of 68) and just over half of classrooms observed (52.9%) had at least



some information and resources displayed that supported independent thinking. Nearly all classes

had information and resources that reflected the content being taught posted around the classroom

(95.6%). Unlike what was observed with Cohort 1 classes, however, the majority of classrooms in

Cohort 2 featured posters and displays that were general to subject areas and not lesson specific.

Students were seldom observed moving around the classroom independently to acquire materials

and resources. Moreover, students generally made infrequent use of different workspaces for

different tasks.

Finally, the only area in which classroom environment observations for spring 2017 were

significantly different from the baseline fall 2015 observations was in Lighthouse Grade 6

classrooms. In spring 2017, 26.7% of these classrooms had lesson specific information and

resources posted whereas in the baseline, 57.1% were observed posting this information (p < .05).

More classrooms were observed posting general subject information (73.3%) in spring 2017 as

opposed to the baseline observations (39.3%).

Figure 4. Frequency of teachers visibly displaying information and resources reflecting content

being taught in Lighthouse Grade 6 classrooms.

Cohort 3. Similar to Cohort 2, observations in spring 2017 indicated few changes across

time points for the classroom environment in Cohort 3. Most classrooms were arranged with desks

in groups (49 of 77). Just under half of classrooms observed (46.8%) had at least some information

and resources displayed that supported independent thinking. Over 90% of classes had information

and resources that reflected the content being taught posted around the classroom and more

featured posters and displays that were general to subject areas (53.2%) as opposed to lesson

specific (40.3%). Students were seldom observed moving around the classroom independently to

acquire materials and resources. Similarly, students also made seldom use of different workspaces

for different tasks as this too was not observed in over three quarters of the classrooms.

Phase 2 Grade 6 classrooms were observed displaying independent thinking materials

significantly less frequently in spring 2017 compared to spring 2016 (p < .05; see Figure 5). In

spring 2017, 57.2% of classrooms had at least some display of these materials (17.9% extensive)

whereas at baseline, 75.0% of classrooms had these displays (64.3% extensive).



Figure 5. Frequency of teachers visibly displaying information and communications that support

independent thinking in Phase 2 Grade 6 classrooms.

Summary. Overall, in the area of classroom environment, observation results were

generally similar in spring 2017 with the baseline observations for each cohort. Across the eight

subgroups that made up the cohorts, only two subgroups demonstrated significant differences on

any items between the baseline and most recent observations. Moreover, classroom environments

were generally similar across all three cohorts during the most recent observations. The item that

varied the most between the cohorts dealt with students using different workspaces for different

learning environments. Cohorts 1 and 2, which both have participated in S.T.A.T. for multiple

years, had more extensive use of this activity than Cohort 3, which was new to S.T.A.T. this year.

Measurable Outcomes: Teacher Practice

This section begins with results of the classroom observations in Cohorts 1, 2, and 3.

Findings related to teacher practice from interviews and focus groups, and the classroom teacher

survey from these groups are then presented.

Observations. Six OASIS-21 items examined teacher practice including teacher

presentation, coaching/facilitating instruction, offering higher order instructional feedback,

student-initiated communication, higher level questioning, and flexible grouping of students.

Results are described by group and comparisons across time points are reported where applicable.

Data from the spring 2017 classroom observations for each of the cohort groups are presented in

Figure 6 and further discussed below.



Figure 6. Frequency of extensiveness observed on OASIS-21 teacher practice items during

spring 2017.

Cohort 1. Results for classroom observation items concerning teacher practice did not

significantly differ between the most recent observations and baseline observations from fall,

2014. The first two teacher practice items examined the extent to which teachers acted as



coaches/facilitators of instruction and provided direct instruction through presentations. Both of

these teaching activities were exhibited relatively frequently during the observations, though

teachers were somewhat more likely to engage in coaching/facilitating as opposed to presenting.

Specifically, three quarters of observed teachers engaged in coaching/facilitating to at least an

occasional extent (28.1% did so extensively), whereas only half of observed teachers provided at

least occasional presentations (6.3% made extensive use of this teaching practice).

Observations of student-teacher interactions rated the frequency of higher-order

instructional feedback given by the teacher, higher-level questioning used by the teacher, and

academically relevant communication initiated by students. Generally, Cohort 1 teachers were

somewhat rarely observed providing higher-order instructional feedback to students and using

higher order questioning strategies. More frequently, however, students were observed initiating

academically purposeful communication with the teacher and each other. Specifically, student-

initiated communication was observed at least occasionally in roughly half of classrooms. In the

area of student grouping, few classrooms were observed utilizing flexible student groups (grouping

based on student and task needs).

Cohort 2. Cohort 2 results for classroom observation items concerning teacher practice in

spring, 2017 did not significantly differ for teachers as compared with baseline observations. As

with Cohort 1, Cohort 2 teachers exhibited relatively frequent use of both coaching/facilitating as

well as presenting, though they were slightly more likely to engage in coaching or facilitating.

Here, 64.7% of observed teachers engaged in coaching to at least an occasional extent (22.1% did

so extensively), whereas less than half of observed teachers provided at least occasional

presentations (8.8% made extensive use of this teaching practice).

Generally, Cohort 2 teachers made somewhat seldom use of higher-order instructional

feedback or using higher order questioning strategies. As was the case with Cohort 1, students,

however, were observed more frequently initiating academically purposeful communication with

the teacher and each other. This behavior was observed of students in over three quarters of the

classrooms and was observed occasionally or more in nearly half of the classrooms. Few

classrooms were observed utilizing flexible student grouping as this was not observed at all in over

two thirds of the classrooms.

Cohort 3. Results for classroom observation items concerning teacher practice in spring

2017 for Cohort 3 were similar with a few notable differences between the most recent

observations and baseline observations. In terms of instructional modes, Cohort 3 teachers were

observed making relatively frequent use of coaching/facilitating while making generally

infrequent use of teacher presentation. Specifically, 84.4% of teachers were observed acting as

coaches/facilitators to some extent during the observations, while 70.2% were observed making

occasional use or more of this approach. Conversely, in nearly half of classrooms, teacher

presentations were not observed at all, while only 36.3% of classrooms had teachers use this

approach occasionally or more. As with Cohorts 1 and 2, Cohort 3 teachers generally made rare

use of higher-order instructional feedback or higher order questioning strategies. As was also the

case with Cohorts 1 and 2, however, more frequently students were observed initiating

academically purposeful communication with the teacher and each other. This behavior was



observed of students in over two thirds of the classrooms and was observed occasionally or more

in roughly half of the classrooms. Very rarely were classrooms observed using flexible grouping.

Unlike Cohorts 1 and 2, Cohort 3 classrooms, across several subgroups, did exhibit several

significant differences in teacher practice during the spring of 2017 as compared with the baseline

observations. First, Phase 2 K, 4, and 5 classrooms exhibited significantly less extensive use of

higher-order instructional feedback in the most recent observations compared to baseline, fall 2016

observations (p < .05; see Figure 7). This same trend was also found with Phase 2 Grade 6

classrooms with a significantly less extensive use of higher-order instructional feedback in the

most recent observations compared to baseline, spring 2016 observations (p < .05; see Figure 8).

Such declines are expected as a function of the transition from teacher- to student-centered

instruction.

Figure 7. Frequency of teachers providing higher-order instructional feedback observed for

OASIS-21 items in Phase 2 Grades K, 4, and 5.

Figure 8. Frequency of teachers providing higher-order instructional feedback observed for

OASIS-21 items in Phase 2 Grade 6 classrooms.

However, in contrast to the trend for higher-order feedback, Lighthouse Grade 7 teachers

were observed more frequently asking higher-level questions in the spring of 2017 than occurred

during the baseline observations in fall 2016 (p < .05; see Figure 9).



Figure 9. Frequency of teachers using higher level questioning techniques observed for OASIS-21

items in Lighthouse Grade 7 classrooms.

In the area of flexible grouping, Phase 2 Grades K, 4, and 5 teachers were observed

grouping students based on student and task needs significantly less often in the spring of 2017

than baseline observations in fall 2016 (p < .05; see Figure 10).

Figure 10. Frequency of teachers using flexible grouping based on student and task needs

observed for OASIS-21 items in Phase 2 Grades K, 4, and 5.

Perceptions regarding the impact on teacher practices. Principals, S.T.A.T. teachers,

and classroom teachers of these groups provided their perceived impact of the S.T.A.T. initiative

on teacher practices through interviews and focus groups. In addition, classroom teachers

responded to survey items regarding their practices.

Principals. During interviews, principals in all groups noted that teachers had experienced

“mindset changes” where, for example, teachers were more willing to collaborate on instructional

planning and seek out guidance and assistance from their peers or S.T.A.T. teacher. Depending on

the grade level, but not necessarily the individual groups, these changes were manifested in slightly

different ways.

At the elementary level, although mindset changes were consistently noted, a variety of

other practices was observed. For example, Lighthouse school principals noted “increased

collaboration,” better “integration of technology,” more “purposeful use of devices,” and “rigorous

instruction.” One principal summed this up by stating, “Teachers are really letting go and

becoming more of a facilitator.”

In comparison, Phase 2 elementary schools were still, in many instances, learning the

process. One principal commented, “There are some teachers who are new to everything. They

have taken that on and they are using that” while a second noted that teachers have “been on a fast



learning curve and are using it (technology) more effectively this year.” Several principals

commented that teachers were also becoming more skilled in “device use,” “co-planning,”

“understanding data,” “differentiation,” and “customized learning”. Although principals observed,

“We’ve improved opportunities for students to show their learning,” there was still awareness in

the group of areas for improvement. As one principal commented, “We still have work to do . . .

Finding those appropriate activities that will support learning needs . . . and those resources that

will fill the achievement gap. We are better but can get better.”

Middle school principals from both Lighthouse and Phase 2 schools consistently noted that

teachers had also experienced mindset changes. In addition, teacher’s approaches to planning were

viewed as another key change in practice. Principals in both groups felt that teachers now planned

more thoughtfully and with more purpose as a result of the initiative. One Lighthouse principal

commented, “It’s not about just the device per se but the whole transformation of the teacher.” A

second Lighthouse principal stated that teachers were “more conscious of how important planning

is to make the initiative successful,” while a third commented teachers were also “investing more

time in planning.” Phase 2 principals also saw this same change, in addition to teachers

“differentiating” more in their planning. One principal summed up the changes noted in teacher

practice by all middle school groups stating, “Goodness, everything is different! With every child

having a device . . . teachers have to change their practices. They are learning from each other new

ways to invite technology into the classroom and assist with the learning process.”

Lighthouse high school principals also agreed that mindset changes and planning were key

practice changes. Principals noted that teachers were more student-centered as well as planning

more methodically and with purpose for instruction. One principal explained that this was the area

where he has seen the greatest improvement. He felt there was an increased emphasis on long

range planning and “teachers were providing more choice for students . . . generally changing their

approaches to instruction to be more student-centered”. A second principal supported this stating,

“I definitely see a wider array of methodologies being utilized, being more experimental, trying

new things”. A third observed, “Compared to last year when we had virtually no one talking about

targeted small group instruction, it’s now driving our conversation in department meetings.

Growth is tenfold from last year and people are being really innovative with their implementation.”

S.T.A.T. teachers. Both Lighthouse and Phase 2 elementary S.T.A.T. teachers conveyed

in interviews that teachers in their schools continued to take risks with implementing meaningful

uses of technology in the classroom. They also reported that while in many instances teacher

practices continue to evolve, teachers were utilizing data more, and more teachers were

collaborating with one another across grade levels. Phase 2 Elementary also described increases

in small group instruction, and differentiation in their classrooms, with one teacher commenting:

Through the digital data teachers are able to collect information in real-time and customize

using the devices. This has really helped the teachers create more leveled experiences for

students immediately using a digital platform. The kids are just more engaged in the fact

that they have more options in the way they learn, maybe they read from a book or utilize

the devices and teachers are giving a lot of choices and students appreciate that.



Lighthouse middle school S.T.A.T. teachers were in agreement with their elementary peers

in their description of increased teacher collaboration, personalization, and small-group instruction

as well as a willingness in teachers to take more risks in the classroom. In addition, teachers from

these two groups of middle schools reported use of backward mapping for long-term lesson

planning and the use of formative assessment in order to modify teaching and learning activities

to improve student achievement. One Phase 2 middle school S.T.A.T. teacher stated that their

teachers had ‘backed off’ on device use in the classroom when they recognized that other

modalities were more successful noting, “In terms of teacher practices, there may be less device

use routinely, but they’re more thoughtfully used.”

Lighthouse high school S.T.A.T. teachers reported their teachers were changing the way

students learn and interpret lessons, with one teacher noting, “There is more of a deliberate sense

of where we want the students to be. My teachers are thinking more about how to meet students at

their current educational level to plan accordingly and more long term.” Like their peers at other

grade levels, they also cited an increase in the use of small-group instruction and in teachers taking

risks with technology and software within their schools.

Classroom teachers. Classroom teachers in Lighthouse and Phase 2 elementary and middle

and Lighthouse high schools responded to survey items regarding teacher practice. Nearly all

classroom teachers agreed (37.3% strongly agreed) that they are skilled at engaging student in

collaborative learning activities using technology as a resource or tool. Predictably, Cohort 1

teachers were more likely to agree (97.0%) than Cohort 2 (93.0%) and Cohort 3 teachers (90.0%).

Further, most teachers (87.2%) indicated their students collaborated with their peers on an in-class

activity or investigation at least once per week. This practice was more frequently reported on a

once-a-week basis by Cohort 1 teachers (95.0%), as compared with Cohort 2 teachers (90.7%) and

Cohort 3 teachers (82.2%). However, there were not significant differences observed across time

points.

Teachers also indicated the frequency that they employed different teaching practices (see

Figure 11).



Figure 11. Frequency that classroom teachers reported employing different teaching practices.

Teachers in all cohorts indicated a moderate to frequent use of direct instruction or lecture in their

classrooms. They also reported a relatively frequent use of cooperative learning in their

classrooms. Student projects, though, varied between cohorts. Cohort 1 teachers reported the most

frequent use of this practice (93.0% at least moderately), followed by Cohort 2 (79.5%) and Cohort

3 (73.1%). A potential difference between cohorts may relate to the grade level within each.

Specifically, elementary school teachers (Cohort 1) may be more likely to incorporate student

projects as compared with the other two cohorts that have a mixture of middle and high school

teachers. Teachers also reported a fairly high occurrence of individualized learning, whether at a

desk or computer. Cohort 1 teachers reported the most frequent use of this practice (98.0%),

followed by Cohort 2 (94.9%) and Cohort 3 (88.7%) teachers. There were not significant

differences pertaining to teacher practice across time points for any grade group participants that

completed both the spring 2016 and spring 2017 survey.



In their focus groups, classroom teachers felt that the initiative provided more opportunities

to personalize, differentiate, and make learning more interactive; made student choice, creativity,

and autonomy easier to initiate; streamlined lesson planning, communication, and helped break

down language barriers; as well as “leveling the playing field” for students who need additional

assistance and confidence building.

All teacher groups remarked that their efficiency had increased as a result of the initiative

and was, in the words of a Lighthouse high school teacher, “giving them back time.” One

Lighthouse middle school teacher commented,

It really increases the efficiency of the whole record keeping and grading process. It cuts

time down. It calculates all your grades for you. You don’t have to compute anything. You

can see who’s turned things in quickly and easily. It really cuts down on all the logistical

organization.

As a result of changes made in teaching practices as compared to last year, teachers also felt that

they were able to deliver higher quality instruction. As one Phase 2 elementary teacher noted,

Lessons are involved and high level… Having access to introduce them [students] to topics

that are probably well above their level, that they have no background knowledge on, and

bring it to their level in some fashion has really helped us as teachers be able to now teach

this lesson, and get them to where we need them to be.

Summary. As conveyed through observations, interviews, focus groups, and survey items,

there has been an impact of S.T.A.T. on teacher practice. Principals in all groups described mindset

changes with their teachers, referencing increased collaboration with peers and more time spent

planning. S.T.A.T. teachers also described increased collaboration among teachers, in addition to

teachers continually willing to take risks with meaningful integration of technology. Teachers

described a change in their practices regarding personalizing instruction more frequently and

creating more interactive learning for their students.

Observation findings revealed that overall, teacher practices in spring 2017 were mostly

similar to the baseline observations for each cohort, with a few notable exceptions. Across the

eight subgroups that made up the cohorts, only three subgroups demonstrated significant

differences on any items between the baseline and most recent observations, all of which were part

of Cohort 3. Across the cohorts, teachers made fairly frequent use of coaching/facilitating and

similar (albeit somewhat less frequent) use of presentations. Higher-level questioning and higher-

order instructional feedback were both exhibited somewhat infrequently in the observations while

student initiated communication was exhibited noticeably more often. Use of flexible grouping

was rarely observed, if ever, in each of the cohorts.

As would be expected given their early experience with S.T.A.T. implementation, Cohort

3 classrooms tended to exhibit a less noticeable impact of the initiative on teacher practice as

compared with Cohorts 1 and 2. Cohort 3 teachers did, though, display coaching/facilitating

behavior to a similar extent as the other cohorts and actually relied less on presentations.



Measurable Outcomes: Digital Content

This section begins with classroom teachers’ reported usage of BCPSOne, then principal,

S.T.A.T. teacher, and classroom teacher focus group responses pertaining to technology

integration are presented.

BCPSOne usage. Classroom teachers in the three cohorts indicated their use of BCPSOne

through survey responses (see Figure 12). Across all responses, the most frequent use of BCPS

One was to deliver instruction customized to students’ needs (79.9% at least once a week),

followed by developing assignments (68.4% at least once a week). Less often was BCPSOne used

for developing formative assessments (55.4% at least once a week) and posting homework

assignments (33.7% at least once a week). Responses were fairly consistent across cohorts, though

Cohort 1 teachers tended to indicate more frequent use of BCPSOne to customize students’ needs

and develop assignments as compared with teachers in other cohorts. In contrast, Cohort 2 and 3

teachers more frequently reported the use of BCPSOne to post homework assignments as

compared with Cohort 1 teachers.

There were significant changes across time points for teachers that responded to both the

spring 2016 and spring 2017 survey items pertaining to BCPSOne usage. Specifically, teachers in

Lighthouse Grades K, 4, and 5 reported significantly less frequent use of the platform to deliver

instruction customized to students’ needs (p < .05) and to develop assignments (p < .05) in spring

2017 as compared with spring 2016. However, usage for both of these activities remained

relatively high. In contrast, teachers in Phase 2 Grades 1-3 reported a significantly more frequent

use of BCPSOne for developing formative assessments (p < .05) and developing assignments (p <

.01) in the spring 2017 survey as compared with the spring 2016 survey. The changes in Phase 2

Grades 1-3 may reflect continued refinement of BCPSOne usage with experience in S.T.A.T.

implementation.



Figure 12. Teachers’ reported use of BCPSOne.

Technology integration. The perceptions by participants of technology integration within

Lighthouse and Phase 2 elementary and middle schools, as well as Lighthouse high schools, as

gathered through interviews, are presented below. In addition, classroom teachers offered their

perspectives through survey responses and focus groups.

Principals. Principals were asked to comment on how technology had been integrated in

their classroom and to describe their greatest successes with technology integration. In addition to

the BCPSOne lesson tiles, several other programs were identified as part of the school’s efforts

towards technology integration. These included DreamBox, iReady, OneNote, and Movie Maker.

All principals at all levels indicated their schools had experienced some level of technology

integration.

Elementary principals for both Lighthouse and Phase 2 schools listed numerous ways their

schools were integrating technology (e.g., “research, organizing, creating rubrics, and success



charts,” “customizing assignments around interests, needs and strengths,” and “across subject

levels”). As one principal commented, “The greatest success is that all teachers have adjusted—

seasoned and new. Students are already wired to use it.”

Middle school principals also saw teachers and students integrating technology, using it to

better assist ESL students, as well as for writing assignments, “video recording,” “book

performances,” “news reports,” “video portfolios,” etc. One Lighthouse principal made a comment

that summed up several principal’s comments from both groups:

I love seeing students create their understanding by using all their digital tools. They are

no longer restricted to one way of doing work, and giving them that piece of autonomy

when it comes to their education has really made all of the difference.

A Phase 2 principal also had a summative comment concerning teachers that reflected other

principal’s views:

The greatest success we’ve had with this initiative is that it’s empowered teachers to share

with others what has been successful for them. Teachers are hosting others in their room

to model how they’re using the devices as a collaborative tool.

Both middle and high school principals also commented that technology integration had “leveled

the playing field” for students in that all students had access to resources and materials to support

their learning.

Two high school principals commented that students and teachers, for a variety of reasons,

had initially met technology integration with resistance. However, over time, this had changed.

One principal explained:

I think the coolest thing for me is that our seniors, when told we were going 1 to 1 were

kind of like: ‘Why? This is our last year in school. We don’t need this’. Some quite honestly

were very concerned about 1 to 1 because they live in some pretty rough neighborhoods

and were concerned about taking the computers home. But what has happened over the

course of the year is they have seen how this resource can be a game changer to them, and

I’m just talking about the device itself. So, when you add that with all the digital resources

that the school system has invested in there is just so much more information at kids’

fingertips.

A second principal noted that initially teachers were also resistant, but were becoming acclimated

to using technology in the classroom through support. The principal stated that teachers were now

supportive because “we have given a lot of support for teachers to be more competent and

confident in their work” and as a result “on every level, we have seen an increase in the seamless

integration.”

S.T.A.T. teachers. During interviews, Lighthouse and Phase 2 elementary school S.T.A.T.

teachers repeatedly reported successful utilization of the BCPSOne platform for lesson tiles,

student data, assessments, instruction, and resources:



We have been successful in looking at all of the digital tools that are offered because there

are so many that are offered in BCPSOne. We’ve come a long way since year one at what

we use and how we use it. We’ve done a lot this year in implementing Maker’s Space in

our building and bringing in devices and recording it along the way and documenting and

sharing that process and that’s been a shift in how kids are getting feedback and seeing

the progress of learning along the way.

They also indicated success in personalizing instruction, increasing student independence,

and the benefits of instant teacher feedback. Many S.T.A.T. teachers conveyed that there had been

a greater emphasis on using technology in a meaningful way this year, rather than just as a

substitute for pencil and paper. Two Phase 2 elementary school S.T.A.T. teachers reported that

their teachers have decreased device use this year with a joint goal of implementing blended

learning and utilizing technology when it is most effective. One S.T.A.T. teacher commented, “We

definitely use the devices, but we’re trying to use them in a purposeful way, not just having kids

on devices all day.”

Lighthouse and Phase 2 middle school S.T.A.T. teachers agreed with their elementary

counterparts that BCPSOne played a strong role in their technology integration during the past

year. One Lighthouse middle school S.T.A.T. teacher observed that their teachers had started out

using the ‘Substitution, Augmentation, Modification, Redefinition’ (SAMR) model to move

towards transformative integration and were progressing towards use of the ‘Technological

Pedagogical Content Knowledge’ (TPACK) model of technology integration. Another

commented:

As a Lighthouse School, a lot of people come to visit us, and one of the first things they ask

us is ‘how do you get kids to bring their devices to school every day?’ We don’t have to do

anything because kids want to bring it in. It’s fully integrated and a beautiful thing to see.

A Phase 2 Middle school S.T.A.T. teacher noted that the S.T.A.T. acronym had given some the

impression that the initiative was all about technology. Their purposeful PD was changing that

impression and allowing for a focus on student-centered learning in their school.

Lighthouse high school S.T.A.T. teachers commented that technology integration has been

up to individual teachers this year. They conveyed that some teachers sought out the technology

and had were eager to incorporate it with it while others had not. One reported that their teachers

were very open to trying new software and trying to see what worked best for their students.

OneNote, a digital note-taking application, was the one and only application mentioned by both of

these S.T.A.T. teachers.

In addition to BCPSOne, other programs or platforms that were identified by S.T.A.T.

teachers as being used in their schools included Board Builder, Digital Learning University,

Discovery Education, DreamBox, Engrade, Google Images, i-Ready, Makerspace, One Note,

Skype, and Socrative.



Classroom teachers. Through the survey, classroom teachers indicated the degree to which

technology was an integral part of their teaching practices and instructional planning and

administration (see Figure 13). Nearly all classroom teachers indicated that the use of technology

was at least a moderate part of their teaching practices (40.1% very strong) and instructional

planning and administration (58.7% very strong). While Cohort 1 teachers tended to indicate a

slightly stronger role of technology with teaching practices as compared with those in other

cohorts, teachers in the three cohorts reported comparable use of technology for planning and

administration. There was, though, a significant increase (p < .05) in the degree to which

Lighthouse Grade 6 teachers (Cohort 2) reported that technology was an integral part of their

instructional planning and administration in the spring 2017 survey as compared with the spring

2016 survey.

Figure 13. Teachers’ survey responses indicating the degree to which technology is an integral

part of their practices.

Classroom teachers indicated through open-ended survey responses their successes with

technology integration. The following themes were identified from Lighthouse and Phase 2

classroom teachers regarding successes with technology integration:

Differentiation was a key success mentioned by elementary, middle, and high

school classroom teachers. Specifically, teachers described their ability to

differentiate content, assignments, and the learning environment based on student

readiness, interest, or learning profile. For example, one Lighthouse elementary

teacher commented, “The greatest success is that the students are able to move at

their own pace. Also, I have been able to extend lessons by providing additional

information through their devices.” A Phase 2 middle school teacher commented:

I created several student-paced and student choice-based units on OneNote where

I could alter materials to suit student needs and provide in-the-moment feedback.

These materials were very engaging and included multimedia. But, what was the



most gratifying was the flexibility it gave me to reach both my higher students and

those who struggle.

As a Lighthouse high school teacher noted, a success is this teacher’s “ability to

differentiate instruction on a more personal level—I was able to push out tutorials

to select students without having to make them feel that they were somehow

inadequate.” Similarly, another high school teacher commented: “Special needs

students often have their needs met through technology.”

Leveraging tools and programs. Classroom teachers in all groups also described

their own use of various tools and programs to support instruction as a success. For

example, teachers mentioned the ability to have stories read to students through

devices, using BCPSOne tiles, content resources, tools for content areas, and using

technology for assessment purposes. Some teachers specifically mentioned their

ability to leverage affordances of technology for instruction. A Lighthouse

elementary teacher offered:

I feel that the greatest success this year is utilizing the visual display and

manipulations of digital objects to illustrate mathematical concepts. Students get a

different perspective of the concept and can relate the visual display to the

mathematical ideas. And at times the manipulation of digital objects is beyond what

we can do with physical manipulatives.

Another commented, “This year I feel as though my students used technology as

more of a modification and redefinition rather than for substitution.” A Phase 2

middle teacher described her use of technology:

I used Discovery Education quite frequently. I was able to post videos and other

activities on lesson tiles. I had a lesson tile for each day, which directly related to

the concept. Students who were absent were able to keep up with their work.

Students were able to preview up-coming lessons.

Teachers in other grade levels also mentioned the use of technology to support

students who had missed class and review materials from previous lessons.

Student impact. A third frequently mentioned theme in many teachers’ responses

pertained to the impact on their students, whether the success was in reference to

students’ technology skills, their ownership of their learning, or the improved level

of engagement. A Lighthouse elementary teacher commented, “Students are now

comfortable with technology and can use it without much teacher support,” while

another offered, “Because the students have used the technology in first grade they

were familiar with the logging in processes, their passwords, and with typing more

easily. This made it easier for the teachers in the next grade to begin use of

technology with the students.” Further, a Lighthouse high school teacher observed,

“Students are leaving this year much more computer-literate than before the year

began.” Teachers also mentioned the use of technology for students to explore



topics or advance their skills. For example, a Lighthouse middle school teacher

commented on the success of “students being able to do self-guided research,”

while another noted the “use of iReady to work on some of the higher-level

comprehension skills for my high achievers.”

Survey responses regarding perceptions of technology integration successes were

corroborated through classroom teacher focus groups. Teachers in all groups and grade levels

emphasized that they were more willing to try new ideas and programs, and that the range and

variety of use had greatly increased. All groups mentioned the use of BCPSOne. In lower grades,

other popular programs included APlus, Brain Pop, Creation Station, RazzKids, EdModo,

Calendar Math, One Note, DreamBox, Wixie, Audacity, and Word Builder. Middle grade teachers

mentioned the use of Kahoot, Quizzizz, Ascend, Duolingo.com, Office 365, DiscoveryEd, iReady,

Board Builder, ClassFlow, and Padlet. High school teachers also mentioned using Padlet and

Office 365, as well as Sketchpad.

Teachers spoke to how new ideas and programs they were willing to try supported student

learning. A Phase 2 elementary teacher noted,

I would say I’m proud of myself that I branched out. I took a Wixie class to learn about

how to use Wixie in my classroom. I am able to bring the stories alive for them more. We’re

able to branch out and do stories that these kids wouldn’t even understand the concept and

haven’t been through these life experiences or to these places and we can Google search

or Safari Video about it.

Teachers also described offering students a variety of resources for learning, and as a Phase 2

elementary teacher stated, the result has “opened up a whole new world of things we can do with

our kids.”

Collectively, teachers in all groups pointed out device use cultivated differentiation and

personalized learning, as well as “empowering” students. A Phase 2 elementary teacher noted,

My successes are through them [students]. I see everything they’re capable of and they

know more than I do [in terms of technology skills]. They teach each other. We made a

specialist chart in terms of what each student is best at. They see what they’re capable of,

what they can do, and are proud of that. When we got devices at the beginning of the year,

I would say ‘all these other people are teachers’. They were so excited when they were able

to help someone at their table. They become the expert, ‘go visit so and so because they

know how to fix that problem’. They get so excited when they’re able to take control of this

. . . It boosts their confidence.

A Phase 2 middle school classroom teacher added:

This is sort of a side note but I feel like part of the success that has come with technology

integration has involved putting me, and my students, on a fluid continuum of

teacher/learner. I think the kids have really enjoyed showing me new things and teaching

me sometimes what works best.



Buying in to the use of technology is also beginning to be seen as successful for teachers, as

illustrated by this response from a Lighthouse middle school classroom teacher:

From the beginning of the year until now, the amount of instruction that I have to give for

these type of things is definitely less. Now I can refer to a lesson tile, try it through the

calendar, put it in the turn-in bin, and you can access testing quizzes for your access ticket.

I can say all that now and no one bats an eye; in the first quarter of the year this was not

as easy, but we are getting better.

Technology integration challenges. Teachers also reported on the degree to which

students were observed using technology inappropriately (e.g., video games, off-topic Internet

browsing, etc.; see Figure 14) through the survey. Over three quarters (76.3%) indicated

inappropriate use of technology occurred at least occasionally. Frequency of occurrence was

greater by Cohort 3 teachers (86.1% at least occasionally), followed by Cohort 2 teachers (70.7%),

then Cohort 1 teachers (58.0%). This finding may relate to the age group in that high school

students (Cohort 3) may be more likely to engage in inappropriate use of technology. Alternatively,

Cohort 1 teachers may have more experience managing off-task behavior since they are in their

third year of S.T.A.T.

Figure 14. Teachers’ reported degree to which students were observed using technology

inappropriately.

If a teacher indicated on the survey at least occasional observed inappropriate use, they

were prompted to describe the issue(s). Just under two thirds of all teachers surveyed provided

examples of inappropriate or off-task device use. Most frequently, elementary, middle, and high

school students were mentioned playing games, visiting various unapproved websites, or using

programs on their devices such as the camera. Games that were mentioned included Cool Math,

ABCYa!, and Minecraft. As an elementary school teacher noted,

Students frequently access games or music websites in a separate tab on their Internet

browser when they are supposed to be using time for in class activities or during time

allocated to specific program usage. When students are asked to get off the websites or are

reprimanded, behavior issues ensue. Because of the limited control of device usage, many

students have access to unwanted/inappropriate websites, preventing learning from

occurring.



Another elementary teacher offered, “If students think you are not watching them they explore

other sites that they were not to be on.” Another noted, “Due to our inability to see what each

student is doing on their computers at all times, several students have gone to websites they were

not assigned to play games, or use the camera feature to take pictures.”

A middle school teacher noted, “While I am instructing (either direct instruction or

reviewing directions) students are playing games on their devices. As soon as students finish their

work, the device comes out and students are playing games.” Another commented,

They play games and Google in place of doing the assigned tasks. The second I walk away

to help or to do small group instruction they are back to playing. Most 6th graders are not

mature enough to be left unmonitored to do independent work while small group

instruction is taking place.

Another noted:

[Students] tend to gravitate to their devices in the attempt to access inappropriate, non-

curricular websites, breaching systemic firewalls, and/or to play non-curricular, but

competitive video games, unless they are regularly, even daily reminded that the devices

are instructional tools which are only to be used to facilitate their learning and mastery of

the learning objectives, and that said devices will be confiscated when and if students use

them inappropriately.

A high school teacher commented that students are “more focused on computers than lessons. No

interest in learning, more interested in using the computer to watch videos, take pictures, and do

their make-up.” Another noted, “While students are working on an assignment, I have frequently

observed them playing video games, doing web searches unrelated to content, and other activities

that distract from the assignment.”

High school teachers also mentioned the use of personal cell phones in addition to device

use for off-task behaviors. As one high school teacher observed,

It is most difficult to manage behavior with phones. As a school with devices, I believe that

there should NO reason for students to have their phones. However, teachers are not

allowed to take phones and students rarely give them up willingly. The devices are fabulous

and should replace the need for students to use their phones, yet that's not how students

use them.

An additional inappropriate use described by a small number of middle and high school

teachers related to irresponsible use of devices academically. As a middle school teacher noted,

“One of the biggest problems is the students' ability to access Internet search tools while taking

online assessments through BCPSOne. It is not uncommon for students to search, copy, and paste

the answers to assessment questions.” As one high school teacher observed, “students copy work

from websites,” and another stated, “I also struggled giving tests online because it is hard to

monitor whether they are looking the answers up or not through Google.” A third high school

teacher commented, “I would like access to my students' computers in order to ensure that they



are staying on task during quiz assignments. Students have easier access to information and have

been caught plagiarizing more frequently.”

Teachers at all levels described challenges with monitoring device use in classrooms,

particularly when students were to complete a task independently or when the teacher was working

with a small group of students. In terms of how students were engaging in these off-task behaviors,

strategies ranged from simple, such as quickly switching windows when a teacher approached, to

more extreme, such as finding workarounds to bypass the BCPS firewall or downloading Psiphon,

a program that allows the user to have open access to the Internet.

Teachers in all grades described the challenge of classroom management and how to

address the issue of inappropriate or off-task device use. For example, an elementary teacher

commented,

We have tried countless ways to get the students to use their devices appropriately. We still

use them during daily instruction at certain points, but the problem is that one teacher

cannot always be monitoring 25 device screens at a time. Even if I was able to just walk

around (which I'm not because I will usually pull small group instruction during that time),

the students can be sneaky. I frequently find them on gaming websites, searching things

completely off topic, on the photo booth app, etc. while they are supposed to be completing

an academic assignment. We have tried consequences for getting caught, positive

reinforcement (such as a "DreamBox Champion of the Week"), but even still, many students

get so distracted by the devices and use them inappropriately.

A middle school teacher noted, “It is impossible to be a full-time educator and a monitor at the

same time,” similar to an observation made by another middle school teacher: “we cannot monitor

student screens. This makes classroom management difficult.” A high school teacher commented,

Students are frequently playing games or finding ways around restrictions set by the

school. It is challenging to monitor 30 screens at once to ensure students are on task and

using the devices to develop a better understanding of the content.

Finally, a high school teacher stated,

Responsibility for addressing these issues is left mostly to the teachers, who are already

overwhelmed with planning, grading, etc. The number of students who commit violations

is so high and the steps that teachers must take before seeking help from administration

makes it difficult for teachers to realistically and consistently address these problems.

Some schools have stricter policies that allow teachers to confiscate devices during class

and to seek immediate help from admin (by calling the office or writing an office referral)

if students refuse to comply, but sadly this is not a universal BCPS policy.

Classroom teachers also described their challenges with technology integration through the

survey and, similar to the frequency of off-task/inappropriate use mentioned above, they viewed

this aspect as the greatest challenge. Additional challenges expressed included technical issues and

a lack of support. These themes are further discussed below.



Off-task device use. Teachers at all levels described the challenge of monitoring

and managing device use during instructional hours, and their comments reflected

those when asked to describe off-task/inappropriate use above.

Technical issues. Some of the technical issues expressed by middle and high school

teachers centered on students’ lack of accountability with devices, such as returning

to school with a depleted battery, forgetting the device at home, or breaking the

devices. Other technical issues mentioned by teachers at all grade level included

slow Internet or BCPSOne not functioning.

Lack of support. Teachers at all levels conveyed feeling overwhelmed and not

supported with technology integration. Some teachers described not having enough

time for planning, as noted by a Lighthouse middle school teacher: “TIME!! More

planning time is definitely needed!!!” Others mentioned the challenge of attempting

to learn new approaches to instruction along with other initiatives. A Lighthouse

elementary teacher described the struggle of “incorporating the new grading system

at the same time as technology,” while another noted, “My greatest challenge is just

not taking on too much at one time. Learning each new innovative ‘thing’ at a time

rather than trying to do it all at once.” Others echoed this sentiment, as a Lighthouse

middle school teacher described the challenge of “deciding which resources to use

and which to pass on. There were plenty of resources available but it felt as though

I was supposed to utilize as many as I could rather than focusing on/mastering a

few. I eventually minimized the resources I utilized.” Similarly, other teachers

specifically referenced the need for further PD on the effective integration of

technology rather than PD on a multitude of tools. As a Lighthouse high school

teacher offered, “I was not introduced very deeply into the different ways I could

integrate technology into my classroom. I want to find new engaging ways to use

technology in my classroom.” This sentiment was echoed by teachers at other grade

levels, such as the Lighthouse elementary school teacher that commented, “More

guidance/examples of what uses of technology fall in the latter part of the SAMR

model.”

Summary. Results from surveys, interviews, and focus groups indicated that all participant

groups perceive an improvement in the access to and use of digital content and resources. In terms

of successes with technology integration, principals referenced the use of BCPSOne and programs,

particularly in terms of student-conducted research and students having options in how they

demonstrated their learning. S.T.A.T. teachers also mentioned the success of using BCPSOne,

such as to personalize instruction and encourage student ownership of their learning. While many

referenced specific digital programs or tools, one S.T.A.T. teacher emphasized the meaningful

integration of technology as a success. Similar to principals and S.T.A.T. teachers, classroom

teachers described their successes with technology integration through their ability to differentiate

learning and provide the opportunity for students to learn at their own pace. Teachers also

described the positive impact on students as a success, such as students taking ownership of their

learning, improving their technology skills, and increased engagement.

Teachers did express challenges with technology integration, most notably through

frequent reports of off-task/inappropriate use of devices. While devices bring students’ access to



a wealth of information and resources for learning, teachers struggle with how to ensure that

students do not abuse this access by engaging in activities that are inappropriate or not

instructional, whether playing games or accessing unacceptable websites. Less frequently

mentioned challenges included technical issues and feeling a lack of support, whether through

planning time or PD.

Measurable Outcomes: Student Engagement

This section begins with results of the classroom observations in Cohorts 1, 2, and 3.

Findings from interviews and focus groups, the classroom teacher survey, and behavioral data are

then presented.

Classroom observations. Observers assessed student engagement using five OASIS-21

items that included the degree to which they use digital tools for learning, multiple modes of

student responses, independent work, collaborative learning, and student discussion. Results are

described by group and comparisons across time points are reported where applicable. Data from

the spring 2017 classroom observations for each of the cohort groups are presented in Figure 15

and further discussed below.



Figure 15. Frequency of extensiveness observed for OASIS-21 items related to student

engagement in spring 2017.

Cohort 1. Results for classroom observation items concerning student engagement in

spring 2017 were mostly similar to baseline observations from fall 2014. Overall, students in

Cohort 1 classrooms were observed with moderate frequency using digital tools for learning such

as laptops and tablets. In just over 70% of classrooms students were observed using digital tools

and technology and in about one-third of classrooms students made frequent or extensive use of

these tools. When used, digital tools were predominantly (87.0%) used for independent activities

and rarely (13.0%) were devices used for collaborative activities. Multiple response types, whether

verbal, physical, or through technology, were observed in fewer than half of all Cohort 1

classrooms, and in the classrooms where this was observed, it was most often used sparingly.



The three remaining student engagement items on the OASIS-21 dealt with specific types

of student learning activities. Independent work was used with much greater frequency than

collaborative learning or student discussion. This activity was observed to at least some extent in

over 85% of the classrooms and was used frequently or extensively in over 60% of the classrooms.

Student discussion, though used with substantially less frequency than independent work, occurred

comparatively more frequently than collaborative learning.

The only area in which student engagement observation ratings for spring 2017 were

significantly different from the baseline fall 2014 observations was with the frequency of student

discussion on a teacher prompted, high-level topic. In this area, student discussion was

significantly more prevalent in classrooms in spring 2017 observations compared with those in fall

2014 (p < .05). In spring 2017, student discussion was observed in roughly 60% of classrooms and

about one quarter of classrooms made frequent or extensive use of this activity (see Figure 16).

Figure 16. Frequency of student discussion observed for OASIS-21 items in Lighthouse Grade 1-

3 classrooms.

Cohort 2. With a few notable exceptions, results for classroom observation items

concerning student engagement in spring 2017 were mostly similar as compared with baseline

observations. Overall, students in just under 60% of Cohort 2 classrooms were using digital

tools/technology at least occasionally and in about 30% of classrooms students made frequent or

extensive use of these tools. As with Cohort 1, most frequently (85.5%) devices were used by

students independently. Most classrooms made only rare or occasional use of this practice.

As with the Cohort 1 classrooms, in Cohort 2 classrooms independent work was used with

much greater frequency than collaborative learning or student discussion. Conversely, both

collaborative learning and student discussion were observed infrequently as these activities were

observed in only about 40% of classrooms.

Phase 2 Grades 1-3 classrooms demonstrated significantly more extensive student use of

digital tools for learning in the spring 2017 observations compared with those at baseline in spring

2015 (p < .05) (see Figure 17).

Figure 17. Frequency of students using digital tools for learning observed for OASIS-21 items in

Phase 2 Grades 1-3 classrooms.



Lighthouse Grade 6 classrooms also demonstrated significantly greater use of students

providing multiple modes of responses in the spring 2017 observations compared with those at

baseline in fall 2015 (p < .05; see Figure 18).

Figure 18. Frequency of multiple modes of student responses observed for OASIS-21 items in

Lighthouse Grade 6 classrooms.

Cohort 3. Results for Cohort 3 were somewhat similar between the most recent

observations and baseline observations from spring and fall 2016. Students were observed

somewhat frequently using digital tools for learning. In just over 60% of classrooms, students were

observed using digital tools for learning, and in 40% of these classrooms students used them

frequently or extensively. As with the other Cohorts, devices were predominantly (87.2%) used

for independent work. Multiple modes of student responses were observed to a lesser extent. As

with Cohort 1 and 2 classrooms, independent work was observed with much greater frequency

than collaborative learning or student discussion.

Select Cohort 3 subgroups had significantly different ratings in the areas of digital tools for

learning, multiple modes of student response, and use of student discussion. First, as shown in

Figure 19. Phase 2 Grade 6 classrooms demonstrated significantly more extensive student use of

digital tools for learning in the spring 2017 observations compared with those at baseline in spring

2016 (p < .05).



Figure 19. Frequency of students using digital tools for learning observed for OASIS-21 items in

Phase 2 Grade 6 classrooms.

Phase 2 Grade 6 classrooms also demonstrated significantly less use of providing multiple

modes for student responses in the spring 2017 observations compared with those at baseline (p <

.05; see Figure 20).

Figure 20. Frequency of multiple modes of student responses observed for OASIS-21 items in

Phase 2 Grade 6 classrooms.

Phase 2 Grade 6 classrooms also demonstrated significantly less frequent use of student

discussion in the spring 2017 observations compared with those in spring 2016 (p < .05; see Figure

21).

Figure 21. Frequency of student discussion observed for OASIS-21 items in Phase 2 Grade 6

classrooms.

Perceptions of student engagement. Classroom teachers in these groups responded to

survey items and conveyed perceptions through focus groups regarding their perceived impact on

student engagement and behavior. In addition, the perceptions of principals and S.T.A.T. teachers

within these schools, as gathered through interviews, are presented.



Principals. According to the principals interviewed, the S.T.A.T. initiative was regarded

as being overwhelmingly positive for all groups, with the exception of one elementary Phase 2

principal, who felt results were “inconsistent” for her school. She explained: “It (S.T.A.T.) is

effective in an inconsistent way at my school and specific to individual teachers as far as how

effectively or what impact it has on student engagement.” For all other groups, the impact was

seen as particularly strong in the areas of student engagement and collaboration. Principals in all

groups, at all levels viewed S.T.A.T. as more than just a technology initiative. One elementary

Lighthouse respondent replied to the question on the level of engagement impact stating,

“Surprisingly, I think it’s really had a very positive effect, and I say surprisingly just because of

the way most people think about computers, devices, etc., and that it becomes a more isolated

individual experience.” Another elementary Lighthouse principal noted:

The levels of engagement have increased, and because of this (students) show deeper

understanding of content . . . I definitely have seen an increase in student ability to

communicate. As they’ve engaged in the instruction, they’re collaborating with one

another, compromising and verbalizing their understanding of different subject areas.

That’s been exciting, to listen to those conversations.

All other principal groups, whether Lighthouse or Phase 2, also reinforced the observation that the

S.T.A.T. initiative increased student engagement and collaboration because “learning is tailored

and customized,” “student-centered,” more “creative,” and students are “motivated” as well as

“empowered”. Principals felt that increases in engagement and collaboration were also due to the

fact that teachers are more “confident”, “less nervous,” and now have a better ability to introduce

“real world learning” and “experiment with options”.

S.T.A.T. teachers. The Lighthouse elementary S.T.A.T. teachers interviewed conveyed

that the initiative was “definitely” having an impact on student engagement and increased

collaboration in their classrooms. S.T.A.T. teachers described an increasingly effective

implementation of expanded student choice, personalized learning, and collaboration between

students, with a move to have students take ownership of their own learning. One S.T.A.T. teacher

stated, “The S.T.A.T. initiative definitely helps student engagement.” Another commented:

They’re really working to communicate with each other and engage in ways they hadn’t

been asked to prior to S.T.A.T. I think teachers have a better sense of what that

collaboration can look like and have a true understanding of collaboration and getting

kids ready for the real world, by providing authentic opportunities to work together in

project-based learning or in maker’s space in meaningful ways.

Phase 2 elementary school S.T.A.T. teachers agreed with their Lighthouse counterparts

that the S.T.A.T. initiative had increased student engagement and collaboration. They stated that

students were embracing student-centered learning, were accessing content in new ways, and were

directing their focus on P21 skills. One S.T.A.T. teacher stated, “I think for the students, as far as

presentation purposes, using the devices, has been really motivating for them and really kept them

on task—different than sitting with pencil and paper and writing.” One respondent felt that

technology use in the classroom was inhibiting student collaboration, stating, “They can’t



collaborate on the device like they can without the device. Even though there are lots of

applications, they’re not sophisticated to use it to collaborate like they’re intended, in my opinion.”

Lighthouse middle school S.T.A.T. teachers also described a positive impact that the

S.T.A.T. initiative was having on both student engagement and collaboration in the classroom.

One commented, “I think it has had a tremendous impact at my school in the way the teachers are

using technology.” Phase 2 middle school S.T.A.T. teachers also observed a positive impact in

their schools, largely due to the introduction of technology. One S.T.A.T. teacher remarked,

“Teachers have started taking more risks and the students, therefore, have been able to collaborate

more often and are engaged more in the activities.”

Lighthouse high school S.T.A.T. teachers stated that this year they had focused more on

technology integration than on student collaboration with one remarking, “At first I believe that it

took away from collaboration, but now teachers have a better idea of how to balance that.” Another

stated, “The amount of time working collaboratively is similar but more effective because of

targeted lesson planning.”

Classroom teachers. Elementary, middle, and high school classroom teachers were almost

evenly split as to the impact S.T.A.T. has had on student engagement. Some teachers felt that,

“engagement had waned,” “students are more engaged with technology than each other,” and that

the initiative “worked better for some subjects than others.” One teacher noted, “Just because they

are using a device or talking does not mean they are engaged.” Several others commented that

there were issues with keeping students on task, and “management trumps engagement.”

Conversely, slightly more than half of Lighthouse teachers did note increased engagement,

commenting, “Overall, engagement has increased”, or “engagement has increased dramatically,”

and “there are positive impacts on engagement.”

Phase 2 elementary classroom teachers overall also seemed to feel engagement had

improved at some level. One teacher commented, “I can keep them (students) engaged differently

every single day. They are not bored. I don’t think any kid comes to school thinking ‘I can’t believe

I’m sitting in this lesson’. I can find different ways to engage them.” However, some teachers

across groups noted, “Collaboration has not improved.” One Lighthouse teacher stated, “Kids are

working individually; the amount of group work and collaboration has gone down significantly.”

Phase 2 middle school responses were also mixed. Several Phase 2 middle school teachers agreed

collaboration “had both improved and worsened as a result of having the devices.”

Another Lighthouse elementary teacher summed up what a variety of others expressed,

“We need to find a balance of when to use it (technology); how often; . . . what is too much, and

what is not enough” in order to stimulate engagement and collaboration. In many respects, teachers

in all groups were still trying to determine what works best for them individually in this area.

Student behavior. Participants from Lighthouse and Phase 2 S.T.A.T. schools provided

their perceptions regarding the impact on student behavior. In addition, behavioral data, including

attendance, suspensions, and office referrals, were obtained from BCPS. Behavioral data for

Cohort 1 did not include May’s Chapel elementary school because the school was not open during

the pre-program year (2013-14).



Principals. Less than half (9 of 22) of principals at all levels interviewed indicated that

S.T.A.T. had not had an impact on student behavior in their classrooms this year. Elementary

principals in both Lighthouse and Phase 2 schools made comments to the fact that, “it’s been a

mix.” One Lighthouse principal stated, “I would not be honest if said that all behavior problems

have been resolved, but the initiative has had a huge impact on academics.” A Phase 2 principal

felt that there were external factors that affected behavior improvement and stated,

If you asked me, the last two or three years we saw a drop-in behavior referrals to the

office, but this year we just haven’t seen it. I don’t think it’s as much about S .T.A.T., but a

changing demographic in our schools. We’re having more and more students arrive that

have had trauma experiences, so you can’t negate that and I don’t believe the S .T.A.T.

initiative is going to reduce that.

A second Phase 2 elementary principal reported that results could often be correlated to teacher

experience and engagement with the process. She commented:

On the positive side of the spectrum, students who are in classrooms where teachers are

using (technology) effectively, students are interested, they’re motivated, responding in a

positive way in the classroom and learning environment and demonstrate appropriate

learning behaviors, whether they are with or without technology, which has positively

colored their learning experience.

A third Phase 2 elementary principal felt that the impact on student behavior had not been positive

or negative, but had “remained steady.”

Lighthouse middle school principals also conveyed mixed views. One principal

commented that the impact had been “minimal” because some students have used their devices to

try to “view prohibited content, download prohibited files, or use it inappropriately”. The principal

commented:

While I love the way it is transforming the way our students learn, I will say that we have

had a few issues in device behavior since the implementation. Our teachers have to be

extra vigilant to ensure devices are used properly.

Another Lighthouse middle principal noted that even though students were engaged and paying

attention in class, there was no observed reduction in fights and disturbances. He stated, “I don’t

know that I’ve seen a change in behavior.”

At the high school level, two Lighthouse principals had equivocal views in terms of

improvement in behaviors. One principal stated:

Even before this, we didn’t have large disruptions, but I can say that some of the referrals

that we’re getting are still related to off-task behaviors on cell phones. As we see

engagement increase we see unwanted behaviors decrease so that’s a positive impact,

certainly.



A second principal noted that, anecdotally, he felt student behavior had probably improved as a

result of the improved instruction he is witnessing, but he does not have “concrete data” necessarily

confirming improved behavior.

Some principals, however, did see improvements in behavior across all groups at all levels.

They noted that behavior referrals had decreased because students “do not want to miss out on

learning and the engagement process.” A Lighthouse elementary principal commented, “with

increases in student engagement, there is a direct correlation to student behavior,” while a third

principal noted, “My behavior referrals have gone down and stayed down at least 70-80% and the

folder now is nowhere as large as it used to be.”

Middle school principals were more in agreement (7 of 9) that behavior had improved. One

Lighthouse middle school principal made a comment that seemed to summarize the positive effect

the program had on the majority of schools:

It [S.T.A. T.] has improved their behavior. Students who have typically been disengaged

feel more empowered. Students who wouldn’t be engaged feel more accepted. Showing

teachers they have other options has really improved student behavior in our school.

A Phase 2 middle school principal went further, stating:

It is my belief that the students understand the material more. Often times we see

behavioral issues with children who are frustrated because they do not understand. Having

fewer children frustrated because they do not understand the material is making a world

of difference in the behavior of students.

Lighthouse high school principals showed the least agreement on behavior improving

either feeling there was negligible or no validation at present to support improved behaviors.

Anecdotally, two principals felt that behavior would improve over time. One principal viewed

behavior improvement through a slightly different lens stating, “Initially it was a change, a

paradigm shift, but we see more and more students acting like they are now the leaders of their

classrooms and active participants.”

S.T.A.T. teachers. Lighthouse elementary S.T.A.T. teachers described the S.T.A.T.

initiative’s positive, indirect impact on student behavior. This improvement was attributed to

several factors including student-centered classrooms, greater student engagement, immediate

teacher feedback, and differentiated/individualized learning. Some of these S.T.A.T. teachers

noted a relationship between the implementation of the initiative and a decrease in negative

behaviors and office referrals, with one teacher stating, “We found with the S.T.A.T. initiative, we

were able to decrease student referrals by 72%, within the very first year.” Phase 2 elementary

S.T.A.T. teachers seemed divided on the impact that was being felt, with one teacher remarking,

“I don’t know that it’s really had any impact on student behavior” and another remarking “Overall,

if there was a trend in my building, I would say that it has been positive, especially in the

community that I serve there is not always a positive association with school and learning.”



The majority of middle school S.T.A.T. teachers from both Lighthouse and Phase 2 schools

reported that the initiative was having a positive effect on student behavior, largely through higher

engagement due to technology use in the classroom. One Phase 2 middle school S.T.A.T. teacher

remarked:

You can tell there is a shift in climate. There is an increase in motivation level and you can

see kids wanting to stay in the classroom because they are more engaged. They don’t take

the opportunities during transition to show off or cause disruption. Our block schedule

also helps with that because we have time to really dig into content. It helps a lot with

discipline that students are motivated.

A Lighthouse middle school S.T.A.T. teacher added:

At times, we have definitely improved behavior in the classroom, but sometimes it also

exacerbates student chatting because sometimes they get really excited about their work

and sometimes students have to learn how to use the devices in successful ways.

Lighthouse high school S.T.A.T. teachers were divided in their perceptions of the impact

on student behavior. One felt that the initiative had improved classroom behavior due to the use of

targeted lessons. The other believed that students needed more time to master the initiative’s new

grading policy, stating, “I feel like there has been a bit more apathy this year because they do not

fully understand the grading model and they have not bought into the initiative.”

Classroom teachers. Survey responses revealed that slightly more than two thirds of

teachers tended to agree that student behavior had improved this year. Cohort 1 teachers, in their

third year of S.T.A.T., were more likely to agree (81.0%) than Cohort 2 teachers (67.4%), and

Cohort 3 teachers (57.7%). Teachers further elaborated during focus groups with several teachers

across Lighthouse and Phase 2 schools noting that “Although technology has its benefits, the

devices add a new layer of distractions that can make management challenging.” These distractions

included students off-task, technology issues (e.g. devices not working, not charged, or in need of

repair; theft and loss, etc.), accessing inappropriate sites, filters that block appropriate sites,

unsuitable advertising popping up in searches, as well as students using devices when they were

not supposed to be online. One Phase 2 elementary teacher remarked “A class can look like it’s

full of busy, actively engaged students but when you walk around you can discover that a large

percentage of them are actually goofing off.” Several others noted that students were adept at

hiding inappropriate use, and often did not take care of devices. Teachers in all groups echoed a

comment made by a Lighthouse middle school teacher that “You have to move around constantly

to make sure they’re doing what they’re supposed to be doing.” Several other teachers across

groups remarked that social interaction was not developing as it should and this affected classroom

management.

At the same time, a number of teachers noted that in many instances, “behavior is better,

they’re quiet, usually engaged,” and that going back would be like stepping into “the dark ages.”

A Phase 2 middle teacher stated,



It is two pronged, or maybe more pronged, perhaps fork level—I’m more organized, I’m

more aware of students’ levels, more in touch with parents, have greater access to exciting

material and that all leads to a more engaged, well-behaved classroom.

Moreover, teachers across groups noted that the technology has helped with lesson

planning and made it much easier to quickly adapt or restructure lessons—which in turn had made

management easier when lessons need to be quickly adapted.

Attendance. The percentage of students meeting the attendance cutoff of at least 94% of

the days enrolled declined slightly between pre-program years and the present years for all three

cohorts (see Figure 22). The decline was greatest within Cohort 3 schools (-5.70pts) who are in

their first year of S.T.A.T. implementation.

Figure 22. Attendance rates for students in Cohorts 1, 2, and 3.

Cohort 1. Students in Cohort 1 schools (Lighthouse Grades 1-3) exhibited a slight

decline (-1.69) in attendance rates. There was not a statistically significant

difference in pre-program attendance to year three of implementation.

Cohort 2. Students in Cohort 2 schools exhibited a slight decline (-2.26) between

the pre-program year (2014-15) and the second year of S.T.A.T. implementation

(2016-17). The decline was greatest in Lighthouse Grade 6 (-2.79), followed by

non-Lighthouse Grades 1-3 (-2.47). Lighthouse Grades K, 4, and 5 exhibited an

increase in attendance rates (0.40). There were not statistically significant

differences in attendance for any group.

Cohort 3. Students in Cohort 3 schools exhibited a decline (-5.7pts) between the

pre-program year and the first year of S.T.A.T. implementation. There was a

statistically significant decline in the percentage of students meeting the 94% cut-

off rate for Lighthouse high schools (-8.13), non-Lighthouse Grades K, 4, and 5 (-

5.62), and non-Lighthouse Grade 6 (-6.52). While Lighthouse Grade 6 (-2.79) and

Grade 7 (-3.50) also exhibited declines in attendance, the differences were not

significant.

Office referrals. The proportion of students that received office referrals across years

changed based on cohort (see Figure 23). While Cohorts 1 and 2 exhibited increases (2.95 and



0.39, respectively) from the pre-program year through the present year, Cohort 3 exhibited a slight

decline (-0.64).

Figure 23. Office referral rates for students in Cohorts 1, 2, and 3.

Cohort 1. The proportion of Cohort 1 students that received office referrals

increased from the pre-program year to the end of the third year of implementation.

There was not a statistically significant difference in the proportion of students

receiving office referrals from the pre-program year through the present year.

Cohort 2. Cohort 2 schools exhibited a slight increase in office referral frequency

between the pre-program year and the second year of implementation. There were

increases in referral frequency within non-Lighthouse Grades 1-3 (+0.18), and

Lighthouse Grades K, 4, and 5 (+4.78), In contrast, Lighthouse Grade 6 exhibited

a decline in the frequency of office referrals from the pre-program year through the

end of the second year of implementation (-1.29). There was not a statistically

significant difference between pre-program and year two referral rates for any

group.

Cohort 3. In contrast with Cohorts 1 and 2, Cohort 3 schools overall exhibited a

decline in office referral frequency during their first year of S.T.A.T.

implementation. Non-Lighthouse Grade 6 had a decline (-9.37) in referral

frequency. There were, though, increases in referral frequencies for non-Lighthouse

Grades K, 4, and 5 (+0.83), Lighthouse Grade 7 (+3.37), and Lighthouse high

school (+0.72). There was not a statistically significant difference between pre-

program and year two referral rates for any group.

Suspensions. There were slight increases in suspension frequency across all three cohorts

from the pre-program year to the present year (see Figure 24).



Figure 24. Suspension rates for students in Cohorts 1, 2, and 3.

Cohort 1. Similar to the trend exhibited for office referrals, there was a slight

increase in suspension frequency from the pre-program year to the third year of

S.T.A.T. (+0.47). There was not a statistically significant difference in suspension

rates between the pre-program year and the third year of implementation.

Cohort 2. Similar to Cohort 1, there was an increase in suspension frequency

between the pre-program year and the second year of S.T.A.T. (+0.62). While

Lighthouse Grade 6 exhibited a decline (-0.34), non-Lighthouse Grades 1-3 and

Lighthouse Grades K, 4, and 5 exhibited increases (+0.64 and +1.30, respectively).

The increases in the elementary grades were statistically significant (p < .05).

Cohort 3. Cohort 2 schools exhibited an increase in suspension frequencies between

the pre-program year and first year of implementation (+0.75). The increases in

non-Lighthouse Grade 6 (+2.43) and Lighthouse high schools (+1.95) were not

statistically significant. The increase in non-Lighthouse Grades K, 4, and 5 (+0.51),

was significant (p < .05). There was, though, a non-significant decline in suspension

frequency within Lighthouse Grade 7 (-1.20).

Summary. Though a few notable differences were found between the spring 2017

observation results and baseline observations, most recent observations for student engagement

were mostly similar with those gathered at baseline for Cohorts 1, 2, and 3. Between the cohorts,

students generally made fairly frequent use of digital tools for learning and when devices were

used, they were predominantly used by students engaging in independent work. It is not surprising,

then, that teachers generally provided somewhat infrequent opportunities for multiple modes of

student response. For each of the Cohorts, independent work was observed with much greater

frequency than collaborative learning or student discussion. While collaborative learning was

observed less frequently, this finding does not mean that students did not interact with one another.

Instead, this finding indicates that students did not work together towards a common goal.

Principals and S.T.A.T. teachers across groups conveyed a positive impact on student

engagement. They attributed this improvement to more student-centered and personalized

learning. Classroom teachers, though, varied in their perceptions of student engagement. Some felt



there was a positive impact whereas others felt that engagement has waned. Teachers appeared to

still be determining how best to integrate technology to engage students as well as how to

encourage collaboration while students used technology.

The perceived impact on student behavior was much more varied between principals,

S.T.A.T. teachers, and classroom teachers across grade levels as compared with student

engagement. While some viewed a positive improvement, others did not and some related behavior

issues to device presence and associated distractions. Behavioral data, though, indicated there were

changes in attendance, referrals, and suspensions. While many increases were slight, there were

some statistically significant increases in suspension and attendance rates within Cohort 2 and 3

schools. Based on the interview and focus group data, attendance, office referrals, and suspension

rates may be influenced by changing demographics and other unknown factors to a much greater

degree than student reactions to S.T.A.T. That is, the extensive principal, teacher, and S.T.A.T.

teacher reactions generally suggest, with exceptions at some schools, improved student classroom

engagement and behavior.

Measurable Outcomes: P21 Skills

This section begins with results of the classroom observations in Lighthouse elementary,

Phase 2 elementary Grades 1-3, and Lighthouse middle schools. Findings from interviews and

focus groups, and the classroom teacher survey from these groups are then presented.

P21 skills require more extensive lesson planning on the part of the teacher and are not

expected to be as common as traditional approaches to instruction such as teacher presentations of

information. Consistent with the temporal logic model, one would expect little impact on P21 skills

in initial years of implementation (such as with Cohort 3) but a stronger impact through experience

with S.T.A.T. implementation (as with Cohorts 1 and 2).

Observations. The first P21 item assessed was problem-solving skills. This is defined as

students using multiple resources, using resources effectively, and engaging in critical thinking in

order to solve a problem. The second and third items under P21 skills pertained to project - and

inquiry-based approaches to instruction. Project-based approaches center on the creation of a

tangible product (e.g., report), which results from an inquiry or question. A distinguishing feature

is that project-based approaches to instruction involve an extended completion time (e.g., more

than a single class period). Inquiry-based approaches involve in-depth student exploration of a

question or topic, development and asking further questions, and conducting research to answer

the question. The final P21 item assessed the extent to which learning incorporated authentic/real

world contexts during the observations.

For these items, results are described by group and comparisons across time points are

reported where applicable. Data from the spring 2017 classroom observations for each of the

cohort groups are presented in Figure 25 and further discussed below.



Figure 25. Frequency of extensiveness observed for OASIS-21 items in spring 2017 observations

related to P21 Skills.

Cohort 1. Results for classroom observation items concerning P21 skills in spring 2017

were similar as compared with baseline observations from fall 2014. During the spring 2017

observations, P21 skills were observed infrequently in the Cohort 1 classrooms. This was

particularly the case with regard to the use of problem-solving, project-based approaches to

instruction, and inquiry-based approaches to instruction. Relatively speaking, learning

incorporating authentic/real world contexts was exhibited more often than these other practices,

but was also observed infrequently.

Cohort 2. Cohort 2 observations of P21 skills were comparable to baseline observations

and were rarely observed. Learning incorporating authentic/real world contexts was exhibited

somewhat more often, but was also seldom observed.

Cohort 3. Results for Cohort 3 were mostly similar to baseline observations. As for Cohort

1 and 2 classrooms, Cohort 3 classrooms seldom exhibited activities associated with P21 skills.

Learning incorporating real-world contexts, however, was evident to at least some extent in about

one-third of classrooms and was observed frequently or extensively in just under 15%.



The only area in which significant differences were found between the most recent

observations and those at baseline occurred in Phase 2 Grade 6 classrooms with regard to learning

incorporating real-world contexts (see Figure 26). In these classrooms, significantly fewer

instances of learning incorporating authentic/real world contexts were observed in the spring of

2017 compared with what was observed in spring 2016 (p < .05). Specifically, at baseline learning

activities of this type were observed to at least some extent in about two-thirds of classrooms and

were observed frequently or extensively in over 40%. In contrast, during the most recent

observations this type of activity was only observed in about 25% of classrooms and only about

20% showed frequent or extensive displays.

Figure 26. Frequency of learning incorporating authentic/real world contexts observed for

OASIS-21 items in Phase 2 Grade 6 classrooms.

Perceptions regarding the impact on P21 skills. Principals, S.T.A.T. teachers, and

classroom teachers of these groups provided their perceived impact of the S.T.A.T. initiative on

P21 skills through interviews and focus groups. In addition, classroom teachers responded to a

survey item regarding the impact.

Principals. Several principals at all levels noted that it was challenging to ascertain whether

P21 skills had increased as a result of S.T.A.T. At the elementary level, one Lighthouse principal

could see “no major difference” on student mastery, and two other principals felt that their schools

were still “growing” in this area or that “it was a work in progress”. Two Phase 2 principals noted

that either results were “inconsistent” or the principal had “to assume right now that boys and girls

are doing more of those skills because the curriculum demands it, but I don’t have any data to

prove it.” A third elementary Phase 2 principal offered:

It depends on teacher capacity and how comfortable they feel using that technology and

giving those experiences to students. I think it looks different in every grade level. I see

more of these opportunities in intermediate than in primary, as far as giving the kids

experiences with the P21 skills.

Middle school perceptions were also mixed. One Lighthouse principal noted that the

impact of the S.T.A.T initiative on P21 skills was “minimal.” The principal explained, “The

initiative does not directly target P21 skills. It is the instruction of highly skilled teachers willing

to modify and inspire these skills in students.” Several other Lighthouse principals felt the mastery

was hard to correlate to the S.T.A.T initiative, while others had “definitely” seen an improvement.

Phase 2 principals made the same types of comments. One summed up several comments stating



that growth may be visible in the classroom but it was “unclear as to whether it could be correlated

to student mastery of P21 skills.”

Lighthouse high school principals also indicated mixed views. One principal felt that she

“would not say that those skills are interwoven as frequently as needed, but we could use more

professional development to integrate them more holistically.” A second principal felt there had

been “positive growth,” while a third stated the impact had been “tremendous.”

S.T.A.T. teachers. Lighthouse and Phase 2 elementary S.T.A.T. teachers stated that they

had seen real progress in the past year in growing P21 skills with one commenting, “Students are

having more experiences like this daily and therefore are mastering these concepts, these skills and

standards more often because of the S.T.A.T. initiative.” Another commented,

If you were to walk into a typical classroom today, I would say it’s filled with those P21

skills. Kids are solving problems on their own, collaborating with peers, students can

articulate what they’re doing, how they’re doing it and why and can explain how they’re

going to solve it.

A third commented,

This had been one of the largest impacts—higher level thinking, problem solving, creativity

occurring throughout the day is indescribable. I think it has to do with student choice and

that they now have options about how they are going to show their mastery. They have to

think about how they are going to demonstrate their learning.

Middle school S.T.A.T. teachers indicated that P21 skills were an area that “needs to grow”

with one noting, “I don’t think that they have mastered P21 skills, but these elements are certainly

encouraged with the help of S.T.A.T.” Many felt that there had been an increase in collaboration,

critical thinking, and problem solving in their schools. One commented:

I know that some of our students are getting better in the critical thinking and problem

solving of P21, but I think that where we are right now, we’re still almost on the surface.

And, our next goal is to get the teachers to dig deeper, to get the kids to dig deeper.

One Lighthouse middle school S.T.A.T. teacher noted that teachers were learning to lead

with inquiry rather than foundational information with the intent that their students would then

start asking questions themselves. Teachers at their school were utilizing Bloom’s Taxonomy as a

resource for promoting higher forms of thinking in their classrooms.

Lighthouse high school S.T.A.T. teachers described an increase in self-awareness,

creativity, and critical in some of their students. One commented:

I think we are able to ask more of our students now. Students aren’t learning material by

simply regurgitating information anymore. They are challenged to absorb the information

and work with what it means. Because they are thinking through the material more, they

are really achieving P21 skills.



Classroom teachers. Through survey responses, teachers were in agreement (88.1% at

least somewhat agreed) that their students improved in mastery of 21st century learning skills.

Consistent with the temporal logic model Cohort 1 teachers were more in agreement (98.0%) than

Cohort 2 (95.3%) and Cohort 3 (79.7%) teachers.

Teacher focus group responses were somewhat mixed with about half of teachers stating

the initiative had improved skill sets, while the remaining half disagreed. First, several teachers

discussed the benefits of S.T.A.T. in developing P21 skills. A Lighthouse elementary teacher

noted, “It got you thinking about the type of questions . . . so that you’re getting more out of the

child than one-word answers.” Lighthouse and Phase 2 teachers commented that the initiative

allowed teachers to better present different approaches to solving a problem using the devices. One

Phase 2 teacher noted,

They [students] are thinking more critically, thinking outside of the box. They remember,

they’re making those connections. It has a lot to do with what we do here with them, they

don’t have a lot of outside experiences and we’re using tech to bring that to them or using

more hands-on things.

Several other teachers across groups noted S.T.A.T. had been a “help with the development of

collaboration and problem-solving skills.”

Challenges were noted and came from all levels. One Lighthouse high school teacher

remarked,

I definitely wouldn’t say it’s [P21 skills] improved; critical thinking is a really massive

problem, in my opinion for the students in this school, especially G&T students. When

they’re in those classrooms and we ask them to critically think and apply ideas to tough

problems because they should know that, they shut down. They can’t do it, and they don’t

want to do it . . . A lot of kids know how to regurgitate information very easily. I think . . .

they don’t know how to apply it to things and I don’t think the technology has assisted in

that. As teachers, we’ve been trying really hard but, unless they have an actual interest in

the topic that is going on, they’re not going to; a lot of them are very reluctant to try to

apply knowledge and answer a true analysis question.

Further, a number of teachers in all groups noted that although the devices provide more resources

for critical thinking and problem solving such as examples and information, there was little or no

change in application to critical thinking skills. A Lighthouse teacher pointed out that the S.T.A.T.

initiative was only as good as the teacher, stating, “It all depends on how the teacher implements

it.” Another Lighthouse teacher expressed a similar perception:

They’re going to be citizens in this world having to make opinions. We have the ability now

to let them out into the Internet and help them figure out what’s real and not real; and help

them develop those opinions, and show them how to look at the world that they’re exposed

to. That’s our job.



In addition, many of the teachers across all groups conveyed that students needed personal

interaction and that although the S.T.A.T. initiative had “a place, it is just one element that leads

to . . . P21 skills.” Finally, one Lighthouse elementary teacher made a comment echoed by others

that because the S.T.A.T. initiative is still relatively new, they are still learning how to fully use

the program.

We think we’re getting there, connecting the S.T.A.T. initiative to P21 skills, but we’re just

at the beginning of the path right now. We’re starting to explore as a school community

how does something like this [device] and all the technology that surrounds it, how does it

fit into communication, creativity, collaboration, critical thinking, and problem solving.

We’re learning as we go.

Summary. Observation results for P21 skills were similar between spring 2017 and

baseline for all three cohorts. While a stronger impact on P21 skills might be expected for Cohort

1 classrooms, now in their third year of S.T.A.T., results were comparable across all cohorts. None

of the four items concerning P21 skills was observed frequently. Inquiry-based approaches to

learning were seen at slightly greater frequency (particularly in Cohort 3), but were also seldom

observed. Of the four P21 items, the presence of learning that incorporates authentic/real world

contexts was observed the most frequently across each of the cohorts. BCPS may consider offering

targeted professional development to teachers regarding these practices, as well as creating sample

lessons that teachers may incorporate to further encourage use of P21 skills.

Though observation results revealed little impact on P21 skills, classroom teachers,

particularly those in Cohort 1, somewhat agreed that these skills had improved this year. However,

principals, S.T.A.T. teachers, and classroom teachers across all groups conveyed slight or varied

impacts on P21 skills during interviews and focus groups. All generally felt that there was more

work to be done in terms of developing these skills in students.


The impact of the S.T.A.T. initiative on student achievement was examined descriptively

through MAP scores in both reading and mathematics for BCPS Grades 1-3. Across each grade

and subject, the average RIT scores are presented for the year prior to S.T.A.T. implementation

through the current year. The RIT (Rasch Unit) score reflects a student’s academic knowledge,

skills, and abilities. RIT scores may range from 100 – 350. In addition to average RIT scores for

mathematics and reading, the national normative scores1 from the 2015 administration are

presented for comparative purposes. The national norm is comprised from a representative sample

drawn from test records in public schools across the U.S. NWEA does not provide normative data

every year and therefore only data from the 2015 administration are presented. In addition to the

RIT score, the proportion of students at least meeting growth expectations was also examined.

NWEA provides growth expectations for each student based on their RIT score, grade level, and

1 Retrieved from https://www.nwea.org/content/uploads/2015/06/2015-MAP-Normative-Data-AUG15.pdf

https://www.nwea.org/content/uploads/2015/06/2015-MAP-Normative-Data-AUG15.pdf



the subject. The growth expectation represents the “median level of growth observed for similar

students in NWEA’s norming sample2.”

The reader should be reminded that Lighthouse Grades 1-3 began S.T.A.T. implementation

during the 2014-15 school year and are presently in their third year of implementation. Non-

Lighthouse Grades 1-3 began S.T.A.T. a year later (2015-16) and are presently in their second year

of implementation. Due to these differences and variations in school characteristics between

Lighthouse and non-Lighthouse schools, no comparisons were made between them. BCPS did not

administer the winter MAP assessment in 2013-14 across all schools and therefore the fall 2013-

14 MAP data serves as pre-program data for Lighthouse schools whereas achievement for all other

years and for non-Lighthouse schools is represented with winter MAP data.

Lighthouse Grades 1-3. As the following sections will present in more detail, Lighthouse

Grades 1-3 generally increased their average MAP mathematics and reading RIT scores across

years while implementing S.T.A.T. All three grades exceeded the national average RIT scores in

both mathematics and reading during their second year of S.T.A.T.

Mathematics. Gains in average MAP RIT scores between years were generally greatest

from the pre-program year to the end of the first year of implementation. Whereas Grades 1 and 2

exhibited year-to-year gains in MAP RIT scores, Lighthouse Grade 3 exhibited a decline from year

two to year three (-0.53pts).

As displayed in Figure 27, the percentage of students at least meeting growth expectations

steadily increased from year one of S.T.A.T. through the present third year of S.T.A.T. While

Grade 1 initially fell below the national average of 50% of students at least meeting expectations,

the percentage of students exceeded the national average by the end of year two and year three.

Grades 2 and 3 consistently exceeded the national average each year of implementation.

2 Retrieved from https://www.nwea.org/blog/2013/using-percentage-students-meeting-exceeding-growth-

projections-evaluration-tool/



Figure 27. Percentage of students meeting mathematics growth expectations for 2014-15, 2015-

16, and 2016-17 in Lighthouse Grades 1-3.

Grade 1. Lighthouse schools' average MAP mathematics RIT scores increased with

each year of implementation (see Figure 28). Lighthouse schools demonstrated the

greatest change in scores from the pre-program year (2013-14) to year one of

implementation (2014-15), averaging a 7.14-point increase in scores. Lighthouse

schools also exceeded the 2015-16 Grade 1 national average (1.23pts).

Figure 28. Average MAP mathematics RIT score of Lighthouse students in Grade 1 for the 2013-

14, 2014-15, 2015-16, and 2016-17 school years.

Note: National norms are only available for 2015-16.

Grade 2. Similar to Grade 1, Lighthouse Grade 2 average MAP mathematics RIT

scores increased with each year of implementation (see Figure 29). Lighthouse schools

demonstrated the greatest change in scores from the pre-program year (2013-14) to

year one of implementation (2014-15), averaging a 10.02-point improvement.



Lighthouse schools also exceeded (5.82pts) the 2015-16 Grade 2 national average RIT

score.

Figure 29. Average MAP mathematics RIT score of Lighthouse students in Grade 2 for the 2013-

14, 2014-15, 2015-16, and 2016-17 school years.


Grade 3. Lighthouse Grade 3 average MAP mathematics RIT scores increased with

each year of implementation, with the exception of year three (see Figure 30).

Lighthouse schools demonstrated the greatest change in average scores from the pre-

program year (2013-14) to year one of implementation (2014-15), averaging a 6.83-

point increase. Lighthouse schools also exceeded (0.73pts) the 2015-16 Grade 1

national average RIT score.

Figure 30. Average MAP mathematics RIT score of Lighthouse students in grade three for the

2013-14, 2014-15, 2015-16, and 2016-17 school years.


Reading. Average RIT scores for reading mirrored those for mathematics. Improvements

in scores between years was generally greatest from the pre-program year through the first year of



implementation. Again, whereas Grades 1 and 2 exhibited year-to-year increases in average MAP

RIT scores, Lighthouse Grade 3 exhibited a decline from year two to year three (-0.88pts).

Consistent with findings for mathematics, the proportion of Grades 1-3 students at least

meeting growth expectations in reading steadily increased each year of S.T.A.T. implementation

(see Figure 31). In addition, despite Grades 1 and 2 initially falling below the national average of

students meeting growth expectations (50%), they exceeded this national average in years 2 and

3. Grade 3 consistently exceeded the national average each year of implementation.

Figure 31. Percentage of students meeting reading growth expectations for 2014-15, 2015-16, and

2016-17 in Lighthouse Grades 1-3.

Grade 1. Similar to Lighthouse MAP mathematics results for Grade 1, Lighthouse

schools demonstrated the largest increase (7.84pts) in average RIT reading scores from

the pre-program year of 2013-14 to year one of implementation in 2014-15 (see Figure

30). Lighthouse schools also exceeded (3.17pts) the 2015-16 Grade 1 national average

RIT score.



Figure 30. Average MAP reading RIT score of Lighthouse students in Grade 1 for the 2013-14,

2014-15, 2015-16, and 2016-17 school years.


Grade 2. Comparable to Grade 1 reading results, Lighthouse Grade 2 average MAP

reading RIT scores steadily increased with each year of implementation (see Figure

31). Lighthouse schools demonstrated the greatest increase in average scores from the

pre-program year (2013-14) to year one of implementation (2014-15), averaging a

8.18-point increase. Lighthouse schools also exceeded (4.37pts) the 2015-16 Grade 1

national average RIT score.


2014-15, 2015-16, and 2016-17 school years.


Grade 3. Grade 3 demonstrated the largest improvement (5.62pts) in average RIT

reading scores from the pre-program year of 2013-14 to year one of implementation in

2014-15 (see Figure 32). The average score change from year one (2014-15) to year

two (2015-16) was 4.23 points, while schools exhibited a decline (-0.88pts) in average



scores from year two of implementation (2015-16) to year three (2016-17). Lighthouse

schools exceeded (2.77pts) the 2015-16 Grade 3 national average RIT score.


2014-15, 2015-16, and 2016-17 school years.


Non-Lighthouse Grades 1-3. As the following sections will present in more detail, non-

Lighthouse Grades 1 and 2 demonstrated positive trends in MAP mathematics scores across the

two years while implementing S.T.A.T. Grades 1 and 2 also demonstrated increases in reading

RIT scores across years.

Mathematics. Grades 1 and 2 exhibited improvements year over year in average MAP RIT

scores. Grade 3, though, scored almost identically (-0.02pts) from year one to year two. As

displayed in Figure 33, the percentage of students at least meeting growth expectations steadily

increased from year one of S.T.A.T. through the present second year of S.T.A.T. In addition, all

three grades exceeded the national average of students meeting growth expectations (50%) each

year of implementation.



Figure 33. Percentage of students meeting mathematics growth expectations for 2015-16 and

2016-17 in Lighthouse Grades 1-3.

Grade 1. Non-Lighthouse Grade 1 average MAP mathematics RIT scores increased

with each year of implementation (see Figure 34). These schools demonstrated the

greatest increase in scores from the pre-program year (2014-15) to year one of

implementation (2015-16), averaging a 2.31-point improvement. Non-Lighthouse

schools fell slightly below (-0.8pts) the 2015-16 Grade 1 national average RIT score.

Figure 34. Average MAP mathematics RIT score of non-Lighthouse students in Grade 1 for the

2014-15, 2015-16, and 2016-17 school years.


Grade 2. Comparable to Grade 1, Grade 2 demonstrated an improvement of 1.71 points

in average RIT mathematics scores from the pre-program year of 2014-15 to year one

of implementation in 2015-16 (see Figure 35). Non-Lighthouse schools exceeded

(3.7pts) the 2015-16 Grade 2 national average RIT score.



Figure 35. Average MAP mathematics RIT score of non-Lighthouse students in Grade 2 for the

2014-15, 2015-16, and 2016-17 school years.

Grade 3. Like Grade 2, non-Lighthouse Grade 3 demonstrated a 2.17-point

improvement in average RIT mathematics scores from the pre-program year of 2014-

15 to year one of implementation in 2015-16 (see Figure 36). There was a slight decline

(-0.02pts) in average scores between year one of implementation (2015-16) to year two

(2016-17). Non-Lighthouse schools fell below (-1.07pts) the 2015-16 Grade 3 national

average RIT score.

Figure 36. Average MAP mathematics RIT scores of non-Lighthouse students in Grade 3 for the

2014-15, 2015-16, and 2016-17 school years.

Reading. In contrast with mathematics, all three grades exhibited increases in average RIT

scores each year. As displayed in Figure 37, the percentage of students at least meeting growth

expectations steadily increased from year one of S.T.A.T. through the present second year of

S.T.A.T. In addition, all three grades exceeded the national average of students meeting growth

expectations (50%) each year of implementation.



Figure 37. Percentage of students meeting reading growth expectations for 2015-16 and 2016-17

in Lighthouse Grades 1-3.

Grade 1. Similar to non-Lighthouse mathematics results, Grade 1 average MAP

reading RIT scores increased with each year of implementation (see Figure 36). They

demonstrated the greatest improvement in scores from the pre-program year (2014-15)

to year one of implementation (2015-16), averaging a 2.39-point increase. Non-

Lighthouse schools exceeded (1.54pts) the 2015-16 Grade 1 national average RIT

score.

Figure 36. Average MAP reading RIT score of non-Lighthouse students in Grade 1 for the 2014-

15, 2015-16, and 2016-17 school years.

Grade 2. Similar to Grade 1 reading results, Grade 2 average MAP reading RIT scores

increased with each year of implementation (see Figure 37). However, unlike Grade 1,

Grade 2 demonstrated the greatest improvement in scores from year one (2015-16) to

year two of implementation (2016-17), averaging a 0.92-point increase. Non-



Lighthouse schools exceeded (2.91pts) the 2015-16 Grade 3 national average RIT

score.


15, 2015-16, and 2016-17 school years.

Grade 3. Similar to Grades 1, Grade 3 demonstrated the largest improvement (2.0pts)

in average RIT reading scores from the pre-program year of 2014-15 to year one of

implementation in 2015-16 (see Figure 38). Non-Lighthouse schools exceeded

(0.70pts) the 2015-16 Grade 3 national average RIT score.


15, 2015-16, and 2016-17 school years.

Perceptions of the impact on CCSS. Through surveys, interviews, and focus groups,

participants provided their perceptions regarding the impact on student achievement, such as

mastery of Common Core State Standards (CCSS).



Principals. During interviews, principals conveyed mixed views as to whether the S.T.A.T.

initiative had affected CCSS. While some principals from Lighthouse and Phase 2 schools reported

a perceived increase in test scores, a number of principals indicated that it was either too early to

tell, or they were uncertain as to the correlation of S.T.A.T. to student growth in scores as other

initiatives were also in place. What principals did consistently feel was that “S.T.A.T. has had an

impact on teacher planning”, “collaboration” and “creativity” . . . “and this has likely enhanced the

schools’ delivery of the CCSS.” Further, principals felt the S.T.A.T. initiative was providing

resources that teachers needed to be successful in preparing students for CCSS. As one elementary

Phase 2 principal commented, “Basically, teachers don’t have to ‘reinvent the wheel’ as much to

teach these students anymore because of the resources and programs provided through BCPSOne.”

S.T.A.T. teachers. S.T.A.T. Teachers across school groups provided information on the

strategies being implemented to achieve impact more than on the impact itself. One Lighthouse

elementary school S.T.A.T. teacher commented, “They are also able to use the technology and

new instructional ways of our teachers in order to show their knowledge, but I can’t really be

specific about Common Core Standards.” A Phase 2 elementary S.T.A.T. teacher noted:

I don’t think it has necessarily caused all students to magically master the standards ,

however, through the S.T.A.T. initiative, there has been a big emphasis making sure our

curricula reflect not only the content standards but also the P21 skills.

S.T.A.T. teachers also commented on the impact of the revised curriculum on CCSS in

schools. Several teachers stressed the importance of formative assessment’s role in modifying

teaching and learning activities to improve student achievement. Elementary school S.T.A.T.

teachers stated that standards-based platforms such as DreamBox Learning and i-Ready had helped

teachers to access and apply standards. BCPSOne was also named as a useful resource in this

regard since it tags standards and teachers can see what has been mastered by students and what

needs to be revisited. One Phase 2 elementary S.T.A.T. teacher felt that the standards and

expectations for students were not appropriate for their developmental stage or age, commenting,

“I don’t see how the S.T.A.T. initiative has helped students to master the standards, because I think

the standards are the problem.” In contrast, a Lighthouse middle school S.T.A.T. teacher remarked:

The S.T.A.T. initiative is changing the way we plan common core education completely.

We are now using formative assessments to line up with common core standards. With

streamlined education, we are certainly seeing mastery improve.

Phase 2 Middle School S.T.A.T. teachers were in agreement with their Lighthouse peer

with three noting gains in student scores. Lighthouse High school S.T.A.T. teachers did not provide

a clear indication of the initiative’s impact on CCSS, in part because of recent implementation.

One teacher stated, “If the teachers are still trying to figure things out, and are not implementing

the program well, (they) are not seeing much impact.”

Classroom teachers. Classroom teachers across all groups tended to agree (90.6% at least

somewhat agreed) that their students had improved in their mastery of Common Core State

Standards this year. Not surprising given tenure of S.T.A.T. implementation, Cohort 1 teachers

were more likely to agree (99.0%) as compared with Cohort 2 (96.7%) and Cohort 3 teachers



(83.5%). In terms of comparisons across time points, Phase 2 Grades 1-3 teachers were

significantly more likely to agree (p < .05) to an improvement in CCSS mastery in spring 2017 as

compared with spring 2016 survey responses.

In focus groups, most teachers in Lighthouse and Phase 2 schools either did not feel that

enough time had elapsed to fully judge impact in this area, or that the use of technology alone did

not specifically result in student mastery. Instead, teachers viewed technology as another tool to

utilize in developing skill sets for CCSS. One Lighthouse elementary teacher made a comment

that summed up many teachers’ remarks, stating, “Technology doesn’t replace good teaching but

it does engage and gives them (students) so much more (information) at their fingertips.”

A Lighthouse middle school classroom teacher provided a more in-depth explanation stating,

Devices have been able to give different learners different outlets to approach CCSS. For

instance, students enjoy using manipulatives . . . There are also a variety of tools available

for students to use in order to meet standards, and the gauging of student ability seems

more accurate on utilizing digital components and assessments. It’s important to

distinguish between skills learned and device usage.

Another Phase 2 middle school teacher agreed stating that standards were easier to access because

of technology. She stated, “Now, they [standards] are constantly in plain sight, and so that makes

it easier to focus on and see mastery and progress.” However, one Lighthouse elementary teacher

noted that although students “get more practice at testing for the Common Core . . . the kids who

are mastering this would master regardless.” Several other teachers in various groups supported

this claim.

Summary. An examination MAP scores in Lighthouse and non-Lighthouse Grades 1 -3

showed some impact on student achievement. Lighthouse students in Grades 1-2 exhibited

improvements in reading and mathematics scores across all three years of implementation and

Grade 3 increased reading and mathematics scores in all but the present year. Further, all grades

exceeded the national average mathematics and reading scores. Non-Lighthouse Grades 1-3 also

exhibited improvements in reading scores across all three years and, similarly, Grades 1-2

increased mathematics scores. Grades 1-3 exceeded the national average reading scores, but only

Grade 2 exceeded the national average mathematics scores.

Principals and S.T.A.T. teachers were hesitant to comment on a direct impact of S.T.A.T.

on student mastery of CCSS. Many conveyed that students should be improving due to enhanced

teaching practices and curriculum that offered access to a variety of resources and options for

learning. Teacher survey responses indicated a perceived improvement in master of CCSS but

during focus groups, many teachers indicated it was too early to if there had been an impact.

Perceptions of the S.T.A.T. Initiative

This section summarizes school district participant perceptions about the strengths of the

S.T.A.T. initiative. In addition, student and parent responses to BCPS climate survey items specific

to the S.T.A.T. initiative are presented.



Principals. Overwhelmingly, at all levels and across all groups, principals conveyed that

the greatest strength of the S.T.A.T. initiative was the increase in student engagement and student-

centered learning. One principal summed this up stating, “It [S.T.A.T.] has greatly increased

student engagement and really allowed children’s spirits to thrive in schools.” A second principal

commented that “the focus on student-centered learning and the focus on changing the

environment” have made the teacher, the facilitator, and the room, “belongs to the students.”

Principals also commented that parents were seeing this change. A principal explained:

Parents have compared older siblings who didn’t have technology with the ones with the

opportunity. Those with technology are much further along. Not concrete data, but soft

data point that parents have seen a difference.

Another point principals mentioned in relation to student engagement was greater access

for all students. As one principal commented, “In some more abstract level, we’ve leveled the

playing field just with the access in that these computers are in the households and these kids use

them quite frequently. Other kids were using them all of the time.” Another principal stated the

S.T.A.T. initiative had created:

The opportunity for an even playing field for every student . . . and the students in my

building are going to have access to the same information across the system. It’s bringing

all of us into the 21st century and that is huge. It forces us as educators to start thinking

creatively.

Second to student engagement, two principals commented on the key role the S.T.A.T.

teacher played in making the S.T.A.T. program successful. As one Lighthouse elementary

principal observed:

If you don’t have a good S.T.A.T. teacher--it’s not going to go well because she’s not only

helping us to navigate, she’s a cheerleader. She keeps everybody motivated because it is

stressful managing this rollout. She makes things fun and interesting and I think the actual

teacher is the key component.

S.T.A.T. teachers. S.T.A.T. teachers provided varied strengths of the initiative. One

Lighthouse middle school S.T.A.T. teacher commented, “One of the major strengths is that it has

gotten the teachers, and really everyone in the county, to think about instruction more.” They

described classrooms that are preparing students for the future in a more digitally focused world

with one respondent observing, “Students have the opportunity to access learning in a way they

didn’t have before. I think our students are becoming 21st century learners.” S.T.A.T. teachers see

students becoming better problem solvers and communicators as a result of the initiative’s move

towards small group instruction and student-centered learning. For example, one S.T.A.T. teacher

commented, “Now they are getting the instruction that they need, in a way they can receive it, and

the teacher is constantly reviewing data to make sure they are moving forward.” Another noted,



The greatest strength I see is what kids are able to do because of it. I’ve seen such an

increase in the way they communicate what they learn, how they communicate with each

other, how they problem solve. They’re thinking outside of the box and we see it every day.

It’s hard to collect as a data point, but they’re eager and invested in their learning in a

whole new way.

Additional initiative strengths named by S.T.A.T. teachers included introduction of more

real-world content, daily P21 skill use, improved teacher practices, and student growth as learners.

A teacher remarked:

[The] overall strength is the growth and the confidence and the reassurance of the

students…To see them grow from year to year, in their P21 skills, the standards of

instruction, the way they’re now more student driven. I remember what I had to do as a

teacher and how it looks now, and there is a huge difference in what that looks like and the

impact it has had. You go into a classroom now and all of the students, they all might be

doing something different, but with the same end goal, it’s just that each student has their

own unique pathway to reach that goal.

Classroom teachers. During focus groups, teachers noted four main strengths of the

initiative: differentiated learning, breadth of resources, choices available to both students and

teachers, and accessibility. Teachers also indicated that incorporating technology into the

classroom has opened up a variety of new options for stakeholders. One Phase 2 middle teacher

explained that the “Possibilities seem endless. I feel like I have only scratched the surface of what

we can do.” A Lighthouse middle teacher also commented:

I’m excited; it gives us new ways to get them [students] excited about learning besides just

classroom activities. You can make classroom activities so much fun . . . they love it. Gets

kids inspired to work together, and find new ways to learn.

These aspects of the program have, in the opinion of most of the teachers interviewed, greatly

increased student engagement and collaboration. One Phase 2 elementary teacher commented that

engagement was the primary strength of the S.T.A.T. initiative stating,

Number one for me is student engagement. I don’t make copies of things nearly as much

anymore because they [students] want to watch a video. If you have let’s say Scholastic

News, they can read it online and watch an embedded video. A lot of digital content has

that. They’re not just reading something; they have color photos to go along with the text.

Before I was making black and white copies. Their interest is higher than it used to be.

Teachers also felt that students are better prepared for the future because of the S.T.A.T. initiative.

One Phase 2 elementary teacher noted the initiative “is preparing students for life and careers in

the ‘modern world’—a world which makes extensive use of technology.”

Across all groups, the majority of teachers interviewed have very positive views of the

S.T.A.T. initiative. Although there are a variety of issues that teachers are learning to deal with

such as device troubleshooting, logistics, and some behavior mismanagement, classroom teachers



feel that S.T.A.T initiative is providing students with the skills they will need for the future. In

addition, S.T.A.T. is “pushing students forward to be more self-directed learners” who have “more

responsibility and accountability for their learning.”

Parents and students. Three S.T.A.T.-specific survey items for parents were included on

the BCPS Climate Survey for the 2016-17 school year. Two of these items were also administered

to students in grades 4 – 12.

The majority of parents (87.9%) indicated agreement that making learning more

personalized for students help teachers meet the academic needs of all students. Elementary school

parents (Lighthouse: 92.6%, non-Lighthouse: 90.0%) were somewhat more in agreement than

middle school (83.0% for both Lighthouse and non-Lighthouse) and high school (Lighthouse:

84.4%) parents (see Figure 39). There were no significant differences between Lighthouse and

non-Lighthouse schools at any school level.

Figure 39. Parents’ responses to survey item regarding personalized learning.

Both parents and students were asked their level of agreement regarding personalized

learning facilitated by technology. A strong majority (84.5%) of parents agreed (38.4% strongly

agreed) that access to technology increases opportunities for making learning more personalized

for students (see Figure 40). There were not significant differences between parents of Lighthouse

and non-Lighthouse elementary or middle school students.



Figure 40. Parents’ responses to survey item regarding students’ access to technology.

Students in Grades 4 through 12 were also in clear agreement (86.4% agreed, 35.6%

strongly agreed) that access to technology increases opportunities for teachers to make learning

more personalized for them (see Figure 41). Cohort 2 students conveyed similar levels of

agreement to Cohort 3 students (84.9% and 84.3% indicated agreement).

Figure 41. Students’ responses to survey item regarding their access to technology.

There were differences noted between Lighthouse and non-Lighthouse Grade 6 students.

Lighthouse students in Grade 6 were also significantly more likely to agree than those in non-

Lighthouse Grade 6 (p < .001). Similar to elementary students, both Lighthouse and non-

Lighthouse middle students tended to agree.

The third climate survey question centered around teachers’ use of technology to meet

student needs. The vast majority (82.0%) of parents agreed (31.4% strongly agreed) that teachers

are able to use technology to meet the academic needs of all students (see Figure 42). Elementary

school parents tended to agree more so than middle school and high school parents. There were no

significant differences between parents of Lighthouse and non-Lighthouse elementary or middle

school students.



Figure 42. Parents’ responses to survey item regarding teachers’ use of technology.

Similar to their parents, nearly all (88.2%) of students agreed (40.3% strongly agreed) that

teachers are able to use technology to meet the academic needs of all students (see Figure 43).

Cohort 2 and Cohort 3 students’ levels of agreement were consistent.

Figure 43. Students’ responses to survey item regarding teachers’ use of technology.

Differences between Lighthouse and non-Lighthouse students were not statistically significant at

any grade level.

Along with identified strengths of the S.T.A.T. initiative, the school district participants

provided recommendations for improvement.

Principals. Several consistent themes of recommendations were offered by principals

during interviews. At all levels and across all groups, communication was cited as needing

improvement. This included communication from the district, within the district, and to parents

regarding the scope of the program. One Lighthouse elementary principal observed: “The

Baltimore County School system is so large with so many schools, principals, teachers and

personalities, needing greater clarity of expectations.”

Within the district, communicating in a more streamlined fashion was cited as a need. A

Phase 2 middle school principal commented that communication was needed “up front” and

“everything should be in one place”, meaning that one should not have to obtain information from

multiple places. A Phase 2 elementary principal observed:



I think the county could communicate that we’re pushing an initiative but we need to

address implementation and network with each other to share ideas rather than teaching

them something new because that feels like something that is added to their plate. So being

mindful about giving the teachers opportunity to focus on implementation is important.

Several principals at the elementary, middle and high school level felt the district needed

to communicate with parents more effectively as to the scope of the initiative. One high school

principal noted the “need to better communicate the initiative as a whole rather than just a tech

adoption.” One Phase 2 principal explained:

There is a perception that the S.T.A.T. initiative is ‘all about the devices’ which it’s not. So

the parents have pushback on ‘Oh your children are on devices all day!’ They’re not. They

use it for research, they use it as an extension, as a remediation [tool], but they’re not on

it all the time and it’s an enhancement. It’s not just all about the devices.

A second Phase 2 principal went further stating there was “A lot of negative press on kids and

screen time, and parents are surprised about how much they’re not staring at the screen. Share with

parents beforehand that screen time depends on a specific activity.”

A second theme that emerged at all levels from principals was the need for more support.

Sustaining what was already in place was deemed important, but many principals stated they

needed more on-site staff. As one Lighthouse middle principal explained, “Making sure that the

support staff are sustained as there are changes in the system. Their support in the schools is so

key.” Four principals at the elementary, middle and high school levels specifically noted the need

for “more staff to support devices” such as repairs, resources, and logistics of collecting devices.”

One high school principal went further stating:

Some middle schools have 500 students and have one technology liaison, and our schools

have over 900 devices and one liaison. As economies of scale increase, we may require

two technology liaisons to service schools, especially around testing when students are

using their devices.

A Phase 2 principal also noted that for low-performing schools, or those with a large threshold of

new teachers, an additional S.T.A.T. teacher was needed “because there is so much to learn.”

Principals in all groups at all levels also noted the need “to refine what we’ve started -- not

just add stuff on” as well as having “the time necessary to be thoughtful in . . . planning,” so that

teachers have the opportunity to “perfect what has been given to them.” Another two principals

had comments concerning this. One Phase 2 elementary principal observed, “Having spent three

years aggressively tackling and assimilating the topic, teachers need more time to actually

implement some of the work talked about . . . If teachers had more time, the outcome would be

tremendous.” The second principal went further, noting, “We need to continue to focus on how we

can build up our repertoire on ways to use technology to support instruction and try not to go off

in too many directions.” Another principal explained, “If you add too much, too quickly, it will

not be effective. Slowing down the implementation process would be a huge help.” A second Phase

2 elementary principal noted that it was important to:



Continue to develop teachers in this work and not add any more layers. Focus on

standards, focus on ways we can support curriculum and grow that way and not branch

off in other directions that will cause confusion and frustration. We are in a place where

people are more comfortable and I want to keep the momentum going. I don’t want to

throw more things on top of them that will muddy the waters.

A fourth and final theme specific to all groups dealt with logistics. Issuing devices was a

concern noted by several principals, particularly at the middle school level. Suggestions included

having assistance in the first weeks of school as to checking out devices, providing a better means

to hold students accountable for devices, and helping in the initial process of teaching students

how to log on. Additionally, there were some concerns from elementary and middle school

principals about the developmental appropriateness of younger students having devices and “the

seemingly arbitrary line drawn regarding [middle school students] taking or not taking home

devices.”

S.T.A.T. teachers. Two Lighthouse elementary S.T.A.T. teachers stated that teachers

needed a break. They felt that they needed time to take in and digest what they have already

received in terms of professional development, resources and materials. Their comments included,

“Sometimes we need just to fine tune and make those priorities. Once we’re doing this well, we

can move on” and, “It’s ever evolving and I think just providing more time for teachers. There are

so many initiatives out there and things to think about.” Two of the Phase 2 elementary S.T.A.T.

teachers agreed with their Lighthouse peers with regard to feeling they are reaching their limit in

absorbing new material. One commented, “I think the recommendation is, give time to learn, or

re-learn, and sort out what the issues were. Do not introduce anything new, for a while, so that we

can get proficient in what we’re doing now.”

And another stated,

I don’t need quite as many initiatives in one year. We are overwhelmed with all of the

things that are available, while it’s wonderful that it’s all there, it can be difficult, trying

to decipher what to bring back to my staff and which things to prioritize.

A Lighthouse middle school S.T.A.T. teacher was in agreement with some of her

elementary school peers in believing that time was needed to better master teaching practices

envisioned through the S.T.A.T. initiative:

In Baltimore County, the S.T.A.T. initiative is one of many. The people who are in the

offices do not understand how much work is involved in keeping these initiatives up. They

need to give teachers time on how to implement this effectively before they add something

else on. It’s sometimes a bit too much to keep adding to the plate before what is given to

us is mastered. There are not enough hours to the day to continue adding more to the

initiative.

Relatedly, other suggestions for improvement included additional district-wide PD, such

as recommended by one S.T.A.T. teacher:



As of right now, I just see more positive changes from year to year. I think it would help if

there were more opportunity for teachers to get extra help and support from professional

development. It can be difficult for teachers to always get it from me, and if they could

meet--maybe quarterly--in their zone as a grade level, I think that would be tremendous

Middle school S.T.A.T. teachers also conveyed that teachers needed more time to “go a little

deeper with the S.T.A.T. initiative,” to provide new S.T.A.T. teachers with more intensive

professional development, and to reduce the number of initiative-derived goals teachers focus on

at any given time. As one S.T.A.T. teacher put it, “From the teachers I have spoken with, a lot of

them feel like they are treading water to keep up and they really need that extra assistance.”

Another Phase 2 elementary S.T.A.T. teacher spoke of her own limitations and the possible

need for additional S.T.A.T. teachers in larger schools:

By nature, we end up wearing many hats. I love that part of my job, and I think most

S.T.A.T. Teachers do. Every day is different and exciting and we get to support teachers in

so many ways. But I think, in schools of a certain size, one person cannot provide enough

support.

Another suggestion from S.T.A.T. teachers pertained to device use. Elementary school

S.T.A.T. teachers conveyed the need to reduce device use for students in the youngest grades so

that foundational skills would not be lost. Two Lighthouse Middle School S.T.A.T. teachers

suggested having students keep their devices at school rather than taking them home at night with

one stating, “We could be so much more productive with the devices if students weren’t taking

them home” and the other adding:

I think we as a middle school, we may benefit from the students not taking the devices home.

The challenge is that they don’t come back with them charged, or have repairs that need

to be done. So, they don’t have the device in class then it’s a challenge to find an

appropriate alternative to a video resource or a collaboration online. We don’t have loaner

devices for all that don’t have the device with them.

A Phase 2 Middle school S.T.A.T. teacher agreed with their Lighthouse peers in believing it would

be advantageous if students kept their devices at school.

Others voiced concerns about their own job security, with one stating “Don’t get rid of

S.T.A.T. teachers” and the inconsistency of the S.T.A.T. teacher job description across schools:

“It’s inconsistent from school to school. I’ve heard others say they are not supported by

administrators or they’re busy doing other things, have other duties and can’t focus one hundred

percent on S.T.A.T.”

Classroom teachers. Classroom teachers provided recommendations for the S.T.A.T.

initiative. The responses below reflect recurring themes in the focus groups:



Clear Expectations. Teachers stated that as the school year begins, BCPS needs to

develop a “clear expectation of what you want the teachers to do with the devices”.

Teachers also stated they needed help in finding the appropriate “balance” in

technology use in terms of what is an appropriate amount of time for students to

use devices and what is not.

Professional Development (PD). Teachers had a number of recommendations

regarding PD. A term that was used frequently in regard to PD was “overwhelming”

The broad scope of PD made teachers feel “too much was being thrown at them.”

One Phase 2 teacher remarked,

There is so much to the S.T.A.T. initiative, and so many programs and applications

that are part of the digital curriculum and BCPSOne that it is really hard to cover

all of it (or even most of it).

Teachers would prefer that their school have them focus on mastering a “couple of

programs each year” as opposed to trying to learn many and integrate too much into

instruction. One Phase 2 elementary teacher cited a specific example stating:

“Instead of giving us five different vocal recording programs—give us one that is

good, and teach us how to use it.”

Time: Teachers overwhelmingly indicated that they need to be provided more

structured time to explore and experiment programs available through BCPSOne

so that they can get a better feel for the ins and outs of the various programs. One

Lighthouse middle teacher noted,

The County needs to keep open mind and know that if we are to do A, B, C you have

to look at the whole picture . . . Giving us more opportunities to learn, continue

with PD, to dive in with what we’re being taught. Having patience with us and

knowing that it’s not going to work for all situations, for all classrooms, for all

students.

Teachers also noted that the time when PD was presented often affected how well

they “buy into training,” as well as their initiative. One Lighthouse elementary

teacher commented,

The S.T.A.T. institute over the summer is incredibly overwhelming. A lot of

information thrown at you, right at the end of the school year before summer. Not

all teachers attend. Confusing as to the purpose. Great way to showcase what we

have, but as far as classroom teachers the expectation and purpose are unclear.

Why in June when I’m not going to be using it for the next several weeks?

Several teachers noted dissatisfaction with summer training in that it affects

personal time, or second jobs. One teacher Lighthouse elementary teacher

commented,



It is after school, when I have to take off from my summer job to attend. When your

administration asks you to go it’s hard to say no. It should be held during the school

year, not fair to ask teachers to go when . . . most teachers work during the summer.

Technology. Teachers noted a number of improvement needs related to

technology. Adequate broadband width was one concern. A second concern noted

by several teachers dealt with better device maintenance (batteries, memory

updates, software, replacing loose parts, etc.). A third concern was centered on

student accountability for devices and how to appropriately manage this issue (e.g.

forgetting or not charging devices, abuse of devices, etc.). A number of teachers

also commented upon the need for a monitoring system to track whether students

were on task. One Phase 2 middle teacher wanted to know if there was a way to

“legitimately link their (student) devices to my device so I get a flag when they’re

not doing what they should.” Teachers across all groups would also like the ability

to better restrict how students use the devices (e.g., the ability to block students

from accessing certain applications, turn off apps when they are not being used, set

and lock the device display preferences, be able to delete applications, and have

better controls on what websites are blocked for students).

S.T.A.T. Teachers. Teachers repeatedly noted that the S.T.A.T. teacher was

critically important in the initiative. However, teachers consistently noted that the

roles and duties of the S.T.A.T. teacher should be clearly defined for consistency

across schools.

Summary. Principals, S.T.A.T. teachers, and classroom teachers described a variety of

strengths of the S.T.A.T. initiative including increases in student engagement, a focus on

instruction and student-centered learning, and the variety of options and resources available for

teachers to support student learning. Overall, these stakeholders appeared to be quite positive

towards the initiative, as were parents and students. The vast majority of parents and students were

positive towards personalized learning and technology, though elementary school parents tended

to be more positive towards teachers’ use of technology to meet students’ academic needs.

There were, though, a variety of suggestions offered by principals, S.T.A.T. teachers, and

classroom teachers that mirror findings from logic model components. First, classroom teachers

conveyed the need to explicitly define the roles and responsibilities of S.T.A.T. teachers. In terms

of teacher practice, all three participant groups referenced the need to allow teachers time to digest

what they had learned and explore how they may better incorporate technology. In addition, there

was a desire for more targeted and focus PD on technology integration practices. Second, S.T.A.T.

teachers suggested the district revisit whether devices should be taken home by students and

classroom teachers expressed the need for assistance with classroom management of device use.

Third, principals indicated the need for improved communication from the district to schools and

particularly from the district to parents about the initiative and its goals and purposes.

Conclusion



In the present section, we draw from the comprehensive results of the third-year study to

present broader conclusions regarding the main findings and their implications. The evaluation

questions that guided the study are used as an organizing framework.

Roles and Practices of S.T.A.T. Teachers and Classroom Teachers

As in prior reports but increasingly so, S.T.A.T. teachers conveyed feeling prepared to

serve in their role of supporting classroom teachers. Their expressed interests in additional PD

pertained to incorporating technology in the curriculum, conducting small-group instruction,

managing behavior, and engaging all teachers in their building. Classroom teachers continue to

view S.T.A.T. teachers as highly valuable, critical to the initiative’s success, and an important

resource as they (the teachers) gain further experience with technology integration and changing

instructional practices. Continuing improvement needs are engaging the S.T.A.T. teachers

according to program expectations (i.e., not for incidental or unrelated school assignments) and

increasing the consistency of S.T.A.T. teacher roles and responsibilities across schools.

Principals and teachers described a wealth of support and preparation for S.T.A.T.

implementation. The vast majority of teachers positively viewed their principal’s role and support.

Nearly all principals were satisfied with how their teachers were implementing S.T.A.T. As would

be expected, isolated concerns were raised by some participants, especially with the amount of

time needed to plan for technology integration in the midst of many other district and school

programs. Teachers expressed the need for future PD on using digital tools and programs,

including BCPSOne, and P21 skills. A positive development is the increased sharing of model

lessons and common lesson planning sessions at many of the S.T.A.T. schools. In this regard,

recent literature on teacher PD has highlighted the emphasis by high-performing educational

systems, such as Japan’s3 (Reference), in supporting teacher PD communities and “lesson study”

(group demonstrations and co-development of lessons). Facilitated by the support of S.T.A.T.

teachers and engaged principals, many teachers conveyed a stronger sense of being part of a

community of practice focused on instructional improvement.

Impact of S.T.A.T. on the Classroom Environment

Across the eight subgroups that made up the cohorts, only isolated differences between the

baseline and most recent observations were evidenced and none (e.g., students using workspaces)

that appear central to S.T.A.T. implementation. For two expected components of S.T.A.T.—

communicating/posting information on independent thinking (seen in over 50% of classrooms)

and on content (over 90%)—the majority of classrooms exhibited moderate to extensive

implementation. Seemingly, a usage plateau for these components is reached early in

implementation (Year 1 or 2), with a minority of teachers remaining holdouts over time. This

finding would be consistent with the logic model with the majority of changes to the classroom

environment exhibited during Year 1. The other two environmental practices –students acquiring

resources and using different workspaces—reflect classroom activities that are favored in certain

classrooms more than others by teacher choice but not essential or necessarily expected S.T.A.T.

practices.

3 Hiebert, J., & Stigler, J. W. (2017). Teaching versus teachers as a level for change: Comparing a Japanese and U.S.

perspective on improving instruction. Educational Researcher, 46(4), 169-176.



Teacher Practice

As conveyed through multiple data sources, teacher practices have changed substantively

as a result of S.T.A.T. Principals in all groups described “mindset changes” in their teachers,

referencing increased collaboration with peers and more time spent in individual and group

planning. Interviews and focus groups with all participant groups conveyed that teachers were

increasingly engaging in meaningful integration of technology, exploring new strategies and

programs, and in general personalizing instruction and creating interactive learning for their

students.

Observation findings, however, reflected few statistically significant differences from the

baseline to the most recent observations in specific strategy use (e.g., coaching/facilitating,

presentations, discussion, etc.). Specifically, only three subgroups, all in the most inexperienced

cohort (Cohort 3) demonstrated significant changes. Notably, across the subgroups, teachers made

fairly frequent use of coaching/ facilitating and moderate to frequent use of presentations. Lessons

that engaged students in using digital tools and independent learning were also highly prevalent.

Based on the three years of observations, our overall conclusion is that the introduction of

digital tools (i.e., initial implementation of S.T.A.T.) has had a substantial early impact on shifting

pedagogy from teacher-centered approaches (presentations, recitation, prompted discussion) to

student-centered approaches (coaching/facilitation and personalized/individualized work). As our

longitudinal findings have shown, once these practices are established in the first year, they tend

to persist at comparable frequency levels. Typical classroom observations are likely to reveal a

mixture of multiple strategies based on teacher preferences and lesson characteristics. Increases in

the quality rather than quantity of priority strategy usage will become the important goal as the

initiative continues.

Digital Content

Results from surveys, interviews, and focus groups indicated that all participant groups

perceived an improvement in the access to and use of digital content and resources. For promoting

successes with technology integration, principals referenced the use of BCPSOne and other

programs, particularly in facilitating student-conducted research and demonstration of learning

achievements. Technology integration was described by all participant groups as progressing each

year. For example, classroom teachers described using technology to more effectively differentiate

learning and provide opportunity for students to learn at their own pace. They also noted the

positive impacts on students, such as taking ownership of their learning, improving technology

skills, and increasing engagement.

While technology integration appears to be advancing in many ways, some teachers are

still struggling with designing and delivering lessons that promote P21 skills and higher-order

learning in general. Teachers also experienced challenges with technology integration, most

notably through frequent reports of students’ off-task or inappropriate uses of devices. Examples

include playing games, obtaining answers during online tests, and accessing prohibited websites.

For some teachers, additional challenges included technical issues and having inadequate planning



time or PD to accomplish instructional goals. These issues seem typical but also resolvable as a

systemic technology integration initiative expands and matures.

Student Engagement

The observation data reflected high student engagement in using digital tools for learning,

particularly for independent work. Because such activity was also frequent at baseline for the

various subgroups, changes over time were not statistically significant. As would be expected

given differing implementation demands and the prevalence of device usage, independent work

was observed much more frequently than formal collaborative learning. Observers, however, noted

frequent informal interactions between students as they completed independent work and projects.

Principals and S.T.A.T. teachers perceived increased student engagement, bolstered by

student-centered and personalized learning, as a beneficial impact of S.T.A.T. Classroom teachers,

although mostly in agreement, conveyed more mixed reactions. Some believed that engagement

had decreased somewhat over time. Possible reasons could be some eroding of the novelty of

personalized learning and technology. Other teachers noted the need to design technology-

enhanced lessons that would be more challenging and interesting for students.

Reactions to behavior were also mixed. Participant reactions did not support an overall

conclusion that behavior in the S.T.A.T. era has changed for the better or worse. Rather,

experiences seem teacher specific, and could largely depend on students’ level of engagement with

lessons. Although the behavioral data (suspensions and referrals) have shown slight upticks over

time, it’s not clear to what degree, if at all, S.T.A.T. contributed relative to other factors (e.g.,

changes in student demographics or discipline policies).

P21 Skills

One of S.T.A.T.’s longer-term goals is to promote student mastery of P21 skills. Logically,

technology integration can be useful for this purpose only to the degree that associated

instructional activities and lesson content address those competencies. Here, our observation data

and reactions by teachers and principals evidenced slower progress than for other implementation

components. As described in several sections of this report, classroom uses of digital devices for

learning and student-centered learning in general are clearly visible and often extensive. But

observed uses of P21-oriented inquiry, problem-solving, and project-based approaches remained

much the same as baseline levels--about 20% of the classrooms. These types of practices are time-

consuming and procedurally difficult to implement, so expected uses relative to traditional

practices would naturally be low. In accord with S.T.A.T. teachers’ perceptions, despite some

perceived improvements this year, there is more work to be done in preparing teachers to develop

students’ P21 skills.


MAP scores in Lighthouse and non-Lighthouse schools showed positive trends in student

achievement. Lighthouse students in Grades 1-2 improved in reading and mathematics across all

three years of implementation and Grade 3 improved in both areas for all but the present year.



Notably, following the initiation of S.T.A.T., all grades exceeded the national average in

mathematics and reading scores. Non-Lighthouse Grades 1-3 demonstrated similar positive trends,

although only Grade 2 exceeded the national average mathematics scores.

Principals and S.T.A.T. teachers perceived that enhanced teaching practices and stronger

curricula were increasing mastery of CCSS. However, they were generally hesitant to attribute the

MAP gains directly or solely to S.T.A.T. We agree with this assessment for several reasons. First,

gains in achievement were not projected by the Logic Model this early in the implementation,

although we cannot rule out more rapidly occurring impacts. Second, there are numerous programs

and initiatives in BCPS, which could contribute to improved student achievement independently

of S.T.A.T. Regardless of the degree of S.T.A.T.’s contribution, the MAP outcomes importantly

reflect positive achievement trends for the district’s schools. The PARCC outcomes for Grade 3,

to be available in the fall of 2017, will provide further evidence for evaluating district performance.

Concluding Perspectives

In Year 3 of S.T.A.T., implementation progress clearly seems to be meeting and often

exceeding expectations. Prominent strengths of the initiative include increases in student

engagement, a focus on instruction and student-centered learning, and a variety of options and

resources available for teachers to support learning. Support and reactions by all stakeholder

groups, including teachers, principals, S.T.A.T. teachers, students, and parents, are remarkably

positive, especially for such a highly comprehensive initiative in its third year. Both personalized

learning and technology usage in instruction have gained broad acceptance as effective

orientations for teaching and learning.

Areas for improvement also emerged from our findings. First, the roles and responsibilities

of S.T.A.T. teachers still require clearer definitions. Second, many teachers feel that too much is

happening too quickly, and consequently, they need more time to digest, plan, and perfect what is

already on their plates. In particular, they desire additional support in designing meaningful lessons

and in integrating technology to teach higher-order skills. Third, strengthening and refining

technology policies and operations are suggested to address student off-task or inappropriate

behaviors, whether devices can be taken home by students at various grade levels, and technical

support needs for different programs and applications. Given the clear implementation progress,

the positive achievement trends on the MAP, and the strong buy-in to the initiative by most

teachers and principals, addressing these weaker areas can only work to make the present

accomplishments even stronger in Year 4.



Appendix A: Classroom Teacher Survey

Professional development items:

1. I was able to apply what I learned during professional development in my classrooms.

2. I received sufficient professional development on the use of technology in my classroom.

3. I received sufficient professional development on the creation of a learner-centered

environment.

Strongly disagree (1)

Somewhat disagree (2)

Somewhat agree (4)

Strongly agree (5)

Teacher practice items:

4. I am skilled at engaging my students in collaborative learning activities using technology

as a resource or tool.

Strongly disagree (1) Somewhat disagree (2) Somewhat agree (4) Strongly agree (5)

5. How often did your students collaborate with other students on an in-class activity or

investigation?

Never (1)

At least once per month (2)

At least once a week (3)

More than once a week (4)

To what extent did you use the following types of teaching practices this year?

6. Direct instruction/lecture

7. Cooperative learning

8. Student projects

9. Individualized learning (at desk or computer)

Never (1)

Rarely (2)

Moderately (3)

Frequently (4)

Student impact:

10. My students have improved in their mastery of CCSS this year.

11. My students have improved in their mastery of P21 skills this year.

12. The student behavior in my classroom has improved this year.



Somewhat agree (4)

Strongly agree (5)



13. To what degree have you observed students using technology inappropriately (e.g., video

games, off-topic/risky Internet searches, instant messaging) during class time?

Not at all (1)

Rarely (2)

Occasionally (3)

Frequently (4)

Extensively (5)

Technology integration:

How often did you use BCPS One to…

14. deliver instruction customized to students’ needs?

15. develop formative assessments?

16. develop assignments?

17. post homework assignments?

Never (1)

At least once per month (2)

At least once a week (3)

At least once a day (4)

18. To what degree is the use of technology an integral part of your teaching practices this

year?

Not at all

Minimal

Moderate

Fairly strong

Very strong

19. To what degree is the use of technology part of your instructional planning and

administration (preparing lessons, grading, data management, etc.)?

Not at all

Minimal

Moderate

Fairly strong

Very strong

Lighthouse preparation: (asked of Lighthouse high school teachers only)

1. I was adequately informed of the expected role of my school as a Lighthouse School.

2. I was adequately prepared to support my school in fulfilling its role as a Lighthouse

School.

3. I feel my school was successful this year in fulfilling its role as a Lighthouse School.



Somewhat agree (4)

Strongly agree (5)



Open-ended items:

1. What do you feel were your greatest successes this year in integrating technology into

instruction?

2. What were the greatest challenges and how could they be overcome in the future?



Appendix B: Descriptive Statistics and Frequencies of Responses to Classroom Teacher

Survey (2017)

I was able to apply what I learned during professional development in my classrooms.

Strongly

disagree

Somewhat

disagree

Somewhat

agree

Strongly

agree

M SD Participant Group % % % %

Lighthouse elementary Gr. 1-3 3.0 3.0 37.0 57.0 4.42 0.89

Lighthouse elementary Gr. K, 4, 5 2.8 7.0 45.1 45.1 4.23 0.97

Phase 2 elementary Gr. 1-3 5.4 4.3 44.6 45.7 4.21 1.04

Lighthouse middle Gr. 6 7.7 3.8 46.2 42.3 4.12 1.13

Phase 2 elementary Gr. K, 4, 5 3.4 2.3 55.7 38.6 4.24 0.86

Phase 2 middle Gr. 6 1.1 7.8 56.7 34.4 4.16 0.86


Lighthouse high school 3.4 3.4 62.5 30.7 4.14 0.86

I received sufficient professional development on the use of technology in my classroom.

Strongly

disagree

Somewhat

disagree

Somewhat

agree

Strongly

agree







Phase 2 middle Gr. 6 2.2 15.6 53.3 28.9 3.91 1.06 Lighthouse middle Gr. 7 2.3 7.0 58.1 32.6 4.12 0.91


I received sufficient professional development on the creation of a learner-centered

environment.

Strongly

disagree

Somewhat

disagree

Somewhat

agree

Strongly

agree






Phase 2 elementary Gr. K, 4, 5 4.5 6.8 45.5 43.2 4.16 1.05 Phase 2 middle Gr. 6 4.4 7.8 57.8 30.0 4.01 1.01





I am skilled at engaging my students in collaborative learning activities using technology as a

resource or tool.

Strongly

disagree

Somewhat

disagree

Somewhat

agree

Strongly

agree



Lighthouse elementary Gr. K, 4, 5 1.4 4.2 49.3 45.1 4.32 0.81 Phase 2 elementary Gr. 1-3 5.4 5.4 59.8 29.3 4.02 1.01



Phase 2 middle Gr. 6 3.3 6.7 58.9 31.1 4.08 0.94



How often did your students collaborate with other students on an in-class activity or

investigation?

Never

At least

once per

month

At least

once a

week

More than

once a

week







Phase 2 middle Gr. 6 2.2 21.1 36.7 40.0 3.14 0.83



To what extend did you use the following types of teaching practices this year: Direct

instruction/lecture. Never Rarely Moderately Frequently







Phase 2 middle Gr. 6 1.1 13.3 56.7 28.9 3.13 0.67





To what extend did you use the following types of teaching practices this year: Cooperative

learning. Never Rarely Moderately Frequently




Phase 2 elementary Gr. 1-3 0.0 0.0 29.3 70.7 3.71 0.46 Lighthouse middle Gr. 6 0.0 0.0 38.5 61.5 3.62 0.49


Phase 2 middle Gr. 6 0.0 2.2 45.6 52.2 3.50 0.55



To what extend did you use the following types of teaching practices this year: Student

projects. Never Rarely Moderately Frequently







Phase 2 middle Gr. 6 3.3 21.1 46.7 28.9 3.01 0.80



To what extend did you use the following types of teaching practices this year: Individualized

learning (at desk or computer). Never Rarely Moderately Frequently







Phase 2 middle Gr. 6 0.0 8.9 55.6 35.6 3.27 0.61 Lighthouse middle Gr. 7 0.0 11.6 60.5 27.9 3.16 0.61


My students have improved in their mastery of CCSS this year.

Strongly

disagree

Somewhat

disagree

Somewhat

agree

Strongly

agree







Phase 2 middle Gr. 6 5.7 18.4 60.9 14.9 3.61 1.12





My students have improved in their mastery of P21 skills this year.

Strongly

disagree

Somewhat

disagree

Somewhat

agree

Strongly

agree







Phase 2 middle Gr. 6 10.5 17.4 60.5 11.6 3.45 1.21



The student behavior in my classroom has improved this year.

Strongly

disagree

Somewhat

disagree

Somewhat

agree

Strongly

agree







Phase 2 middle Gr. 6 18.0 34.8 38.2 9.0 2.85 1.34



To what degree have you observed students using technology inappropriately (e.g., video

games, off-topic/risky Internet searches, instant messaging) during class time?

Not

at all

Rarel

y Occasionally Frequently

Extensivel

y

M SD % % % % %

Lighthouse elementary Gr. 1-3 3.0 39.0 39.0 17.0 2.0 2.76 0.84

Lighthouse elementary Gr. K, 4,

5 12.7 25.4 31.0 25.4 5.6 2.86 1.11

Phase 2 elementary Gr. 1-3 4.3 28.3 41.3 22.8 3.3 2.92 0.90 Lighthouse middle Gr. 6 0.0 11.5 25.0 42.3 21.2 3.73 0.93

Phase 2 elementary Gr. K, 4, 5 10.2 26.1 42.0 15.9 5.7 2.81 1.02

Phase 2 middle Gr. 6 2.2 3.3 25.6 46.7 22.2 3.83 0.89

Lighthouse middle Gr. 7 0.0 4.7 32.6 44.2 18.6 3.77 0.81

Lighthouse high school 0.0 4.5 18.2 48.9 28.4 4.01 0.81



How often did you use BCPS one to deliver instruction customized to students’ needs?

Never

At least

once per

month

At least

once a

week

More

than once

a week







Phase 2 middle Gr. 6 6.7 21.1 41.1 31.1 2.97 0.89



How often did you use BCPS one to develop formative assessments?

Never

At least

once per

month

At least

once a

week

More

than once

a week





Lighthouse middle Gr. 6 1.9 36.5 46.2 15.4 2.75 0.74 Phase 2 elementary Gr. K, 4, 5 19.3 27.3 29.5 23.9 2.58 1.06

Phase 2 middle Gr. 6 10.0 35.6 40.0 14.4 2.59 0.86



How often did you use BCPS one to develop assignments?

Never

At least

once per

month

At least

once a

week

More

than once

a week







Phase 2 middle Gr. 6 6.7 26.7 38.9 27.8 2.88 0.90





How often did you use BCPS one to post homework assignments?

Never

At least

once per

month

At least

once a

week

More

than once

a week







Phase 2 middle Gr. 6 30.7 22.7 27.3 19.3 2.35 1.11



To what degree is the use of technology an integral part of your teaching practices this year?

Not at

all Minimal Moderate

Fairly

strong

Very

strong

M SD % % % % %


Lighthouse elementary Gr. K, 4, 5 0.0 7.0 22.5 33.8 36.6 4.00 0.94

Phase 2 elementary Gr. 1-3 0.0 2.2 22.8 29.3 45.7 4.18 0.86

Lighthouse middle Gr. 6 0.0 3.8 23.1 34.6 38.5 4.08 0.88 Phase 2 elementary Gr. K, 4, 5 0.0 6.9 28.7 35.6 28.7 3.86 0.92

Phase 2 middle Gr. 6 0.0 6.7 24.4 32.2 36.7 3.99 0.94



To what degree is the use of technology part of your instructional planning and

administration (preparing lessons, grading, data management, etc.)?

Not at

all Minimal Moderate

Fairly

strong

Very

strong

M SD % % % % %


Lighthouse elementary Gr. K, 4, 5 0.0 2.8 14.1 21.1 62.0 4.42 0.84

Phase 2 elementary Gr. 1-3 0.0 0.0 14.1 27.2 58.7 4.45 0.73


Phase 2 elementary Gr. K, 4, 5 0.0 1.1 11.5 35.6 51.7 4.38 0.74

Phase 2 middle Gr. 6 0.0 1.1 16.7 32.2 50.0 4.31 0.79



I was adequately informed of the expected role of my school as a Lighthouse School.

Strongly

disagree

Somewhat

disagree

Somewhat

agree

Strongly

agree



I was adequately prepared to support my school in fulfilling its role as a Lighthouse School.

Strongly disagree

Somewhat disagree

Somewhat agree

Strongly agree





I feel my school was successful this year in fulfilling its role as a Lighthouse School.

Strongly

disagree

Somewhat

disagree

Somewhat

agree

Strongly

agree





Appendix C: Principal Interview Protocol

Classroom impact

1. What impact, if any, has the S.T.A.T. initiative had on student engagement? For example,

the degree to which students work collaboratively and interact with peers

2. What impact, if any, has S.T.A.T. had on student behavior in your school’s classrooms?

3. What sort of changes have you observed in teacher practices this year as compared with

last year?

4. Please describe how your school has integrated technology into your classrooms this

year. What have been your greatest successes with technology integration?

Student impact

5. What impact, if any, has the S.T.A.T. initiative had on student mastery of Common Core

State Standards?

6. What impact, if any, has the S.T.A.T. initiative had on student mastery of P21 skills such

as problem solving and critical thinking?

Professional development

7. Were your teachers adequately prepared to implement S.T.A.T. this year? Why or why

not?

8. What professional development do you feel your teachers are still in need of in order to

implement S.T.A.T.?

Lighthouse School preparation (asked to Lighthouse middle school principals only)

9. How, if at all, have you been prepared to support your school in its designation as a

Lighthouse school?

10. What additional preparation, if any, do you recommend for future Lighthouse schools?

S.T.A.T. Teacher role

11. How, if at all, has your S.T.A.T. teacher supported your implementation of S.T.A.T.

(beyond PD sessions)?

12. What are the benefits of the S.T.A.T. teacher role?

13. What improvements, if any, do you suggest for the S.T.A.T. teacher program?

Principal role

14. How have you supported your teacher’s implementation of S.T.A.T.?



15. How have others within the district (peers, curriculum specialists, technology directors)

supported your school’s implementation of S.T.A.T.?

S.T.A.T. perceptions

16. What are the overall strengths of the S.T.A.T. initiative?

17. What changes, if any, do you recommend in the S.T.A.T. initiative?



Appendix D: S.T.A.T. Teacher Interview Protocol

Classroom impact



2. What impact, if any, has S.T.A.T. had on student behavior in your school’s classrooms?

3. What sort of changes have you observed in teacher practices this year as compared with

last year?

4. Please describe how your school has integrated technology into your classrooms this

year. What have been your greatest successes with technology integration?

Student impact


State Standards?




7. What sort of professional development have you received in terms of the S.T.A.T.

initiative?

8. Do you feel adequately prepared to serve as a S.T.A.T. teacher? Why or why not?

9. What professional development do you feel you are still in need of in order to better serve

as a S.T.A.T. teacher?

Lighthouse School preparation (asked to Lighthouse middle school S.T.A.T. teachers only


Lighthouse school?


STAT Teacher role

12. Describe your role as a S.T.A.T. teacher in terms of supporting the S.T.A.T. initiative

within your school.

13. What sort of professional development have you offered to the teachers within your

school?

14. What has been the impact of the professional development you’ve offered?



Principal role

15. How, if at all, has your principal supported your implementation of S.T.A.T.?

16. How, if at all, have others within the district supported your implementation of S.T.A.T.?

STAT perceptions





Appendix E: Teacher Focus Group Protocol

Classroom impact



2. What sort of changes have you made in your teaching this year as compared with last

year?

3. Please describe how you have integrated technology into your classrooms this year. What

have been your greatest successes with technology integration?

Student impact


State Standards?



6. How, if at all, has technology impacted your ability to manage the classroom?


7. Were you adequately prepared to implement S.T.A.T. this year? Why or why not?

Lighthouse School preparation (asked of Lighthouse high school teachers only)


Lighthouse school?


STAT Teacher perceptions

10. How, if at all, has your S.T.A.T. teacher supported your implementation of S.T.A.T.

(beyond PD sessions)?

11. What are the benefits of the S.T.A.T. teacher role?

12. What improvements, if any, do you suggest for the S.T.A.T. teacher program?

Principal role

13. How, if at all, has your principal supported your implementation of S.T.A.T.?

STAT perceptions




Appendix F: OASIS-21


Information and communications that support independent thinking are highly visible in the

classroom.

□ Not observed □ Somewhat □ Extensive

Information and resources that reflect content being taught is visibly displayed in classroom. □ Not observed □ General Subject □ Lesson-specific

Students move around the room independently acquiring material and resources.

□ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

Students utilize different workspaces for different learning environments (e.g. collaborative, independent, receiving direct instruction).


Student Engagement

Students using digital tools for learning. □ Not observed □ Rarely □ Occasionally □ Frequently □ Extensively

Multiple modes of student responses (e.g. verbal, written, through technology, active votes,

texting, physical response.)


Independent work.


Collaborative learning.


Student discussion.


P21 Skills

Problem solving.


Project-based approaches to instruction.


Inquiry-based approaches to instruction.


Learning incorporates authentic/real world contexts.




Teacher Practice

Teachers acting as coach/facilitator. (Teacher facilitates the efficient and effective use of digital tools and content.)


Teacher presentation.


Higher-order instructional feedback given.


Communication is initiated by students.


Higher-level questioning.


Flexible grouping based on student and task needs.




Appendix G: OASIS-21 Reference Guide

Classroom Environment Student Engagement

Information supporting independent thinking

Quotes, slogans conveying that inquiry is valued

Multiple modes of student responses

Verbal, written, through technology, active

votes, texting, physical response

More than one mode used when responding to

other students or to teacher.

Information reflecting content being taught

Dependent on subject matter of lesson

Ex: Lesson is on multiplication and a poster

conveying steps for multiplication is displayed

Students using digital tools

Using devices independently or in group

Watching a video, reading, writing

Non-ex: Teacher using of digital tools

Students move around the room independently

Students acquire materials needed for a task or

project they’re working on.

Non-ex: Teacher directs students to obtain notebooks

from the bookshelf.

Independent work

Students working alone on an assignment or

practicing content

Non-ex: Students working on non-instructional task

should not be coded

Students utilize different work spaces

Spaces for collaboration, independent work, etc.

are utilized by students

At least two different workspaces are being used

Ex: Students working in a group at a cluster of desks

while another group is seated on a reading mat doing independent work.

Non-ex: All students seated in front of white board

for teacher presentation, though other areas are

present.

Collaborative learning

Students working in pairs or small groups to

complete a task or project.

Involves collaboration, helping each other.

Non-ex: Students talking to each other on topic not

related to the lesson.

Student discussion

Discussion amongst students (pairs, groups,

class) on a prompted topic or higher-level

question

Non-ex: collaborating to complete a task.

Ratings:

Not observed (NO): Strategy never observed.

Rarely (R): Received little emphasis, not a dominant instructional or learning component

Somewhat/Occasionally

(S/O):

Receives modest emphasis or time in class

Frequently (F): Receives substantial emphasis or time in class, dominant component

Extensive(ly) (E): Highly prevalent in class, strongly emphasized



P21 Skills Teacher Practice

Problem solving

Students work together to solve problems

May be prompted by teacher, but teacher is not

directly involved.

Higher standard than problems involving recall.

Multiple resources used, using resources

effectively, critical thinking involved

Non-ex: Mathematics problems.

Teacher as coach/facilitator.

Teacher facilitates the efficient and effective use

of digital tools and content.

Teacher is supportive

Non-ex: Teacher disciplining students.

Project-based approaches

Instructional focus is centered on an inquiry or

question

Projects may result in tangible product (research

report, presentation, etc.)

Students are seen working on the project

Non-ex: Project as part of the day’s lesson.

Teacher presentation

Teacher lecture, teacher offering direct

instruction

Do not code classroom management.

Inquiry-based approaches

Students explore a question/topic/theme in-depth, develop and ask further questions, and

conduct research and problem-solve to answer

the questions

Ex: Students given a topic to explore, students

develop questions, use the Internet to research the

topic.

Higher-order instructional feedback

Feedback related to learning process

Provides elaborative feedback

Offers an explanation, provides new information

Ex: Teacher agrees that student response is correct,

then extends student response by adding new

information.

Non-ex: Only stating correctness of response and

moving on. Motivational/encouraging phrases.

Authentic/real world contexts

Problems that students investigate may relate (or

stem from) problems students can relate to in

their own world

Lesson or problems are specifically tailored to

students’ world.

Communication is initiated by students

Asking questions of peers or teacher

Communicate beyond what is asked

Higher level questioning

Questions beyond factual recall

Questions that stimulate discussion

Ex: Questions that involve producing an explanation,

providing an example, making a prediction,

compare/contrast.

Non-ex: Questions that involve memorization to

produce a correct answer.

Flexible grouping of students

Grouping based on ability level

Grouping based on tasks

Differentiated instruction

Will likely need to ask teacher how student groups

were formed.



Appendix H: OASIS-21 Results


Not

Observed

Somewhat Extensive

M SD % % %

Information and communications that support independent thinking are highly visible in the classroom.

LH elementary Grades 1-3

Fall 2014 50.0 37.5 12.5 1.63 0.71

Spring 2017 37.5 43.8 18.8 1.81 0.74

LH elementary Grades K, 4, 5

Fall 2015 44.4 37.0 18.5 1.74 0.76

Spring 2017 36.0 40.0 24.0 1.88 0.78

Phase 2 elementary Grades 1-3

Spring 2015 55.0 35.0 10.0 1.55 0.68

Spring 2017 50.0 42.9 7.1 1.57 0.63

Phase 2 elementary Grades K, 4, 5

Fall 2016 39.3 32.1 28.6 1.89 0.83

Spring 2017 54.2 29.2 16.7 1.63 0.77

LH middle Grade 6 Fall 2015 35.7 28.6 35.7 2.00 0.86

Spring 2017 60.0 26.7 13.3 1.53 0.74

Phase 2 middle Grade 6

Spring 2016 25.0 10.7 64.3 2.39 0.57

Spring 2017 42.9 39.3 17.9 1.75a 0.75

LH middle Grade 7

Fall 2016 41.7 16.7 41.7 2.00 0.95

Spring 2017 76.9 23.1 0.0 1.23 0.44

LH high school

Fall 2016 33.3 25.0 41.7 2.08 0.90

Spring 2017 50.0 41.7 8.3 1.58 0.67 aSpring 2017 was significantly lower than baseline p < .05



Not

Observed

General

Subject

Lesson-

specific

M SD % % %

Information and resources that reflect content being taught is visibly displayed in classroom.


Fall 2014 2.5 70.0 27.5 2.25 0.49

Spring 2017 9.4 40.6 50.0 2.41 0.67


Fall 2015 11.1 37.0 51.9 2.41 0.69

Spring 2017 12.0 48.0 40.0 2.28 0.68


Spring 2015 12.5 42.5 45.0 2.33 0.69

Spring 2017 0.0 71.4 28.6 2.29 0.46


Fall 2016 7.1 42.9 50 2.43 0.63

Spring 2017 4.2 54.2 41.7 2.38 0.58

LH middle Grade 6

Fall 2015 3.6 39.3 57.1 2.54 0.58

Spring 2017 0.0 73.3 26.7 2.27a 0.46


Spring 2016 10.7 21.4 67.9 2.57 0.31

Spring 2017 7.1 50.0 42.9 2.36 0.62

LH middle Grade 7

Fall 2016 0.0 58.3 41.7 2.42 0.51

Spring 2017 15.4 53.8 30.8 2.15 0.69

LH high school

Fall 2016 0.0 50.0 50.0 2.50 0.52

Spring 2017 0.0 58.3 41.7 2.42 0.51 a Spring 2017 was significantly lower than baseline observations, p <.05



Not

Observed Rarely

Somewhat/

Occasionally Frequently Extensively

M SD % % % % %

Students move around the room independently acquiring materials and resources.


Fall 2014 52.5 22.5 12.5 7.5 5.0 1.90 1.19

Spring 2017 68.8 0.0 31.3 0.0 0.0 1.63 0.94


Fall 2015 51.9 14.8 22.2 7.4 3.7 1.96 1.19

Spring 2017 68.0 0.0 32.0 0.0 0.0 1.64 0.95


Spring 2015 70.0 17.5 12.5 0.0 0.0 1.43 0.71 Spring 2017 67.9 0.0 32.1 0.0 0.0 1.64 0.95


Fall 2016 57.1 25.0 10.7 3.6 3.6 1.71 1.05

Spring 2017 79.2 0.0 16.7 4.2 0.0 1.46 0.93

LH middle Grade 6

Fall 2015 78.6 10.7 10.7 0.0 0.0 1.32 0.67 Spring 2017 73.3 0.0 26.7 0.0 0.0 1.53 0.92


Spring 2016 82.1 10.7 0.0 7.1 0.0 1.32 0.53

Spring 2017 78.6 0.0 17.9 3.6 0.0 1.46 0.92

LH middle Grade 7

Fall 2016 50.0 33.3 8.3 8.3 0.0 1.75 0.97

Spring 2017 69.2 0.0 23.1 7.7 0.0 1.69 1.11

LH high school

Fall 2016 66.7 25.0 0.0 0.0 8.3 1.58 1.16

Spring 2017 75.0 0.0 25.0 0.0 0.0 1.50 0.90



Not

Observed Rarely

Somewhat/


% % % % % M SD

Students utilize different work spaces for different learning environments


Fall 2014 40.0 5.0 22.5 25.0 7.5 2.55 1.43

Spring 2017 40.6 0.0 21.9 25.0 12.5 2.69 1.53


Fall 2015 55.6 7.4 14.8 3.7 18.5 2.22 1.60

Spring 2017 60.0 0.0 20.0 20.0 0.0 2.00 1.29


Spring 2015 50.0 17.5 2.5 22.5 7.5 2.20 1.45

Spring 2017 39.3 0.0 35.7 10.7 14.3 2.61 1.47


Fall 2016 57.1 21.4 10.7 3.6 7.1 1.82 1.22

Spring 2017 54.2 0.0 37.5 0.0 8.3 2.08 1.32

LH middle Grade 6

Fall 2015 89.3 3.6 7.1 0.0 0.0 1.18 0.55

Spring 2017 73.3 0.0 26.7 0.0 0.0 1.53 0.92


Spring 2016 85.7 7.1 7.1 0.0 0.0 1.21 0.37

Spring 2017 96.4 0.0 3.6 0.0 0.0 1.07 0.38

LH middle Grade 7

Fall 2016 91.7 0.0 8.3 0.0 0.0 1.17 0.58

Spring 2017 69.2 0.0 23.1 0.0 7.7 1.77 1.30

LH high school

Fall 2016 91.7 0.0 0.0 8.3 0.0 1.25 0.87

Spring 2017 83.3 0.0 8.3 0.0 8.3 1.50 1.24



Teacher Practice

Not

Observed Rarely

Somewhat/


M SD % % % % %

Teacher acting as coach/facilitator.


Fall 2014 20.0 17.5 30.0 25.0 7.5 2.83 1.24 Spring 2017 9.4 15.6 28.1 18.8 28.1 3.41 1.32


Fall 2015 0.0 3.7 14.8 51.9 29.6 4.07 0.78

Spring 2017 28.0 4.0 24.0 20.0 24.0 3.08 1.55


Spring 2015 7.5 5.0 57.7 30.0 0.0 3.10 0.81

Spring 2017 21.4 17.9 35.7 25.0 0.0 2.64 1.10


Fall 2016 14.3 28.6 35.7 14.3 7.1 2.71 1.12

Spring 2017 20.8 12.5 29.2 25.0 12.5 2.96 1.33

LH middle Grade 6

Fall 2015 17.9 21.4 46.4 10.7 3.6 2.61 1.03

Spring 2017 13.3 20.0 0.0 6.7 60.0 3.80 1.66


Spring 2016 10.7 14.3 17.9 21.4 35.7 3.57 0.80

Spring 2017 21.4 21.4 25.0 14.3 17.9 2.86 1.41

LH middle Grade 7 Fall 2016 16.7 16.7 16.7 41.7 8.3 3.08 1.31

Spring 2017 0.0 15.4 23.1 15.4 46.2 3.92 1.19

LH high school

Fall 2016 0.0 33.3 41.7 16.7 8.3 3.00 0.95

Spring 2017 8.3 0.0 16.7 25.0 50.0 4.08 1.24



Not

Observed Rarely

Somewhat/


% % % % % M SD

Teacher presentation. LH elementary Grades 1-3

Fall 2014 27.5 15.0 27.5 20.0 10.0 2.7 1.34

Spring 2017 28.1 21.9 37.5 6.3 6.3 2.41 1.16


Fall 2015 44.4 11.1 29.6 11.1 3.7 2.19 1.24

Spring 2017 32.0 24.0 32.0 4.0 8.0 2.32 1.22


Spring 2015 17.5 10.0 25.0 32.5 15.0 3.18 1.32

Spring 2017 32.1 25.0 17.9 17.9 7.1 2.43 1.32


Fall 2016 35.7 7.1 21.4 3.6 32.1 2.89 1.71

Spring 2017 29.2 29.2 25.0 16.7 0.0 2.29 1.08

LH middle Grade 6

Fall 2015 32.1 28.6 25.0 7.1 7.1 2.29 1.21

Spring 2017 26.7 26.7 13.3 20.0 13.3 2.67 1.45


Spring 2016 71.4 7.1 14.3 3.6 3.6 1.61 0.35 Spring 2017 57.1 14.3 25.0 0.0 3.6 1.79 1.07

LH middle Grade 7

Fall 2016 41.7 25.0 16.7 16.7 0.0 2.08 1.16

Spring 2017 30.8 23.1 15.4 23.1 7.7 2.54 1.39

LH high school

Fall 2016 50.0 16.7 25.0 0.0 8.3 2.00 1.28

Spring 2017 58.3 8.3 16.7 16.7 0.0 1.92 1.24



Not

Observed Rarely

Somewhat/


% % % % % M SD

Higher-order instructional feedback given LH elementary Grades 1-3

Fall 2014 37.5 17.5 32.5 10.0 2.5 2.23 1.14

Spring 2017 46.9 9.4 12.5 25.0 6.3 2.34 1.45


Fall 2015 48.1 11.1 25.9 3.7 11.1 2.19 1.39

Spring 2017 36.0 16.0 24.0 20.0 4.0 2.40 1.29


Spring 2015 42.5 25.0 27.5 2.0 0.0 1.95 0.96

Spring 2017 42.9 17.9 17.9 10.7 10.7 2.29 1.41


Fall 2016 32.1 21.4 17.9 21.4 7.1 2.50 1.35

Spring 2017 62.5 16.7 8.3 4.2 8.3 1.79a 1.28

LH middle Grade 6

Fall 2015 46.4 21.4 21.4 10.7 0.0 1.96 1.07

Spring 2017 40.0 6.7 20.0 6.7 26.7 2.73 1.71


Spring 2016 17.9 28.6 10.7 35.7 7.1 2.86 0.75 Spring 2017 67.9 17.9 7.1 3.6 3.6 1.57a 1.03

LH middle Grade 7

Fall 2016 50.0 16.7 25.0 0.0 8.3 2.00 1.28

Spring 2017 46.2 0.0 15.4 23.1 15.4 2.62 1.66

LH high school

Fall 2016 16.7 58.3 8.3 8.3 8.3 2.33 1.15

Spring 2017 25.0 8.3 25.0 25.0 16.7 3.00 1.48 a Spring 2017 was significantly lower than baseline observations, p <.05



Not

Observed Rarely

Somewhat/


% % % % % M SD

Communication is initiated by students. LH elementary Grades 1-3

Fall 2014 12.5 47.5 35.0 2.5 2.5 2.35 0.83

Spring 2017 25.0 25.0 28.1 21.9 0.0 2.47 1.11


Fall 2015 29.6 33.3 25.9 7.4 3.7 2.22 1.09

Spring 2017 20.0 28.0 28.0 16.0 8.0 2.64 1.22


Spring 2015 37.5 40.0 20.0 2.5 0.0 1.88 0.82

Spring 2017 28.6 28.6 35.7 7.1 0.0 2.21 0.96


Fall 2016 28.6 46.4 10.7 14.3 0.0 2.11 0.99

Spring 2017 37.5 12.5 33.3 12.5 4.2 2.33 1.24

LH middle Grade 6

Fall 2015 32.1 42.9 21.4 3.6 0.0 1.96 0.84

Spring 2017 20.0 26.7 13.3 33.3 6.7 2.80 1.32


Spring 2016 17.9 21.4 32.1 25.0 3.6 2.75 0.90 Spring 2017 39.3 21.4 25.0 10.7 3.6 2.18 1.19

LH middle Grade 7

Fall 2016 33.3 0.0 50.0 8.3 8.3 2.58 1.31

Spring 2017 30.8 15.4 23.1 23.1 7.7 2.62 1.39

LH high school

Fall 2016 0.0 25.0 41.7 33.3 0.0 3.08 0.79

Spring 2017 0.0 25.0 33.3 41.7 0.0 3.17 0.83



Not

Observed Rarely

Somewhat/


% % % % % M SD

Higher-level questioning.


Fall 2014 12.5 17.5 35.0 32.5 2.5 2.95 1.06

Spring 2017 46.9 9.4 15.6 12.5 15.6 2.41 1.56


Fall 2015 37.0 7.4 25.9 25.9 3.7 2.52 1.34

Spring 2017 44.0 16.0 24.0 8.0 8.0 2.20 1.32

Phase 2 elementary Grades 1-3 Spring 2015 30.0 17.5 27.5 20.0 5.0 2.53 1.26

Spring 2017 42.9 14.3 7.1 21.4 14.3 2.50 1.58


Fall 2016 42.9 10.7 10.7 21.4 14.3 2.54 1.57

Spring 2017 41.7 25.0 12.5 12.5 8.3 2.21 1.35

LH middle Grade 6

Fall 2015 57.1 14.3 14.3 14.3 0.0 1.86 1.15

Spring 2017 40.0 20.0 26.7 0.0 13.3 2.27 1.39


Spring 2016 21.4 39.3 25.0 10.7 3.6 2.36 0.75

Spring 2017 71.4 10.7 10.7 3.6 3.6 1.57 1.07

LH middle Grade 7

Fall 2016 58.3 16.7 8.3 8.3 8.3 1.92 1.38

Spring 2017 30.8 15.4 0.0 23.1 30.8 3.08a 1.75

LH high school

Fall 2016 50.0 25.0 0.0 16.7 8.3 2.08 1.44

Spring 2017 50.0 16.7 16.7 16.7 0.0 2.00 1.21 a Spring 2017 mean was significantly higher than baseline, p < .05



Student Engagement

Not

Observed Rarely

Somewhat/


M SD % % % % %

Students using digital tools for learning.


Fall 2014 22.5 12.5 17.5 32.5 15.0 3.05 1.41

Spring 2017 28.1 0.0 37.5 18.8 15.6 2.94 1.41 LH elementary Grades K, 4, 5

Fall 2015 33.3 7.4 22.2 14.8 22.2 2.85 1.59

Spring 2017 48.0 0.0 28.0 0.0 24.0 2.52 1.66


Spring 2015 60.0 17.5 22.5 0.0 0.0 1.63 0.84

Spring 2017 50.0 0.0 35.7 3.6 10.7 2.25a 1.40


Fall 2016 39.3 14.3 10.7 14.3 21.4 2.64 1.64

Spring 2017 45.8 0.0 12.5 25.0 16.7 2.67 1.66

LH middle Grade 6

Fall 2015 25.0 0.0 14.3 17.9 42.9 3.54 1.64

Spring 2017 20.0 0.0 20.0 20.0 40.0 3.60 1.55


Spring 2016 85.7 3.6 3.6 0.0 7.1 1.39 0.50

Spring 2017 32.1 0.0 21.4 10.7 35.7 3.18a 1.70

LH middle Grade 7

Fall 2016 50.0 8.3 16.7 16.7 8.3 2.25 1.48 Spring 2017 30.8 0.0 38.5 15.4 15.4 2.85 1.46

LH high school

Fall 2016 41.7 16.7 8.3 0.0 33.3 2.67 1.83

Spring 2017 50.0 0.0 33.3 8.3 8.3 2.25 1.42 a Spring 2017 mean was significantly higher than baseline mean, p < .05



Not

Observed Rarely

Somewhat/


% % % % % M SD

Multiple modes of student responses.


Fall 2014 22.5 12.5 35.0 30.0 0.0 2.73 1.13

Spring 2017 56.3 25.0 12.5 6.3 0.0 1.69 0.93


Fall 2015 59.3 11.1 25.9 3.7 0.0 1.74 0.98

Spring 2017 48.0 20.0 16.0 12.0 4.0 2.04 1.24


Spring 2015 32.5 30.0 35.0 2.5 0.0 2.08 0.89 Spring 2017 32.1 28.6 21.4 17.9 0.0 2.25 1.11


Fall 2016 35.7 14.3 28.6 7.1 14.3 2.50 1.43

Spring 2017 54.2 12.5 20.8 12.5 0.0 1.92 1.14

LH middle Grade 6

Fall 2015 60.7 10.7 21.4 0.0 7.1 1.82 1.22

Spring 2017 20.0 40.0 33.3 6.7 0.0 2.27a 0.88


Spring 2016 21.4 28.6 10.7 3.6 35.7 3.04 0.83

Spring 2017 46.4 28.6 21.4 3.6 0.0 1.82b 0.90

LH middle Grade 7

Fall 2016 33.3 33.3 33.3 0.0 0.0 2.00 0.85

Spring 2017 23.1 23.1 46.2 7.7 0.0 2.38 0.96

LH high school

Fall 2016 25.0 75.0 0.0 0.0 0.0 1.75 0.45

Spring 2017 25.0 33.3 41.7 0.0 0.0 2.17 0.83 a Spring 2017 mean was significantly higher than baseline mean, p < .05 b Spring 2017 mean was significantly lower than baseline mean, p < .05



Not

Observed Rarely

Somewhat/


M SD % % % % %

Independent work


Fall 2014 15.0 0.0 32.5 42.5 10.0 3.33 1.16

Spring 2017 12.5 9.4 15.6 40.6 21.9 3.50 1.30


Fall 2015 29.6 0.0 22.2 25.9 22.2 3.11 1.55

Spring 2017 24.0 16.0 20.0 20.0 20.0 2.96 1.49


Spring 2015 30.0 17.5 22.5 27.5 2.5 2.55 1.26

Spring 2017 17.9 17.9 28.6 21.4 14.3 2.96 1.32


Fall 2016 35.7 7.1 10.7 21.4 25.0 2.93 1.68

Spring 2017 12.5 4.2 25.0 41.7 16.7 3.46 1.22

LH middle Grade 6

Fall 2015 21.4 3.6 14.3 35.7 25.0 3.39 1.47

Spring 2017 0.0 6.7 20.0 6.7 66.7 4.33 1.05 Phase 2 middle Grade 6

Spring 2016 10.7 7.1 17.9 3.6 60.7 3.96 0.81

Spring 2017 14.3 3.6 17.9 25.0 39.3 3.71 1.41

LH middle Grade 7

Fall 2016 8.3 8.3 8.3 33.3 41.7 3.92 1.31

Spring 2017 15.4 15.4 30.8 7.7 30.8 3.23 1.48

LH high school

Fall 2016 0.0 8.3 16.7 16.7 58.3 4.25 1.06

Spring 2017 16.7 0.0 0.0 16.7 66.7 4.17 1.53



Not

Observed Rarely

Somewhat/


M SD % % % % %

Collaborative learning.


Fall 2014 57.5 22.5 7.5 12.5 0.0 1.75 1.06


Fall 2015 37.0 3.7 14.8 29.6 14.8 2.81 1.57

Spring 2017 48.0 28.0 8.0 8.0 8.0 2.00 1.29


Spring 2015 42.5 15.0 30.0 12.5 0.0 2.13 1.11

Spring 2017 64.3 3.6 17.9 14.3 0.0 1.82 1.19


Fall 2016 57.1 14.3 25.0 0.0 3.6 1.79 1.07

Spring 2017 75.0 8.3 12.5 4.2 0.0 1.46 0.88

LH middle Grade 6

Fall 2015 53.6 3.6 17.9 14.3 10.7 2.25 1.51

Spring 2017 73.3 13.3 0.0 6.7 6.7 1.60 1.24


Spring 2016 78.6 7.1 0.0 7.1 7.1 1.57 0.69 Spring 2017 67.9 14.3 7.1 10.7 0.0 1.61 1.03

LH middle Grade 7

Fall 2016 75.0 8.3 8.3 8.3 0.0 1.50 1.00

Spring 2017 76.9 0.0 15.4 0.0 7.7 1.62 1.26

LH high school

Fall 2016 83.3 8.3 8.3 0.0 0.0 1.25 0.62

Spring 2017 58.3 25.0 0.0 0.0 16.7 1.92 1.51



Not

Observed Rarely

Somewhat/


M SD % % % % %

Student discussion.


Fall 2014 82.5 7.5 7.5 2.5 0.0 1.30 0.72

Spring 2017 40.6 18.8 15.6 18.8 6.3 2.31a 1.35


Fall 2015 51.9 3.7 29.6 14.8 0.0 2.07 1.21

Spring 2017 64.0 12.0 12.0 8.0 4.0 1.76 1.20


Spring 2015 52.5 15.0 20.0 12.5 0.0 1.93 1.12

Spring 2017 50.0 17.9 17.9 10.7 3.6 2.00 1.22


Fall 2016 71.4 7.1 14.3 7.1 0.0 1.57 1.00

Spring 2017 41.7 33.3 20.8 4.2 0.0 1.88 0.90

LH middle Grade 6

Fall 2015 57.1 14.3 14.3 7.1 7.1 1.93 1.30

Spring 2017 80.0 0.0 6.7 6.7 6.7 1.60 1.30 Phase 2 middle Grade 6

Spring 2016 25.0 21.4 39.3 7.1 7.1 2.50 0.46

Spring 2017 82.1 10.7 0.0 7.1 0.0 1.32b 0.82

LH middle Grade 7

Fall 2016 75.0 8.3 8.3 8.3 0.0 1.50 1.00

Spring 2017 53.8 23.1 0.0 7.7 15.4 2.08 1.55

LH high school

Fall 2016 91.7 0.0 8.3 0.0 0.0 1.17 0.58

Spring 2017 83.3 8.3 0.0 8.3 0.0 1.33 0.89 a Spring 2017 mean was significantly higher than baseline mean, p <.05 b Spring 2017 mean was significantly lower than baseline mean, p <.05



P21 Skills

Not

Observed Rarely

Somewhat/


M SD % % % % %

Problem solving.


Fall 2014 90.0 7.5 2.5 0.0 0.0 1.13 0.40


Fall 2015 70.4 3.7 11.1 7.4 7.4 1.78 1.34

Spring 2017 88.0 0.0 8.0 0.0 4.0 1.32 0.95


Spring 2015 92.5 0.0 2.5 5.0 0.0 1.20 0.72

Spring 2017 89.3 7.1 3.6 0.0 0.0 1.14 0.45


Fall 2016 85.7 3.6 3.6 3.6 3.6 1.36 0.99

Spring 2017 83.3 4.2 8.3 4.2 0.0 1.33 0.82

LH middle Grade 6

Fall 2015 67.9 3.6 21.4 7.1 0.0 1.68 1.06

Spring 2017 100.0 0.0 0.0 0.0 0.0 1.00 0.00


Spring 2016 96.4 0.0 3.6 0.0 0.0 1.07 0.19

Spring 2017 96.4 0.0 0.0 3.6 0.0 1.11 0.57

LH middle Grade 7

Fall 2016 91.7 0.0 8.3 0.0 0.0 1.17 0.58 Spring 2017 92.3 0.0 0.0 0.0 7.7 1.31 1.11

LH high school

Fall 2016 83.3 8.3 0.0 8.3 0.0 1.33 0.89

Spring 2017 75.0 0.0 8.3 8.3 8.3 1.75 1.42



Not

Observed Rarely

Somewhat/


M SD % % % % %

Project-based approaches to instruction.


Fall 2014 92.5 0.0 2.5 0.0 5.0 1.25 0.93

Spring 2017 90.6 0.0 0.0 3.1 6.3 1.34 1.10


Fall 2015 88.9 0.0 0.0 3.7 7.4 1.41 1.19

Spring 2017 88.0 0.0 0.0 4.0 8.0 1.44 1.23


Spring 2015 95.0 0.0 0.0 5.0 0.0 1.15 0.66

Spring 2017 89.3 3.6 0.0 3.6 3.6 1.29 0.94


Fall 2016 89.3 0.0 0.0 0.0 10.7 1.43 1.26

Spring 2017 87.5 0.0 0.0 4.2 8.3 1.46 1.25

LH middle Grade 6

Fall 2015 82.1 0.0 0.0 0.0 17.9 1.71 1.56

Spring 2017 93.3 0.0 0.0 0.0 6.7 1.27 1.03


Spring 2016 96.4 0.0 0.0 0.0 3.6 1.14 0.38

Spring 2017 85.7 0.0 0.0 3.6 10.7 1.54 1.35

LH middle Grade 7

Fall 2016 91.7 0.0 0.0 0.0 8.3 1.33 1.15

Spring 2017 84.6 0.0 0.0 0.0 15.4 1.62 1.50

LH high school

Fall 2016 58.3 0.0 0.0 8.3 33.3 2.58 1.98 Spring 2017 66.7 0.0 8.3 8.3 16.7 2.08 1.68



Not

Observed Rarely

Somewhat/


% % % % % M SD

Inquiry-based approaches to instruction.

LH elementary Grades 1-3 Fall 2014 90.0 2.5 2.5 2.5 2.5 1.25 0.84

Spring 2017 84.4 6.3 3.1 3.1 3.1 1.34 0.94


Fall 2015 96.3 0.0 0.0 0.0 3.7 1.15 0.77

Spring 2017 72.0 8.0 12.0 0.0 8.0 1.64 1.22


Spring 2015 95.0 0.0 0.0 2.5 2.5 1.18 0.78

Spring 2017 96.4 0.0 0.0 0.0 3.6 1.14 0.76


Fall 2016 92.9 0.0 0.0 3.6 3.6 1.25 0.93

Spring 2017 87.5 0.0 0.0 8.3 4.2 1.42 1.14

LH middle Grade 6

Fall 2015 67.9 3.6 7.1 10.7 10.7 1.93 1.49

Spring 2017 66.7 6.7 13.3 0.0 13.3 1.87 1.46


Spring 2016 96.4 0.0 0.0 0.0 3.6 1.14 0.38

Spring 2017 71.4 0.0 3.6 10.7 14.3 1.96 1.60

LH middle Grade 7

Fall 2016 83.3 0.0 16.7 0.0 0.0 1.33 0.78 Spring 2017 53.8 15.4 7.7 7.7 15.4 2.15 1.57

LH high school

Fall 2016 58.3 0.0 0.0 16.7 25.0 2.50 1.88

Spring 2017 50.0 0.0 8.3 8.3 33.3 2.75 1.91



Not

Observed Rarely

Somewhat/


M SD % % % % %

Learning incorporates authentic/real world contexts.


Fall 2014 50.0 27.5 20.0 0.0 2.5 1.78 0.95

Spring 2017 56.3 3.1 9.4 12.5 18.8 2.34 1.68


Fall 2015 63.0 0.0 29.6 0.0 7.4 1.89 1.28

Spring 2017 64.0 12.0 8.0 12.0 4.0 1.80 1.26

Phase 2 elementary Grades 1-3 Spring 2015 45.0 30.0 15.0 7.5 2.5 1.93 1.07

Spring 2017 67.9 3.6 17.9 3.6 7.1 1.79 1.29


Fall 2016 53.6 7.1 10.7 10.7 17.9 2.32 1.63

Spring 2017 70.8 0.0 12.5 12.5 4.2 1.79 1.32

LH middle Grade 6

Fall 2015 71.4 3.6 7.1 14.3 3.6 1.75 1.29

Spring 2017 73.3 13.3 0.0 6.7 6.7 1.60 1.24


Spring 2016 32.1 14.3 10.7 10.7 32.1 2.96 0.94

Spring 2017 75.0 3.6 14.3 7.1 0.0 1.54a 1.00

LH middle Grade 7

Fall 2016 66.7 0.0 8.3 16.7 8.3 2.00 1.54

Spring 2017 61.5 7.7 15.4 15.4 0.0 1.85 1.21

LH high school

Fall 2016 66.7 16.7 8.3 0.0 8.3 1.67 1.23

Spring 2017 41.7 16.7 16.7 0.0 25.0 2.50 1.68 a Spring 2017 mean was significantly lower than the baseline mean, p <.05



Not

Observed Rarely

Somewhat/


M SD % % % % %

Flexible grouping based on student and task needs.


Fall 2014 62.5 10.0 12.5 7.5 7.5 1.88 1.32

Spring 2017 71.9 0.0 9.4 9.4 9.4 1.84 1.44


Fall 2015 85.2 3.7 3.7 0.0 7.4 1.41 1.12

Spring 2017 64.0 8.0 12.0 4.0 12.0 1.92 1.44


Spring 2015 82.5 5.0 5.0 5.0 2.5 1.40 0.98

Spring 2017 71.4 3.6 14.3 3.6 7.1 1.71 1.27


Fall 2016 67.9 10.7 0.0 7.1 14.3 1.89 1.52

Spring 2017 87.5 0.0 8.3 0.0 4.2 1.33a 0.96

LH middle Grade 6

Fall 2015 96.4 0.0 0.0 3.6 0.0 1.11 0.57 Spring 2017 93.3 0.0 0.0 0.0 6.7 1.27 1.03


Spring 2016 71.4 3.6 3.6 0.0 21.4 2.02 1.29

Spring 2017 92.9 3.6 3.6 0.0 0.0 1.11 0.42

LH middle Grade 7

Fall 2016 91.7 8.3 0.0 0.0 0.0 1.08 0.29

Spring 2017 100.0 0.0 0.0 0.0 0.0 1.00 0.00

LH high school

Fall 2016 100.0 0.0 0.0 0.0 0.0 1.00 0.00

Spring 2017 91.7 0.0 0.0 0.0 8.3 1.33 1.15 a Spring 2017 mean was significantly lower than the baseline mean, p <.05



Appendix I: Lighthouse School Behavioral Data

Office Referrals and Suspensions

Office Referrals Suspensions

Participant Group

Baseline

%

2016-17

%

Baseline

%

2016-17

%

Cohort 1 Lighthouse Grades 1-3 4.56 7.51 1.03 1.16

Cohort 2 Lighthouse Grades K,4, and 5 3.82 8.60 0.54 1.84 a

Non-Lighthouse Grades 1-3 6.85 7.03 0.73 1.37a

Lighthouse Grade 6 39.76 38.46 6.18 5.84

Cohort 3 Non-Lighthouse Grades K, 4, and 5 8.26 9.09 1.41 1.92 a

Lighthouse Grade 7 42.74 46.11 8.37 7.16

Non-Lighthouse Grade 6 84.41 75.05 8.08 10.51

Lighthouse Grades 9-12 56.32 57.04 10.23 12.17 a Significantly higher than baseline, p < .05

Attendance

Meeting 94% Attendance Rate

Participant Group Baseline

%

2016-17

%

Cohort 1 Lighthouse Grades 1-3 70.51 68.82

Cohort 2

Lighthouse Grades K,4, and 5 65.34 65.74

Non-Lighthouse Grades 1-3 72.01 69.54

Lighthouse Grade 6 71.90 69.11

Cohort 3 Non-Lighthouse Grades K, 4, and 5 74.83 69.22 a

Lighthouse Grade 7 71.20 67.70

Non-Lighthouse Grade 6 72.12 65.59 a

Lighthouse Grades 9-12 59.30 51.17 a a Significantly lower than baseline, p < .05



Appendix J: Descriptive Statistics and Frequencies of Responses to BCPS Climate Survey

Making learning more personalized for students helps teachers to meet the academic needs of

all students.

Strongly

Disagree Disagree Agree

Strongly

Agree M SD

Participant % % % % Parents

Lighthouse elementary 2.48 4.95 44.55 48.02 3.38 0.70 Non-Lighthouse elementary 2.68 6.40 45.72 45.20 3.33 0.72

Lighthouse middle 5.71 11.26 50.90 32.13 3.09 0.81

Non-Lighthouse middle 6.54 10.41 54.03 29.02 3.06 0.81

Lighthouse high 4.44 11.11 55.56 28.89 3.09 0.76

Access to technology increases opportunities for making learning more personalized for

students.

Strongly


Strongly

Agree M SD


Lighthouse elementary 2.64 8.42 44.55 44.39 3.31 0.74

Non-Lighthouse elementary 3.54 8.40 45.04 43.02 3.28 0.76



Lighthouse high 4.44 6.67 49.63 39.26 3.24 0.76

Students

Lighthouse Gr. 4-5 3.32 8.80 42.94 44.94 3.30 0.76 Non-Lighthouse Gr. 4-5 3.46 9.69 43.90 42.96 3.26 0.77

Lighthouse Gr. 6 5.18 12.34 44.71 37.77 3.15 0.83

Non-Lighthouse Gr. 6 5.26 10.98 42.68 41.08 3.20 a 0.83

Lighthouse Gr. 7 11.47 16.66 43.75 28.13 2.89 0.95

Lighthouse high 6.50 16.09 50.42 26.99 2.98 0.83 a non-Lighthouse students were significantly more likely to agree, p < .001

Teachers are able to use technology to meet the academic needs of all students.

Strongly


Strongly

Agree M SD


Lighthouse elementary 2.97 10.54 49.42 37.07 3.21 0.74

Non-Lighthouse elementary 3.54 10.28 50.48 35.70 3.18 0.75



Lighthouse high 4.48 8.96 55.97 30.60 3.13 0.75

Students

Lighthouse Gr. 4-5 2.84 5.27 45.88 46.02 3.35 0.71

Non-Lighthouse Gr. 4-5 3.03 6.53 46.39 44.04 3.31 0.73

Lighthouse Gr. 6 3.47 7.74 50.51 38.28 3.24 0.74 Non-Lighthouse Gr. 6 4.28 8.23 48.47 39.03 3.22 0.77

Lighthouse Gr. 7 7.94 14.01 52.46 25.58 2.96 0.84

Lighthouse high 6.43 14.80 57.52 21.25 2.97 0.80

PR

running head: bcps s.t.a.t. year three evaluation …...key inputs to the initiative are represented...

Documents