the implementation of house bill 22 - region one esc€¦ · the implementation of house bill 22...

The Implementation of House Bill 22

COLLABOR AT I NG TO BU I LD A BETTER ACCOU NTABI L I TY SY STEM

T H E S C H O O L P R O G R E S S D O M AI N

Stakeholder Feedback ForumDomain II

Presentation to School System Stakeholder Teams

October 25, 2017

Region One ESC - Division of Instructional, School Improvement, and College Readiness Support

HB 22 Stakeholder Forum: Session Overview

• New Accountability System: House Bill 22

• Scheduled Stakeholder Feedback Forums

• Three Domain Assessment and A-F Accountability System

• Domain II: School Progress

• Part A and Part B Components/Indicators

• Computational Methodology

• Component Weighting

• Stakeholder Feedback

• School System Leadership Team Discussions

• Stakeholder Feedback Opportunity

2

House Bill 22, 85th Texas Legislature

“The commissioner shall evaluate school district and campus

performance and assign each district and campus an overall

performance rating of”

A–F Accountability: Legislative Context

A B C D or F

HB

2804

HB

22

3

House Bill 22, 85th Texas Legislature

“The commissioner shall solicit input statewide from persons . . . , including school district boards of trustees, administrators and teachers employed by school districts, parents of students enrolled in school districts, and other interested stakeholders.”

A–F Accountability: Gathering Stakeholder Input

4

Feedback Opportunities

• Will solicit input on the

aspects over which

commissioner has

authority

• Won’t solicit input on

aspects that are

required by statute


• Will solicit input on the

aspects over which

commissioner has

authority

• Won’t solicit input on

aspects that are

required by statuteTrusteesParentsAdministrators

Teachers

Closing

The Gaps

School

Progress

Student

Achievement

Best of Achievement or Progress Minimum 30%

Three Domains: Combining to Calculate Overall Score

55


• Certain methodology

decisions in each

domain

• Cut points for each

grade in each domain

• Weight (30% or more)

to Closing the Gaps

Domain


• Certain methodology

decisions in each

domain

• Cut points for each

grade in each domain

• Weight (30% or more)

to Closing the Gaps

Domain

Approaches or Above

Meets or Above

Masters

Student Achievement: Performance

66

Student

Achievement

Closing

The Gaps

School

Progress

School Progress: Growth

7

School Progress

Closing

The Gaps

Student

Achievement

Closing the Gaps: Ensuring Educational Equity

8

Closing

The Gaps

Student

Achievement

School

Progress

Local Accountability Plan

Closing

The Gaps

School

Progress

Student

Achievement

*Example

SaExtra-

Curricular

Activities

*Example

Local

Assessments

Local Accountability

9

School Progress: Growth

10

School Progress

Closing

The Gaps

Student

Achievement

School Progress: Two Aspects to Progress

Part A: Student Growth Part B: Relative Performance

11



12

STAAR: Test Inclusion Methodology

1

313

• Includes all tests (STAAR with and without accommodations and STAAR Alternate 2) combined

• Combines reading and mathematics

• Uses STAAR Progress Measure

• Includes ELs (except in their first year in US schools)

• Uses same STAAR Progress Measure for ELs and non-Els

• Because the first STAAR tests are given in third grade, we can’t assess growth using the STAAR Progress Measure until fourth grade.

• In high school, there are limitations to measuring growth with STAAR. It can only possibly be done for 9th graders who take Algebra I, and then only for 9th and 10th graders taking English I or English II. At this point, only Relative

Performance will be analyzed in high school.

Student Growth: Measuring Advancement

STA

AR

Pe

rfo

rma

nc

e L

eve

l

3rd Grade Example 4th Grade Example

Does Not MeetDoes Not Meet

Approaches

Approaches

Meets

Meets

MastersMasters

Exceeds

Expected

+ 1 Point AwardedFor meeting or exceeding expected growth

+ .5 Points AwardedFor maintaining proficiency but

failing to meet expected growth

+ 0 Points AwardedFor falling to a lower level

Maintains

Limited

1

414

Student Growth: Percentage of Students Gaining

Does Not Meet Grade Level

Approaches Grade Level

Meets Grade Level

Masters Grade Level


Met/Exceeded

Growth Measure = 1 pt

Did not meet = 0 pts

Met/Exceeded


Did not meet = .5 pts

1 pt 1 pt


Met/Exceeded



Met/Exceeded



1 pt 1 pt

Meets Grade Level

0 pts 0 pts 1 pt 1 pt

Masters Grade Level

0 pts 0 pts 0 pts 1 pt

15

Current Year

Pre

vio

us

Ye

ar

Student Growth: Not Maintaining Proficiency Level and

Not Meeting STAAR Progress Measure



Meets Grade Level

Masters Grade Level


Met/Exceeded



Met/Exceeded



1 pt 1 pt


Met/Exceeded



Met/Exceeded



1 pt 1 pt

Meets Grade Level


Masters Grade Level


Current Year

Pre

vio

us

Ye

ar

1

616

No Points• Does Not Meet to

Does Not Meet

(without meeting growth expectations)

• Approaches to Does Not Meet

(without meeting

growth expectations)

• Meets to

Does Not Meet

• Meets to

Approaches

• Masters to

Does Not Meet

• Masters to

Approaches

• Masters to Meets

Student Growth: Meeting Base Proficiency Level

and Not Meeting STAAR Progress Measure


ApproachesGrade Level

Meets Grade Level

Masters Grade Level


Met/Exceeded



Met/Exceeded



1 pt 1 pt


Met/Exceeded



Met/Exceeded



1 pt 1 pt

Meets Grade Level


Masters Grade Level


Pre

vio

us

Ye

ar

1

717

Half Point• Does Not Meet to

Approaches

(without meeting growth expectations)

• Approaches to Approaches

(without meeting


Current Year

Student Growth: Meeting Base Proficiency Level and

Meeting or Exceeding STAAR Progress Measure



Meets Grade Level

Masters Grade Level


Met/Exceeded



Met/Exceeded



1 pt 1 pt


Met/Exceeded



Met/Exceeded



1 pt 1 pt

Meets Grade Level


Masters Grade Level


Pre

vio

us

Ye

ar

1

818

One Point• Does Not Meet to

Approaches

(meeting/exceeding growth expectations)

• Approaches to Approaches

(meeting/exceeding


Current Year

Student Growth: Not Meeting Base Proficiency Level and

Meeting or Exceeding STAAR Progress Measure*



Meets Grade Level

Masters Grade Level


Met/Exceeded



Met/Exceeded



1 pt 1 pt


Met/Exceeded



Met/Exceeded



1 pt 1 pt

Meets Grade Level


Masters Grade Level


Pre

vio

us

Ye

ar

1

919

One Point• Does Not Meet to Does

Not Meet

(meeting/exceeding growth expectations)

• Approaches to Does Not Meet

(meeting/exceeding


Current Year

*Very rare but statistically possible.

Student Growth: Achieving High Proficiency Level or

Maintaining High Proficiency Level



Meets Grade Level

Masters Grade Level


Met/Exceeded



Met/Exceeded



1 pt 1 pt


Met/Exceeded



Met/Exceeded



1 pt 1 pt

Meets Grade Level


Masters Grade Level


Pre

vio

us

Ye

ar

2

020

One Point• Does Not Meet to

Meets

• Does Not Meet to

Masters

• Approaches to Meets

• Approaches to Masters

• Meets to Masters

• Meets to Meets

• Masters to Masters

Current Year

21

Student Growth: Sample Calculation

One Hundred Students

• Each with reading and mathematics results for last year and this year

• Denominator = 200 STAAR Progress Measures

?

200

22


No Points

• Does Not Meet to Does Not Meet(without meeting growth expectations)

• Approaches to Does Not Meet(without meeting growth expectations)

• Masters to Meets

Previous Year Current Year Count of Tests

20

15

14

49

+

+

23


Half Point

• Does Not Meet to Approaches(without meeting growth expectations)

• Approaches to Approaches(without meeting growth expectations)

7

10

17

+


24


One Point

• Does Not Meet to Does Not Meet(meeting/exceeding growth expectations)

• Approaches to Does Not Meet(meeting/exceeding growth expectations)*

• Approaches to Approaches(meeting/exceeding growth expectations)

*Very rare but statistically possible

23

7

+

22

+

52


25


One Point

• Meets to Meets

• Meets to Masters

• Masters to Masters

33

32

82

+

17

+


26


(49× 0)+(17 × .5) +(52 × 1)+(82× 1)

200=

142.5

20071=

In this case, we loosely conclude that 71% of students have gained a

year academically. Technically, however, this is the percentage of tests

taken, with some adjustment for maintaining proficiency.

49 results that

earned no points

17 results that

earned half a point134 results that

earned one point

School Progress Domain: Feedback Opportunities

27

• New approach to growth

• Additional ways to measure growth in high school

• Percentage of students who need to grow to constitute

▪ Excellent performance

▪ Minimally acceptable performance

Part A Scores: Frequency by Campus Type

Elementary

(4,219)

Middle School

(1,653)

K–12

(334)

District

(1,203)

Quantile Part A Score (based on modeling data from 2017 accountability)

100% (Max) 100 96 100 100

99% 88 85 87 86

95% 84 81 83 79

90% 82 78 80 77

75% (Q3) 78 75 76 73

50% (Med) 73 70 70 70

25% (Q1) 68 65 64 66

10% 63 61 59 62

5% 59 59 56 59

1% 52 54 45 49

0% (Min) 34 41 0 24

Common Questions: School Progress Domain, Part A

28

Q: Is there no additional credit for meeting or

exceeding growth at the Meets and

Masters levels?

A: Students at Meets or Masters are given the

same one point as students who show

growth at Does Not Meet and

Approaches.

Q: Slide 14 shows an example of a student

who falls from Approaches Grade Level

one year to Does Not Meet the next year

and still meets STAAR Progress Measure

expectations. Can this really happen?

A: It’s very rare, but, statistically, it’s possible

when a student skips a grade. Our

modelling with 2017 data produced ten

such instances in the entire state.

Q: Why are high schools only scored on relative

performance? Is there no growth measure for

high school?

A: The relatively few STAAR Progress Measures for

high school make them an unreliable measure

of a high school’s progress with students. But

the STAAR Progress Measure scores will be

available on TAPR.



29

30

Texas Higher Education Strategic Plan: 2015-2030

31

Economic Status is Major Factor for Completion

Stu

de

nt A

ch

ieve

me

nt

Do

ma

in S

co

re fo

r A

ll St

ud

en

ts

% Economically Disadvantaged Students

Higher Levels

of Student

Achievement

Higher Rates of

Economically

Disadvantaged Students

Relative Performance: Measuring School Progress

3

232

Includes STAAR, CCMR, and graduation rates for districts and campuses that have that data

Stu

de

nt A

ch

ieve

me

nt

Do

ma

in S

co

re fo

r A

ll St

ud

en

ts


Higher Levels

of Student

Achievement

Higher Rates of

Economically



3

333


Stu

de

nt A

ch

ieve

me

nt

Do

ma

in S

co

re fo

r A

ll St

ud

en

ts


Higher Levels

of Student

Achievement

Higher Rates of

Economically


A campus with fewer economically disadvantaged students on average has higher levels of student achievement.

A campus with more economically

disadvantaged students tends to have lower levels of student achievement.


3

434


Stu

de

nt A

ch

ieve

me

nt

Do

ma

in S

co

re fo

r A

ll St

ud

en

ts


Higher Levels

of Student

Achievement

Higher Rates of

Economically


A campus with fewer economically disadvantaged students on average has higher levels of student achievement.

A campus with more economically

disadvantaged students tends to have lower levels of student achievement.


3

535


Stu

de

nt A

ch

ieve

me

nt

Do

ma

in S

co

re fo

r A

ll St

ud

en

ts


Higher Levels

of Student

Achievement

Higher Rates of

Economically

Disadvantaged


A

BCD

F

3

636

Common Questions: School Progress Domain

38

Q: Does the Student Achievement domain

score (y-axis in relative performance)

include CCMR and graduation rates?

A: Yes, for schools that have that data.

Q: House Bill 22 specifically says that the method

used to evaluate performance should provide

for the mathematical possibility that all districts

and campuses receive an A, but this looks like

a forced distribution that guarantees a set

percentage of schools will get Ds and Fs.

A: Once the cut points are set using 2016–17

accountability data, the cut points will stay

fixed for five years. That way any district or

campus will be able to earn an A.


3

939

• Scatter plot of each district and campus (by campus type) comparing

▪ Student Achievement domain score

▪ Percentage of students who are economically disadvantaged

• Trendline showing average relationships

• Sliding cut points for campuses and districts based on

▪ Student Achievement domain

score

▪ Percentage of students who are economically disadvantaged

• Cut points for each grade based on bands below and above the average line

• Separate cut points

▪ Elementary Schools

▪ Middle Schools

▪ High Schools/K–12

▪ AEAs

• Cut points based on slope-intercept form

▪ Based on 2016–17 performance

▪ Intended to stay fixed for five years

• Cut points will be known before ratings release

Relative Performance: Sample Calculation

4

040

• 𝑦 = 𝑚𝑥 + 𝑏

▪ 𝑦 is the predicted Student Achievement domain score.

▪ 𝑥 is the percentage of students who are economically disadvantaged.

▪ 𝑚 is the slope of the trendline.

▪ 𝑏 is the distance from the trendline (what decides the grade); it is based on average variance from trendline.

• Sample Middle School

▪ 94.4% economically disadvantaged (𝑥)

▪ 𝑦 = –.15666(𝑥) + 45.789

▪ 𝑦 = –.15666(94.4) + 45.789

▪ 𝑦 = –14.79 + 45.789

▪ Predicted Student Achievement domain score (𝑦 ) = 31

▪ Actual Student Achievement domain score: 25

▪ Score in relative performance: D

School Progress Domain: Feedback Opportunities

41

• New approach to growth

• Additional ways to measure growth in high school

• Percentage of students who need to grow to constitute


▪ Minimally acceptable performance

• Combining two parts

▪ Best of

▪ Weighted average

▪ Average

• For Part B, what is the right cut points for


▪ Unacceptable performance

HB 22 Stakeholder Forum: Feedback Survey

Forum Participant Information

• Q1: ESC Region [Multiple Choice: 1–20]

• Q2: Participant Role: [Board Member, Parent, District Administrator, Campus Administrator, Teacher, Community Stakeholder, ESC Administrator]

42


Part A Stakeholder Feedback

• Q3: On a scale of 1 to 5 (five being the highest), how would you rate the

new approach to growth (Part A of the School Progress domain)?

[1,2,3,4,5]

• Q4: What changes, if any, do you think are needed to Part A?

• Q5: What reliable ways to measure growth in high school could be used

in Part A?

• Q6: What percentage of students need to demonstrate growth for a

district or campus to show excellent performance in Part A?

• Q7: What percentage of students need to demonstrate growth for a

district or campus to show minimally acceptable performance in Part A?

43


Part A and B Stakeholder Feedback

• Q8: How should Parts A and B be combined to determine the overall

domain score? [Best of A or B, Weight each part equally, weighted on

part more than the other.]

• Q9: What should the weights of each part be?

44


Session Recommendations

• Q10: What could be done to make the training more effective?

• Q11: What aspects of the training were most helpful?

45

HB 22 Stakeholder Forum: Feedback Survey Link

46

https://www.surveymonkey.com/r/5RBLDFM

https://www.surveymonkey.com/r/5RBLDFM

4

747

TEA Questions and Feedback

Feedback

• Survey link to come by email

• [email protected]

Resources

• http://tea.texas.gov/A-F

• http://tea.texas.gov/accountability


• (512) 463-9704

mailto:[email protected]

http://tea.texas.gov/A-F

http://tea.texas.gov/accountability

mailto:[email protected]

4

848

Region One ESC Questions and Feedback

Contact

• Dr. Belinda Gorena, Administrator for School Improvement, Accountability, and Compliance

• (956) 984-6173


• Kelly VanHee, Administrator for Curriculum, Instruction, and Assessment

• (956) 984-6151


the implementation of house bill 22 - region one esc€¦ · the implementation of house bill 22...

Documents