racial concentration and school effectiveness in sfusd
Post on 14-Jan-2016
55 Views
Preview:
DESCRIPTION
TRANSCRIPT
Racial Concentration and School Effectiveness in SFUSD
SFUSD Board Meeting, 12/14/09
Stephen NewtonProfessor Linda Darling-HammondSchool Redesign Network at Stanford University
1
Background SFUSD’s Strategic Plan sets a goal to “disrupt the
predictive power of demographics.” This problem has also been called the “achievement gap” between historically underserved populations of students and others.
SFUSD schools with high concentrations of African-American, Latino, and Samoan (AA/L/S) students generally have lower achievement levels than other schools. Findings presented to Ad Hoc Committee on 2/28/09.
Key question: do these lower outcomes reflect a relationship between racial concentration and school effectiveness?
2
Lens for this study
These analyses focus on whether school composition, specifically racial concentration of AA/L/S students, plays a role in increasing the achievement gap for historically underserved populations.
3
On average, schools with greater proportions of AA/L/S students generally have lower API scores, but there were also exceptions to this trend
School AA, Latino & Samoan Enrollment vs API Growth 2008Correlation Coefficient = -0.69993
400
600
800
1000
0 20 40 60 80 100 120
School AA, Latino & Samoan Enrollment %
Sch
oo
l AP
I Gro
wth
200
8
Source: Presented to Board, February 2009:4
-0.74 **API Score
AA/L/Samoan
Enrollment
-0.25 *Staff Satisfaction
0.33 **Suspension Rate
0.48 **Teacher Turnover
-0.43 **Attendance (seat time)
-0.54 **Teacher Average Years of Services
Correlations between racial concentration and other school factors
* Correlation is significant at the 0.05 level (2-tailed).
** Correlation is significant at the 0.01 level (2-tailed).
Concentration of AA/L/S students is strongly correlated with a range of measures related to school quality
Source: Ad Hoc Committee Presentation, March 2009:
5
District ELA achievement gap has grown (Gap in percent proficient has grown by 5% for Latinos and 7% for African-Americans)
0
10
20
30
40
50
60
% A
t o
r A
bo
ve
Pro
fic
ien
t
AA 12 13 14 15 19 21 22 23
L 15 16 19 20 24 27 28 28
SFUSD 33 35 39 40 45 48 49 51
2001 2002 2003 2004 2005 2006 2007 2008
CST English Language Arts:8 Year Trends for Proficient and Above (Grade 2 to 11)
18%
21%
23%
28%
Legend
African-American Achievement
Latino Achievement
SFUSD AchievementSource: Ad Hoc Committee Presentation, March 2009:
6
District math achievement gap has also grown (Gap in percent proficient has grown by 3% for Latinos and 6% for African-Americans)
0
10
20
30
40
50
60
70
80
% A
t o
r A
bo
ve
Pro
fic
ien
t
AA 10 14 14 21 24 25 26
L 16 22 23 31 34 35 35
SFUSD 37 44 46 53 57 58 59
2002 2003 2004 2005 2006 2007 2008
CST Mathematics:7 Year Trends for Proficient and Above (Grade 2 to 7)
21%
27%
24%
33%
Legend
African-American Achievement
Latino Achievement
SFUSD AchievementSource: Ad Hoc Committee Presentation, March 2009:
7
Review of findings
Three conclusions from these background slides:1. Racial concentration of AA/L/S students is related to lower
average school performance.2. The achievement gap for historically underserved students is
widening.3. Racial concentration is also related to other school quality
factors. This analysis delves deeper into the role of racial
concentration by using inferential statistics to control for other factors that can influence student outcomes.
8
Our charge – The impact of school composition on school effectiveness
Key Question: On average, are schools with concentrations of African-American, Latino, and Samoan (AA/L/S)* students as effective as other schools in SFUSD? Effectiveness/value-added/productivity compares whether, on
average, students gain more or less than similar students in other SFUSD schools. In other words, it focuses on fair peer-to-peer comparisons.
Positive effectiveness means students gain at a faster rate than average, and negative effectiveness means students gain at a slower than average rate.
If you want to know a given school’s effectiveness, it is better to look at its matrix gap, a value-added estimate, than to consider its racial concentration.
9
Analyses conducted to answer the key question1. Did students in AA/L/S concentrated schools have different academic
outcomes compared with similar students in other SFUSD schools?• School-level outcomes - Matrix gaps.• Student-level outcomes – Productivity, Propensity score matching.
2. When a school’s proportion of AA/L/S students changed, did its achievement also change?
3. Was concentration of AA/L/S students related to non-academic outcomes?
• Graduation rates, Mobility.
4. Was concentration of AA/L/S students related to teacher experience and stability?
• Years experience.• Percent first- and second-year teachers.• Teacher retention.
10
Question 1: Did students in AA/L/S concentrated schools have different academic outcomes compared with similar students in other SFUSD schools?
Methods Compare outcomes while controlling for prior year
achievement in ELA and math and student demographic characteristics.
Value-Added/Productivity - Use statistical models to estimate future achievement and then compare actual with estimated achievement (matrix, productivity analysis).
Propensity Score Matching - Find similar students and compare outcomes.
These methods provide an estimate of school effectiveness.
11
Question 1 (school-level): School value-added in ELA is lower, on average, in concentrated AA/L/S schools (2007-2008)
-1-.5
0.5
0 20 40 60 80 100AALASa08_pct
95% CI Fitted valuesEnglish
t= -5.65, p<.001
School ELA Matrix Gap (sds)
Percent AA/L/S Students in School
12
Question 1 (school-level): School value-added in math is also lower, on average, in concentrated AA/L/S schools (2007-2008)
-.4
-.2
0.2
.4
0 20 40 60 80 100AALASa08_pct
95% CI Fitted valuesMath
t= -8.11, p<.001
Percent AA/L/S Students in School
School Math Matrix Gap (sds)
13
Question 1 (student-level): Racial concentration is not related to lower elementary school productivity for all students, on average, except for highly concentrated schools (2003-04 to 2008-09)
Student Value-Added Gains (Grades 3-5, All Students)
-0.1
-0.05
0
0.05
0.1
0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%
School Percent AA/L/SA
Ave
rag
e V
alu
e-A
dd
ed (
sds)
ELA
Math
Question 1 (student-level): Elementary schools do show lower productivity for AA/L/S students, on average, with increasing racial concentration (2003-04 to 2008-09)
Student Value-Added Gains (Grades 3-5, AA/L/S Only)
-0.1
-0.05
0
0.05
0.1
0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%
School Percent AA/L/SA
Ave
rag
e V
alu
e-A
dd
ed (
sds)
ELA
Math
Question 1 (student-level): Middle schools show a mixed productivity picture for all students, with lower gains at schools with 80% -100% AA/L/S students (2003-04 to 2008-09)
Student Value-Added Gains (Grades 6-8, All Students)
-0.1
-0.05
0
0.05
0.1
0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%
School Percent AA/L/SA
Ave
rag
e V
alu
e-A
dd
ed (
sds)
ELA
Math
Question 1 (student-level): Middle schools also show a mixed productivity picture for AA/L/S students (2003-04 to 2008-09)
Student Value-Added Gains (Grades 6-8, AA/L/S Only)
-0.1
-0.05
0
0.05
0.1
0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%
School Percent AA/L/SA
Ave
rag
e V
alu
e-A
dd
ed (
sds)
ELA
Math
Question 1 (student-level): High schools show lower productivity for all students, on average, with increasing racial concentration (2003-04 to 2008-09)
Student Value-Added Gains (Grades 9-11, All Students)
-0.1
-0.05
0
0.05
0.1
0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%
School Percent AA/L/SA
Ave
rag
e V
alu
e-A
dd
ed (
sds)
ELA
Question 1 (student-level): High schools also show lower productivity for AA/L/S students, on average, with increasing racial concentration (2003-04 to 2008-09)
Student Value-Added Gains (Grades 9-11, AA/L/S)
-0.1
-0.05
0
0.05
0.1
0% to <20% 20% to <40% 40% to <60% 60% to <80% 80% to 100%
School Percent AA/L/SA
Ave
rag
e V
alu
e-A
dd
ed (
sds)
ELA
Question1 (student-level matched*): Students in concentrated (60%+) AA/L/S schools do less well than matched students at other SFUSD schools Students show smaller gains in ELA and math in AA/L/S
concentrated schools than similar students in other SFUSD schools (2003-04 to 2008-09). This comparison combines grades 3-11 in ELA, and all math courses above
grade 2. Students were matched using a statistical technique called propensity score
matching. All Students
ELA: -.04 sds (t=-8.56, p <.001) Math: -.02 sds (t=-3.93, p <.001)
AAL Students ELA: -.04 sds (t=-6.81, p <.001) Math: -.01 sds (t=-1.98, p <.05)
*Students matched on prior achievement in ELA and math, gender, race/ethnicity, parent education, EL status, retained in grade, special education, and poverty
20
Differences in effectiveness add up over time (hypothetical)
A student starting at 50th percentile in 2nd grade in a school with -.04 sd effectiveness would decline to 36th percentile, on average, by 11th grade.
Similarly, with -.02 effectiveness, the student would decline to 43rd percentile by 11th grade.
0
10
20
30
40
50
60
70
80
90
100
2nd 3rd 4th 5th 6th 7th 8th 9th 10th llth
Grade
Per
cen
tile
ELA
Math
Example based on effectiveness difference found for concentrated AALS schools
21
Question 2: Changing demographics and achievement When a school’s demographics changed, what
happened to its average achievement? We focused on the change between 1999 and 2008 in SFUSD schools.
This has important implications for accountability because the state accountability system requires schools to demonstrate adequate yearly progress (AYP) regardless of their demographic composition.
22
-1-.
50
.51
-40 -20 0 20 40AALSchange_pct
95% CI Fitted valuesZELA_change
Schools that increased in percent AALS students tended to decrease in average ELA achievement (1999 to 2008)
Change in ELA Achievement for All Students (z-scores)
Change in %AA/L/S Students in School
t=-5.87, p<.001
23
Schools that increased in percent AALS students also tended to decrease in average math achievement (1999 to 2008)
-1-.
50
.51
-40 -20 0 20 40AALSchange_pct
95% CI Fitted valuesZMath_change
Change in Math Achievement for All Students (z-scores)
Change in %AA/L/S Students in School
t=-6.96, p<.001
24
Question 3 (non-academic outcomes): Graduation and student mobility
Concentrated AA/L/S schools had an 11% lower graduation rate than other SFUSD schools in 2007-08 (controlling for poverty).
Concentrated AA/L/S schools had a 3.8% higher student mobility rate (students entering or leaving a school) than other SFUSD schools in 2007-08 (controlling for poverty).
25
How can we explain differences in school effectiveness? Research has suggested segregated schools are often less
effective: In Florida, segregation mattered in predicting school-level performance on
Florida’s state tests (Borman, 2004). In Texas, high racial concentrations of African American students in schools
reduced achievement for African American students, and racial composition of a school explains a meaningful portion of the racial achievement gap (Hanushek, et al., 2007).
The reasons for lower effectiveness are not fully-understood but appear to be complex.
These were average effects, and not all schools fit these trends.
We decided to explore differences in teacher experience and stability in AA/L/S concentrated schools.
26
Question 4 (teachers): Schools with more AA/L/S students, on average, had less experienced teachers and more teacher turnover
Teacher experience and stability (2004-05 to 2007-08) AALS concentrated schools had teachers with significantly less
experience than other SFUSD schools (average 10.3 years exp. vs. 13.4 years exp.) (t= -11.26, p<.001).
AALS concentrated schools had significantly more first and second year teachers (3.7%) than other SFUSD schools (2.1%) (t= 4.23, p<.001).
AALS concentrated schools had a significantly lower rate of teacher retention than other SFUSD schools (73.4% vs. 83.9%, t=-7.61, p<.001).
27
020
4060
8010
0
0 20 40 60 80 100AALS_pct
95% CI Fitted valuesteacher_return_pct
Schools with higher concentrations of AA/L/S students had lower average rates of teacher retention (2003-04 to 2007-08)
t=-8.7, p<.001
Percent AA/L/S Students in School
Percent of Teachers Retained in School
28
Schools with concentrated poverty – Checking an alternative possibility
Schools with concentrated poverty also had lower effectiveness than other schools. Results were similar to racial concentration when using a
propensity score matching model.
Racial concentration and poverty contribute separate effects When put into the same model, both factors independently
contribute to lower effectiveness.
29
Conclusions (1 of 3) – Racial concentration reduces school effectiveness for historically underserved students
SFUSD schools with concentrations of AA/L/S students have been less effective, on average, in raising student achievement overall. This difference does not depend on differences in the individual
students, because students who are demographically similar still show smaller gains, on average, at concentrated AA/L/S schools.
The clearest patterns were seen in high school for all students and AA/L/S students specifically, and in elementary schools for AA/L/S students.
This effect is larger in ELA than math. Concentrated schools also had lower effectiveness for AA/L/S students.
Lower effectiveness increases the achievement gap. Because effectiveness measures rate of academic progress, the racial
achievement gap is increased if AA/L/S students attend less effective schools.
30
Conclusions (2 of 3) – Racial concentration reduces school effectiveness for historically underserved students
On average, teachers at racially concentrated schools are less experienced and have higher mobility.
Racial concentration is one factor influencing a school’s effectiveness, but there are others. Some schools with high concentrations of AA/L/S students had
strong effectiveness, whereas other schools with low concentrations of AA/L/S students had weak effectiveness.
It is important for the district to study why schools are effective or ineffective. We have found some initial clues that could be further pursued:
Disproportionate allocations of inexperienced teachers, plus high teacher turnover in concentrated AA/L/S schools – both found in other research to reduce student achievement – may be contributors to lower productivity in these schools.
31
Conclusions (3 of 3) – Racial concentration reduces school effectiveness for historically underserved students
SRN case studies of SFUSD schools which effectively serve low-income students of color have identified other factors that may play a role, including:
quality of school leadership; coherent curriculum providing rich literacy and learning
experiences; and extensive professional development focused on
equitable instruction. It may be especially useful to further study effective
schools with high concentrations of AA/L/S students, as well as those that are currently struggling.
Thank you.
32
Citations
Borman, et al. (2004). Accountability in a postdesegregation era: The continuing significance of racial segregation in Florida’s schools,” American Educational Research Journal, v41, n3, p. 605.
Hanushek, E.A., Kain, J.F., Rivkin, S. G. (June 2007) “New Evidence about Brown v. Board of Education: The Complex Effects of School Racial Composition on Achievement.”
33
top related