when is disproportionality truly disproportionate? clifford v. hatt, ed.d., abpp & jon thompson,...
Post on 16-Dec-2015
219 Views
Preview:
TRANSCRIPT
When is Disproportionality Truly When is Disproportionality Truly Disproportionate?Disproportionate?
Clifford V. Hatt, Ed.D., ABPPClifford V. Hatt, Ed.D., ABPP&&
Jon Thompson, Psy.D.Jon Thompson, Psy.D.Virginia Beach City Public SchoolsVirginia Beach City Public Schools
Virginia Beach, VAVirginia Beach, VA
About the PresentersAbout the Presenters
Clifford V. Hatt, Ed.D., ABPPClifford V. Hatt, Ed.D., ABPP
– Coordinator, Psychological Services Virginia Coordinator, Psychological Services Virginia Beach City Public SchoolsBeach City Public Schools
– Licensed Clinical Psychologist; Diplomate in Licensed Clinical Psychologist; Diplomate in School Psychology, American Board of School Psychology, American Board of Professional PsychologyProfessional Psychology
– Adjunct Professor of Psychology, Norfolk State Adjunct Professor of Psychology, Norfolk State University & The College of William & MaryUniversity & The College of William & Mary
– Member, Virginia Department of Education Member, Virginia Department of Education Task Force on DisproportionalityTask Force on Disproportionality
About the PresentersAbout the Presenters
Jon C. Thompson, Psy.D.Jon C. Thompson, Psy.D.
– Licensed Clinical Psychologist, Virginia Beach Licensed Clinical Psychologist, Virginia Beach City Public SchoolsCity Public Schools
– Private Practice, Churchland Psychiatric Private Practice, Churchland Psychiatric Associates, Portsmouth, VAAssociates, Portsmouth, VA
– Listed in National Register of Health Service Listed in National Register of Health Service Providers in PsychologyProviders in Psychology
– Member, Virginia Task Force on Member, Virginia Task Force on DisproportionalityDisproportionality
Participants will …Participants will …
Be able to discuss the three most Be able to discuss the three most common methods for determining common methods for determining the presence of disproportionality the presence of disproportionality and understand the relative and understand the relative strengths and weaknessesstrengths and weaknesses
Be able to discuss the importance of Be able to discuss the importance of significance testing when data is significance testing when data is analyzed for differences between analyzed for differences between groupsgroups
Participants will …Participants will …
Learn how to apply a formula to Learn how to apply a formula to convert simple proportions to z-convert simple proportions to z-scores for significance testingscores for significance testing
Review how the application of a Review how the application of a conversion to z-scores would change conversion to z-scores would change the interpretation of the interpretation of disproportionality data from Virginiadisproportionality data from Virginia
DisproportionalityDisproportionality
““Disproportionality refers to comparisons Disproportionality refers to comparisons made between groups of students by race made between groups of students by race or ethnicity or language who are identified or ethnicity or language who are identified in special education. Where students from in special education. Where students from particular ethnic or linguistic groups are particular ethnic or linguistic groups are identified either at a greater or lesser rate identified either at a greater or lesser rate than all other students then that group than all other students then that group may be said to be disproportionately may be said to be disproportionately represented in special education.”represented in special education.”
Draft Rubric for Looking at District PracticesDraft Rubric for Looking at District Practices
DisproportionalityDisproportionality
How do you measure it?How do you measure it? How should it be measured?How should it be measured? What constitutes disproportionality?What constitutes disproportionality? Is there a “correct” or “ideal” Is there a “correct” or “ideal”
proportion?proportion?
Disproportion vs. IncidenceDisproportion vs. Incidence
Do disabilities and illnesses occur in Do disabilities and illnesses occur in the general population and in the general population and in subpopulations at the same rate?subpopulations at the same rate?
If the incidence of mental retardation If the incidence of mental retardation occurs in 1% of the general school occurs in 1% of the general school age population (based on USDOE age population (based on USDOE data), can we expect it to occur at data), can we expect it to occur at that same rate in specific school-that same rate in specific school-aged ethnic groups? aged ethnic groups?
2004: % of Students Age 6-21 2004: % of Students Age 6-21 Served Under IDEA, Part BServed Under IDEA, Part B
AllAll LDLD SLISLI MRMR EDED OtherOther
United United StatesStates
9.169.16 4.244.24 1.731.73 0.840.84 0.730.73 1.621.62
VirginiaVirginia 9.629.62 4.124.12 1.451.45 0.800.80 0.780.78 2.472.47
Incidence Rates Reported in Incidence Rates Reported in DSM-IV-TRDSM-IV-TR
Learning Disability:Learning Disability:Approximately 5% of students in Approximately 5% of students in public schools are diagnosed with a public schools are diagnosed with a learning disability (reading, written learning disability (reading, written language, math)language, math)
Mental Retardation:Mental Retardation:Approximately 1% of general Approximately 1% of general population is diagnosed with mental population is diagnosed with mental retardationretardation
Is Special Education Data Is Special Education Data Consistent?Consistent?
MRMR LDLD
United States 2004 United States 2004 IDEA DataIDEA Data
0.84%0.84% 4.24%4.24%
Virginia 2004 IDEA Virginia 2004 IDEA DataData
0.80%0.80% 4.12%4.12%
DSM-IV-TR DSM-IV-TR Reported Incidence Reported Incidence RateRate
1.0%1.0% 5.0%5.0%
Consistency of Consistency of Comparative DataComparative Data
If Special Education Data is If Special Education Data is consistent, which incidence rate consistent, which incidence rate should be used for comparison?should be used for comparison?– United States DataUnited States Data– Virginia DataVirginia Data– DSM-IV-TR DSM-IV-TR
Current Methods: Which One Current Methods: Which One Do You Use?Do You Use?
Risk IndexRisk Index Odds/Relative Risk RatioOdds/Relative Risk Ratio Composition IndexComposition Index
Proportional ConceptProportional ConceptEntire Student Population of LEAville
10,000
White6000(60%)
African/Amer3,000(30%)
Hispanic750
(7.5%)
Asian/PIsland150
(1.5%)
Amer Ind100(1%)
Proportional ConceptProportional Concept
Assumes same ethnic proportions Assumes same ethnic proportions will/should be reflected in all will/should be reflected in all educational situationseducational situations– Special Education ClassificationsSpecial Education Classifications– Referrals for educational assistanceReferrals for educational assistance– Discipline referralsDiscipline referrals– AP classesAP classes– Drop out ratesDrop out rates
Seems simple and self-evidentSeems simple and self-evident
For every complex problem there is For every complex problem there is an answer that is clear, simple, and an answer that is clear, simple, and wrong. wrong.
H. L. Mencken (1880-1956)H. L. Mencken (1880-1956)
LEAville, USALEAville, USA
# Students# Students # Identified # Identified Sp.EdSp.Ed
CaucasianCaucasian 60006000 550550
African AmericanAfrican American 30003000 270270
HispanicHispanic 750750 7070
Asian/Pac. Asian/Pac. IslanderIslander
150150 2020
American IndianAmerican Indian 100100 1010
Risk IndexRisk Index
Obtained by dividing # of students Obtained by dividing # of students from a given ethnic background from a given ethnic background identified with a disability by the identified with a disability by the total # of students that exist in a total # of students that exist in a given populationgiven population– Similar to incidence rateSimilar to incidence rate
CaveatCaveat– Purely descriptive analysisPurely descriptive analysis
Odds RatioOdds Ratio(Relative Risk Index)(Relative Risk Index)
Uses the Risk Index of an ethnic minority Uses the Risk Index of an ethnic minority as the nominator and the Risk Index of as the nominator and the Risk Index of whites as the denominatorwhites as the denominator
Gives simple comparison of groups with Gives simple comparison of groups with whites serving as the control groupwhites serving as the control group
Can determine whether a particular ethnic Can determine whether a particular ethnic group will be identified as having a group will be identified as having a disability more or less often compared to disability more or less often compared to whiteswhites
Odds RatioOdds Ratio
CaveatCaveat– No way to evaluate for significanceNo way to evaluate for significance– Assumes whites are the majority and Assumes whites are the majority and
are representative of a disability group are representative of a disability group as a wholeas a whole
– Assumes all disabilities are equally Assumes all disabilities are equally represented across ethnic groupsrepresented across ethnic groups
LEAville, USALEAville, USA
# Students# Students # Identified # Identified Sp.EdSp.Ed
CaucasianCaucasian 60006000 550550
African AmericanAfrican American 30003000 270270
HispanicHispanic 750750 7070
Asian/Pac. Asian/Pac. IslanderIslander
150150 2020
American IndianAmerican Indian 100100 1010
Is LEAville Disproportionate?Is LEAville Disproportionate?
Risk IndexRisk Index Odds/Relative RiskOdds/Relative Risk
RatioRatio
WhiteWhite 9.179.17 9.17/9.17 = 1.09.17/9.17 = 1.0
BlackBlack 9.009.00 9.00/9.17 = 0.989.00/9.17 = 0.98
HispanicHispanic 9.339.33 9.33/9.17 = 1.029.33/9.17 = 1.02
AsianAsian 13.313.3 13.3/9.17 = 1.4513.3/9.17 = 1.45
AmericanAmerican
IndianIndian10.010.0 10.0/9.17 = 1.0910.0/9.17 = 1.09
Composition IndexComposition Index
Calculated by dividing the # of students of Calculated by dividing the # of students of a racial or ethnic group in a category by a racial or ethnic group in a category by the total # of students in that ethnic or the total # of students in that ethnic or racial group in the total student population.racial group in the total student population.
Provides composition for a pre-determined Provides composition for a pre-determined population so that when all compositions of population so that when all compositions of the different ethnic groups are added the different ethnic groups are added together, total equals 100%together, total equals 100%
Composition of the identified category is Composition of the identified category is then compared to composition of the then compared to composition of the population as a wholepopulation as a whole
Composition IndexComposition Index
CaveatCaveat– Once again, no way to evaluate for Once again, no way to evaluate for
significancesignificance– Highly susceptible to interpretation Highly susceptible to interpretation
flaws when population under flaws when population under consideration is smallconsideration is small
– Assumes all disabilities occur with equal Assumes all disabilities occur with equal frequency across all populationsfrequency across all populations
What About the Composition What About the Composition Index?Index?
Looks at whether the composition of a Looks at whether the composition of a particular ethnic group in a category is particular ethnic group in a category is consistent with the composition of that consistent with the composition of that ethnic group in the total student bodyethnic group in the total student body
Let’s examine African-American for Let’s examine African-American for LEAvilleLEAville– Remember that according to the Risk Ratio, Remember that according to the Risk Ratio,
African-American’s may be UNDER-represented African-American’s may be UNDER-represented in Special Education!in Special Education!
LEAville Composition IndexLEAville Composition IndexCategoryCategory African-African-
American American (270)(270)
Total Student Total Student Body Body
(Assumes (Assumes Incidence Rate Incidence Rate
reported by reported by 2004 IDEA 2004 IDEA
data)data)
% % African-African-
AmericanAmerican
in the in the Spec Ed Spec Ed CategoryCategory
% African-% African-American American
in the in the Total Total
Student Student BodyBody
MRMR 4040 100100 4040 3030EDED 4040 100100 4040 3030LDLD 160160 400400 4040 3030SLISLI 1010 200200 55 3030OTHEROTHER 2020 100100 2020 3030
Arbitrary NumbersArbitrary Numbers
It may be recognized that composition It may be recognized that composition ratios are not very useful when comparing ratios are not very useful when comparing subgroups.subgroups.
Some type of “statistical” adjustment is Some type of “statistical” adjustment is sometimes suggested without any clear sometimes suggested without any clear rationale.rationale.
Some states may use “+/- 20 percentage Some states may use “+/- 20 percentage points” to suggest significant points” to suggest significant disproportionality disproportionality (Coutinho & Oswald, 2004).(Coutinho & Oswald, 2004).
Proposal for Statewide AnalysisProposal for Statewide Analysis
Use commonly applied analysis Use commonly applied analysis techniques, but provide a more techniques, but provide a more sophisticated method to analyze for sophisticated method to analyze for significancesignificance
Look at Virginia data as a whole, Look at Virginia data as a whole, using procedure similar to a Chi-using procedure similar to a Chi-Square analysis where we compare Square analysis where we compare the expected composition of data to the expected composition of data to the obtained composition of datathe obtained composition of data
Is Special Education Data Is Special Education Data Consistent?Consistent?
MRMR LDLD
United States 2004 United States 2004 IDEA DataIDEA Data
0.84%0.84% 4.24%4.24%
Virginia 2004 IDEA Virginia 2004 IDEA DataData
0.80%0.80% 4.12%4.12%
Preliminary Analysis for Virginia DataPreliminary Analysis for Virginia DataCategoryCategory 22 dfdf pp
American American IndianIndian
25.0325.03 131131 <1.0<1.0
AsianAsian 83.4983.49 131131 <.99<.99
African-African-AmericanAmerican
127.4127.4 131131 <.57<.57
HawaiianHawaiian 3.613.61 131131 <1.0<1.0
HispanicHispanic 102.0102.0 131131 <.97<.97
UnspecifiedUnspecified 1.521.52 131131 <1.0<1.0
WhiteWhite 56.0656.06 131131 <1.0<1.0
Proposed Data Analysis of Individual Proposed Data Analysis of Individual Districts (Something Old)Districts (Something Old)
Use the commonly applied Risk Index to Use the commonly applied Risk Index to determine what percentage of an ethnic determine what percentage of an ethnic group is identified with a particular disabilitygroup is identified with a particular disability– # of students of Hispanic ethnicity that are # of students of Hispanic ethnicity that are
identified as having a disability divided by the # identified as having a disability divided by the # of students of Hispanic ethnicity in the populationof students of Hispanic ethnicity in the population
Compare the Risk Index to the percentage of Compare the Risk Index to the percentage of a given ethnic group that makes up the a given ethnic group that makes up the population as a whole population as a whole – # of students of Hispanic ethnicity divided by the # of students of Hispanic ethnicity divided by the
total number of students in a populationtotal number of students in a population
Proposed Data Analysis of Individual Proposed Data Analysis of Individual Districts (Something New)Districts (Something New)
Use a formula that converts Use a formula that converts proportions to proportions to zz-scores-scores
Identify those districts that have a z Identify those districts that have a z score greater than 2 (i.e., 2 standard score greater than 2 (i.e., 2 standard deviations above the mean)deviations above the mean)– z-z-scores are based upon standard scores are based upon standard
deviation units, with a deviation units, with a zz-score of 0 = -score of 0 = average, +/- 1 = one standard deviation average, +/- 1 = one standard deviation above/below the mean, etc.above/below the mean, etc.
Standard normal distributionStandard normal distribution
-2 -1 0 1 2
Within 2Within 2** SDs ( SDs (** really 1.96) really 1.96)
-2 -1 0 1 2
95%
Formula KeyFormula Key
PopulationPopulation Population Population proportionproportion
SampleSample
sizesize# ID # ID Sp EdSp Ed
SampleSample
proportionproportion
11 pp11 nn11 XX11
22 pp22 nn22 XX22
1
11ˆn
xp
2
22ˆn
xp
Formula for Formula for zz
Dp
Dp
SE
ppz
nnppSE
nn
xxp
ppD
2ˆ1ˆ
)11
)(ˆ1(ˆ
ˆ
ˆˆ
21
21
2
21
1
Formula in Special Ed TermsFormula in Special Ed Terms
z = (% min spec ed - % min z = (% min spec ed - % min students)/ sqrt[(x * 1-x)*(1/n1 + students)/ sqrt[(x * 1-x)*(1/n1 + 1/n2)]1/n2)]
x = (# min students + # spec ed min x = (# min students + # spec ed min students)/(n1 + n2)students)/(n1 + n2)
n1 = total # minority studentsn1 = total # minority students n2 = total # spec ed studentsn2 = total # spec ed students
ResultsResultsCurrent Current MethodMethod
New MethodNew Method
# Districts # Districts Demonstrating Demonstrating
DisproportionalityDisproportionality
# Districts # Districts Demonstrating Demonstrating
DisproportionalityDisproportionality
Amer. Amer. IndianIndian
2323 2525
AsianAsian 44 99BlackBlack 9696 4747HawaiianHawaiian 77 99HispanicHispanic 3737 22WhiteWhite 5858 1717
ImplicationsImplications
Different choices in the use of the Different choices in the use of the comparison group make a difference!comparison group make a difference!
The method you chose to compare data The method you chose to compare data makes a difference!makes a difference!
Disproportionality is a problem regardless Disproportionality is a problem regardless of method.of method.
More specific methodology allows for more More specific methodology allows for more appropriate, targeted remedies.appropriate, targeted remedies.
Can be easily calculated in an Excel Can be easily calculated in an Excel spreadsheet format.spreadsheet format.
ImplicationsImplications
Modified z-score method provides a way to Modified z-score method provides a way to determine whether the differences determine whether the differences between predicted and obtained between predicted and obtained proportions are meaningful.proportions are meaningful.
More appropriately identifies districts that More appropriately identifies districts that need to have corrective action.need to have corrective action.
Also provides a way to determine progress Also provides a way to determine progress or change following interventions.or change following interventions.
Accounts for differences in size or number Accounts for differences in size or number of students under consideration.of students under consideration.
More on the MethodMore on the Method
This type of significance testing is This type of significance testing is used in public health settings as well.used in public health settings as well.
The University of North Carolina The University of North Carolina Chapel Hill School of Public Health Chapel Hill School of Public Health has a learning module about has a learning module about inferences in the difference in two inferences in the difference in two population proportions atpopulation proportions at– Preparedness Center Training Site: ComPreparedness Center Training Site: Com
paring Two Proportionsparing Two Proportions
A Little Comic WisdomA Little Comic Wisdom
Politics is the art of looking for trouble, Politics is the art of looking for trouble, finding it everywhere, diagnosing it finding it everywhere, diagnosing it incorrectly and applying the wrong incorrectly and applying the wrong remedies.remedies.
Groucho Marx (1890-1977)Groucho Marx (1890-1977)
Contact UsContact Us
Cliff Hatt, Ed.D. ABPPCliff Hatt, Ed.D. ABPP– Clifford.Hatt@vbschools.comClifford.Hatt@vbschools.com
Jon Thompson, Psy.D.Jon Thompson, Psy.D.– Jon.Thompson@vbschools.comJon.Thompson@vbschools.com
Virginia Beach City Public SchoolsVirginia Beach City Public SchoolsPsychological ServicesPsychological Services1413 Laskin Road1413 Laskin RoadVirginia Beach, VA 23451Virginia Beach, VA 23451Phone: 757-263-2700Phone: 757-263-2700Fax: 757-263-2702Fax: 757-263-2702
top related