When is Disproportionality Truly When is Disproportionality Truly Disproportionate?Disproportionate?

Clifford V. Hatt, Ed.D., ABPPClifford V. Hatt, Ed.D., ABPP&&

Jon Thompson, Psy.D.Jon Thompson, Psy.D.Virginia Beach City Public SchoolsVirginia Beach City Public Schools

Virginia Beach, VAVirginia Beach, VA

About the PresentersAbout the Presenters

Clifford V. Hatt, Ed.D., ABPPClifford V. Hatt, Ed.D., ABPP

– Coordinator, Psychological Services Virginia Coordinator, Psychological Services Virginia Beach City Public SchoolsBeach City Public Schools

– Licensed Clinical Psychologist; Diplomate in Licensed Clinical Psychologist; Diplomate in School Psychology, American Board of School Psychology, American Board of Professional PsychologyProfessional Psychology

– Adjunct Professor of Psychology, Norfolk State Adjunct Professor of Psychology, Norfolk State University & The College of William & MaryUniversity & The College of William & Mary

– Member, Virginia Department of Education Member, Virginia Department of Education Task Force on DisproportionalityTask Force on Disproportionality

About the PresentersAbout the Presenters

Jon C. Thompson, Psy.D.Jon C. Thompson, Psy.D.

– Licensed Clinical Psychologist, Virginia Beach Licensed Clinical Psychologist, Virginia Beach City Public SchoolsCity Public Schools

– Private Practice, Churchland Psychiatric Private Practice, Churchland Psychiatric Associates, Portsmouth, VAAssociates, Portsmouth, VA

– Listed in National Register of Health Service Listed in National Register of Health Service Providers in PsychologyProviders in Psychology

– Member, Virginia Task Force on Member, Virginia Task Force on DisproportionalityDisproportionality

Participants will …Participants will …

Be able to discuss the three most Be able to discuss the three most common methods for determining common methods for determining the presence of disproportionality the presence of disproportionality and understand the relative and understand the relative strengths and weaknessesstrengths and weaknesses

Be able to discuss the importance of Be able to discuss the importance of significance testing when data is significance testing when data is analyzed for differences between analyzed for differences between groupsgroups

Participants will …Participants will …

Learn how to apply a formula to Learn how to apply a formula to convert simple proportions to z-convert simple proportions to z-scores for significance testingscores for significance testing

Review how the application of a Review how the application of a conversion to z-scores would change conversion to z-scores would change the interpretation of the interpretation of disproportionality data from Virginiadisproportionality data from Virginia

DisproportionalityDisproportionality

““Disproportionality refers to comparisons Disproportionality refers to comparisons made between groups of students by race made between groups of students by race or ethnicity or language who are identified or ethnicity or language who are identified in special education. Where students from in special education. Where students from particular ethnic or linguistic groups are particular ethnic or linguistic groups are identified either at a greater or lesser rate identified either at a greater or lesser rate than all other students then that group than all other students then that group may be said to be disproportionately may be said to be disproportionately represented in special education.”represented in special education.”

Draft Rubric for Looking at District PracticesDraft Rubric for Looking at District Practices

DisproportionalityDisproportionality

How do you measure it?How do you measure it? How should it be measured?How should it be measured? What constitutes disproportionality?What constitutes disproportionality? Is there a “correct” or “ideal” Is there a “correct” or “ideal”

proportion?proportion?

Disproportion vs. IncidenceDisproportion vs. Incidence

Do disabilities and illnesses occur in Do disabilities and illnesses occur in the general population and in the general population and in subpopulations at the same rate?subpopulations at the same rate?

If the incidence of mental retardation If the incidence of mental retardation occurs in 1% of the general school occurs in 1% of the general school age population (based on USDOE age population (based on USDOE data), can we expect it to occur at data), can we expect it to occur at that same rate in specific school-that same rate in specific school-aged ethnic groups? aged ethnic groups?

2004: % of Students Age 6-21 2004: % of Students Age 6-21 Served Under IDEA, Part BServed Under IDEA, Part B

AllAll LDLD SLISLI MRMR EDED OtherOther

United United StatesStates

9.169.16 4.244.24 1.731.73 0.840.84 0.730.73 1.621.62

VirginiaVirginia 9.629.62 4.124.12 1.451.45 0.800.80 0.780.78 2.472.47

Incidence Rates Reported in Incidence Rates Reported in DSM-IV-TRDSM-IV-TR

Learning Disability:Learning Disability:Approximately 5% of students in Approximately 5% of students in public schools are diagnosed with a public schools are diagnosed with a learning disability (reading, written learning disability (reading, written language, math)language, math)

Mental Retardation:Mental Retardation:Approximately 1% of general Approximately 1% of general population is diagnosed with mental population is diagnosed with mental retardationretardation

Is Special Education Data Is Special Education Data Consistent?Consistent?

MRMR LDLD

United States 2004 United States 2004 IDEA DataIDEA Data

0.84%0.84% 4.24%4.24%

Virginia 2004 IDEA Virginia 2004 IDEA DataData

0.80%0.80% 4.12%4.12%

DSM-IV-TR DSM-IV-TR Reported Incidence Reported Incidence RateRate

1.0%1.0% 5.0%5.0%

Consistency of Consistency of Comparative DataComparative Data

If Special Education Data is If Special Education Data is consistent, which incidence rate consistent, which incidence rate should be used for comparison?should be used for comparison?– United States DataUnited States Data– Virginia DataVirginia Data– DSM-IV-TR DSM-IV-TR

Current Methods: Which One Current Methods: Which One Do You Use?Do You Use?

Risk IndexRisk Index Odds/Relative Risk RatioOdds/Relative Risk Ratio Composition IndexComposition Index

Proportional ConceptProportional ConceptEntire Student Population of LEAville

10,000

White6000(60%)

African/Amer3,000(30%)

Hispanic750

(7.5%)

Asian/PIsland150

(1.5%)

Amer Ind100(1%)

Proportional ConceptProportional Concept

Assumes same ethnic proportions Assumes same ethnic proportions will/should be reflected in all will/should be reflected in all educational situationseducational situations– Special Education ClassificationsSpecial Education Classifications– Referrals for educational assistanceReferrals for educational assistance– Discipline referralsDiscipline referrals– AP classesAP classes– Drop out ratesDrop out rates

Seems simple and self-evidentSeems simple and self-evident

For every complex problem there is For every complex problem there is an answer that is clear, simple, and an answer that is clear, simple, and wrong. wrong.

H. L. Mencken (1880-1956)H. L. Mencken (1880-1956)

LEAville, USALEAville, USA

# Students# Students # Identified # Identified Sp.EdSp.Ed

CaucasianCaucasian 60006000 550550

African AmericanAfrican American 30003000 270270

HispanicHispanic 750750 7070

Asian/Pac. Asian/Pac. IslanderIslander

150150 2020

American IndianAmerican Indian 100100 1010

Risk IndexRisk Index

Obtained by dividing # of students Obtained by dividing # of students from a given ethnic background from a given ethnic background identified with a disability by the identified with a disability by the total # of students that exist in a total # of students that exist in a given populationgiven population– Similar to incidence rateSimilar to incidence rate

CaveatCaveat– Purely descriptive analysisPurely descriptive analysis

Odds RatioOdds Ratio(Relative Risk Index)(Relative Risk Index)

Uses the Risk Index of an ethnic minority Uses the Risk Index of an ethnic minority as the nominator and the Risk Index of as the nominator and the Risk Index of whites as the denominatorwhites as the denominator

Gives simple comparison of groups with Gives simple comparison of groups with whites serving as the control groupwhites serving as the control group

Can determine whether a particular ethnic Can determine whether a particular ethnic group will be identified as having a group will be identified as having a disability more or less often compared to disability more or less often compared to whiteswhites

Odds RatioOdds Ratio

CaveatCaveat– No way to evaluate for significanceNo way to evaluate for significance– Assumes whites are the majority and Assumes whites are the majority and

are representative of a disability group are representative of a disability group as a wholeas a whole

– Assumes all disabilities are equally Assumes all disabilities are equally represented across ethnic groupsrepresented across ethnic groups

LEAville, USALEAville, USA

# Students# Students # Identified # Identified Sp.EdSp.Ed

CaucasianCaucasian 60006000 550550

African AmericanAfrican American 30003000 270270

HispanicHispanic 750750 7070

Asian/Pac. Asian/Pac. IslanderIslander

150150 2020

American IndianAmerican Indian 100100 1010

Is LEAville Disproportionate?Is LEAville Disproportionate?

Risk IndexRisk Index Odds/Relative RiskOdds/Relative Risk

RatioRatio

WhiteWhite 9.179.17 9.17/9.17 = 1.09.17/9.17 = 1.0

BlackBlack 9.009.00 9.00/9.17 = 0.989.00/9.17 = 0.98

HispanicHispanic 9.339.33 9.33/9.17 = 1.029.33/9.17 = 1.02

AsianAsian 13.313.3 13.3/9.17 = 1.4513.3/9.17 = 1.45

AmericanAmerican

IndianIndian10.010.0 10.0/9.17 = 1.0910.0/9.17 = 1.09

Composition IndexComposition Index

Calculated by dividing the # of students of Calculated by dividing the # of students of a racial or ethnic group in a category by a racial or ethnic group in a category by the total # of students in that ethnic or the total # of students in that ethnic or racial group in the total student population.racial group in the total student population.

Provides composition for a pre-determined Provides composition for a pre-determined population so that when all compositions of population so that when all compositions of the different ethnic groups are added the different ethnic groups are added together, total equals 100%together, total equals 100%

Composition of the identified category is Composition of the identified category is then compared to composition of the then compared to composition of the population as a wholepopulation as a whole

Composition IndexComposition Index

CaveatCaveat– Once again, no way to evaluate for Once again, no way to evaluate for

significancesignificance– Highly susceptible to interpretation Highly susceptible to interpretation

flaws when population under flaws when population under consideration is smallconsideration is small

– Assumes all disabilities occur with equal Assumes all disabilities occur with equal frequency across all populationsfrequency across all populations

What About the Composition What About the Composition Index?Index?

Looks at whether the composition of a Looks at whether the composition of a particular ethnic group in a category is particular ethnic group in a category is consistent with the composition of that consistent with the composition of that ethnic group in the total student bodyethnic group in the total student body

Let’s examine African-American for Let’s examine African-American for LEAvilleLEAville– Remember that according to the Risk Ratio, Remember that according to the Risk Ratio,

African-American’s may be UNDER-represented African-American’s may be UNDER-represented in Special Education!in Special Education!

LEAville Composition IndexLEAville Composition IndexCategoryCategory African-African-

American American (270)(270)

Total Student Total Student Body Body

(Assumes (Assumes Incidence Rate Incidence Rate

reported by reported by 2004 IDEA 2004 IDEA

data)data)

% % African-African-

AmericanAmerican

in the in the Spec Ed Spec Ed CategoryCategory

% African-% African-American American

in the in the Total Total

Student Student BodyBody

MRMR 4040 100100 4040 3030EDED 4040 100100 4040 3030LDLD 160160 400400 4040 3030SLISLI 1010 200200 55 3030OTHEROTHER 2020 100100 2020 3030

Arbitrary NumbersArbitrary Numbers

It may be recognized that composition It may be recognized that composition ratios are not very useful when comparing ratios are not very useful when comparing subgroups.subgroups.

Some type of “statistical” adjustment is Some type of “statistical” adjustment is sometimes suggested without any clear sometimes suggested without any clear rationale.rationale.

Some states may use “+/- 20 percentage Some states may use “+/- 20 percentage points” to suggest significant points” to suggest significant disproportionality disproportionality (Coutinho & Oswald, 2004).(Coutinho & Oswald, 2004).

Proposal for Statewide AnalysisProposal for Statewide Analysis

Use commonly applied analysis Use commonly applied analysis techniques, but provide a more techniques, but provide a more sophisticated method to analyze for sophisticated method to analyze for significancesignificance

Look at Virginia data as a whole, Look at Virginia data as a whole, using procedure similar to a Chi-using procedure similar to a Chi-Square analysis where we compare Square analysis where we compare the expected composition of data to the expected composition of data to the obtained composition of datathe obtained composition of data

Is Special Education Data Is Special Education Data Consistent?Consistent?

MRMR LDLD

United States 2004 United States 2004 IDEA DataIDEA Data

0.84%0.84% 4.24%4.24%

Virginia 2004 IDEA Virginia 2004 IDEA DataData

0.80%0.80% 4.12%4.12%

Preliminary Analysis for Virginia DataPreliminary Analysis for Virginia DataCategoryCategory 22 dfdf pp

American American IndianIndian

25.0325.03 131131 <1.0<1.0

AsianAsian 83.4983.49 131131 <.99<.99

African-African-AmericanAmerican

127.4127.4 131131 <.57<.57

HawaiianHawaiian 3.613.61 131131 <1.0<1.0

HispanicHispanic 102.0102.0 131131 <.97<.97

UnspecifiedUnspecified 1.521.52 131131 <1.0<1.0

WhiteWhite 56.0656.06 131131 <1.0<1.0

Proposed Data Analysis of Individual Proposed Data Analysis of Individual Districts (Something Old)Districts (Something Old)

Use the commonly applied Risk Index to Use the commonly applied Risk Index to determine what percentage of an ethnic determine what percentage of an ethnic group is identified with a particular disabilitygroup is identified with a particular disability– # of students of Hispanic ethnicity that are # of students of Hispanic ethnicity that are

identified as having a disability divided by the # identified as having a disability divided by the # of students of Hispanic ethnicity in the populationof students of Hispanic ethnicity in the population

Compare the Risk Index to the percentage of Compare the Risk Index to the percentage of a given ethnic group that makes up the a given ethnic group that makes up the population as a whole population as a whole – # of students of Hispanic ethnicity divided by the # of students of Hispanic ethnicity divided by the

total number of students in a populationtotal number of students in a population

Proposed Data Analysis of Individual Proposed Data Analysis of Individual Districts (Something New)Districts (Something New)

Use a formula that converts Use a formula that converts proportions to proportions to zz-scores-scores

Identify those districts that have a z Identify those districts that have a z score greater than 2 (i.e., 2 standard score greater than 2 (i.e., 2 standard deviations above the mean)deviations above the mean)– z-z-scores are based upon standard scores are based upon standard

deviation units, with a deviation units, with a zz-score of 0 = -score of 0 = average, +/- 1 = one standard deviation average, +/- 1 = one standard deviation above/below the mean, etc.above/below the mean, etc.

Standard normal distributionStandard normal distribution

-2 -1 0 1 2

Within 2Within 2** SDs ( SDs (** really 1.96) really 1.96)

-2 -1 0 1 2

95%

Formula KeyFormula Key

PopulationPopulation Population Population proportionproportion

SampleSample

sizesize# ID # ID Sp EdSp Ed

SampleSample

proportionproportion

11 pp11 nn11 XX11

22 pp22 nn22 XX22

1

11ˆn

xp

2

22ˆn

xp

Formula for Formula for zz

Dp

Dp

SE

ppz

nnppSE

nn

xxp

ppD

2ˆ1ˆ

)11

)(ˆ1(ˆ

ˆ

ˆˆ

21

21

2

21

1

Formula in Special Ed TermsFormula in Special Ed Terms

z = (% min spec ed - % min z = (% min spec ed - % min students)/ sqrt[(x * 1-x)*(1/n1 + students)/ sqrt[(x * 1-x)*(1/n1 + 1/n2)]1/n2)]

x = (# min students + # spec ed min x = (# min students + # spec ed min students)/(n1 + n2)students)/(n1 + n2)

n1 = total # minority studentsn1 = total # minority students n2 = total # spec ed studentsn2 = total # spec ed students

ResultsResultsCurrent Current MethodMethod

New MethodNew Method

# Districts # Districts Demonstrating Demonstrating

DisproportionalityDisproportionality

# Districts # Districts Demonstrating Demonstrating

DisproportionalityDisproportionality

Amer. Amer. IndianIndian

2323 2525

AsianAsian 44 99BlackBlack 9696 4747HawaiianHawaiian 77 99HispanicHispanic 3737 22WhiteWhite 5858 1717

ImplicationsImplications

Different choices in the use of the Different choices in the use of the comparison group make a difference!comparison group make a difference!

The method you chose to compare data The method you chose to compare data makes a difference!makes a difference!

Disproportionality is a problem regardless Disproportionality is a problem regardless of method.of method.

More specific methodology allows for more More specific methodology allows for more appropriate, targeted remedies.appropriate, targeted remedies.

Can be easily calculated in an Excel Can be easily calculated in an Excel spreadsheet format.spreadsheet format.

ImplicationsImplications

Modified z-score method provides a way to Modified z-score method provides a way to determine whether the differences determine whether the differences between predicted and obtained between predicted and obtained proportions are meaningful.proportions are meaningful.

More appropriately identifies districts that More appropriately identifies districts that need to have corrective action.need to have corrective action.

Also provides a way to determine progress Also provides a way to determine progress or change following interventions.or change following interventions.

Accounts for differences in size or number Accounts for differences in size or number of students under consideration.of students under consideration.

More on the MethodMore on the Method

This type of significance testing is This type of significance testing is used in public health settings as well.used in public health settings as well.

The University of North Carolina The University of North Carolina Chapel Hill School of Public Health Chapel Hill School of Public Health has a learning module about has a learning module about inferences in the difference in two inferences in the difference in two population proportions atpopulation proportions at– Preparedness Center Training Site: ComPreparedness Center Training Site: Com

paring Two Proportionsparing Two Proportions

A Little Comic WisdomA Little Comic Wisdom

Politics is the art of looking for trouble, Politics is the art of looking for trouble, finding it everywhere, diagnosing it finding it everywhere, diagnosing it incorrectly and applying the wrong incorrectly and applying the wrong remedies.remedies.

Groucho Marx (1890-1977)Groucho Marx (1890-1977)

Contact UsContact Us

Cliff Hatt, Ed.D. ABPPCliff Hatt, Ed.D. ABPP– Clifford.Hatt@vbschools.comClifford.Hatt@vbschools.com

Jon Thompson, Psy.D.Jon Thompson, Psy.D.– Jon.Thompson@vbschools.comJon.Thompson@vbschools.com

Virginia Beach City Public SchoolsVirginia Beach City Public SchoolsPsychological ServicesPsychological Services1413 Laskin Road1413 Laskin RoadVirginia Beach, VA 23451Virginia Beach, VA 23451Phone: 757-263-2700Phone: 757-263-2700Fax: 757-263-2702Fax: 757-263-2702