potential utility of actuarial methods for identifying specific learning disabilities

Psychology in the Schools, Vol. 47(6), 2010 C© 2010 Wiley Periodicals, Inc.Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/pits.20489

POTENTIAL UTILITY OF ACTUARIAL METHODS FOR IDENTIFYING SPECIFICLEARNING DISABILITIES

NICHOLAS BENSON

The University of South Dakota

ISADORE NEWMAN

Florida International University

This article describes howactuarialmethods can supplant discrepancymodels and augment problemsolving and Response to Intervention (RTI) efforts by guiding the process of identifying speciÞclearning disabilities (SLD). Actuarial methods use routinized selection and execution of formulasderived from empirically established relationships to make predictions that fall within a plausiblerange of possible future outcomes. In the case of SLD identiÞcation, the extent to which predictionsare reasonable can be evaluated by their ability to categorize large segments of the population intosubgroups that vary considerably along a spectrum of risk for academic failure. Although empiricalcomparisons of actuarial methods to clinical judgment reveal that actuarial methods consistentlyoutperform clinical judgment, multidisciplinary teams charged with identifying SLD currently relyon clinical judgment. Actuarial methods provide educators with an empirically veriÞable indicatorof student need for special education and related services that could be used to estimate the relativeeffects of exclusionary criteria. This indicator would provide a defensible endpoint in the processof identifying SLD as well as a means of informing and improving the SLD identiÞcation process.C© 2010 Wiley Periodicals, Inc.

Human judgment is prone to systematic errors (i.e., biases) of reasoning (e.g., Tversky &Kahneman, 1974). The process of identifying students with speciÞc learning disabilities (SLD) isfraught with complexities and thus remarkably prone to systematic errors of reasoning. To complicatematters, although SLD identiÞcation is considered among the most difÞcult diagnostic tasks, it oftenis performed by practitioners with limited diagnostic expertise (Reynolds, 2003).

SLD refer to substantial deÞcits in speciÞc (as opposed to general) aspects of learning and aca-demic achievement (U.S. Department of Education, 2004). SLD encompass several domain-speciÞcproblems such as reading, arithmetic, and written expression (Stanovich, 1999). The deÞnitionof SLD excludes academic deÞcits determined to result primarily from visual, auditory, motor,or emotional impairment; mental retardation; or environmental disadvantage (U.S. Department ofEducation, 2004). States, local educational agencies, and individual practitioners have shown consid-erable variance with interpretation and implementation of federal criteria and measurement modelsfor identiÞcation of SLD (Reynolds, 2003). Such variance creates confusion and results in decisionsthat are unreliable and potentially arbitrary.

Practitioners typically have not evaluated many variables with known relations to SLD duringthe identiÞcation process (Mather & Kaufman, 2006). Given the importance of learning contextfor student achievement, the SLD identiÞcation process should consider variables that elucidate themultiple contexts in which children learn and develop (Reschly, Coolong-ChafÞn, Christenson, &Gutkin, 2007). Decision-making processes that integrate information gathered from multiple hier-archical ecological contexts (Bronfenbrenner, 1977) with biological and psychological informationare likely to withstand legal and scientiÞc scrutiny (Weissman & DeBow, 2003). Moreover, inte-gration of multiple types and sources of information for the purpose of contextualizing academicachievement is consistent with the Individuals with Disabilities Education Improvement Act (IDEIA;

Correspondence to: Nicholas Benson, Division of Counseling and Psychology in Education, The Universityof South Dakota, 414 East Clark Street, Delzell Education Center Room 205D, Vermillion, SD 57069. E-mail:[email protected].

538

Actuarial Methods 539

U.S. Department of Education, 2004), federal policy (e.g., federal rules of evidence and other caselaw), professional standards (e.g., codes of ethics), as well as the International ClassiÞcation ofFunctioning, Disability and Health (ICF; World Health Organization, 2001).

The ICF views human functioning as a complex interaction between biological, environmental,social, and personal factors. The ICF provides both a classiÞcation system and philosophical frame-work that are used to both describe human functioning and guide service delivery. Comprehensiveassessments integrating biological, environmental, social, and personal factors are assumed integralto gaining an informed understanding of how disabilities and health status affect human functioning,and informed understanding is assumed to promote independent functioning as well as inclusionin society (World Health Organization, 2001). Adoption of an ICF framework for assessment andidentiÞcation of SLD, in synergy with the use of actuarial methods to identify all variables relevantto academic functioning, may improve diagnostic and intervention outcomes.

This article describes how actuarial methods can supplant discrepancy models and augmentproblem solving and Response to Intervention (RTI) by guiding the process of SLD identiÞcation.In the Þrst section of this article, the rise and fall of discrepancy models are discussed. In the sectionthat follows, RTI and problem solving are described and concerns regarding the adequacy of RTImodels for SLD identiÞcation are discussed. Next, actuarial methods are described and empiricalcomparisons of these methods to human judgment are discussed. Finally, the potential beneÞts ofactuarial methods are presented.

DISCREPANCY MODELS

The acceptability of criteria for SLD identiÞcation continues to be a contentious issue (Vaughn& Fuchs, 2003). Historically, national and state departments of education seemingly have vieweddiscrepancy models as the sine qua non of SLD identiÞcation. Discrepancy models utilize discrep-ancy between intelligence and achievement as the diagnostic cornerstone of SLD (Dombrowski,Kamphaus, & Reynolds, 2004).

The U.S. Department of Education established rules and regulations formalizing discrepancybetween intelligence and academic achievement as a diagnostic cornerstone in 1977 (Mercer, Jordan,Allsopp, &Mercer, 1996). The basis for formalizing this cornerstone was based, in part, on empiricalcomparisons of the observed rate of children with academic difÞculties to the predicted rate basedon statistical data (e.g., Rutter & Yule, 1975). Although empirical comparisons suggest that anappreciable number of studentswith learning difÞculties evidence a discrepancy between intelligenceand achievement, empirical and conceptual literature has not supported the validity of discrepancymodels (e.g., Aaron, 1997; Dombrowski et al., 2004; Fletcher, Francis, Morris, & Lyon, 2005;Vellutino, Scanlon, & Lyon, 2000). Accordingly, discrepancy models for SLD diagnosis are unlikelyto be used in the future (Stanovich, 2005).

RTI AS AN APPROACH FOR SLD DIAGNOSIS

RTI has been proposed as an alternative approach for SLD diagnosis (Gresham, 2002). RTIrefers to the systematic integration of high-quality instruction with formative assessment. Integrationof instruction and assessment is accomplished using a problem-solving model, which yields data-based instructional decisions. Although there are multiple perspectives on problem solving (e.g.,Bergan & Kratochwill, 1990; Bransford & Stein, 1984; Salvia and Hughes, 1990), these modelsshare three characteristics: (a) a focus on formulating the problem affecting a student�s achievement,(b) identifying and selecting interventions that address the problem, and (c) monitoring the extentto which intervention achieves desired outcomes (Deno, 2005).

RTI methods are delivered using a multitier model in which students move from general to in-dividualized assistance based on educational need and responsiveness to evidence-based instruction

Psychology in the Schools DOI: 10.1002/pits

540 Benson and Newman

or other interventions (Brown-Chidsey & Steege, 2005). Educators are encouraged to begin withhigh-quality instruction, use formative assessment to identify any gap between educators� expecta-tions for students (educational or behavioral) and students� actual behavior (academic or behavioral),employ supplemental instruction and interventions to address identiÞed gaps between expectationsand behavior, and use ongoing progress monitoring to determine the extent to which the gap hasbeen narrowed.

Some children inevitability fail to respond to intervention despite receiving intensive, protractedintervention that directly addresses the target behavior. Although positive outcomes for children withSLD are likely to be increased by using a problem-solving model to link assessment information anddata-driven decisions to problem-solving goals (Reschly & Ysseldyke, 2002), additional researchis needed to determine the adequacy of RTI models for SLD diagnosis (Burns, Jacob, & Wagner,2008; Fuchs, Moch, Morgan, & Young, 2006; Reynolds & Shaywitz, 2009). An important stepin determining the adequacy of RTI models for SLD diagnosis involves establishing a deÞnitionfor unresponsiveness to intervention (Vaughn & Fuchs, 2003). The deÞnition of unresponsivenessis inherently linked to the intervention process (Compton, 2006). That is, we cannot deÞne unre-sponsiveness to intervention without knowledge of (a) who received a speciÞc intervention; (b) theactivities that comprised the speciÞc intervention; (c) who implemented these activities; (d) the on-set, frequency, and duration of these activities; and (e) the integrity (i.e., consistency and accuracy)with which these activities were implemented.

There presently are at least three challenges that prevent practitioners from obtaining knowledgeneeded to deÞne unresponsiveness. First, empirical support for procedures used to guide selectionof appropriate intervention strategies for children with academic skill problems (i.e., functionalanalysis, the keystone behavior strategy, and the diagnostic strategy) is limited, and practitionerstend to rely on clinical judgment (Shapiro, 2004). Second, integrity of implementation exists on acontinuum ranging from failure to implement a treatment to perfect implementation, and the integritywith which students� interventions are implemented will vary on this continuum as a function ofteachers and schools. Third, it is difÞcult to specify an appropriate reference group that can be usedwhen deÞning unresponsiveness to intervention.

The concept of RTI assumes that if a child does not respond to instruction found to be effectivefor most children, contextual variables such as poor instruction or poor implementation integritydo not explain academic failure and the problem resides in the child (Vaughn & Fuchs, 2003). Itis difÞcult, however, to specify an appropriate reference group because the integrity with whichinterventions are implemented is unlikely to be equivalent across teachers, schools, districts, andstates. If reference groups were limited to classroom peers, interpreting unresponsiveness wouldbecome increasingly difÞcult as integrity deviates from perfection, as this inference would depend asmuch on the personal characteristics of peers in the reference group as on the personal characteristicsof the child compared to this reference group. If the reference group includes a large sample ofstudents randomly selected from multiple classrooms, interpretation of unresponsiveness wouldresult in overidentiÞcation of students attending classrooms in which interventions are deliveredwith below average integrity.

To summarize, limited empirical evidence exists to guide decisions about the selection of ap-propriate interventions, the extent to which unresponsiveness is a result of inadequate integrity ofimplementation of interventions, and the appropriateness of reference groups. Thus, equitable al-location of special education services seemingly requires evidence beyond unresponsiveness tointervention data. In the following sections, we describe how actuarial methods can augmentRTI and problem solving by providing educators with an empirically veriÞable indicator, de-rived from multiple types and sources of data, of student need for special education and relatedservices.



ACTUARIAL METHODS

Actuarial methods produce conclusions using routinized selection and execution of formulasderived from empirically established relationships between data and an event or condition of interest(Dawes, Faust, &Meehl, 1989). Actuarial models can produce reasonable predictions that fall withina plausible range of possible future outcomes (American Academy of Actuaries, 2006). In the case ofSLD identiÞcation, the extent to which predictions are reasonable can be evaluated by their ability tocategorize large segments of the population into subgroups that vary considerably along a spectrumof risk for academic failure. By focusing on the aggregate probability of academic failure associatedwith more or less homogeneous population subgroups, actuarial models estimate the probability ofoutcomes for groups of individuals with a certain conÞguration of risk markers rather than for anyone individual (American Academy of Actuaries, 2006).

Adopting the ICF framework, with its emphasis on the importance of considering all variablesrelevant to human functioning and inclusion in society, implies that all variables known to alter theprobability of SLD should be included when calculating statistical probability estimates. The inclu-sion of as many contributing variables as possible can be expected to reduce error of prediction byaccounting for variables that alter relations between other predictor variables and outcome variables.Even the most powerful predictors of SLD identiÞed within a large population representative ofU.S. students will not predict well in cases where another variable or variables substantially affectthe direction and/or strength of relations with outcome variables. To maintain consistency with theprinciple of parsimony, modern actuarial practices can prevent the inclusion of unnecessary vari-ables by identifying variables that �behave like independent random phenomena and do not containadditional modeling information� (Hickman, 1997, p. 1).

Actuarial models can be validated over time by comparing model predictions to observedoutcomes (Hickman, 1997). In other words, model predictions of achievement can be compared tolongitudinal observations of students� actual achievement. Additionally, states, local school districts,and individual practitioners can obtain cross-validation estimates of how well predictor equationswork across cases gathered in various reference groups (e.g., students in a particular state or schooldistrict or students acquiring English as a second language). Cross-validation estimates can beused to guide decisions pertaining to the modiÞcation of statistical parameters (Grove & Meehl,1996). The goal of cross-validation is to make sure that the actuarial model works well whenapplied in different, albeit similar, groups than those on which it was built. A statistical techniqueknown as bootstrapping can be used for cross-validation of actuarial models. Bootstrapping involvesresampling results previously obtained from an actuarial model and estimating the standard deviationor conÞdence interval of the model�s accuracy (Efron, 1982; Gong, 1986).

Assumptions

Actuarial models require assumptions about variables, their interrelationships with each other,and their statistical relationship with outcomes. Actuarial models are highly sensitive to the assump-tions used (American Academy of Actuaries, 2006). If a variable that in reality affects an outcome isomitted from an actuarial model, the predictions that result from that model will not be an accuratereßection of reality.

Actuarial predictions are dependent on the comprehensiveness and empirical validity of the dataon which the actuarial model is based (American Academy of Actuaries, 2006). Decision supportsystems (i.e., interactive systems for compiling data intended to help professionals make decisions)can serve as a comprehensive source of data when making actuarial predictions. Decision supportsystems provide decision makers with relevant, reliable, and valid data compiled from a wide rangeof sources (National Forum on Education Statistics, 2006).



The relevance of various types of data to the SLD identiÞcation process has been contested(e.g., Fletcher & Reschly, 2005; Kavale, Kaufman, Naglieri, & Hale, 2005). From the perspectiveof actuarial modeling, any variable that may affect predictions or decisions is potentially relevant.Examples of potentially relevant types of data include developmental and instructional history data(Fletcher & Reschly, 2005), academic engaged time and classroom contingencies (Shapiro, 2004),teacher effectiveness (Sanders & Horn, 1998), responsiveness to intervention data (Gresham, 2002),general intellectual ability (Jensen, 1984; Swanson, 2008), and speciÞc cognitive abilities (Flanagan,Ortiz, Alfonso, &Mascolo, 2002). Relevant types of data are not limited to those that are quantitative,as qualitative data may be coded and subsequently used in actuarial equations (McNeil, Newman,& Kelly, 1996; Newman & Ramlo, in press). Some types of data are relevant only in speciÞccircumstances. For example, it would be appropriate to use acculturation and language proÞciencydata when diagnosing a culturally and linguistically diverse child (Rhodes, Ochoa, & Ortiz, 2005).All information considered valuable to the decision-making process should be gathered (McNeilet al., 1996), even if this information does not have incremental validity or only contributes toprediction in a subgroup of students rather than the entire population in which it will be applied.Exclusionary information is particularly important to consider. For example, a test of intellectualfunctioning should be administered to rule out the alternative explanation of mental retardation(Kavale & Flanagan, 2007).

Actuarial Methods

A discriminate analysis application of multiple regression (Huberty & Olejnik, 2006) canbe used to identify substantial deÞcits in speciÞc aspects of learning and academic achievement(e.g., reading, arithmetic, and written expression) that (a) result from a set of logical predictorvariables and (b) do not result primarily from visual, auditory, motor, or emotional impairment;mental retardation; or environmental disadvantage. The proposed discriminate analysis applicationof multiple regression involves setting up equations with Þxed and random predictor variables(designated as X�s). Values of appropriate academic criteria then are determined for each value of aset of predictors. The following is an illustrative regression equation (Equation 1) specifying a setof predictors for an academic criterion (designated as Y):

Y = a0U+ a1X1 + a2X2 + a3X3 + a4X4 + a5X5 + a6X6 + a7X7 + E1 (1)

where:

a0 . . . a8 = partial regression weights;

U = unit vector (coded �1� for each student);

Y = score derived from a standardized, norm-referenced test of reading achievement;

X1 = number of nuclear family members reported to have a history of reading problems;

X2 = number of extended family members reported to have a history of reading problems;

X3 = discrepancy between level of oral reading ßuency and the average level of oral reading ßuency foran appropriate reference group during Tier 1 intervention;

X4 = discrepancy between oral reading ßuency growth rate and the average growth rate for an appropriatereference group during small-group standard protocol (i.e., Tier 2) intervention;

X5 = discrepancy between oral reading ßuency level and growth rate and the average level and growthrate for an appropriate reference group during intensive, individualized (i.e., Tier 3) intervention;

X6 = estimate of crystallized intelligence;

X7 = estimate of auditory processing; and

E1 = error vector.



The set of predictors in Equation 1 were identiÞed from select literature pertaining to SLDidentiÞcation (i.e., Flanagan et al., 2002; Fletcher & Reschly, 2005; Fuchs & Fuchs, 1997). Thesepredictors were selected for illustrative purposes, and likely are not an optimal set of predictors.Assuming that Equation 1 was found to be an accurate, parsimonious actuarial formula for predictingperformance with a standardized, norm-referenced test of reading achievement for the populationof students within a speciÞc geographical region, it may still be appropriate to include additionalpredictors. For example, teacher effectiveness has been found to affect academic progress, and thiseffect appears to be additive and cumulative (Sanders & Horn, 1998). Thus, fair and equitabledecision making may be dependent on consideration of teacher effectiveness. Fair and equitableidentiÞcation of SLD also may depend on consideration of the integrity with which interventions areimplemented, although at present this is difÞcult to accomplish because the deÞnition and assessmentof implementation integrity are not well established (Jimerson, Burns, & VanDerHeyden, 2007). Ifteacher effectiveness and intervention integrity can be measured reliably, then their effects onstudents� learning could be estimated using value-added models (Raudenbush, 2004). These modelscompare the value added to student learning by variables of interest (e.g., teachers, interventions,schools).

Interpretation

R2 (i.e., the variance in the criterion accounted for by a set of predictor variables) tends tohave an upward bias when regression equations are applied in samples rather than to a populationas a whole (Nunnally, 1967). Upward bias can be addressed using shrinkage estimates to adjustR2. A Monte Carlo evaluation of shrinkage estimates found that formulas by Darlington (1978) andLord (1950) provide the most conservative estimates (Newman, Seymour, Garver, &McNeil, 1979).Upward bias notwithstanding, R2 tends to be highly stable and interpretable, even when regressionweights are not, because multicollinearity has minimal effects on R2.

Evaluation of equations can be aided by interpretation of regressionweights. Actuarial equationsfor SLD identiÞcation are likely to contain highly correlated predictors. As multicollinearity (i.e.,linear correlation among two or more explanatory variables) increases, the stability of regressionweights decreases. As predictors of SLD tend to be highly correlated, regression weights for actuarialequations developed to aid SLD identiÞcation will be prone to ßuctuation across samples. Severalformulas can be used to increase the stability of regression weights when predictors are highlycorrelated. Examination of the geometric structure of these methods suggests that ridge regression(Hoerl & Kennard, 1970) provides the most accurate estimate of shrinkage (Druilhet & Mom,2006). Thus, ridge regression estimates, which use the standard deviation of regression weights asa shrinkage criterion, are likely to be useful when interpreting the regression weights of actuarialequations developed to aid SLD identiÞcation. Syntax for performing ridge regression has beendeveloped for commercial statistical programs available from companies such as SPSS Incorporated(Chicago, IL) and SAS Institute Incorporated (Cary, NC).

Generalizability

Once a parsimonious equation has been identiÞed, it is important to examine the generalizabilityof the equation. First, examining stability (i.e., replicability of the equation across time when appliedrepeatedly within the same population) is useful because random variation may affect prediction(Newman, McNeil, & Fraas, 2004). The probability of a statistically signiÞcant exact replication canbe estimated by executing a three-step approach: (a) subtracting the critical signiÞcance value fromthe observed value of an appropriate signiÞcance test, (b) performing a one-tailed signiÞcance test



on the resulting value to obtain a one-tailed probability, and (c) subtracting this probability valuefrom 1.00 (Greenwald, Gonzalez, Harris, & Guthrie, 1996; Posavac, 2002).

If predictions are found to have adequate stability, the next step is to test generalizability acrosspopulations. Thus, the equation should be cross-validated by examining its stability across samplesdrawn from target populations. If R2 shrinks by 10% or more, the equation should be modiÞed.Decisions regarding removal of predictors can be guided by interpretation of regression weights.If variables previously not included in an equation are believed to affect prediction in the newpopulation, a variety of models, each representing a different but plausible prediction equation, canbe calculated and integrated. Then, the overall effects of mediator and moderator variables can beestimated by evaluating the means and standard deviations of the integrated set of models.

The generalizability of actuarial equations can be optimized by using an approach designedto identify homogenous subgroups using decision-tree models (Biggs, de Ville, and Suen, 1991).Decision-tree models consist of a collection of decision nodes, each containing a test of a predic-tor variable�s value, as well as terminal nodes that contain the predicted output variable values.Exhaustive Chi-square Automatic Interaction Detector (CHAID) modeling can be used to identifyactuarial equations that optimally predict outcomes and are robust to violations of standard statisticalassumptions (Biggs et al., 1991). Exhaustive CHAID modeling is used by commercially availablesoftware, such as the Answer Tree Program (version 3.1; SPSS, 2002).

The identiÞcation of homogenous subgroups involves obtaining historical data from decisionsupport systems and using decision-tree models to identify subgroups in a top-down fashion. Thisprocess segments the overall population into subgroups that, with respect to the relations betweenpredictor variables and outcomes, are homogeneous (i.e., further subdividing the subgroups byintroducing additional predictor variables should not alter the prediction substantially). Identifyinghomogenous subgroups avoids the pitfall of predicting an event (in this case, academic failure) forindividuals by allowing estimation of the probability of this event for subgroups of individuals withunique conÞgurations of risk markers (American Academy of Actuaries, 2006).

ACTUARIAL METHODS VERSUS HUMAN JUDGMENT

Studies comparing actuarial methods to clinical judgment (i.e., decisions made by humanexperts) consistently support the conclusion that actuarial methods result in conclusions of equal orgreater reliability than those obtained using clinical judgment (Grove &Meehl, 1996). The accuracygap between actuarial methods and clinical judgment is likely to widen because of advances incomputer technology and actuarial science. Actuarial methods are believed (e.g., Meehl, 1954) tobe more accurate than clinical judgment alone because clinical judgments are bounded by a myriadof constraints (e.g., time, information, and computational constraints).

Limitations of Practitioners� Judgment

When making predictions or decisions, practitioners tend to assign nonoptimal weights tovariables (i.e., underestimate or overestimate the effects of a variable) and apply these subjectiveweights inconsistently (Grove & Meehl, 1996). This problem likely is even more pronounced in thecase of SLD diagnosis. As little agreement exists with respect to the deÞnition and speciÞcation ofdiagnostic criteria (Flanagan, Ortiz, Alfonso, & Dynda, 2006), practitioners will not only be inclinedto assign subjective, nonoptimal weights, they also will be inclined to apply subjective, nonoptimaldiagnostic criteria.

Although the conÞdence level of practitioners tends to increase with years of experience, thelevel of conÞdence that practitioners have in their diagnoses has been found to correlate negativelywith diagnostic accuracy (Dawes, 1994). This paradox seemingly is a result of overconÞdence bias.



OverconÞdence bias refers to the tendency for people to rate their abilities, skills, and performanceto be above average (Gilovich, 1993) and thus display a systematic propensity to overestimate theaccuracy of their decisions (Bishop & Trout, 2002). OverconÞdence bias has been described as oneof the most robust phenomena in psychology (Bishop & Trout, 2002).

Limitations of Team Judgments

Within public education, decisions as to whether a student has a disability must be madeby a team consisting of qualiÞed professionals as well the child�s parents (U.S. Department ofEducation, 2004). Although a team approach may have intuitive appeal, team determinations maybe less accurate than are determinations made by individual practitioners. Scholarship comparingteam decisions to actuarial methods could not be located, but scholarship regarding group behaviorhas revealed that group approaches to decision making are subject to numerous imperfections.

Imperfections in group approaches to decision making include diffusion of responsibility(Darley & Latane, 1968), conformity (Asch, 1951, 1956; Milgram, 1974), passive and uncriti-cal acceptance of group decisions (LaBarre, 1972), unquestioned belief in the moral correctness ofthe group (Janis, 1972), and a propensity for groups to take more extreme positions than individ-uals (Freedman, Carlsmith, & Sears, 1974). Additionally, the most inßuential team members havenot necessarily received sufÞcient specialized training that would facilitate systematic, data-basedproblem solving or accurate diagnosis. Team leadership may be based on arbitrary appointment,communication networks, and similarity to other group members rather than expertise (Freedmanet al., 1974). Thus, team members who have had no formal training pertaining to the identiÞcationof SLD may lead teams charged with identifying students for special education.

As Fletcher and Reschly (2005) have noted, �Ultimately, the decision to identify a child forspecial education is a team judgment involving integration of a variety of sources of informationabout the student and his or her development, instructional history, family and environmental factors,and test scores� (p. 10). Research suggests, however, that teams frequently ignore many sources ofinformation that are relevant to the diagnostic and exclusionary criteria for SLD. Multidisciplinaryteams charged with determining special education eligibility rarely override child performancedata when environmental, cultural, or economic disadvantage is likely to be the primary cause ofSLD (Fletcher & Naverrete, 2003). Conversely, multidisciplinary teams have tended to overrideevidence that does not conÞrm the presence of SLD with clinical judgments supporting need forservices (Bocian, Beebe,MacMillan,&Gresham, 1999). Thus, when formulating clinical judgments,multidisciplinary teams appear to value referral information over child-performance data (Ysseldyke& Algozzine, 1982).

POTENTIAL BENEFITS OF ACTUARIAL METHODS

The potential beneÞts of applying actuarial modeling to special education eligibility determi-nation have been discussed by others (Riccio & Rodriguez, 2007); however, to our knowledge thisis the Þrst article that proposes a detailed methodology describing how actuarial modeling can beused for this purpose. Although empirical research that examines the utility of actuarial modelingfor diagnosis in schools is needed, there are studies that demonstrate the utility of actuarial modelingfor similar purposes. For example, actuarial methods have been used to simultaneously evaluate theroles of rehabilitation services and demographic variables on employment outcomes for people withorthopedic disabilities (Chan, Cheing, Chan, Rosenthal, & Chronister, 2006), spinal cord injuries(Marini, Lee, Chan, Chapin, & Romero, 2008), and traumatic brain injuries (Catalano, Pereira, Wu,Ho, & Chan, 2006).



The actuarial models we propose allow simultaneous evaluation of multiple diagnostic andexclusionary criteria for SLD diagnosis. Diagnosis of SLD requires multiple diagnostic criteria,including evidence of past and current academic difÞculty, as well as evidence that any behaviorsthat require the deÞcient academic skills do indeed manifest in functional impairment. Diagnosis ofSLD also requires multiple exclusionary criteria, including evidence that such impairments are notprimarily the result of alternative causes such as visual impairment, auditory impairment, emotionaldisturbance, or environmental disadvantage (U.S. Department of Education, 2004).

Problem solving and RTI models are now used in schools across the nation for the purpose ofimproving educational outcomes and preventing students without disabilities from being identiÞedas having SLD (Reschly & Ysseldyke, 2002). Actuarial methods can augment the roles of problemsolving and RTI in SLD identiÞcation by providing empirically veriÞable models of diagnosis.Evidence of inadequate academic achievement by itself is not sufÞcient evidence to concludethat a child has SLD. Additional evidence supporting a plausible cause of inadequate academicachievement is necessary because diagnoses based solely on inadequate academic achievementwould be fallacious due to circularity. As the effectiveness of instruction and interventions have notbeen demonstrated to be consistent across schools and classrooms, it is not possible to determine ifinadequate academic achievement is due to contextual variables or SLD. Thus, it is inappropriate touse inadequate academic achievement as the sole piece of evidence for itself.

Assessment of contextual variables is necessary when making determinations regarding therelative roles of intrinsic (i.e., student) and extrinsic factors in learning difÞculties. Thus, in additionto RTI data, it is necessary to gather, integrate, and assign weights (i.e., judgments of the relativeeffect that variables have on academic achievement) to data from other sources (e.g., clinical in-terviews, review of historical records, cognitive ability tests, behavioral observations, broad-bandrating scales). As practitioners tend to assign nonoptimal weights to information and apply thesesubjective weights inconsistently (Grove & Meehl, 1996), actuarial methods should be used whenmaking diagnostic decisions.

Actuarial methods provide (a) an objective method for applying weights to various types ofdata and (b) an empirically veriÞable method of investigating plausible causes of SLD and rulingout alternate causes of academic failure (e.g., environmental deprivation, cultural or linguisticissues, health-related variables). Although diagnostic criteria and exclusionary factors also needto be evaluated individually to establish necessary conditions for SLD diagnosis (e.g., intellectualfunctioning and adaptive behavior will need to be evaluated in some cases to rule out mentalretardation), the combination of diagnostic criteria and exclusionary factors within actuarial modelscan be used to establish what has been proposed as a sufÞcient condition for SLD diagnosis,namely interference with academic achievement or other daily activities that are dependent on theseskills (Flanagan et al., 2002). If results obtained from an appropriate actuarial formula indicate thatdiagnostic criteria account for most of this interference, then diagnosis of SLD is supported. Ifexclusionary factors account for most of this interference, then diagnosis of SLD is not supported.

CONCLUSION

The process of identifying students with SLD is complex and requires integrating and assign-ing weights to multiple types and sources of information. Actuarial models have the potential toimprove SLD identiÞcation because they allow simultaneous evaluation of multiple diagnostic andexclusionary criteria. SLD identiÞcation requires multiple diagnostic criteria, including evidence ofpast and current academic difÞculty, as well as evidence that any behaviors that require the deÞ-cient academic skills do indeed manifest in functional impairment. SLD identiÞcation also requiresmultiple exclusionary criteria, including evidence that such impairments are not primarily the resultof alternative causes such as visual impairment, auditory impairment, emotional disturbance, or



environmental disadvantage (U.S. Department of Education, 2004). Assigning optimal weights tothese diagnostic and exclusionary criteria would improve the objectivity and accuracy of diagnosticdecisions.

The use of actuarial methods for optimizing predictions would provide educators with a veriÞ-able indicator, derived from multiple types and sources of data, of student need for special educationand related services. As need for special education and need for related services are important criteriafor SLD eligibility (U.S. Department of Education, 2004), this empirically veriÞable indicator wouldprovide a defensible endpoint in the process of identifying SLD. Additionally, actuarial methods canbe used to inform and improve clinical judgment. Actuarial methods provide a means of testing thehypotheses on which practitioners base diagnostic decisions. Thus, actuarial methods can be usedas part of an iterative learning process in which they provide feedback to practitioners regardingthe accuracy of their assumptions. Practitioners, in turn, can use this feedback to improve the SLDidentiÞcation process.

School psychology developed largely in response to the need for schools to identify childrenwho fail to proÞt adequately from regular educational instruction (Reynolds, 1986); therefore, schoolpsychologists are the professionals most likely to possess the unique set of skills needed to developand implement actuarial methods for identifying SLD. School psychologists are likely the onlyprofessionals with requisite knowledge and skills for developing and implementing actuarial modelsthat account for the effects of biological, environmental, social, and personal factors, as well astheir interactions, on academic achievement. Indeed, school psychologists are the only school-basedpersonnel with requisite competencies in testing, measurement, and assessment and the only appliedpsychologists with requisite foundations in the knowledge bases of both psychology and education.

REFERENCES

Aaron, P. G. (1997). The impending demise of the discrepancy formula. Review of Educational Research, 67, 461�502.

American Academy of Actuaries. (2006). The roles of the actuary in the selection and application of actuarial models.Retrieved February 6, 2008, from www.actuary.org.

Asch, S. E. (1951). Effects of group pressure upon the modiÞcation and distortion of judgments. Oxford, England: CarnegiePress.

Asch, S. E. (1956). Studies of independence and conformity: A minority of one against a unanimous majority. PsychologicalMonographs, 70, 70.

Bergan, J. R., & Kratochwill, T. R. (1990). Behavioral consultation and therapy. New York: Plenum Press.

Biggs, D., de Ville, B., & Suen, E. (1991). A method of choosing multiway partitions for classiÞcation and decision trees.Journal of Applied Statistics, 18, 49�62.

Bishop, M. A., & Trout, J. D. (2002). 50 years of successful predictive modeling should be enough: Lessons for philosophyof science. Philosophy of Science, 59, S197�S208.

Bocian, K. M., Beebe, M. E., MacMillan, D. L., & Gresham, F. M. (1999). Competing paradigms in learning disabilitiesclassiÞcation by schools and the variations in the meaning of discrepant achievement. Learning Disabilities Research &Practice, 14, 1�14.

Bransford, J. J., & Stein, B. S. (1984). The IDEAL problem solver. New York: Freeman.

Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. American Psychologist, 32, 513�531.

Brown-Chidsey, R., & Steege, M. W. (2005). Response to intervention: Principles and strategies for effective practice.New York: Guilford.

Burns, M. K., Jacob, S., & Wagner, A. R. (2008). Ethical and legal issues associated with using response-to-intervention toassess learning disabilities. Journal of School Psychology, 46, 263�279.

Catalano, D., Pereira, A. P., Wu, M., Ho, H., & Chan, F. (2006). Service patterns related to successful employment outcomesof persons with traumatic brain injury in vocational rehabilitation. NeuroRehabilitation, 21, 279�293.

Chan, F., Cheing, G., Chan, J. Y. C., Rosenthal, D. A., & Chronister, J. (2006). Predicting employment outcomes ofrehabilitation clients with orthopedic disabilities: A CHAID analysis. Disability and Rehabilitation, 28, 257�270.

Compton, D. L. (2006). How should �unresponsiveness� to secondary intervention be operationalized? It is all about thenudge. Journal of Learning Disabilities, 39, 170�173.



Darley, J. M., & Latane, B. (1968). Bystander intervention in emergencies: Diffusion of responsibility. Journal of Personalityand Social Psychology, 8, 377�383.

Darlington, R. B. (1978). Reduced-variance regression. Psychological Bulletin, 85, 1238�1255.

Dawes, R. M. (1994). House of cards: Psychology and psychotherapy built on myth. New York: Free Press.

Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668�1674.

Deno, S. L. (2005). Problem-solving assessment. New York: Guilford Press.

Dombrowski, S. C., Kamphaus, R. W., & Reynolds, C. R. (2004). After the demise of the discrepancy: Proposed learningdisabilities diagnostic criteria. Professional Psychology: Research and Practice, 35, 364�372.

Druilhet, P., & Mom, A. (2006). Shrinkage structure in biased regression. Journal of Multivariate Analysis, 99, 232�244.

Efron, B. (1982). The jackknife, the bootstrap and other resampling plans. Philadelphia: Society for Industrial and AppliedMathematics.

Flanagan, D. P., Ortiz, S. O., Alfonso, V. C., & Dynda, A. M. (2006). Integration of response to intervention and norm-referenced tests in learning disability identiÞcation: Learning from the Tower of Babel. Psychology in the Schools, 43,807�825.

Flanagan, D. P., Ortiz, S. O., Alfonso, V., &Mascolo, J. (2002). The achievement test desk reference (ATDR): Comprehensiveassessment and learning disabilities. Boston: Allyn & Bacon.

Fletcher, J. M., Francis, D. J., Morris, R. D., & Lyon, G. R. (2005). Evidence-based assessment of learning disabilities inchildren and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 506�522.

Fletcher, J. M., & Reschly, D. J. (2005). Changing procedures for identifying learning disabilities: The danger of perpetuatingold ideas. The School Psychologist, 15, 10�15.

Fletcher, T., & Navarrete, L. (2003). Learning disabilities or difference: A critical look at the issues associated with themisidentiÞcation and placement of Hispanic students in special education programs. Rural Special Education Quarterly,22, 37�46.

Freedman, J. L., Carlsmith, J. M., & Sears, D. O. (1974). Social psychology (2nd ed.). Oxford, England: Prentice Hall.

Fuchs, D., Moch, D., Morgan, P. L., & Young, C. L. (2006). Responsiveness-to-intervention: DeÞnitions, evidence, andimplications for the learning disabilities construct. Learning Disabilities Research & Practice, 18, 157�171.

Fuchs, L., & Fuchs, D. (1997). Use of curriculum-based measurement in identifying students with disabilities. Focus onExceptional Children, 30, 1�16.

Gilovich, T. (1993). How we know what isn�t so. New York: Free Press.

Gong, G. (1986). Cross-validation, the jackknife, and the bootstrap: Excess error estimation in forward logistic regression.Journal of the American Statistical Association, 81, 108�113.

Greenwald, A. G., Gonzalez, R., Harris, R. J., & Guthrie, D. (1996). Effect sizes and p values: What should be reported andwhat should be replicated? Psychophysiology, 33, 175�183.

Gresham, F. M. (2002). Responsiveness to intervention: An alternative approach to the identiÞcation of learning disabilities.In R. Bradley, L. Danielson, & D. P. Hallahan (Eds.), IdentiÞcation of learning disabilities: Research to practice.(pp. 467�519). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.

Grove,W.M.,&Meehl, P. E. (1996). Comparative efÞciency of informal (subjective, impressionistic) and formal (mechanical,algorithmic) prediction procedures: The clinical-statistical controversy. Psychology, Public Policy, and Law, 2, 293�323.

Hickman, J. C. (1997). Introduction to actuarial modeling. North American Actuarial Journal, 1, 1�5.

Hoerl, A., & Kennard, R. (1970). Ridge regression: Applications to nonorthogonal problems. Technometrics, 12, 69�82.

Huberty, C. J., & Olejnik, S. (2006). Applied MANOVA and discriminate analysis (2nd ed.). New York: John Wiley & Sons.

Janis, I. L. (1972). Victims of groupthink: A psychological study of foreign-policy decisions and Þascoes. Oxford, England:Houghton Mifßin.

Jensen, A. R. (1984). Test validity: g versus the speciÞcity doctrine. Journal of Social & Biological Structures, 7,93�118.

Jimerson, S. R., Burns, M. K., & VanDerHeyden, A. M. (2007). Response to intervention at school: The science and practiceof assessment and intervention. In S. R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Handbook of responseto intervention: The science and practice of assessment and intervention (pp. 3�9). New York: Springer.

Kavale, K. A., & Flanagan, D. P. (2007). Ability-achievement discrepancy, response to intervention, and assessment ofcognitive abilities/processes in speciÞc learning disability identiÞcation: Toward a contemporary operation deÞnition.In S. R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Handbook of response to intervention: The science andpractice of assessment and intervention (pp. 130�147). New York: Springer.

Kavale, K. A., Kaufman, A. S., Naglieri, J. A., & Hale, J. B. (2005). Changing procedures for identifying learning disabilities:The danger of poorly supported ideas. The School Psychologist, 15, 16�25.

LaBarre, W. (1972). The ghost dance. New York: Delta.

Lord, F. M. (1950). EfÞciency of prediction when a regression equation from one sample is used in a new sample. ResearchBulletin, 50, 40.



Marini, I., Lee, G. K., Chan, F., Chapin, H. H., & Romero, M. G. (2008). Vocational rehabilitation service patterns related tosuccessful competitive employment outcomes of persons with spinal cord injury. Journal of Vocational Rehabilitation,28, 1�13.

Mather, N., & Kaufman, N. (2006). Introduction to the special issue, part one: It�s about the what, the how well, and the why.Psychology in the Schools, 43, 747�752.

McNeil, K., Newman, I., & Kelly, F. J. (1996). Testing research hypotheses with the general linear model. Carbondale andEdwardsville: Southern Illinois University Press.

Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis,MN: University of Minnesota Press.

Mercer, C. D., Jordan, L., Allsopp, D. H., & Mercer, A. R. (1996). Learning disabilities deÞnitions and criteria used by stateeducation departments. Journal of Learning Disabilities, 19, 217�232.

Milgram, S. (1974). Obedience to authority. New York: Harper & Row.

National Forum on Education Statistics. (2006). Forum guide to decision support systems: A resource for educators. Jessup,MD: ED Pubs.

Newman, I., McNeil, K., & Fraas, J. (2004). Two methods of estimating a study�s replicability. Mid-Western EducationalResearcher, 17, 36�40.

Newman, I., & Ramlo, S. (in press). The use of twomultivariate techniques, Qmethodology and Q factor analysis, to facilitateinterpretation of mixed methods research. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social& behavioral research (2nd ed., pp. TBD). Thousand Oaks, Calif.: Sage Publications.

Newman, I., Seymour,G.,Garver, T.,&McNeil,K. (1979).AMonteCarlo evaluation of estimated parameters ofÞve shrinkageestimate formulas. Paper presented at the meeting of the American Educational Research Association, San Francisco,CA, April, 1979.

Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.

Posavac, E. J. (2002). Using p values to estimate the probability of statistically signiÞcant replications. UnderstandingStatistics, 1, 101�112.

Raudenbush, S. (2004). What are value-added models estimating and what does this imply for statistical practice? Journal ofEducational and Behavioral Statistics, 29, 121�129.

Reschly, A. L., Coolong-ChafÞn, M., Christenson, S. L., & Gutkin, T. (2007). Contextual inßuences and response tointervention: Critical issues and strategies. In S. R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Handbook ofresponse to intervention: The science and practice of assessment and intervention. (pp. 148�160). New York: Springer.

Reschly, D. J., & Ysseldyke, J. E. (2002). School psychology paradigm shift. In A. Thomas, & J. Grimes (Eds.), Best practicesin school psychology (4th ed., pp. 3�20). Washington, DC: National Association of School Psychologists.

Reynolds, C. R. (1986). The elusive professionalization of school psychology: Lessons from the past, portents for the future.Professional School Psychology, 1, 41�46.

Reynolds, C. R. (2003). Conceptual and technical problems in learning disability diagnosis. In C. R. Reynolds, & R.W.Kamphaus (Eds.), Handbook of psychological and educational assessment of children: Intelligence, aptitude, andachievement (2nd ed., pp. 475�497). New York: Guilford Press.

Reynolds, C. R., & Shaywitz, S. E. (2009). Response to intervention: Ready or not? Or, from wait-to-fail to watch-them-fail.School Psychology Quarterly, 24, 130�145.

Rhodes, R. L., Ochoa, S. H., & Ortiz, S. O. (2005) Assessing culturally and linguistically diverse students: A practical guide.New York: Guilford.

Riccio, C. A., & Rodriguez, O. (2007). Integration of psychological assessment approaches in school psychology. Psychologyin the Schools, 44, 243�255.

Rutter, M., & Yule, W. (1975). The concept of speciÞc reading retardation. Journal of Child Psychology and Psychiatry, 16,181�197.

Salvia, J., & Hughes, C. (1990). Curriculum-based assessment: Testing what is taught. New York: Macmillan.

Sanders, W. L., & Horn, S. P. (1998). Research Þndings from the Tennessee Value-Added Assessment System (TVAAS)Database: Implications for educational evaluation and research. Journal of Personnel Evaluation in Education, 12,247�256.

Shapiro, E. S. (2004). Academic skills problems: Direct assessment and intervention (3rd ed.). New York: Guilford Press.

SPSS. (2002). Answer Tree 3.1. user�s guide. Chicago: SPSS Inc.

Stanovich, K. (1999). The sociometrics of learning disabilities. Journal of Learning Disabilities, 32, 302�319.

Stanovich, K. (2005). The future of a mistake: Will discrepancy measurement continue to make the learning disabilities Þelda pseudoscience? Learning Disabilities Quarterly, 28, 103�106.

Swanson, H. L. (2008). Neuroscience and response to instruction (RTI): A complementary role. In E. Fletcher-Janzen, &C. R. Reynolds (Eds.), Neuropsychological perspectives on learning disabilities in the era of RTI: Recommendations ofdiagnosis and intervention (pp. 28�53). New York: John Wiley & Sons.



Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124�1131.

U.S. Department of Education. (2004). The Individuals with Disabilities Education Improvement Act of 2004 (IDEIA 2004).Washington, DC: Author.

Vaughn, S., & Fuchs, L. S. (2003). RedeÞning learning disabilities as inadequate response to instruction: The promise andpotential problems. Learning Disabilities Research & Practice, 18, 137�146.

Vellutino, F. R., Scanlon, D. M., & Lyon, G. R. (2000). Differentiating between difÞcult-to-remediate and readily remediatedpoor readers: More evidence against the IQ-achievement discrepancy deÞnition of reading disability. Journal of LearningDisabilities, 33, 223�238.

Weissman, H. N., & DeBow, D. M. (2003). Ethical principles and professional competencies. In A. M. Goldstein (Ed.),Handbook of psychology: Forensic psychology (Vol. 11, pp. 33�53). New York: John Wiley & Sons.

World Health Organization. (2001). International classiÞcation of functioning, disability and health (ICF). Geneva,Switzerland: Author.

Ysseldyke, J. E., & Algozzine, B. (1982). Bias among professionals who erroneously declare students eligible for specialservices. Journal of Experimental Education, 50, 223�228.


potential utility of actuarial methods for identifying specific learning disabilities

Documents