core academic language skills (cals): an expanded

Core Academic Language Skills (CALS):An expanded operational construct and

a novel instrument to chart school-relevant language proficiency in per-adolescent and adolescent learners.

The Harvard community has made thisarticle openly available. Please share howthis access benefits you. Your story matters

Citation Uccelli, Paola, Christopher D. Barr, Christina L. Dobbs, EmilyPhillips Galloway, Alejandra Meneses, and Emilio Sanchez. (2014, tobe published). Core Academic Language Skills (CALS): An expandedoperational construct and a novel instrument to chart school-relevant language proficiency in per-adolescent and adolescentlearners. Applied Psycholinguistics xx, no. x:xx-xx.

Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:11380186

Terms of Use This article was downloaded from Harvard University’s DASHrepository, and is made available under the terms and conditionsapplicable to Open Access Policy Articles, as set forth at http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#OAP

http://osc.hul.harvard.edu/dash/open-access-feedback?handle=&title=Core%20Academic%20Language%20Skills%20(CALS):%20An%20expanded%20operational%20construct%20and%20a%20novel%20instrument%20to%20chart%20school-relevant%20language%20proficiency%20in%20per-adolescent%20and%20adolescent%20learners.&community=1/3345927&collection=1/3345928&owningCollection1/3345928&harvardAuthors=67786e02eb133f5313ac5933b0a0801a&department

http://nrs.harvard.edu/urn-3:HUL.InstRepos:11380186

http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#OAP



ACADEMIC LANGUAGE DEVELOPMENT

1

CORE ACADEMIC LANGUAGE SKILLS (CALS): AN EXPANDED OPERATIONAL CONSTRUCT AND A NOVEL INSTRUMENT TO CHART SCHOOL-RELEVANT LANGUAGE PROFICIENCY IN PRE-ADOLESCENT AND ADOLESCENT LEARNERS PAOLA UCCELLI, HARVARD GRADUATE SCHOOL OF EDUCATION CHRISTOPHER D. BARR, UNIVERSITY OF HOUSTON CHRISTINA L. DOBBS, BOSTON UNIVERSITY EMILY PHILLIPS GALLOWAY, HARVARD GRADUATE SCHOOL OF EDUCATION ALEJANDRA MENESES, PONTIFICIA UNIVERSIDAD CATÓLICA DE CHILE & EMILIO SÁNCHEZ , UNIVERSIDAD DE SALAMANCA ABSTRACT Beyond academic vocabulary, the constellation of skills that comprise academic language proficiency has remained imprecisely defined. This study proposes an expanded operationalization of this construct referred to as ‘Core Academic Language Skills’ (CALS). CALS refers to the knowledge and deployment of a repertoire of language forms and functions that co-occur with school learning tasks across disciplines. Using an innovative instrument, we explored CALS in a cross-sectional sample of 235 students in grades 4-8. Results revealed between- and within-grade variability in CALS. Psychometric analyses yielded strong reliability and supported the presence of a single CALS factor, which was found to be predictive of reading comprehension. Findings suggest that the CALS construct and instrument appear promising for exploring students' school-relevant language skills.


2

Introduction In the educational linguistics literature, school-relevant language proficiency has long

been hypothesized to contribute to academic success, specifically to skill in comprehending

school texts throughout the upper elementary school years and beyond (Snow & Uccelli, 2009;

Bailey, 2007; Biancarosa & Snow, 2004; Chamot & O’Malley, 1994; Cummins, 1981, 2001;

Scarcella, 2003; Schleppegrell, 2004, 2012; Shanahan & Shanahan, 2008; Valdés, 2004; Wong-

Fillmore & Fillmore, 2012). However, direct empirical evidence of this relationship has remained

elusive, in part, because the construct of school-relevant language --or academic language

proficiency-- has been either imprecisely delineated or too reductively defined to inform

educational research, assessment, or pedagogical practice (Nagy & Townsend, 2012; National

Research Council, 2010; Valdés, 2004). Studies exploring reading comprehension in populations

of both bilingual and monolingual students suggest that for large proportions of struggling

adolescent readers, one source of reading comprehension difficulty is not necessarily or

exclusively decoding the words on the page, but understanding the meanings of words and

comprehending the syntactic and discourse constructions in which words are embedded

(August & Hakuta, 1997; August & Shanahan, 2006; Biancarosa & Snow, 2004; Lesaux, Crosson,

Kieffer & Pierce, 2010). Although researchers have often surmised that this comprehension

challenge is, in large part, due to the barriers posed by school-relevant lexical, syntactic, and

discourse skills, to our knowledge neither a construct to define this wider set of developmental

language skills nor an assessment to capture these skills presently exists. In fact, proof of this

relationship has resided mostly in indirect evidence (Nagy & Townsend, 2012; National

Research Council, 2010; Valdés, 2004). For instance, an insightful line of research has


3

documented that among students with equal content-area knowledge, English proficient

students in the U.S. consistently outperform English language learners1 (Abedi & Herman, 2010;

Martiniello, 2008). These findings have pointed to academic language as an obstacle to

students’ demonstration of –or access to-- content-area knowledge, but have not yet directly

and comprehensively assessed adolescents' academic language skills.

In the absence of such a construct, academic language proficiency has often been

narrowly operationalized as knowledge of academic words –and occasionally, word parts-- both

in developmental research and in intervention studies focused on the relationship between

language proficiency and reading comprehension. Developmental studies have consistently

reported robust predictive relations between academic vocabulary knowledge and reading

comprehension (Dickinson & Tabors, 2001; Mancilla-Martinez & Lesaux, 2010). In contrast, the

results of a majority of research-based vocabulary intervention studies so far have shown

improvements in students’ vocabulary knowledge but somewhat less satisfactory effects for

improving students’ reading comprehension (Proctor, Dalton, Uccelli, Biancarosa, Mo, Snow, &

Neugebauer, 2011; Deshler, Palincsar, Biancarosa, & Nair, 2007; Lesaux, Kieffer, Faller, & Kelley,

2010). This contrast between developmental and intervention studies, however, might not be

that surprising after all. A highly literate person who has developed an extensive and rich

repertoire of academic vocabulary has also learned how to use those words in particular ways,

(e.g., to pack dense information through subordination and nominalization, to mark conceptual

relationships explicitly through precise connectives, and to use various referential strategies to

1 The Englihs language learner (ELL) designation is used by U.S. schools to identify language minority students who

require intervention to support the development of grade-appropriate English skills.


4

link themes throughout a text). Therefore, in developmental studies, vocabulary knowledge

may serve as a proxy indicator of a wider set of language skills that individuals have developed

in synchrony. Among other alternatives, we hypothesize that the unsatisfactory outcomes of

some intervention studies targeting reading comprehension might be the result of pedagogical

practices that miss additional key academic language skills that need to be attended in

synchrony with academic vocabulary instruction. In fact, recent definitions of academic

language proficiency proposed by different authors support a more comprehensive approach to

language and literacy instruction (see for example, Bailey, 2007; Scarcella, 2003; Schleppegrell,

2004; or the Understanding Language Initiative website at Stanford University). Yet, to date,

few researchers have tried to systematically measure a broader set of academic language skills

with the goal of informing language and literacy instruction.

In this pilot study, we propose a broader construct of academic language proficiency

that includes many of these key academic language skills and we use a set of innovative tasks to

evaluate two hypotheses: that academic language skills are still developing during the

adolescence years, and that these skills positively contribute to upper elementary and middle

school students’ text comprehension. The endeavor to more precisely define the skills that

comprise the broad domain of academic language proficiency began in 2010, when the authors

of this article embarked on the design of an instrument that could capture growth in school-

relevant language skills within- and across-grades 4 to 8. This work is part of a larger, ongoing

initiative to investigate the impact of an intervention designed to improve students’ reading

comprehension, which is one of the fundamental skills that supports the learning of complex

content and ideas from texts in upper elementary school and beyond (Kintsch, 2004). One of


5

the main malleable factors hypothesized to contribute to students' reading comprehension in

this intervention is students' academic language skills. Thus, it is in the context of this larger

research project that the authors of this paper proposed the Core Academic Language Skills

construct and designed and piloted the CALS instrument described in this study.1

We define Core Academic Language Skills (CALS) as knowledge and deployment of a

repertoire of language forms and functions that co-occur with oral and written school learning

tasks across disciplines. While other ongoing research initiatives are investigating the

development of discipline-specific academic language skills (e.g., the language of science or

history), we focus instead on cross-discipline academic language skills, which we hypothesize to

be critical for pre-adolescent and adolescent students’ participation in the oral learning

exchanges of the classroom, the reading of school texts, and the production of school writing

assignments across content areas (Bailey, 2007). The CALS instrument (CALS-I) encompasses a

set of theoretically grounded tasks designed to measure a subset of CALS believed to support

text comprehension in upper elementary and middle school students. Although we believe that

CALS might underlie a host of school-based communicative capabilities (classroom talk,

classroom listening, academic writing, and reading comprehension), we acknowledge the long-

standing hypothesis that links school-relevant language skills with text comprehension

(Biancarosa & Snow, 2004; Kintsch, 2004; Schleppegrell, 2004) and focus in this initial study on

exploring reading comprehension as our outcome of interest. Following the RAND report (2002)

we define reading comprehension as "the process of simultaneously extracting and

constructing meaning through interaction and involvement with written language" (RAND,

2002:11). The most recent theoretical models characterize the process of reading


6

comprehension as complex and multicomponential, with language knowledge as a key

contributing factor, among many others, such as reading fluency, world knowledge, or working

memory (Kintsch, 2004; Perfetti, Landi, & Oakhill, 2005). In this study, we explore CALS as a

particularly relevant subset of language skills hypothesized to be associated with the

comprehension of school texts.

An important motivation of our work is to make visible for educators and researchers a

repertoire of language skills that might continue to develop throughout adolescence and might

play a significant role in academic success. These cross-discipline language skills are particularly

relevant because, in contrast to disciplinary academic language, they are rarely considered in

instruction (Fang, 2012). Mature users of academic language—such as teachers—see language

as a transparent medium of communication; yet students, as relatively inexperienced academic

language users, struggle not only with the abstract content they need to learn but also with

acquiring a language that they often perceive as obscure (Bailey, Burkett, & Freeman, 2008). It

is important to clarify, though, that we are interested in academic language as a tool for precise

communication that supports effective school learning. Consequently, we focus on high utility

language forms across content areas; not on what some have called academic gibberish, i.e.,

rarely used forms and unnecessarily intricate structures widely acknowledged to obstruct

instead of facilitate communication (Krashen, 2013).

Three main questions guided this study:

1) Based on 4th-to-8th-grade students’ performances in the CALS-I tasks, does the

constellation of Core Academic Language Skills function as a unitary or

multidimensional construct?


7

2) Do 4th-to-8th-grade students’ Core Academic Language Skills—as measured by the

CALS-I—vary across- and within-grades?

3) Controlling for reading fluency, are 4th-to-8th-grade students’ Core Academic

Language Skills—as assessed by the CALS-I—predictive of students’ reading

comprehension levels?

Given the limitations of studying development with cross-sectional samples, we are

cautious to report our results as between-grade variability instead of referring to

developmental trends or developmental trajectories. This study can only offer initial findings

relevant to inform future longitudinal research. However, to our knowledge, no study has

embarked so far on the direct assessment of a sample of upper elementary and middle school

students’ core academic language skills, thus an initial test of our proposed operational

construct seems justified before embarking on more ambitious longitudinal studies.

In the next section, we briefly introduce the sociocultural pragmatics-based view of

language that has guided our study. In the subsequent sections, we introduce our operational

definition of CALS in more detail, and we briefly describe the CALS instrument. After the

introductory sections, we move on to explain the study design and the results of exploring the

CALS instrument’s reliability, unidimensionality, and validity, focusing on CALS as a predictor of

reading comprehension in a cross-sectional sample of 4th-to-8th-graders from an urban, public

K-8 (kindergarten through grade 8) school located in the Northeastern region of the United

States.

A sociocultural pragmatics-based view: Later language development as rhetorical flexibility

A sociocultural pragmatics-based view of language development entails understanding


8

language as inseparable from social context, and language learning as the result of individuals’

socialization and enculturation histories (Halliday, 2004; Heath, 1983, 2012; Ninio & Snow 1996;

Ochs, 1993; Ravid & Tolchinsky, 2002; Snow & Uccelli, 2009). Two key developmental

consequences emerge from this framework. First, language development is not seen as

complete after the first few years of life but as continuing throughout adolescence and

potentially throughout life as language users develop new skills to navigate an increasing

number of social contexts (Berman & Ravid, 2009). Second, this view of language suggests that

being a skilled language user in some social contexts does not guarantee adequate language

proficiency in other social contexts. In fact, whereas speakers are enculturated at home into the

language of face-to-face interaction which typically prepares them well for colloquial

conversations in their respective communities (Heath, 1983; Ochs, 1993), participating

successfully in academic discourses appears to be challenging for many colloquially fluent

students who, inside or outside of school, may not have been granted ample opportunities to

be socialized into more academic ways of using language (Cummins, 2000; Schleppegrell, 2004;

Snow & Uccelli, 2009)2.

Needless to say, the goal of later language development is not restricted to school-

relevant language discourses. Instead, during the adolescent years, progress in language

development entails developing “rhetorical flexibility,” i.e., the ability to use an increasing

repertoire of lexico-grammatical and discourse resources appropriately and flexibly in an

expanding variety of social contexts (Ferguson, 1994; Ravid & Tolchinsky, 2002). In other words,

language learning involves expanding overlapping sets of skills that are deployed to participate

in different registers across contexts (e.g., family conversations, youth discourses, sportscasts).


9

A register is defined as the assemblage of lexical, grammatical, and discourse features prevalent

in particular uses of language (Biber, 1995; Halliday & Hassan, 1976). In the present study, CALS

is understood as the knowledge and deployment of language features characteristic of registers

that co-occur with learning-related tasks across content areas in the social contexts of school

(Bailey, 2007; Chamot & O’Malley, 1994; Halliday, 1980; Scarcella, 2003; Schleppegrell, 2001,

2004, 2012; Snow & Uccelli, 2009).

Despite extensive research on early language acquisition, research on school-relevant

adolescent language development has received comparably minimal attention (Berman &

Ravid, 2009; Nippold, 2007). A recent and illuminating line of developmental linguistics

research, though, points to adolescence as a period of substantial growth in the deployment of

register features typically encountered in school texts, for example the use of more complex

noun phrases and more academic discourse markers (Berman & Ravid, 2009; Derewianka,

2003; Nippold, 2007; Ravid & Tolchinsky, 2002). The present study seeks to extend this research

by focusing not on skilled academic language users, but rather on a diverse group of pre-

adolescent and adolescent learners presumed to have varying levels of academic language

skills. In addition, in contrast to most research on developmental linguistics, instead of

analyzing features of texts produced by adolescents, in this study we identified a

comprehensive --yet not exhaustive--repertoire of core academic language skills, and tested

them directly to explore the associations between students’ CALS and their reading

comprehension abilities.

Core Academic Language Skills (CALS): An innovative construct


10

As stated above, CALS refers to the language knowledge and effective deployment of

language forms and functions prevalent in school texts and tasks across content areas.

Following Bailey (2007), we hypothesized the existence of overlapping but distinguishable

constructs included under the umbrella term academic language proficiency. By ‘proficiency,’

we refer to both knowledge of language forms and skill in deploying these forms. Discipline-

specific academic language skills refer to the different language forms and functions that

highlight the key concepts and reasoning moves of specific disciplines. In contrast, CALS refers

to language skills that cut across content areas and are used to fulfill similar language goals,

such as communicating or understanding precise meanings, concisely packed information, and

explicitly-marked conceptual relations (Bailey, 2007; Hyland, 2009; Schleppegrell, 2004; Uccelli,

Dobbs, & Scott, 2013). For the operationalization of the CALS construct proposed in this study,

we drew from different research traditions: (1) textual linguistics focused on English academic

discourses, (2) educational research focused on the language demands of U.S. classrooms,

standards, and assessments; and (3) developmental linguistics focused on adolescents’ skills in

text production.

To develop the CALS construct, we first engaged in a comprehensive review of the

literature that included different lines of research in textual linguistics, such as empirical studies

in contrastive analysis of language corpora (e.g., Biber & Reppen, 2002; Chafe & Danielewicz,

1987; Hyland, 2004), evolutionary analyses of scientific language (e.g., Halliday & Martin, 1993),

analysis of performances at different levels of expertise or in different disciplines

(Schleppegrell, Achugar, & Oteíza, 2004; Schleppegrell, 2001) or analysis of genres used for

specific purposes (Swales, 1990). These extensive linguistic analyses have produced inventories


11

of common features identified in English texts produced by academic writers across different

disciplines. A review and integration of such studies provided an initial map of linguistic

expectations and textual features characteristic of experts’ academic texts across disciplines.

For instance, this literature suggests that academic texts across disciplines are expected to be

lexically precise; concise and densely packed; and explicit in the marking of conceptual

relations, among other characteristics. These expectations are reflected in texts’ specific and

diverse academic vocabulary, the prevalence of complex morphologically-derived words (e.g.,

nominalizations, such as evaporation) and syntactically intricate structures (such as embedded

clauses or extended noun phrases); and the ubiquity of a range of contrastive and causal

connectives used to mark relations among complex ideas (for a synthesis and more detailed

inventory of features, please see Snow & Uccelli, 2009). By extension, these are the aspects of

text that proficient readers must learn to comprehend if they are to have access to the

academic content these texts contain.

As a second step, we reviewed the limited educational research on the language

demands of reading and learning during the middle school years in the U.S. Notably, many of

the cross-disciplinary linguistic features that characterize academic texts written by experts are

also found in the texts written for student readers. Indeed, in keeping with the common

features derived from our comprehensive literature review of textual linguistics studies, the

more limited research on the language demands of U.S. classroom discourse, textbooks,

assessments, and educational standards, has documented so far “remarkable similarities across

disciplinary discourses” during the middle school years (Bailey, 2007, p. 10). For instance, across

social science and science disciplines, Bailey and colleagues report an overlap of all-purpose


12

academic vocabulary, clause connectors, and extended noun phrases. Bailey and colleagues

also report the ubiquity of certain expository discourses (i.e., definitions, explanations, and

argumentation), which they describe as types of texts that draw from a common linguistic

repertoire across content areas (Bailey, 2007; Butler, Bailey, Stevens, Huang & Lord, 2004).

Given the existence of common features that characterize school texts across disciplines

(Schleppegrell, 2001), in developing the CALS tasks presented here, we reasoned that the

language skills that constitute the counterpart of such textual features should be relevant to

support the comprehension of school texts. For instance, knowledge of connectives would

support the comprehension of conceptual relationships marked through connectives in an

academic text. In fact, a few studies focused on measuring discrete skills –e.g., knowledge of

connectives, syntactic or morphological knowledge—already support the relevance of looking

at cross-disciplinary language skills as important contributors to reading comprehension

(Crosson, Martiniello, & Lesaux, 2008; Kieffer & Lesaux, 2008; Mokhtari & Thompson, 2006).

Yet, instead of looking at discrete skills subsumed under –and divided by-- formal linguistic

categories (morphology, syntax), the goal of this study was to explore a set of functionally

related language skills hypothesized to be particularly relevant for school reading and learning.

On the basis of our literature review, not all types of connective, morphological, or syntactic

knowledge seem to be equally relevant for the gradual mastery of academic ways of using

language. Thus, we focused instead on a particular subset of school relevant skills, such as

knowledge of connectives which are prevalent in academic texts; skills in derivational

morphology, particularly skill in nominalizations; and syntactic knowledge of embedded

clauses. As discussed below, prior research has identified these skills as particularly relevant in


13

comprehending the precise, densely packed, explicitly connected ways of using language

characteristic of academic texts.

Finally, we reviewed empirical evidence from developmental linguistics focused on

adolescent language development (Bailey, 2004; Benelli, Belacchi, Gini, & Lucangeli, 2006;

Berman, 2004; Berman & Ravid, 2009; Berman & Verhoeven, 2002; Christie & Derewianka,

2008; Derewianka, 2003; Nippold, 2007; Ravid & Tolchinsky, 2002; Schleppegrell, 2001, 2004;

Uccelli, Scott, & Dobbs, 2013). These studies have revealed that adolescence is a period of

substantial growth in the production of register features typically encountered in school texts,

such as the use of more complex noun phrases and embedded clauses, more academic linking

devices, the construction of referential chains, and the gradual learning of the overall

organization of expository texts (for a review, see Blum-Kulka, 2008; Berman & Ravid, 2009;

Nippold, 2007). However, these studies tend to focus on spontaneously generated texts usually

produced by skilled language users from middle class environments. In our study, we do not

rely on the spontaneous display of these linguistic skills, but instead focus on directly assessing

them in an ethnically and socio-economically diverse sample of students.

The Common Core State Standards (CCSS) --recently adopted by the majority of U.S.

states-- also offered an additional resource to corroborate the relevance of the selection of

CALS areas. From the upper elementary grades up, the CCSS place special emphasis on the

language demands students face and explicitly refer to the need to prepare students to engage

in argumentative discourse and, more specifically, in writing and language proficiency. For

instance, the CCSS include “use precise language… use words, phrases and clauses to create

cohesion and clarify relations among claims… establish a formal style… organize ideas, concepts


14

and information, using strategies such as definitions” (CCSS, 2010, Writing standards: p. 20, 42);

“acquire... general academic… words that signal contrast, addition, and other logical relations,”

(CCSS, 2010, Language standards: p. 29), among others.

Below we offer an overview for six CALS areas, which, based on the prior literature

reviews, underlie skilled comprehension of academic texts.

1. Unpacking complex words. Morphological skills, in particular skills in derivational

morphology, are essential for understanding and producing complex words used to pack

information in academic texts. Recent research has shown that skills in decomposing

morphologically complex words (e.g., contribution, cultural) contribute positively to

reading comprehension in upper elementary/middle school students (Carlisle, 2000; Kieffer

& Lesaux, 2007, 2008, 2010; Nagy, Berninger, & Abbott, 2006).

2. Comprehending complex sentences. Even though a few studies have questioned the

relationship between syntactic awareness skills and reading comprehension (Cain, 2007;

Layton, Robinson, & Lawson, 1998), this area was included because numerous studies have

provided evidence of later syntactic skills—such as extended noun phrases and complex

sentences—positively contributing to reading comprehension in children, adolescents, and

adults (e.g., Mokhtari & Thompson, 2006; Nation & Snowling, 2000; Taylor, Greenberg,

Laures-Gore, & Wise, 2011).

3. Connecting ideas. Academic discourse markers (such as connectives and lexical cue

phrases, e.g., although, in other words) constitute prevalent signaling devices that explicitly

mark intra-sentential conceptual relations and text transitions in academic discourse

(Hyland, 2004). Several sources—from lexical databases to corpus linguistics studies—


15

document the most prevalent causal and discourse makers in academic texts at school

(Zeno, Ivans, Millard, Duvvuri, 1995; Dale & O’Rourke, 1981) and beyond school, in

academic texts exchanged by experts across disciplines (Hyland, 2005; Simpson-Vlach &

Ellis, 2010). Although not without some controversy, several studies have provided

evidence to suggest that discourse markers affect online processing, text memory, and

learning from academic text (Meyer & Rice, 1982; Hyönä & Lorch, 2004; Meyer & Poon,

2001).

4. Tracking themes. Anaphors, words or phrases appearing in a text that refer to a prior

participant or idea, can be interpreted as instructions to the reader/listener to link a

previous idea with an element in the text (Givón, 1992). Whereas concrete anaphoric

elements are ubiquitous in colloquial language (e.g., ‘she’ refers to ‘Mary’), one type of

anaphor, conceptual anaphora, is particularly characteristic of academic text, where it has

been estimated to account for approximately 20% of all anaphoric references (Biber,

Conrad & Reppen, 1998). Conceptual anaphora consists of a demonstrative determiner

(e.g., this) with or without a hypernoun, i.e., a noun that encapsulates meanings expressed

in prior discourse (e.g., The evaporation of water occurs to due to rising temperatures. This

process…) (Flowerdew, 2003; Hunston & Francis, 2000). A positive relationship between

skill in resolving conceptual anaphora and reading comprehension has been documented

recently for upper elementary school students (Sánchez & García, 2009).

5. Organizing argumentative texts: Research has described later language development as

moving along a continuum at the interface of modality and text type (i.e., genre): from the

early mastery of oral narrative to gradual progress in written narrative, followed by


16

proficiency in oral expository and subsequently, written expository texts (Berman & Ravid,

2009). Previous research suggests that narrative organization is well-achieved by age 9 to

10, whereas knowledge of the global structure of expository discourse progresses gradually

and consolidates only around high school age (Berman & Nir-Sagiv, 2007, p. 96). Skills in

structuring narratives have been found to contribute to reading comprehension during the

primary school years, when children read mostly narrative texts (Oakhill & Cain, 2000).

Among the many prevalent non-narrative text types in school discourses, we selected only

the argumentative text to assess knowledge of global discourse structure. Given the

argumentative nature of academic language3 (Rex, Thomas & Engel, 2010; Toulmin, 1958),

skills in structuring argumentative texts (i.e., thesis, arguments, examples, conclusion)

were hypothesized to be associated to school literacy during the middle school years.

6. Awareness of academic register: Definitions provide an optimal minimal genre to assess

identification and production of short texts that display core academic language features,

including those used to achieve lexical precision (i.e., the use of less vs. more precise

superordinates, bicycle is a thing vs. bicycle is a vehicle), and concise information packing

(complex grammatical structures, such as nominalizations and center-embedded clauses).

Prior research has documented the predictive power of productive definitional skills for

later academic success (Benelli, Belacchi, Gini, & Lucangeli, 2006; Kurland & Snow, 1997).

We selected this minimal genre to focus not only on production, but also on identification

skills, i.e., students’ skill in identifying more academic forms of discourse in comparison to

more colloquial alternatives.


17

We conceive of these proposed areas as an initial selection to begin to delineate an

operational construct of CALS. Whereas the particular selection is so far supported by extensive

research and widely-adopted educational standards, we see this set as only provisional and,

most likely, in need of further expansion through future research.

Core academic language skills: An innovative assessment instrument

In order to assess students’ CALS, we embarked on the design of a set of tasks to

assess the six areas listed above (see Appendix 1). The research-based CALS Instrument

(CALS-I) was the result of a process that included expert linguists, psychometricians,

psychologists, educators, and students. The target words, grammatical structures and

discourse structures included in the CALS-I were systematically selected to represent

high-utility language forms and functions prevalent in school texts during the upper

elementary and middle school years (for more information, see Appendix 1).

Once the construct had been defined, the development of the CALS instrument

followed four phases. First, the Task Design Phase involved designing items to minimize

decoding, vocabulary knowledge, prior knowledge, reading comprehension and

productive writing demands.4 Also, to overcome the logistical difficulties of individual

language assessment (a procedure that schools find enormously disruptive), we

designed items that could be group- administered.

In the second phase, the Pilot Phase, all tasks and items were first piloted in

individual or small-group interviews with a sample of 32 students (grades 4-8). The

design of items followed an iterative process of generation, testing, incorporation of

students’ feedback (obtained through multi-party interviews using a structured


18

protocol), and retesting that resulted in modifying, recalibrating, or discarding individual

items and in improving task instructions. The third phase, Study 1, consisted of an initial

full battery of 110 items administered in three 45-minute sessions by four native

English-speaking research assistants with experience in language testing who used a

scripted administration protocol. The data reported in the present study were collected

during this phase of data collection. The final set of CALS tasks was the result of

selecting 44 items and 4 constructed response definitions identified as tapping

developmental differences across grades and individual variability within grade (for a

full detailed psychometric report of all items and tasks, please see Uccelli & Barr, 2011).

To select the 48 items that best captured variability in students’ performance, a Rasch

unidimensional measurement model was conducted on students’ item-level

performances across the larger pool of 110 items. From this model, we obtained both

individual item difficulty estimates, as well as item fit information. Items with

substantial misfit, e.g., items the model identified as easy but where many high

performing students endorsed incorrect responses, were eliminated from further

consideration in the final item set. Once we had a set of items that demonstrated good

model fit, the items were further reduced by eliminating items that had similar difficulty

estimates, because items with similar difficulty estimates provide redundant

information about student performance at a given point on the academic language

ability continuum. Finally, the test information function (TIF) was examined to

determine the number of items that needed to be retained to ensure that student

performance on the CALS tasks was a reliable measure of student ability across the


19

entire targeted ability range (2 SD below the 4th grade mean and 2 SD above the 8th

grade mean). Based on the examination of the TIF and in light of theoretical

considerations, the final set of 44 multiple-choice items and 4 constructed response

items provided adequate information at all points of the targeted ability range and thus,

constituted the final selection for the CALS instrument (See Appendix 1 for a more

detailed description of tasks and specific skills measured). The fourth phase consisted of

an Expert Panel Review solicited to establish content validity. The CALS instrument along

with a content validation survey was sent to five experts in the field of academic

language for their independent review. Overall, the team of experts gave the CALS

instrument a mean score of 3.5 out of 5-point scale, with a median score of 4. Experts’

valuable feedback and recommendations were considered and, to the extent possible,

incorporated in the assessment. In this discussion, we report the design and results of

the Study 1, the third phase in the CALS instrument development.

Methods

Participants A total of 235 4th-to-8th grade students attending one urban K-8 school

in the Northeast region of the United States participated in this study (see Table 1). The

sample was relatively balanced by gender. Substantial portions of the students came

from low-income families, with 80% receiving free or reduced-price lunch. Participants

were from a diversity of ethnic backgrounds, with 66% African American or black, 22%

Caucasian, and 7% Latino/a students, according to school records. Only 18% of students

in the sample were officially classified by the school as English Language Learners (ELLs),

but 28% were identified as language minority students with a range of home language


20

resources. From a total of 66 language minority students, the majority came from

families with Cape Verdean Creole as a home language (36), followed by Haitian Creole

(14), Spanish (5) Portuguese (4) and Somali (3) as home languages. Other home

languages, such as French, Swahili, and Vietnamese, were also represented by single

students (for one language minority student, the home language could not be

identified). Based on school records, approximately 15% of the sample had a special

education designation. The grade-specific socio-demographic distributions closely

resembled the distribution of the overall sample for gender, SES, home language, and

special education. There was a smaller proportion of ELLs in the higher grades (and no

ELLs in 8th grade) due most likely to students being reclassified as English-proficient in

the later years. In any case, the number of ELLs was relatively small in the overall

sample.

----------------------------------INSERT TABLE 1 ABOUT HERE----------------------------------

Measures

Participants were administered the following three assessments:

(1) Reading Comprehension - MASSACHUSETTS COMPREHENSIVE ASSESSMENT

SYSTEM—ENGLISH LANGUAGE ARTS (MCAS–ELA): A criterion-referenced, statewide

assessment of English language arts, the MCAS-ELA focused on reading

comprehension and contained selected readings followed by multiple-choice and

open-response questions. Previous studies have shown the MCAS-ELA to be strongly

associated with other standardized measures of reading comprehension (see

Proctor, Uccelli, Dalton, & Snow, 2009). Possible scores ranged from 200 to 280. The


21

MCAS-ELA scores are classified into four performance levels (Advanced: 260-280;

Proficient: 240-259; Needs improvement: 220-239; Warning: 200-219).

(2) Reading Fluency - Test of Silent Word Reading Fluency (TOSWRF) (Mather, Hammill, Allen,

& Roberts, 2004): A group-administered measure of silent fluency. The TOSWRF is designed

to measure reading fluency, i.e., the ability to recognize printed words accurately and

efficiently, in students in grades 1 and above. On this task, students are asked to draw a

dividing lead between words (e.g. ‘inyesgomesee’ should be parsed as ‘in/yes/go/me/see’)

and receive one point for each word correctly identified. We used Silent Word Reading

Fluency standard scores, based on a mean of 100 with a standard deviation of 15.

(3) Core Academic Language Skills Instrument (CALS-I): Researcher-designed group-

administered instrument that measures Core Academic Language Skills (see Appendix 1 for

a detailed description of tasks and specific skills measured). A total of 44 items and 4

constructed responses comprised the CALS instrument used in this study. In order to

equally weight the items when creating the AL total score, the few items that were not

scored dichotomously as correct/incorrect were rescaled to be between 0 and 1. Task-

specific and total summative CALS-I scores were computed as a percentage metric, i.e., the

number of items correct divided by the number of items a given student was administered5.

Reliability for the CALS-I was investigated and found to be robust at .92 as indexed by

coefficient alpha and at .82 by split-half reliability of even vs. odd numbered items.

Analytic plan

Descriptive statistics were generated for the CALS-I task-specific and total scores, as well

as for each administered measure. Subsequently, pairwise correlational analyses were


22

conducted to explore associations between CALS-I scores, reading fluency (TOSWRF),

and reading comprehension (MCAS-ELA). In order to investigate the potential

multidimensionality of the CALS construct, factor analyses at the task level were

conducted using principal components analysis (PCA). Using the factor score generated

for each student, we explored the variability of CALS-I student scores within and across

grades. Finally, to assess the predictability of the CALS-I scores, multiple regression

analyses were conducted with MCAS-ELA scores as outcome and reading fluency

(TOSWRF) as a covariate.

Results

A unitary but multifaceted construct of CALS (Research Question 1)

As displayed in Table 2, pairwise correlational analyses revealed that all of the CALS-I

task-specific scores were significantly and positively correlated. Across the first five CALS-I tasks

displayed in Table 2, all bivariate correlations ranged from moderate to high. The Identifying

academic register task was positively correlated to all tasks, but displayed the lowest

correlations of all tasks administered. Finally, for the Producing academic register task, all

within-task dimensions used to score the definition productions were, as expected, highly

correlated with each other. In addition, each scoring dimension of this production task

displayed mostly positive and moderate correlations with all other CALS-I tasks.

Between reading fluency and each CALS-I task, most correlations were moderate. Except for the

correlation with Decomposing words (morphological skills), correlations did not exceed .36.


In order to further investigate the potential multidimensionality of the CALS construct, a


23

factor analysis at the task level was conducted using principal components analysis (PCA). The

seven CALS task-specific scores (Unpacking Complex Words, Comprehending Complex

Sentences, Connecting Ideas, Tracking Themes, Structuring Argumentative Texts, Identifying

Academic definitions and Producing Academic Definitions) were entered into the PCA. The

results of the PCA indicated that there was one eigenvalue greater than 1. This first principal

component (eigenvalue = 3.44) accounted for 49% of the variability in the data (see Table 3).

Confirmatory factor analysis results also suggested that the general academic language data

supported a single factor solution (CFI=.93, TLI = .92, RMSEA <.05). These findings suggest that

aggregate core academic language tasks scores load onto a single factor.

-------------------------------------INSERT TABLE 3 ABOUT HERE---------------------------

Variability in the development of CALS (Research Question 2)

Operationalized as a unitary construct, we examined total CALS-I scores to gain insight into the

developmental variability exhibited by this sample. Table 4 displays students’ means and

standard deviations for the CALS instrument, the MCAS-ELA, and the reading fluency test

(TOSWRF). Total CALS-I scores displayed evidence of variability across grades and within-grade.

The mean CALS-I score across grades was .61 with a standard deviation of .19. As displayed in

Table 2 and in Figure 1, the mean CALS-I total scores per grade revealed that students exhibited

higher performances in the higher grades. The mean total CALS-I score was lowest in 4th grade,

with a mean of .44 (SD = .14), and progressively higher across grades, with the highest mean of

.76 (SD = .12) displayed by 8th graders. Interestingly, as displayed in Figure 2, an analogous

increasing progression was detected for mean task-specific CALS-I scores per grade, indicating

that students in higher grades performed better on average in the CALS-I as a whole, as well as


24

within each specific task. In addition, total CALS-I scores also revealed considerable individual

variability, as illustrated by the normal distribution of scores within grade displayed in the bar

graphs in Figure 1. Standard deviations fluctuated from .12 to .19 across grades, displaying

approximately normal distributions in each grade with a wide range of scores that progressively

moved towards the higher scores in the later grades. A one-way ANOVA indicated that CALS-I

scores differed significantly as a function of grade, F(4, 230) = 40.08, MSE = .14, p < .0001. Post-

hoc Tukey’s HSD tests suggested that students in 4th grade and 5th grade had CALS-I scores that

were significantly lower than students in higher grades at the .05 level of significance. However,

there was no significant difference observed between the performances of 4th graders when

compared to their 5th grade peers. Similarly, there were no statistically significant differences

between the CALS performance of students in grades 6, 7 and 8. Yet, it is important to

emphasize that CALS-I scores at each grade displayed considerable individual variability.

As shown in Table 4, reading fluency scores were available for 217 students in the

sample and the overall mean was 109.07 (SD = 25.94), indicating that students tended to

perform slightly above average on this measure, with 4th, 5th, and 6th grader typically displaying

average performances, and at 7th and 8th graders performing above average (7th grade mean

123, SD = 28.81; 8th grade mean 121.46, SD = 17.74).6

Although MCAS-ELA scores are valuable metrics of within-grade reading performance,

this measure was not designed using an equated scale across grades and so cannot be used for

cross-grade comparison. Results of MCAS-ELA assessment are reported on a scale (200-280

points) with thresholds that serve to delineate achievement levels. Participants’ distribution

across performance levels in the MCAS-ELA indicated a larger proportion of low-performing


25

students in our sample when compared to the overall state distribution: with only 26% of

students scoring ‘proficient or advanced,’ (compared to 69% for the state); 54% receiving the

designation of ‘needs improvement,’ (compared to 23% for the state); and 20% with scores that

placed them in the ‘warning’ range (compared to 8% for the state)7. Despite the differences

with overall state means, this distribution of MCAS-ELA scores is not atypical of urban public

schools in the region.


----------------------------------INSERT FIGURE 1 ABOUT HERE----------------------------------

Figure 1: Histograms of distribution of CALS total scores by grade

----------------------------------INSERT FIGURE 2 ABOUT HERE----------------------------------

Figure 2: Mean CALS task-specific scores per grade

CALS as a predictor of reading comprehension (Research Question 3)

Multiple linear regression models were run at each grade level using the CALS-I total

score to predict MCAS-ELA scores, controlling for fluency. Prior to conducting regression

analyses, the correlations between students’ MCAS-ELA scores and CALS-I scores were

examined. In all grades, correlations were moderate to strong and statistically significant

(p<.0001) (grade 4=.54; grade 5=.61; grade 6=.77; grade 7=.41; grade 8=.65). At each grade

level, students’ CALS-I scores were found to be significant predictors of MCAS-ELA scores, even

after controlling for reading fluency. As displayed in Table 5, these regression models

accounted for an important amount of variability in MCAS-ELA scores, ranging from explaining

25% of the variability in 7th grade MCAS-ELA scores to 61% of the variability for 6th grade

students.


26

To investigate the potential impact of ELL status, the contribution of ELL status was

explored within each grade, as was the interaction between ELL and CALS-I scores. No

significant contribution of ELL status was detected either as a main effect or an interaction. This

is likely a power issue given the small sample of ELLs in this study. In addition, regression

analyses were also conducted with a sample that excluded the students classified as ELLs. We

compared these results to those for the full sample and found that the findings, with regard to

statistical significance, were identical, and the magnitude of the parameter estimates appeared

to be quite comparable.8


Discussion

Starting with the premise that knowledge of academic vocabulary constitutes a highly

relevant developmental and pedagogical domain, yet only a fragment of a wider repertoire of

academic language skills, the goal of this study was to identify additional developmental skills

involved in mastering academic language throughout the upper elementary and middle school

years. Building from Bailey’s (2007) data-driven conceptualization of academic language, in this

study we offer preliminary evidence to support the construct of Core Academic Language Skills

(CALS), defined as knowledge and deployment of an integrated set of prevalent language forms

and functions that co-occur with learning-related tasks across content areas at school. On the

basis of an extensive integration of different lines of textual, educational, and developmental

linguistics research, in this study we measured CALS in the following areas: (1) unpacking

complex words; (2) comprehending complex sentences; (3) connecting ideas; (4) tracking

themes; (5) organizing argumentative texts; (6a) identifying academic register; and (6b)


27

producing academic register. We administered the innovative and theoretically-grounded CALS

instrument (CALS-I) to 235 4th-to-8th graders in order to answer three main questions: Do CALS-I

task-specific performances support a unitary or a multidimensional operational construct of

Core Academic Language Skills? Do students’ performances in the CALS-I tasks yield evidence of

variability within and across the upper elementary and middle school grades? Are students’

CALS-I scores predictive of reading comprehension? Results revealed variability within- and

across-grades in an expanded—but certainly not exhaustive—set of core academic language

skills. Results from the principal component analysis offer initial empirical evidence that

supports a unitary construct of CALS, as defined in this study. Regression analyses showed that

CALS-I total scores predicted reading comprehension, as measured by the MCAS-ELA, at each

grade level from 4th to 8th grade, even after controlling for students’ reading fluency levels.

Our results move the field forward by documenting not only a potential developmental

progression, but also considerable individual variability in a precise set of CALS that expand our

understanding of academic language proficiency. At the level of construct building, this study

offers promising initial results that support a construct that is worth exploring in future

research. At the level of assessment, the CALS instrument offers a preliminary tool focused on

testable cross-discipline academic language skills that can be highly relevant for the design and

development of research and pedagogical instruments. We discuss our findings in more detail

below and then consider research and educational implications. We close the article by

highlighting the limitations of our study and presenting some new questions raised by the

current findings.

Core Academic Language Skills as a unitary construct


28

This study tackled the question of whether CALS --as measured by the CALS

instrument—is a unitary or a multidimensional construct. Results offer preliminary evidence

pointing to a unitary construct. Scores across the tasks administered revealed positive pairwise

associations, and confirmatory factor analyses showed that task-specific scores loaded onto a

single factor. These results are in keeping with our view of language use as register

participation. Within this perspective, language users expand their resources as they participate

in different registers, i.e., a constellation of prevalent language features that occur together to

serve particular purposes. Thus, adolescent language learning is viewed not as a process of

learning discrete individual skills independently, but instead as learning a repertoire of language

features in synchrony as they are used in particular contexts and for specific purposes. As

students are provided with opportunities to participate in the discourses of school, they would

learn linguistic forms and functions as a constellation of features that typically occur together in

oral or written school texts in response to the particular goals of academic learning. For

instance, the morphologically complex words and academic connectives usually employed

when constructing academic argumentative texts would be hypothesized to be learned

together as students participate in reading and writing such texts.

On the other hand, caution should be invoked given the novel and exploratory nature of

this construct. Certainly, in our CALS construct, the skills assessed represent only a selective

repertoire of those involved in reading school texts. In addition, only three highly prevalent

school text types are used across the CALS-I tasks: definitions, argumentative text, and

fragments of expository texts. The conceptualization of CALS as unitary or multidimensional will

depend on how the boundaries of this construct are defined. Thus, more exploration is


29

necessary to investigate the nature of this construct. Some important questions emerge: are

there additional tasks or general cross-content area text types that should be considered as

part of this construct? Would the evidence that points to Core Academic Language Skills as

unitary hold if the boundaries of the construct were slightly expanded, for example by including

additional text types? Only future research can answer these pending questions.

Finally, these results do not imply that the global construct of academic language

proficiency entails a single dimension. Given that this larger construct also includes discipline-

specific academic language skills, which might not be correlated within individuals (e.g.,

understanding the structure of a story problem in math vs. that of a literary analysis), we would

in fact predict that discipline-specific academic language skills would not be part of a

unidimensional construct.

In sum, we interpret the findings pointing towards a unitary construct of CALS as

insightful, yet preliminary and of course, as a result of the particular construct investigated in

this study. These results align with a pragmatics-based view of language development. Instead

of viewing development as progress in fragmented or isolated skills, or as divided by traditional

linguistic levels (morphology, syntax, discourse), we view the development of CALS as a

synchronous progress that takes place as language users' participate in oral and written school

discourses around learning.

Core academic language skills: Development and individual variability

The present results are encouraging as they support an empirically-based construct and

instrument that appear promising for capturing change over time and variability across

individuals. In line with prior developmental linguistics research on adolescent language


30

(Berman & Ravid, 2009; Nippold, 2007), the current findings support a view of adolescence as a

period of substantial growth, in particular in this case, in a specific subset of language skills –

those relevant for school-related learning tasks. Core academic language skills, as measured by

the CALS instrument, manifested a general upward trend across both CALS-I total scores and

task-specific mean scores. Our findings suggest that repertoires of forms and functions continue

to expand throughout development in each of the language-learning areas measured by the

CALS instrument. Whereas vocabulary knowledge is widely understood as a potentially ever-

expanding domain with thousands of words to be learned in a given language (Stahl & Nagy,

2006), other language domains are sometimes viewed as fully mastered by the end of primary

school (e.g., syntactic or morphological skills). Adding to a recently emerging line of research on

adolescent language, our findings clearly point to the continuing expansion of school-relevant

language knowledge across all our tested tasks. Interestingly, statistically significant differences

in performance were only evident when comparing the performance of elementary grade to

middle grade students. While it is possible that this intriguing lack of difference is an a artifact

of the CALS instrument or sampling idiosyncrasies, these results may also suggest some

interesting patterns in pre-adolescent and adolescent language development, for example that

school-relevant language growth is more subtle in the period from grade 6-8. Further research

needs to carefully examine this issue by studying development in longitudinal samples, ideally

with larger and more diverse groups of students and by further refining the CALS instrument.

Although chronological ages for students were not available to us, future studies may also

examine variability in CALS scores not only by grade, but also by age.

Core Academic Language Skills scores revealed considerable variability not only across


31

grades, but also within grades. The within-grade variability in CALS was considerable, with even

a few 4th graders outperforming some 8th graders in this sample. These results might not be

surprising in light of prior research that has highlighted considerable individual variability in

vocabulary knowledge across and within bilingual and monolingual populations (Biemiller &

Slonim, 2001; Graves, 2007; Kieffer & Lesaux, 2007; Stahl & Nagy, 2006). Even if not surprising,

these results are revealing. Our findings extend the repertoire of academic language skills that

seem to be highly vulnerable to individual variability beyond vocabulary knowledge. In addition,

while individual differences in language skills have been documented mostly during the early

years (Hart & Risley, 1995; Bowers & Vasilyeva, 2011), our results indicate that even during the

middle school years students are differentially equipped to face the language demands of

school.

Core Academic Language Skills as predictors of reading comprehension

Our findings suggest that the variability in CALS accounts for a significant variability in

students’ academic reading comprehension in each of the grades assessed (4th – 8th grade). It

should be of no surprise that, analogous to academic vocabulary knowledge, a broader set of

general academic language skills would be predictive of reading comprehension. The innovation

of this study resides in having identified a precise set of cross-disciplinary academic language

skills that offer initial evidence that this proficiency can be measured in a way that captures

enough variability associated with advanced literacy skills. Prior research has pointed to

academic language as a key predictor in reading comprehension mostly indirectly by comparing

skilled vs. unskilled readers, or English proficient vs. English language learners, or by narrowly

operationalizing academic language as academic vocabulary. To our knowledge, our study is the


32

first one to directly assess pre-adolescents and adolescents from grades 4th to 8th with a

comprehensive –even if not exhaustive-- set of core academic language tasks.

Limitations of the current study

Adding to other well documented skills (e.g., prior knowledge, motivation, strategy use,

vocabulary knowledge), this study expands the range of known contributors to reading

comprehension and suggests that core academic language proficiency—understood as a

repertoire of skills—is a relevant construct for understanding students’ academic literacy. Our

results are promising but require further research in a variety of directions.

First, further research is required to explore if these findings can be replicated

with larger groups of diverse students. Caution should be exerted in drawing inferences

beyond our sample, which included only 235 students. This is particularly important in

light of the fact that our sample comprises a low-performing group of students. Thus,

even though our sample’s MCAS-ELA performance is representative of urban public

schools in the region, understanding cross-disciplinary academic language development

requires the inclusion of a sample with a wider range of academic literacy levels.

Second, this study reports variability across grades in a cross-sectional sample.

Developmental research, however, requires longitudinal studies. The next step in this

research entails following students from 4th to 8th grade to document individual

variability in developmental trajectories. Our finding that students in grades 6, 7 and 8

did not demonstrate a statistically significant difference in CALS performance might

suggest also that future versions of this instrument might require further refinement to


33

include items that present greater challenge to older students who are more adept

academic language users.

Furthermore, the CALS construct and instrument need to be refined, possibly expanded,

and further explored. It is necessary to consider other potentially relevant areas of proficiency,

such as epistemic stance (Uccelli, Dobbs, & Scott, 2013), or the inclusion of a wider variety of

text types, such as explanations or school-relevant paragraph patterns (compare-and-contrast,

classification, etc.) (Sánchez & García, 2009).

In addition, in order to further understand the contribution of CALS to reading

comprehension, future studies may also examine additional covariates, such as academic

vocabulary knowledge or listening comprehension skills. As in the case of vocabulary

knowledge and reading comprehension, a reciprocal relationship between CALS and reading

comprehension should also be considered. Finally, the relationship of CALS to other academic

literacy outcomes encompassed in our proposed construct, such as skill in speaking, writing,

and comprehending oral language in school contexts should be explored in future studies.

Finally, this study offers evidence of considerable cross-grade and within-grade variation

in CALS skills, yet cannot answer the question of what accounts for such variability as this is

beyond the scope of the present study. Some researchers have argued that school-relevant

language proficiency is acquired mostly in interaction with academic texts (Ravid & Tolchinsky,

2002; Wong-Fillmore and Fillmore, 2012), however there is also evidence to suggest that

participation in certain oral exchanges, such as debates and text-based discussions, might

support the expansion of school-relevant oral and written language proficiency (Dickinson &

Tabors, 2001; Reznistkaya, Anderson, & Kuo, 2007). From a socio-cultural pragmatics-based


34

perspective, we interpret this range of performances as the result of individuals’ histories of

socialization and enculturation, i.e., their opportunities to participate in particular forms of

language and text interactions. The goal of this research was not to examine the impact of

these language experiences or the methods by which CALS is acquired, but rather to examine

the impact of having acquired CALS on a fundamentally important element of school literacy—

text comprehension. Yet, we acknowledge that examining the social and pedagogical conditions

under which academic language skills develop is a crucial area of study that deserves attention.

Only future research, though, can shed light on what in-school and out-of-school opportunities

–as well as other factors- are associated with school language proficiency.

Implications for educational research, assessment, and pedagogy

This set of CALS tasks constitutes a promising tool that can inform the design of research

instruments and, ultimately, a pedagogically informative assessment. This study was designed

on the premise that academic vocabulary is only a single component of a constellation of

academic language skills and the results provide initial empirical evidence that can help us

move beyond the widespread metonymical confusion that equates academic vocabulary

knowledge with academic language proficiency.

For research purposes, instruments that directly assess cross-disciplinary academic

language skills can further enrich current research programs. For instance, the research

conducted by Kintsch (2004), and also by McNamara and colleagues, suggests that text features

(i.e., level of cohesion) and readers’ characteristics (i.e., background knowledge,

comprehension skills) interactively impact reading comprehension. O’Reilly and McNamara

(2007) found that skilled comprehenders with high background knowledge benefited from a


35

highly cohesive text whereas a low cohesion text was beneficial for less skilled readers with

high background knowledge. However, in these studies students’ proficiency in the linguistic

features manipulated in texts—in other words, proficiency in certain aspects of core academic

language—is not directly assessed, and is consequently implicitly assumed to be homogenous

across students. Measuring students’ academic language proficiency as an additional variable in

order to explore interactions between textual and reader’s characteristics could be illuminating.

For educational purposes, the CALS instrument represents an initial step towards

developing a tool that might be useful for making some school-relevant language skills visible to

educators, curriculum writers, assessment developers, and students. As stated by Bailey et al.

(2008) and Wong-Fillmore and Fillmore (2012), instead of an accessible medium of instruction,

the language of school is often opaque to students, while at the same time being often

transparent to the teachers. Research that has documented opportunities to learn as

significantly contributing to the variability of students' academic achievement (Abedi &

Herman, 2010) might expand its focus by investigating classroom discourse (exposure and

participation) as an additional dimension relevant for assessing the quality of instructional

environments.

We should clarify that we see the CALS construct and instrument as complementary to

other ongoing efforts focused on discipline-specific language skills. Bailey and colleagues, for

instance, have worked on academic language as embedded in disciplinary learning which

subsequently led to the design of assessments used to measure the language skills required to

understand typical standardized-assessment questions in 5th-grade math, social studies or

science (Butler et al., 2004; Bailey 2007). CALS, in our view, is an essential common


36

denominator, but certainly not sufficient to participate successfully in the increasing variety of

discipline-specific discourses that students encounter at school.

The findings of this study seem to be particularly timely in light of the current

climate of the Common Core Standards (CCSS) in the United States. The CCSS call for

exposing students to more complex texts, which places academic language at the

forefront of the pedagogical conversation about how to best prepare students to be

successful readers (and writers). In fact, in the reading and writing competencies

outlined in the CCSS, students are called upon to both understand and use

'appropriate transitions' and 'precise words and phrases.' Thus, a potential pedagogical

application of the CALS instrument might be to help make the crucial role of students’

academic language skills visible to educators and researchers and to support them in (1)

identifying language features in the complex texts that students must read at school and

(2) in designing lessons that incorporate the expansion of students’ academic language

skills as an important pedagogical goal. Such cross-disciplinary academic language

consciousness might support educators in identifying instructional instances where

paraphrasing, unpacking and paying explicit attention to language structures may

support students in reading comprehension, written composition or oral discussion

always in the service of expanding students' content knowledge and conceptual

understandings (see van Lier & Walqui, 2012 and Wong-Fillmore and Fillmore, 2012 for

discussion of such pedagogies). The potential malleability of CALS skills and the best

ways to scaffold them is an imminent question that emerges from this work. The current


37

results offer a promising construct and instrument, but constitute only the first step in a

journey still filled with numerous questions to be answered.


38

Appendix 1: Core Academic Language Skills – Instrument (CALS-I) CALS-I Tasks Skill measured Items’

Description Sources

for research-based design

1. Unpacking Complex Words Selected items from Kieffer’s (2009) adaptation of Carlisle (2000)

Skill in decomposing morphologically-derived words

SAMPLE ITEM: Students are read a set of morphologically derived words followed by an incomplete sentence and are asked to complete the sentence by extracting the base from the derived word:

ethnicity. The city had many _______groups.

This task includes a subset of Kieffer’s Morphological Decomposition Task, an adaptation of Carlisle’s (2000) measure (Kieffer & Lesaux, 2007, 2008, 2010). Twelve out of the 18 words in Kieffer’s assessment were selected. Kieffer’s (2009) scoring protocol was followed to score responses as correct or incorrect. Correct responses included phonetically logical versions of the word (e.g., both ‘popular’ and ‘populer’ were scored as correct).

2. Comprehending Complex Sentences Selected and adapted items from the TROG-2 (Bishop, 2003)

Skill in understanding complex syntax

SAMPLE ITEM: Administrator reads a sentence and students select the corresponding picture from among four options that accurately represent the lexical and grammatical contrasts conveyed in the sentence, e.g., The sheep the girl looks at is running.

This task consists of a selective adaptation of the Test of Receptive Grammar-2 (TROG-2) (Bishop, 2003), a test suitable for ages 4 to young adulthood. From a total of 80 items, 10 were selected to assess five syntactic structures prevalent in academic texts (e.g., relative clause in object, center-embedded relative clause). In our adaptation, the test items were group administered.

3. Connecting Ideas

Skills in understanding of school-relevant connectives and discourse markers

SAMPLE ITEM A: Students select a missing marker: Kim was sick_______she stayed home and did not go to school Options: OTHERWISE, YET, IN CONTRAST AS A RESULT SAMPLE ITEM B: Students select the best continuation for an incomplete sentence, from among three options: Most teachers think that homework is important. ON THE OTHER HAND …

Design informed by researcher-designed assessments used in prior studies (Uccelli, Rosenthal, & Barr, 2011; Sánchez & García, 2009). A selection of frequent academic markers that vary in levels of difficulty was informed by databases of students’ word knowledge (Biemiller, 2010; LWV, Dale & O’Rourke, 1981), word frequency in school texts (Zeno et al., 1995) and academic lexical bundles derived from corpus analyses (Biber et al., 2004; Cortes, 2004, 2006; Simpson-Vlach & Ellis, 2010).

4. Tracking themes

Skill in anaphoric resolution

SAMPLE ITEM: Students match the underlined text with its antecedent by selecting among three options: China resisted the move for change. In 1989 students protested to demand changes, but the army opposed these changes. Troops were sent to stop the movement…

Design informed by a prior researcher-designed assessment used in studies of middle-school students’ reading comprehension (Sánchez & García, 2009). Widely-used grade-specific school texts were consulted to confirm that the structures included were representative of U.S. middle school textbooks. Fragments were selected from middle school textbooks or grade-appropriate Time for Kids magazine.

5. Organizing argumentative texts

Skill in argumentative text organization

Students order six fragments of a brief essay (each sentence was introduced by conventional markers such as: ‘in my opinion,’ ‘one reason,’ ‘another reason,’ ‘in conclusion’) to display a conventional argumentative text structure.

Design informed by the story anagram task used by Stein & Glenn (1978), and by Cain, Oakhill and colleagues in their reading comprehension studies (Cain & Oakhill, 2006; Cain, Oakhill & Bryant, 2000). Construction of this task was informed by research on the development of argumentative texts (Crowhurst, 1990: Christie & Derewianka, 2008).

6a. Awareness of Academic Register-Identifying academic definitions

Skill in identifying academic register

This task asks students to identify an academic definition in comparison to two more colloquial alternatives.

This task was inspired by research on children’s register awareness (Andersen, 1996; Gibbons, 1998). The specific task design, however, was not modeled after any prior research.


39

6b. Awareness of Academic Register-Producing written academic definitions

Skill in producing academic register

This task elicits written definitions for a dictionary for adults. Four written definitions were elicited and, subsequently, scored for: (1) superordinate precision; (2) structural density; (3) informativeness:

bicycle winter debate anger

This task was informed by extensive research on definitions (Benelli, Belacchi, Gini, & Lucangeli , 2006; Kurland & Snow, 1997; Marinellie, 2001; Nippold, Hegel, Sohlberg, & Schwarz, 1999). The goal of this task was not to measure students’ understanding of the meaning of words, but to evaluate their deployment of core academic language. Using a research-based and data-driven innovative scoring manual, definitions were scored for superordinate precision, structural density (i.e., morphosyntactic complexity), and informativeness (content features included in the definition). Inter-rater reliability was estimated for all three dimensions with all kappa coefficients reaching .9 or higher.


40

Acknowledgments The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305F100026, awarded to the Strategic Education Research Partnership as part of the Reading for Understanding Research Initiative. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. We would like to express our gratitude to the students and teachers who shared their valuable time and insights with us and to our numerous colleagues for their helpful comments as we conducted this work. Finally, we want to thank the valuable comments from three anonymous reviewers.


41

References Abedi, J., & Herman, J. (2010). Assessing English Language Learners' Opportunity to Learn

Mathematics: Issues and Limitations. Teachers College Record, 112, 723-746. Andersen, E. S. (1996). A cross-cultural study of children's register knowledge. In D. I. Slobin, J.

Gerhardt, A. Kyratzis & J. Guo (Eds.), Social interaction, social context, and language: Essays in honor of Susan-Ervin-Tripp (pp. 125-142). Mahwah, NJ: Erlbaum.

August, D., & Hakuta, K. (Eds.). (1997). Improving schooling for language-minority children: A research agenda. Washington, DC: National Academies Press.

August, D., & Shanahan, T. (2006). Developing literacy in second language learners: Report of the National Literacy Panel on language minority and youth. Mahwah, NJ: Lawrence Erlbaum.

Bailey, A. L. (2007). The Language Demands of School: Putting academic English to the test. New Haven, CT: Yale University Press.

Bailey, F., Burkett, B., & Freeman D. (2008). The mediating role of language in teaching and learning: A classroom perspective. In B. Spolsky & F. Hult (Eds.), The handbook of educational linguistics (pp. 606-625). Oxford: Blackwell.

Benelli, B., Belacchi, C., Gini, G., & Lucangeli, D. (2006). "To define means to say what you know about things": The development of definitional skills as metalinguistic acquisition. Journal of Child Language, 33, 71-97.

Berman, R. A. (2004). Language development across childhood and adolescence (Vol. 3). Amsterdam: John Benjamins Publishing Company.

Berman, R. & Verhoeven, L. (2002). Cross-linguistic perspectives on the development of text-production abilities: Speech and writing. Written Language and Literacy, 5, 1-44.

Berman, R. A., & Nir-Sagiv, B. (2007). Comparing narrative and expository text construction across adolescence: A developmental paradox. Discourse Processes, 43, 79-120.

Berman, R. A., & Ravid, D. (2009). Becoming a literate language user: Oral and written text construction across adolescence. In D. R. Olson, & N. Torrance (Eds.), Cambridge handbook of literacy (pp. 92-111). Cambridge, UK: Cambridge University Press.

Biancarosa, G., & Snow, C. E. (2004). Reading Next—A vision for action and research in middle and high school literacy: A report from Carnegie Corporation of New York. Washington, DC: Alliance for Excellent Education.

Biber, D. (1995). Dimensions of register variation: A cross-linguistic comparison. Cambridge, UK: Cambridge University Press.

Biber, D., Conrad, S., & Reppen, R. (1998). Corpus Linguistics: Investigating Language Structure and Use. Cambridge, UK: Cambridge University Press.

Biber, D. and Reppen, R. (2002). What does frequency have to do with grammar teaching? Studies in Second Language Acquisition, 24, 199–208.

Biemiller, A. (2010). Words worth teaching: Closing the vocabulary gap. Columbus, OH: SRA/McGraw-Hill.

Biemiller, A., & Slonim, N. (2001). Estimating root word vocabulary growth in normative and advantaged populations: Evidence for a common sequence of vocabulary acquisition. Journal of Educational Psychology, 93, 498 –520.


42

Bishop, D. V. (2003). Test for Reception of Grammar (version 2). London, UK: The Psychological Corporation.

Blum-Kulka, S. (2008). Language, Communication and Literacy: Major steps in the development of literate discourse. In P. Klein, & Y. Yablon (Eds.), From research to practice in early education (pp.117-154). Jerusalem: The Israeli Academy of Science.

Bowers, E., & Vasilyeva, M. (2011). The relation between teacher input and lexical growth of preschoolers. Applied Psycholinguistics, 32, 221-241.

Butler, F.A., Bailey, A. L., Stevens, R., Huang, B. and Lord, C. (2004). Academic English in 5th-grade Mathematics Science, and Social Studies Textbooks (CSE report # 642). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

Cain, K., & Oakhill, J. V. (2006). Profiles of children with specific reading comprehension difficulties. British Journal of Educational Psychology, 76, 683- 696.

Cain, K., Oakhill, J. V., & Bryant, P. E. (2000). Investigating the causes of reading comprehension failure: The comprehension-age match design. Reading and Writing: An Interdisciplinary Journal, 12, 31-40.

Cain, K. (2007). Syntactic awareness and reading ability: Is there any evidence for a special relationship? Applied Psycholinguistics, 28, 679–694.

Carlisle, J. F. (2000). Awareness of the structure and meaning of morphologically complex words: Impact on reading. Reading and Writing: An Interdisciplinary Journal, 12, 169-190.

Chafe, W. and Danielewicz, J. (1987). Properties of Spoken and Written Language. In R. Horowitz and S.J. Samuels (Eds.), Comprehending Oral and Written Language (pp. 83-113). San Diego, CA: Academic Press.

Chamot, A. U., & O'Malley, J. M. (1994). The CALLA handbook: Implementing the cognitive academic language learning approach. Reading, MA: Addison-Wesley.

Christie, F. & Derewianka, B. (2008). School Discourse: Learning to Write across the Years of Schooling. London, UK: Continuum.

Common Core State Standards Initiative. (2010). English language arts standards. Retrieved from http://www.corestandards.org/

Cortes, V. (2006). Teaching lexical bundles in the disciplines: An example from a writing intensive history class. Linguistics and Education, 17, 391-406.

Cortes, V. (2004). Lexical bundles in published and student disciplinary writing: Examples from history and biology. English for Specific Purposes, 23, 397–423.

Crosson, A., Lesaux, N., & Martiniello, M. (2008). Factors that influence comprehension of connectives among language minority children from Spanish-speaking backgrounds. Applied Psycholinguistics, 29, 603-624.

Crowhurst, M. (1990). Teaching and learning the writing of persuasive/argumentative discourse. Canadian Journal of Education, 15, 348-359.

Cummins, J. (1981). Bilingualism and minority-language children. Toronto, Canada: OISE Press. Cummins, J. (2000). Language, power, and pedagogy: Bilingual children in the crossfire.

Clevedon, England: Multilingual Matters. Cummins, J. (2001). Negotiating identities: Education for empowerment in a diverse society (2nd

ed.). Los Angeles: California Association for Bilingual Education.

http://www.corestandards.org/


43

Dale, E., & O'Rourke, J. (1981). The living word vocabulary. Chicago: World Book-Childcraft International.

Derewianka, B. M. (2003). Grammatical metaphor in the transition to adolescence. In A. Simon-Vandenbergen, M. Taverniers & L. Ravelli (Eds.), Grammatical Metaphor: Views from Systemic Functional Linguistics (pp. 185-220). Amsterdam: John Benjamins Publishing Company.

Deshler, D. D., Palincsar, A. S., Biancarosa, G., & Nair, M. (2007). Informed Choices for Struggling Adolescent Readers: A Research-Based Guide to Instructional Programs and Practice. New York, NY: International Reading Association.

Dickinson, D. K., & Tabors, P. O. (2001). Beginning literacy with language: Young children learning at home and school. Baltimore, MD: Paul H. Brookes Publishing.

Fang, Z. (2012). Approaches to developing content area literacies: A synthesis and a critique. Journal of Adolescent and Adult Literacy, 56, 111-116.

Ferguson, C. (1994). Dialect, register, and genre: Working assumptions about conventionalism. In D. Biber & E. Finegan (Eds.), Sociolinguistic perspectives on register (pp. 15-30). New York, NY: Oxford University Press.

Flowerdew, L. (2003). A combined corpus and systemic-functional analysis of the Problem-Solution pattern in a student and professional corpus of technical writing. TESOL Quarterly, 37, 489–511.

Gibbons, P. (1998). Classroom talk and the learning of new registers in a second language. Language and Education, 12, 99-118.

Givón, T. (1992). The Grammar of Referential Coherence as Mental Processing Instructions. Linguistics, 30, 5-55.

Graves, M. F. (2007). Vocabulary instruction in the middle grades. Voices from the Middle, 15, 13–19.

Halliday, M. A. K. (1980). Three aspects of children’s language development: learning language, learning through language, learning about language. In M.A.K.Halliday (Ed.) The language of early childhood (pp. 308-326). New York, NY: Continuum.

Halliday, M. A. K. (2004). The Language of Science - Collected Works of M.A.K. Halliday (Volume 5). New York, NY: Continuum.

Halliday, M. A. K. & Hasan, R. (1976). Cohesion in English, London, UK: Longman. Halliday, M. A. K. & Martin, J. R. (1993). Writing Science: Literacy and Discursive Power. London,

UK: The Falmer Press. Hart, B., & Risley, R. T. (1995). Meaningful differences in the everyday experience of young

American children. Baltimore, MD: Paul H. Brookes. Heath, S.B. (1983). Ways with words: Language, life, and work in communities and

classrooms. New York, NY: McGraw-Hill. Heath, S. B. (2012). Words at work and play: Three decades in family and community life.

Cambridge, UK: Cambridge University Press. Hunston, S. & Francis, G. (2000). Pattern grammar: A corpus-driven approach to the lexical

grammar of English. Amsterdam: John Benjamins Publishing Company. Hyland, K. (2004). Disciplinary discourses: Social interactions in academic writing. Ann Arbor,

MI: University of Michigan Press. Hyland, K. (2005). Metadiscourse. London, UK: Continuum.

https://misprd.uow.edu.au/ris_public/WebObjects/RISPublic.woa/wa/Staff/selectPerson?id=3324&group=9


44

Hyland,K. (2009). Academic discourse. London: Continuum Hyöna, J., & Lorch, R. F. (2004). Effects of topic headings on text processing: Evidence from

adult readers' eye fixation patterns. Learning and Instruction, 14, 131-152. Kieffer, M. J. (2009). The development of morphological awareness and vocabulary knowledge

in adolescent language minority learners and their classmates. (Doctoral Dissertation). Cambridge, MA: Harvard Graduate School of Education.

Kieffer, M. J. & Lesaux, N. K. (2007). Breaking words down to build meaning: Vocabulary, morphology, and reading comprehension in the urban classroom. The Reading Teacher, 61, 134-144.

Kieffer, M. J. & Lesaux, N. K. (2008). The role of derivational morphological awareness in the reading comprehension of Spanish-speaking English language learners. Reading and Writing: An Interdisciplinary Journal, 21, 783-804.

Kieffer, M. J., & Lesaux, N. K. (2010). Morphing into adolescents: Active word learning for English language learners and their classmates in middle school. Journal of Adolescent & Adult Literacy, 54, 47-56.

Kintsch, W. (2004) The Construction-integration model of text comprehension and its implications for instruction. In R. Ruddell & N. Unrau (Eds.), Theoretical models and processes of reading (5th ed.), (pp.1720-1328). Newark, DE: International Reading Association.

Krashen, S. (2012). Academic gibberish. RELC Journal: A Journal of Language Teaching and Research, 43, 283-285.

Kurland, B. F., & Snow, C. E. (1997). Longitudinal measurement of growth in definitional skill. Journal of Child Language, 24, 603-625.

Layton, A., Robinson, J., & Lawson, M. (1998). The relationship between syntactic awareness and reading performance. Journal of Research in Reading, 21, 5–23.

Lesaux, N. K., Crosson, A., Kieffer, M. J., & Pierce, M. (2010). Uneven profiles: Language minority learners' word reading, vocabulary, and reading comprehension skills. Journal of Applied Developmental Psychology, 31, 475-483.

Lesaux, N. K., Kieffer, M. J., Faller, S. E., & Kelley, J. G. (2010). The effectiveness and ease of implementation of an academic vocabulary intervention for sixth graders in urban middle schools. Reading Research Quarterly, 45, 196-228.

Mather, N., Hammill, D. D., Allen, E. A., Roberts, R. (2004). Test of Silent Word Reading Fluency (TOSWRF). Austin, TX: Pro Education.

Mancilla-Martinez, J. & Lesaux, N. K. (2010). Predictors of reading comprehension for struggling readers: The case of Spanish-speaking language minority learners. Journal of Educational Psychology, 102, 701-711.

Marinellie, S. A. (2001). What does "apple" mean? Learning to define words. Young Exceptional Children, 4(2), 2-11.

Martiniello, M. (2008). Language and the Performance of English Language Learners in Math Word Problems. Harvard Educational Review, 78, 333-368.

Meyer, B. J. F. & Rice, G. E. (1982). The interaction of reader strategies and the organization of text. Text: Interdisciplinary Journal for the Study of Discourse, 2, 155-192.

Meyer, B. J. F., & Poon, L. W. (2001). Effects of structure training and signaling on recall of text. Journal of Educational Psychology, 93, 141-159.


45

Mokhtari, K., & Thompson, H. B. (2006). How problems of reading fluency and comprehension are related to difficulties in syntactic awareness skills among fifth graders. Reading Research and Instruction, 46, 73–94.

Nagy, W., Berninger, V. W., & Abbott, R. D. (2006). Contributions of morphology beyond phonology to literacy outcomes of upper elementary and middle-school students. Journal of Educational Psychology, 98, 134.

Nagy, W. M. & Townsend, D. (2012). Words as tools: ‘Learning academic vocabulary’ as language acquisition. Reading Research Quarterly, 47, 91-108.

Nation, K., & Snowling, M. J. (2000). Factors influencing syntactic awareness skills in normal readers and poor comprehenders. Applied Psycholinguistics, 21, 229–241.

National Research Council (2010). Language Diversity, School Learning, and Closing Achievement Gaps: A Workshop Summary. Committee on the Role of Language in School Learning: Implications for Closing the Achievement Gap. Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. Retrieved from http://www.nap.edu/catalog.php?record_id=12907

Ninio, A. & Snow, C. E. (1996). Pragmatic Development. Boulder, CO: Westview Press. Nippold, M. A. (2007). Later language development: School-aged children, adolescents, and

young adults (3rd ed.). Dallas, TX: Pro-Ed. Nippold, M., Hegel, S., Sohlberg, M., & Schwarz, L. (1999). Defining abstract entities:

Development in pre-adolescent, adolescent, and young adults. Journal of Speech, Language and Hearing Research, 42, 473-481.

Oakhill, J., & Cain. K. (2000) Children’s difficulties in text comprehension: Assessing causal issues. Journal of Deaf Studies and Deaf Education, 5, 51-59.

Ochs, E. (1993). Constructing social identity: a language socialization perspective. Research on Language and Social Interaction, 26, 287-306.

O’Reilly, T., & McNamara, D. S. (2007). The impact of science knowledge, reading skill, and reading strategy knowledge on more traditional “high-stakes” measures of high school students’ science achievement. American Educational Research Journal, 44, 161–196.

Perfetti, C. A., Landi, N., Oakhill, J. (2005). The acquisition of reading comprehension skills. In M. J. Snowling, & C. Hulme (Eds.), The science of reading: A handbook, (pp. 227-247). Malden, MA: Blackwell Publishing.

Proctor, P., Uccelli, Dalton, B., & Snow, C. (2009). Understanding Depth of Vocabulary and Improving Comprehension Online with Bilingual and Monolingual Children. Reading and Writing, 25, 311-333.

Proctor, P., Dalton, B., Uccelli, Biancarosa, G., Mo, E., Snow, C.E., & Neugebauer, S. (2011). Improving comprehension online: Effects of deep vocabulary instruction with bilingual and monolingual fifth graders. Reading and Writing: An interdisciplinary journal, 24, 517-544.

RAND Reading Study Group (2002). Toward an R&D program in reading comprehension. Santa Monica, CA: RAND.

Ravid, D. & Tolchinsky, L. (2002). Developing linguistic literacy: A comprehensive model. Journal of Child Language, 29, 419-448.

Rex, L . A., Thomas, E. E., & Engel, S. (2010). Applying Toulmin: Teaching logical reasoning and argumentative writing, English Journal, 99, 56-62.


46

Reznitskaya, A., Anderson, R., & Kuo, L. (2007). Teaching and learning argumentation. The Elementary School Journal, 107(5), 449-472.

Sánchez, E. & García, J.R. (2009). The relation of knowledge of textual integration devices to expository text comprehension under different assessment conditions. Reading and Writing: An Interdisciplinary Journal, 22, 1081–1108.

Scarcella, R. (2003). Academic English: A conceptual framework. Davis, CA: The University of California Linguistic Minority Research Institute.

Schleppegrell, M. J. (2001). Linguistic features of the language of schooling. Linguistics and Education, 12, 431–459.

Schleppegrell, M. J. (2004). The language of schooling: A functional linguistics perspective. Mahwah, NJ: Lawrence Erlbaum.

Schleppegrell, M. J. (2009). Language in Academic Subject Areas and Classroom Instruction: What Is Academic Language and How Can We Teach It? Paper presented at the Workshop on the Role of Language in School Learning: Implications for Closing the Achievement Gap, October 15-16, Hewlett Foundation, Menlo Park, CA.

Schleppegrell, M. J. (2012). Academic Language in Teaching and Learning Introduction to Special Issue. The Elementary School Journal, 112, 409-418.

Schleppegrell, M. J., Achugar, M., & Oteíza, T. (2004). The grammar of history: Enhancing content-based instruction through a functional focus on language. TESOL Quarterly, 38, 67-93.

Scott, C. M. (2004). Syntactic contributions to literacy. In B. Shulman, K. Apel, B. Ehren, E. Silliman & A. Stone (Eds.), (pp. 340-362). New York, NY: Guilford.

Shanahan, T. & Shanahan, C. (2008). Teaching Disciplinary Literacy to Adolescents: Rethinking Content- Area Literacy. Harvard Educational Review, 78, 40-59.

Simpson-Vlach, R. & Ellis, N. C. (2010): An Academic Formulas List: New Methods in Phraseology Research. Applied Linguistics, 31(4), 487-512.

Snow, C. E., & Uccelli, P. (2009). The challenge of academic language. In D. R. Olson, & N. Torrance (Eds.), The Cambridge handbook of literacy, (pp. 112-133). New York: Cambridge University Press.

Stahl, S. A. & Nagy, W. E. (2006). Teaching Word Meanings. Mahwah, NJ: Erlbaum. Stanford University. Graduate School of Education. (n.d.) Understanding Language: Language,

Literacy and Learning in the Content Areas. Retrieved November 20, 2013 from http://ell.stanford.edu/

Stein, N. L. & Glenn, C. G. (1978). An analysis of story comprehension in elementary school children. In R. Freedle (Ed.), Discourse processing: Multidisciplinary perspectives (pp. 53-120). Hillsdale, NJ: Erlbaum.

Swales, J. M. (1990). Genre analysis: English in academic and research settings. Cambridge, UK: Cambridge University Press.

Taylor, N. A., Greenberg, D., Laures-Gore, J., & Wise, J. C. (2011). Exploring the syntactic skills of struggling adult readers, Reading and Writing: An Interdisciplinary Journal, 25, 1385-1402.

Toulmin, S. (1958). The Uses of Argument. Cambridge, UK: Cambridge University Press. Uccelli, P., Rosenthal, F. & Barr, C. D. (2011). Promising connections: Play, multimodality,

and more than one language in learning connectives at school. Poster presented at

http://ell.stanford.edu/


47

the Annual meeting of the American Educational Research Association, New Orleans, LA.

Uccelli, P., & Barr, C. D. (2011). Psychometric report of the academic language evaluation. Unpublished report.

Uccelli, P., Dobbs, C., & Scott, J. (2013). Mastering academic language: Organization and stance in the persuasive writing of high school students. Written Communication, 30(1), 36-62.

Valdés, G. (2004). Between support and marginalization: The development of academic language in linguistic minority children. International Journal Of Bilingual Education and Bilingualism, 7, 102-132.

van Lier, L., & Walqui, A. (2012, January). Language and the common core state standards. Paper presented at the Understanding Language Conference, Stanford, CA. Retrieved from http://ell.stanford.edu/papers/language.

Wong-Fillmore, L., & Fillmore, C. J. (2012, January). What Does Text Complexity Mean for English Learners and Language Minority Students? Paper presented at the Understanding Language Conference, Stanford, CA. Retrieved from http://ell.stanford.edu/papers/language.

Zeno, S. M., Ivens, S. H., Millard, R. T., & Duvvuri, R. (1995). The educator's word frequency guide. Brewster, NY: Touchstone Applied Science Associates.

http://ell.stanford.edu/papers/language

http://ell.stanford.edu/papers/language


48

TABLES AND FIGURES Table 1: Student Demographic Characteristics (n=235*)

n (%) Gender Female 114 (52%) Male 104 (48%) SES Free/reduced lunch 175 (80%) No free/reduced lunch 43 (20%) Ethnicity Black/African American 143 (66%) White 48 (22%) Latino/Hispanic 15 (7%) Asian 2 (1%) American Indian/Alaskan Native 1 (.5%) Two or more races 9 (4%) Language Status Classified as English Language

Learners 39 (18%)

Classified as English proficient 179 (82%) Language minority students 66 (28%) Special Education Status Classified as SPED 34 (15%) Not classified as SPED 184 (84%) Grade 4th 52 (22%) 5th 55 (23%) 6th 39 (17%) 7th 50 (21%) 8th 39 (17%) * Demographic data were unavailable for 17 students.


49

Table 2. Pairwise correlations between Core Academic Language Skills-Instrument tasks

CALS Subtests 1 2 3 4 5 6 7 8 9 10

1: Unpacking complex words 1

2: Comprehending complex sentences .5** 1

3: Connecting ideas .61** .45** 1

4: Track themes .4**

.4**

.5** 1

5: Organizing argumentative texts .58**

.23** .54**

.36**

1

Producing written academic definitions

6: Definition-Structural density .36**

.33** .43**

.29**

.33**

1

7: Definition-Superordinate precision .36**

.32** .44**

.3**

.4**

.69** 1

8: Definition-Informativeness .43** .33**

.46**

.34**

.48**

.63**

.55** 1

9:Total Definition score .50** .32**

.48**

.31**

.46**

.59** .64** 55** 1

10: Identifying academic definitions .42** .18** .29** .17* .38** .25** .23** .3** .23** 1

11: Fluency

.48** .19** .34**

.15*

.36** .24** .16* .29** .32**

.19**

*p < .05, ** < .01, *** < .0001


50

Table 3. Results from Principal Component Analysis for Core Academic Language Skills - Instrument Tasks

Variable Eigenvalue Explained Variation

Loading

Core AL Skills construct 3.44 .49

Unpacking complex words 0.81417 Comprehending sentences 0.64459 Connecting ideas 0.80623 Tracking themes and participants 0.66574 Organizing argumentative texts 0.72464 Producing academic definitions 0.68314 Identifying academic definitions 0.52233


51

Table 4: Descriptive statistics for Core Academic Language Skills Instrument (CALS-I), MCAS-ELA and Fluency scores by grade

Grade

Core Academic Language Skills-

Instrument (CALS-I)

MCAS-ELA Scaled Scores

Reading fluency

4th grade N Mean (s.d.) N Mean (s.d.) N Mean (s.d.) 52 .44 (.14) 48 234.54 (14.82) 48 98.36 (26.20)

5th grade 55 .56 (.19) 50 228.72 (10.45) 49 99.36 (22.70)

6th grade 39 .65 (.14) 35 229.09 (12.21) 36 105.88 (22.37)

7th grade 50 .72 (.12) 48 230.29 (14.36) 47 123.00 (28.81)

8th grade 39 .76 (.12) 37 230.59 (12.93) 37 121.46 (17.74)

Total sample 235 .61 (.19) 218 230.72 (13.14) 217 109.07 (25.94)


52

Table 5. Regression models by grade: Contribution of CALS-I scores to MCAS-ELA scores, controlling for reading fluency Sample Variable B (Coefficient) Standard

Error (S.E) P value R2

Reading fluency -.06 .04 .167 4th CALS-I 34.76 8.15 <.0001 (N=44) .33** Reading fluency .094 .05 .065 5th CALS-I 27.36 6.33 <.0001 (N=45) .47*** Reading fluency -.010 .04 .818 6th CALS-I 40.91 6.82 <.0001 (N=33) .56*** Reading fluency .008 .04 .857 7th CALS-I 37.94 10.04 <.0001 (N=45) .25** Reading fluency .083 .07 .257 8th CALS-I 55.08 10.24 <.0001 (N=35) .48*** *p < .05, ** < .01, *** < .00


53

FIGURES Figure 1: Series of histograms displaying the distribution of total CALS-I scores within grade

Grade 4

Grade 5

Grade 6

Grade 7

Grade 8


54

Figure 2: CALS-I task-specific mean scores per grade

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1. UnpackingWords

(morphology)

2. ComprehendingSentences

3. ConnectingIdeas (connectives)

4. Tracking Themes 5. OrganizingArgumentative

Texts

6A. IdentifyingWritten Academic

Definitions

6B. DefinitionProduction Score

(total)


55

1 The work reported here was part of the project entitled Catalyzing Comprehension through Discussion and

Debate. This project was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305F100026 awarded to the Strategic Education Research Partnership as part of the Reading for Understanding Research Initiative.

2 Of course, equally plausible is the case of highly-skilled academic language users who would struggle in other contexts, such as producing effective sportscasts.

3 Because academic audiences approach texts from a skeptical orientation, academic writing across disciplines manifests explicitly, through the use of language that conveys certainty or the citing of evidence, or implicitly, through the use of a distant, authoritative voice, a persuasive or argumentative agenda in the writer’s rhetorical choices (Rex, Thomas & Engel, 2010; Toulmin, 1958).

4 To minimize vocabulary knowledge and decoding demands beyond the tested targets, the surrounding vocabulary in the tasks was selected from a pool of words reported to be known by a majority of 4th graders (Dale & O’Rourke, 1981), giving preference to easily decodable words. Whenever possible, the impact of prior knowledge was reduced by selecting simple topics familiar to students from their everyday experiences of being children and going to school (e.g., taking a test, being sick and missing school). Finally, in our effort to measure language proficiency without involving high-inference reading comprehension processes, reading comprehension demands were minimized by first, reducing the length of texts included in the assessment (the longest text includes 5 sentences, but most items involve single sentences); and, second, by including items whose comprehension demands did not go beyond what Kintsch (2004) has dubbed the ‘surface level of comprehension’ and which, consequently, did not require high-level order processing. Finally, authentic sources (e.g., middle school textbooks, Time for kids) were used or adapted, whenever possible.

5 To avoid invalid inferences on students’ CALS, a percentage metric was used instead, because, due to students’ absences, not all students were administered all tasks.

6 The lowest average observed fluency scores were observed for ELLs and Special Education students, but their average fluency scores were still reasonably high with the average fluency for ELLs being 96.9 words per minute (WPM) (112.0 WPM for non ELLs) and the average fluency for Special Education students being 95.9 WPM (111.9 WPM for non Special Education students).

7 According to the state, students designated as ‘needs improvement’ manifest a modest reading vocabulary, have a partial understanding of abstract ideas in text, and show partial understanding of genre-specific text organization and produce partially organized compositions. MCAS-ELA thresholds establish proficiency levels (<210=warning, <230=needs improvement, <240 Proficient, >260 advanced).

8 We also conducted analyses that included the variable, special education status as well as the interaction between special education status and CALS-I score, to predict the MCAS score in each grade. We found no main effect except in grade 8, for which the effect was negative for both special education status and for the interaction between special education status and CALS-I score.

core academic language skills (cals): an expanded

Documents