oral proficiency standards & teacher candidate standards flan12030

7/27/2019 Oral Proficiency Standards & Teacher Candidate Standards Flan12030

1/26


2/26

IntroductionIn the eld of education, the past decadehas seen an increasing emphasis on teacheraccountability for student learning, at leastpartially in response to the national de-mand for highly quali ed teachers pro-posed through the No Child Left BehindAct (NCLB) (2001). The need to identifyteachers of high quality has resulted inmore rigorous standards related to thepreparation of teachers across disciplines,particularly with respect to the develop-ment of their content knowledge and skills(see, for example, NCATE, 2008). Thisnational endeavor to develop teacher stand-ards has prompted widespread discussionand research exploring the type of content speci c knowledge teachers should pos-sess. This discussion has been particularlyprevalent in the eld of mathematics (Ball,Thames, & Phelps, 2008; Kleickmann et al.,2013), but it has also occurred in mostother elds, including science (Lederman,1999) and physical education (Siedentop,2002).

Although dialogue concerning thecontent knowledge and skills of foreignlanguage teachers dates back to the late1980s (ACTFL, 1988), more recently threesets of teacher standards have sparked arenewed discussion regarding the contentknowledge and skills that foreign languageteachers should possess as they moveacross the career continuum from teacher

candidates to beginning teachers to accom-plished teachers. In 2002, NCATE ap-proved the ACTFL/NCATE ProgramStandards for the Preparation of ForeignLanguage Teachers (ACTFL, 2002) (here-after referred to as the ACTFL/NCATEProgram Standards), which de ne theexpectations for teacher candidates whocomplete a teacher preparation programand earn teacher certi cation. NCATE,

which is recognized by the U.S. Departmentof Education and the Council for HigherEducation as an accrediting agency forschools, colleges, and departments of education, determines which colleges of education meet rigorous national standards

for teacher preparation. Foreign languageteacher preparation programs that undergoNCATE review and seek national recogni-tion by ACTFL/NCATE must provideevidence that their programs are preparingcandidates who can demonstrate the per-formance described in the standards. Inconjunction with the development of theACTFL/NCATE Program Standards, in2002, the Interstate New Teacher Assess-ment and Support Consortium (INTASC 1 ),with support from ACTFL, designed itsModel Licensing Standards for BeginningForeign Language Teachers, which mayalso be used by participating states tolicense teachers and induct them into theprofession. To complete the career contin-uum, in 2001, the National Board forProfessional Teaching Standards (NBPTS)created the World Languages Other ThanEnglish Standards, which describe theknowledge and skills necessary for accom-plished language teachers who elect to earnNational Board certi cation; these stand-ards were updated in 2010 and renamed the World Languages Standards (NBPTS, 2001,2010).

One area that all three sets of standardsaddress is the need for teachers to demon-strate a high level of oral pro ciency in theforeign language to be taught in theclassroom, the target language (TL). 2 How-ever, the standards that have garnered themost attention in this regard are the ACTFL/

NCATE Program Standards, because theystipulate that foreign language teacherpreparation programs desiring nationalrecognition must assume responsibility fordetermining whether their teacher candi-dates reach speci c levels of oral pro ciencyas outlined in the standards.

The requirement that teacher candi-dates demonstrate the requisite level of pro ciency on the Oral Pro ciency Inter-

view (OPI) was the result of two years of professional dialogue, during which timeforeign language professionals participatedin extensive discussions at state, regional,and national levels before reaching consen-sus. According to Glisan (2013), the

Foreign Language Annals VOL. 46, NO. 2 265


3/26

standard regarding the oral pro ciency of teacher candidates re ects the voice of theforeign language education profession in-asmuch as it materialized through bottom up consensus building in the eld. However,since the publication of the ACTFL/NCATEProgram Standards in 2002 and their use byprograms of foreign language teacher prep-aration seeking national recognition byACTFL/NCATE, professional discussionhas ensued regarding (1) the extent towhich teacher candidates are achieving therequisite levels of oral pro ciency prior tocompleting their preparation programs (see,for example, Burke, 2013), and (2) the roleof the preparation program in assisting itscandidates in reaching these levels (Glisan,2013; Huhn, 2012). This discussion hastaken on increased importance as expect-ations de ning teacher candidates oralpro ciency have evolved into a high stakestesting requirement, not only for teacherpreparation programs seeking ACTFL/ NCATE recognition but also for teachercandidates who reside in the 23 states thatcurrently require a speci c level of oralpro ciency for teacher licensure 3 (Chamb-less, 2012).

A cogent professional dialogue regard-ing the success with which teacher candi-dates and programs of teacher preparationare meeting national oral pro ciency ex-pectations be they for purposes of pro-gram completion or state licensure must

be based on empirical data rather thananecdotal evidence and speculation. Up tothis point, however, there has been a dearthof published research examining the extentto which candidates in foreign languageteacher preparation programs are meetingthe ACTFL/NCATE pro ciency expect-ations. This study analyzes OPI data oversix years, from 2006 to 2012, in an effort todescribe the level at which foreign language

teacher candidates are meeting the ACTFL/ NCATE Oral Pro ciency Standard andto determine how these results may differacross languages, years in the sample,and university programs of teacherpreparation.

Oral Proficiency Expectations of ACTFL/NCATE ProgramStandards

The ACTFL/NCATE Program Standards forthe Preparation of Foreign Language Teach-ers establish the expectations that teachercandidates must demonstrate prior to com-pleting college or university programs of teacher preparation in foreign languages(ACTFL, 2002). To this end, foreignlanguage teacher preparation programsseeking national recognition must:

1. Require that their teacher candidatescomplete an ACTFL OPI, which isadministered and double rated throughLanguage Testing International (LTI),the ACTFL testing of ce. LTI schedules,administers, and reports ACTFL certi edratings for all academic, commercial, andgovernments testing clientele.

2. Establish exit levels of pro ciency accord-ing to the ACTFL/NCATE Program Stand-

ards

Advanced Low for most languagesor Intermediate High in Arabic, Chinese, Japanese, and Korean. The expected levelsof oralpro ciency are de ned according tothe ACTFL Pro ciency Guidelines 2012 Speaking (ACTFL, 2012).

The setting of a pro ciency level bylanguage was based on research conductedby the Foreign Service Institute (FSI) in the1980s regarding the length of time necessaryto reach speci c levels of oral pro ciency ina target language and considered howsimilar the TL is to the learner s nativelanguage (Liskin Gasparro, 1982). For ex-ample, learners will typically need con-siderably more instructional time to reachthe Advanced level of pro ciency in Arabicthan in Spanish when the learners nativelanguage is English (Liskin Gasparro,1982, as cited in Chambless, 2012, p.S144). The U.S. government developed asystem for grouping languages into fourcategories: Group I includes languages suchas French, Italian, Portuguese, and Spanish;Group II includes German, Greek, and

266 SUMMER 2013


4/26


5/26

interviewee s level of oral pro ciency ac-cording to the criteria of the ACTFLPro ciency Guidelines Speaking (ACTFL,2012). The OPI is a global assessmentinasmuch as it establishes a speaker s levelof consistent functional ability (patterns of strength) as well as the upper limits of thatability (patterns of weakness) and does notmeasure discrete pieces of knowledge aboutthe language (Swender & Vicars, 2012, p. 2).There are four major assessment criteriaused to rate an OPI:

Global tasks or functions performed inthe language. Asking and answering simple questions,narrating and describing, supporting opin-ions, etc.Contexts/Content Areas the sets of circumstances, linguistic or situational,in which these tasks are performed, andtopics that relate to these contexts.Context (in a restaurant in Mexico)Content (food and drink)The accuracy with which the tasks areperformed, i.e., the comprehensibility of the message.How grammar, vocabulary, and other language features affect the precision,clarity, and appropriateness of the message.Who can understand the message and whatdegree of empathy is required on the part of the interlocutor .The oral text type that is produced in the

performance of the tasks.Discrete words or phrases, sentences, para-graphs, or extended discourse(Swender &Vicars, 2012, p. 2).

As each speech sample is rated accord-ing to the criteria described above and not inrelation to the performances of other speak-ers, the OPI is a criterion referenced, ratherthan a norm referenced, assessment.

The Advanced Level on the OPIAs explained earlier, the various sets of national standards for foreign languageteachers include an expectation that teach-ers speak the TL at a minimum pro ciency

level of Advanced Low. In short, AdvancedLow speakers can:

Participate in most informal and someformal conversation on topics related toschool, home, and leisure activities andabout some topics related to employment,current events, and matters of public andcommunity interest;Narrate and describe in the major timeframes of past, present, and future inparagraph length discourse with somecontrol of aspect; 7

Handle appropriately the linguistic chal-lenges presented by a complication orunexpected turn of events in a socialsituation;Speak with suf cient control of basicstructures and generic vocabulary to beunderstood by native speakers of thelanguage, including those unaccustomedto nonnative speech (Swender & Vicars,2012, p. 42; see the Appendix for adetailed description of the Advanced Lowlevel).

As a rationale for requiring AdvancedLow, the ACTFL/NCATE standards presentthe need for teachers to speak in spontane-ous, connected discourse and be able toprovide the essential TL input and type of classroom environment that is necessary forlanguage acquisition to occur (ACTFL,2013, n.p.). Because the decision regarding

the acceptable minimum level of pro ciencycame about through professional consensusin our eld, it is widely believed that teacherswho cannot speak in connected discourseand in major time frames will not be effectivein planning, teaching and assessing commu-nication tasks in the three modes as de nedin the Standards for Foreign LanguageLearning in the 21st Century (NationalStandards, 2006), nor will they be able to

guide students in interacting with others ininterpersonal contexts.

As some states have established Inter-mediate High as the minimum level for statelicensure a requirement that has sparkedsome professional debate in the eld it is

268 SUMMER 2013


6/26

important to highlight the differences be-tween Intermediate High and AdvancedLow. Intermediate High speakers:

Converse with ease and con dence whendealing with the routine tasks and socialsituations of the Intermediate level;Handle successfully uncomplicated tasksand social situations requiring an ex-change of basic information related totheir work, school, recreation, particularinterests, and areas of competence;Handle a substantial number of tasksassociated with the Advanced level, butthey are unable to sustain performance of all of these tasks all of the time; i.e.,typically when they attempt Advanced level tasks, their speech exhibits one ormore features of breakdown, such as thefailure to carry out fully the narration ordescription in the appropriate major timeframe, an inability to maintain paragraph length discourse, or a reduction in breadthand appropriateness of vocabulary;Can generally be understood by nativespeakers unaccustomed to dealing withnonnatives, although interference fromanother language may be evident and apattern of gaps in communication mayoccur (Swender & Vicars, 2012, p. 61).

Foreign language teachers with Inter-mediate High pro ciency function mostcomfortably at the Intermediate level, using

sentence level speech in the present timeframe and speaking within concrete, every-day contexts and uncomplicated situations. When attempting the tasks across thecontexts and content areas of the Advancedlevel, while they may be able to perform atthe Advanced level much of the time, theyare not able to consistently sustain this levelat all times. A main distinguishing feature of Intermediate High speakers is that their

language demonstrates patterns of break-down, i.e., their linguistic accuracy declines,their uency suffers, they are unable toconsistently communicate their intendedmessage in the appropriate time frame, andthere are typically gaps in their compre-

hensibility. In sum, teachers with oralpro ciency at the Advanced level are ableto communicate both in larger quantity andat a higher level of quality. Their linguisticskills allow them to have the spontaneityand automaticity that teachers need tomodel and engage students in sustainedclassroom communication. Because theirspeech breaks down when attemptingmore complicated and sustained speech,teachers with pro ciency at the Intermediatelevel, even at Intermediate High, may lackspontaneity and automaticity, and, in fact,may not consistently communicate theintended message.

Review of the Literature on OralProficiency LevelsResearch examining the oral pro ciencylevels of foreign language teachers is scantat best, as acknowledged by Chambless(2012) in her recent literature review of this topic. In the 1980s and 1990s, severalstudies were published that examined theoral pro ciency levels of students aftercompleting a particular number of years of language study in high school (Glisan &Foltz, 1998; Huebner & Jensen, 1992) or incollege (Magnan, 1986; Tschirner & Heilen-man, 1998). However, the vast majority of this research did not include the use of of cial OPIs, conducted by certi ed OPItesters and double rated through LTI, nor

did it seek to examine the pro ciency levelsof teachers or teacher candidates. In what isperhaps the rst publication of of cial OPIratings, Swender (2003) reported the levelsof undergraduate foreign language majorsre ecting data from 501 of cial OPIs con-ducted between 1998 and 2002 in Mandarin,French, German, Italian, Japanese, Russian,and Spanish. These interviews were con-ducted either in the face toface format or via

telephone, and according to current ACTFLOPI testing protocol, they were double ratedand certi ed by ACTFL through LTI.Although the students who completed theseOPIs were all language majors in their junioror senior years, the data were not



7/26

disaggregated for teacher education candi-dates. Swender reported that the greatestconcentration of ratings (55.8%) occurred inthe Intermediate High/Advanced Low range,with 47% of language majors demonstratingoral pro ciencyat the Advanced Low level orhigher (2003, p. 523).

The rst large scale study examiningthe OPI scores of teacher candidates wasconducted by Swender, Surface, and Ham-lyn in 2007, using OPI data collected on3,198 teacher candidates from 10 states overa twoyear period, from 2005 to 2007. 8 Thecandidates in this study, which includedboth heritage and native speakers of the TL,had completed traditional programs of teacher preparation, and some were licensedby their states through alternative certi ca-tion routes. The study analyzed the numberof candidates who reached the ACTFL / NCATE required level as well as those whomet the state licensure level. Languagesincluded French, German, Italian, Manda-rin, and Spanish, as well as a category called other world languages that groupedtogether languages such as Arabic, Canton-ese, Haitian, Hebrew, Japanese, Korean,Polish, Portuguese, and Russian. OPI resultscon rmed that 59.5% of teacher candidatesachieved the ACTFL/NCATE requirementof Advanced Low (or Intermediate High inthe case of Group IV languages) on their rstattempt and that of those who did not meetthe level on the rst try, 38.5% met the

requirement on their second OPI attempt. Inaddition, the study illustrated that 85% of teacher candidates reached the minimumOPI level set by their respective states ontheir rst attempt. In analyzing data forindividual languages, the following repre-sent the percentages of candidates whoreached the Advanced Low level as requiredby ACTFL/NCATE: French: 73.3%,German: 72.8%, Italian: 84.8%, and Spanish:

81%. In Mandarin, 100% of teacher candi-dates achieved the ACTFL/NCATE mini-mum level of Intermediate High, possiblybecause they were primarily native orheritage speakers of the language, and inthe category of other world languages

92.2% achieved Advanced Low or higher,possibly for the same reasons (pp. 29 34).

Sullivan (2011) conducted theonly otherstudy that has reported OPI data speci callyfor teacher preparation candidates in anattempt to examine the strategies used byteacher candidates in preparation for takingthe OPI. Data from her survey of 734 teachercandidates in Spanish, Mandarin, French,English, Italian, German, Arabic, and Russianrevealed that 72% reported OPI ratings of Advanced Low or higher (p. 243). Thispercentage is much higher than the resultsof the Swender et al. (2007) study, perhaps atleast in part because 200 candidates reportedbeing native speakers of the TL and wouldpresumably obtain a high pro ciency rating.Further, it is important to note that OPIresults in Sullivan s study were self reportedby teacher candidates, rather than data thatwere obtained through LTI, i.e., not of cialOPI ratings. An interesting nding in Sulli-vans investigation was that the candidateswho reported having been successful onthe OPI reported spending an average of 19 hours per week using the TL outside of class, more than triple the number of hoursspent by candidates who were unsuccessful inreaching the requisite OPI level.

In sum, the principal ndings of the fewstudies that have examined OPI data fromforeign language teacher candidates are asfollows:

The majority of teacher candidates inthese studies have demonstrated pro -ciency at a minimum level of AdvancedLow, thereby meeting the ACTFL/ NCATE standard.There are differences across TLs withrespect to the number of candidatesachieving the requisite OPI levels.Of the teacher candidates who do notreach the ACTFL/NCATE level on their

rst attempt at the OPI, it appears that apercentage of them are able to reach thelevel in a second OPI.There is some evidence that particularstrategies for preparing teacher candidatesto succeed on the OPI may have a role to

270 SUMMER 2013


8/26

play in whether or not they demonstrateoral pro ciency at the requisite level.

The current study seeks to build on theSwender et al. (2007) study by analyzingOPI data trends from a longer six yearperiod, investigating factors not examinedearlier (e.g., attainment of the Oral Pro -ciency Standard across university teacherpreparation programs) and deriving futureresearch needs.

The Study

As the primary focus of this study was toascertain how well foreign language teachercandidates met the ACTFL/NCATE OralPro ciency Standard, the rst researchquestion was:

RQ1. How many teacher candidates metthe ACTFL/NCATE Oral Pro ciencyStandard for their language for years2006 2012?

In view of the fact that languages differ interms of time required to reach certain levelsof pro ciency as noted above, the secondresearch question sought to determinewhether teacher candidates in certain lan-guages were more likely to meet thestandard for their language:

RQ2. Are there signi cant differencesacross languages in the rates at whichteacher candidates met the appropriate

ACTFL/NCATE Oral Pro ciencyStandard?

Given that it has been nearly a decade sincethe release of the ACTFL/NCATE ProgramStandards, the study also sought to deter-mine whether there have been changes overtime in the number of teacher candidateswho met the oral pro ciency requirement.Therefore, the third research questionaddressed the extent to which teachercandidates met the appropriate ACTFL/ NCATE Oral Pro ciency Standard over thecourse of the 2006 2012 period:

RQ3. How have the rates at whichteacher candidates have met the appro-

priate ACTFL/NCATE oral pro ciencystandards changed over time from 2006to 2012?

Finally, in addition to exploring how manycandidates met the ACTFL/NCATE Pro-gram Standards requirement, the study alsoexamined possible differences in candidates attainment of the standard across universityteacher preparation programs:

RQ4. Are there signi cant differences inthe number of candidates attaining theACTFL/NCATE Oral Pro ciency Stan-dard across university programs of teacher preparation?

MethodologySample and Data CollectionACTFL provided OPI data for 2,881 foreignlanguage teacher candidates who testedfrom 2006 to 2012 as part of meeting theACTFL/NCATE Oral Pro ciency Standardfrom 48 identi ed university teacher prepa-ration programs. Because 734 candidates didnot report their university af liation, thesetest results were notused foranalyses relatedto program results, although these datawere used in all other analyses. Teachercandidates tested in 11 different languages:Arabic, Mandarin, French, German, He-brew, Italian, Japanese, Korean, Portuguese,Russian, and Spanish (see Table 1). Ninecandidates tested in more than one lan-guage. For these cases, each language wastreated as a separate test taker. Thus, therewere 2,890 tests included in the analysis forRQ1 3 and 2,156 included in the analysisfor RQ4.

MeasuresACTFL OPI and Advanced Level Check The ACTFL OPI is the test of record for the

ACTFL/NCATE Oral Pro ciency standard.All OPIs in the sample were conducted andrecorded by ACTFL certi ed OPI testers inthe respective languages and were ratedaccording to the ACTFL Pro ciency Guide-lines (1999, 2012). All of cial ACTFL OPIs



9/26

administered through LTI receive twoindependent ratings to con rm the leveland maintain interrater reliability; in cases of disagreement on the two ratings, a thirdrating is obtained. This procedure has beenfound to yield highly reliable ratings of oralpro ciency (Surface & Dierdorff, 2003;SWA Consulting Inc. 2012).

Of the 2,890 OPIs in the sample, 2,828were conducted via telephone, scheduled byLTI, conducted by an ACTFL certi ed OPItester, rated by the tester, and then indepen-dently second rated by a different ACTFL

certi ed tester for a blind second rating. Theremaining 62 tests were academic up-grades, in which a certi ed tester at an

academic institution conducted the inter-view in a face to face format, assigned anadvisory rating, and submitted the recordedinterview to ACTFL for a blind secondrating. Both telephonic interviews andacademic upgrades are consideredof cial OPIs and were treated as such inthis study.

Of the 2,890 interviews, 2,396 were fullOPIs, and 494 were Advanced Level Checks,a truncated version of the OPI designed todetermine whether or not a candidate hasmet the Advanced Low level (see Table 2).The outcome of the Advanced Level Checkis stated in terms of quali ed ( Q ) if thecandidate meets or exceeds an AdvancedLow level of speaking pro ciency or notquali ed ( NQ ) if the candidate s speechdoes not meet the criteria for Advanced Low(Language Testing International [LTI],2013). It is important to note that, becausethe goal of the Advanced Level Check is todetermine whether or not the candidate hasmet the minimal level of Advanced Low, theinterview does not yield results to determinethe exact OPI rating that is, how far belowor above the Advanced level the candidate sspeaking ability actually is. However, fordescriptive purposes, ACTFL often classi esa rating of Q as Advanced Low and NQ asIntermediate High. Figure 1 depicts theoverall distribution of OPI and AdvancedLevel Check ratings in the study using thisconvention. Figure 2 illustrates the distribu-

tion of OPI ratings only. Because theAdvanced Level Check does not result in aspeci c OPI rating, these data were not

TABLE 1

Languages Tested

Language Sample

Arabic 9Chinese (Mandarin) 142French 400German 113Hebrew 1Italian 105 Japanese 8Korean 1Portuguese 4

Russian 13Spanish 2,094

n 2,890

TABLE 2

OPI and Advanced Level Checks 2006 2012

2006 2007 2008 2009 2010 2011 2012 TotalOPIs 141 261 326 365 521 403 379 2,396Advanced Level Checks 39 89 99 52 73 68 74 494Total 180 350 425 417 594 471 453 2,890

272 SUMMER 2013


10/26

included in analyses that examined actualOPI ratings of candidates.

Meeting StandardThe research questions focus on whether ornot candidates attained the standard (Ad-

vanced Low or Intermediate High) for theirrespective languages on the ACTFL OPI orAdvanced Level Check. Therefore, theratings on these assessments were dichoto-mized into a two level dependent variable(met standard, did not meet standard) for

FIGURE 2

ACTFL OPI Score Distribution for Teacher Candidates in Sample

n 2396. Sample contains testing data from 2006 2012 and across test takers regardless of whether they indicated a specific program when testing. This excludes the 494 Advanced LevelChecks included in the overall sample.

FIGURE 1

ACTFL OPI and Advanced Level Check Score Distribution for TeacherCandidates in Sample

n 2,890. Sample contains testing data from 2006 2012 and across test takers regardless of whether they indicated a specific program when testing. This includes 494 Advanced LevelChecks, using the convention of coding Q as AL and NQ as IH.



11/26

analysis. If a candidate received AdvancedLow or higher in French, German, Hebrew,Italian, Portuguese, Russian, and Spanish,the candidate was coded as meeting stan-dard, whereas a candidate who received arating of Intermediate High or lower in theselanguages was coded as not meeting stan-dard. In Arabic, Mandarin, Japanese, andKorean, candidates who were rated Inter-mediate High or above were coded asmeeting the standard while candidateswhose scores were Intermediate Mid orbelow were coded as not meeting thestandard. The use of the dichotomousdependent variable permitted inclusion of the data from OPIs and Advanced LevelChecks in the analyses.

Categorical Independent VariablesSeveral categorical independent variableswere included in this study to determinewhether or not structural factors in uencedstandard attainment in the sample. Speci -cally, language, year tested, and universityprogram af liation were investigated. Thesefactors may be proxies for unmeasuredfactors that in uence pro ciency or stan-dard attainment. This in uence is acknowl-edged in the ACTFL/NCATE OralPro ciency Standard to some degree inthat differences in the time it takes a nativespeaker of English to learn the language(language learning dif culty) is incorporat-ed into the pro ciency level set for lan-

guages. Although year tested could betreated as a continuous independent vari-able and actually was in one analysis(correlation for RQ3) all three variableswere treated as categorical independentvariables for the main analyses. Languageand university program were nominalvariables that is, there was no intrinsicordering of the categories. The numbersindicate category membership; higher num-

bers do not indicate more of something,but merely a new category. Treating yeartested as a categorical variable in our mainanalyses was used to determine whetherdifferences between years were blips ortrends.

At the university program level, thecategorical variable of whether a universityprogram was a public or private institutionwas included in a post hoc analysis for RQ4.The university program level variables werecontinuous and calculated from the data inthe study the number of years theircandidates had been testing, the overallnumber of candidates tested, and thenumber of languages tested.

Analytic ProcedureThe choice of a dichotomous dependent

variable (two levels)

met standard or didnot meet standard had an impact on theanalytic procedure choices.

RQ1To address the rst research question,frequencies are presented for candidateswho met and did not meet the ACTFL/ NCATE oral pro ciency standard. In addi-tion, because the university program was

not indicated for some candidates, resultsare presented in two categories: institutionknown and institution unknown. To deter-mine whether the form in which theassessment was taken (OPI or AdvancedLevel Check) had a statistically signi canteffect on candidates success in meeting theACTFL/NCATE Oral Pro ciency Standardin this sample, a chi square analysis wasconducted. The same approach was used todetermine whether there was a statisticallysigni cant difference in standard attainmentfor candidates with identi ed institutionsand those candidates without.

RQ24To address RQ2 4, a series of logisticregressions was conducted to investigate if pre existing categorically independent var-iables (language, year tested, and program)

signi cantly in uenced the odds of whethercandidates met or did not meet the ACTFL/ NCATE Oral Pro ciency Standard. Logisticregression can handle any type of indepen-dent variable and a dichotomous dependentvariable (i.e., two categories; met standard or

274 SUMMER 2013


12/26

did not meet standard). Logistic regressionprovides an omnibus test of model t on thechi square distribution ( x 2 ) to determine if there is a statistically signi cant overalleffect of the independent variable or varia-bles in the model (i.e., p value for theomnibus x 2 is < or 0.05). Logistic re-gression has certain advantages over othertechniques used with dichotomous depen-dent variables (e.g., chi square analysis of anindividual independent variable), such asallowing the researcher to control for otherfactors and to approximate the percentage of variance in the dependent variable ac-counted for by each independent variable(i.e., McFadden s Rhosquared). Given theRQ2 4 goals and the data, logistic regressionwas an appropriate method of analysis(Cohen, Cohen, West, & Aiken, 2003).

For RQ3, because year can be treated asa continuous variable as well a categoricalvariable, a correlation analysis ( r ) wasconducted to determine whether year wasrelated to standard attainment. A statisti-cally signi cant r coef cient would indicatea signi cant increasing (positive sign) ordecreasing (negative sign) trend in attain-ment. The logistic regression allowed for theinvestigation of the effect of each year.

For RQ2 4, the language, year, anduniversity program were used as categoricalvariables in the logistic regressions. Thesevariables were recoded and entered into thelogistic regression as a series of dichotomous

variables for each category or level inthe variable (e.g., for language: whethera candidate tested in French, whether acandidate tested in German, etc.), except forthe category or level chosen as the referentgroup. For language, for example, eachcandidate would receive a 1 for the languagetested and 0 for all other languages, unless itwas the referent group. In analysis, thereferent group (Spanish in the case of

language) did not receive a dichotomousvariable as did the other categories or levels(other languages in this example) because itserved as a category or level against whichthe others were compared. Members of the referent group received 0 for all the

dichotomous independent variables createdfrom the other categories (all the otherlanguages in this example). This recodingapproach is standard practice in logisticregression with categorical independentvariables with multiple levels (Cohenet al., 2003).

For RQ2, Spanish was used as thereferent group because it was the largestlanguage in terms of tests in the sample. ForRQ3, 2006 was used as the referent groupbecause it was the rst year of testing in thesample. For RQ4, the largest single universityprogram by candidates who tested in thesample was used as the referent group. For thelogistic regression results, the interpretationof p values for the signi cance of thedichotomous independent variable s effecton the odds of the dependent variable is inreference to the referent group s relationshipwith the dependent variable. Continuing ourlanguage example above, the p values pre-sentedforallother languageswere interpretedin comparison to the referent group (Span-ish). Given our focus on description (notprediction), the p values along with thepercentages of candidate attainment shouldbe interpreted to indicate whether candidatesin each language met the standard at asigni cantly higher or lower percentagethan candidates in Spanish.

For RQ4, a post hoc multilevel logisticregression was conducted to explore theimpact of several university program level

variables public/private status, the numberof years their candidates have been testing, theoverall number of candidates tested, and thenumber of languages tested. Multilevel logis-tic regression is similar to logistic regressionexcept it acknowledges that data can bein uenced by hierarchical structure, such ascandidates being nested within universityprograms. A two level model splitting thevariance in candidate attainment (dependent

variable) between the university program andcandidate levels was conducted. This modelallowed for approximating variance associat-ed with university program membership andthe individual candidates. It also focused theindependent variables on the appropriate



13/26

level of variance such as program levelindependent variables predicting program level variance in attainment providing amore appropriate test of the in uence of program level variables.

ResultsRQ1. How many teacher candidatesmet the ACTFL/NCATE OralProficiency Standard for their language for 2006 2012?As seen in Table 3, of the 2,890 tests

administered from 2006 to 2012, 54.8%(n 1,584) met the appropriate ACTFL/ NCATE Oral Pro ciency Standard for theirlanguage. For those who indicated theuniversity they attended, 52.2% met thestandard, while 62.3% of those who did notindicate an institution met the standard.This nding represented a statisticallysigni cant difference in attainment betweenthe two groups ( x 2 (1) 22.261,

p < 0.001). Across languages for which arating of Advanced Low ACTFL/NCATEOral Pro ciency is required, a signi cantstatistical difference was found between thepercentage of candidates meeting the stan-dard on the ACTFL Advanced Level Checkand those candidates meeting standardon the ACTFL OPI (x 2 (1) 18.857, p < 0.001). This nding indicates that agreater proportion of candidates achieved

the standard on the ACTFL Advanced LevelCheck (62.2%; n 298 of 479) than on theACTFL OPI (51.3%; n 1,155 of 2,251).

RQ2: Are there significant differencesacross languages in the rates at whichteacher candidates met the

appropriate ACTFL/NCATE OralProficiency Standard?Signi cant differences in the rates at whichcandidates attained the standard were foundacross different languages ( x 2 (10) 155.013, p < 0.001). As seen from Table 4,the rate of attaining the standard forSpanish the referent group in thisanalysis was $ 51%. Mandarin, French,and Italian had signi cantly higher attain-

ment rates than Spanish (all p values< 0.001), while German and Portuguesehad a similar attainment rate as Spanish( p 0.248, 0.329, respectively; i.e., not astatistically signi cant difference). Due tolow sample sizes, results were inconclusivefor the other languages. Of note, the ndingsfor languages with the same underlyingstandard (Advanced Low) as Spanish weremixed, with some having similar levels of

standard attainment (e.g., German) andothers having signi cantly higher levels of standard attainment (e.g., French).

RQ3: To what extent have the rates atwhich teacher candidates met theappropriate ACTFL/NCATE OralProficiency Standard changed over time from 2006 to 2012?At rst glance, the data in Table 5, Table 6,and Figure 3 seem to suggest that the rates of meeting the standard appeared to be

TABLE 3

Results for Meeting the ACTFL/NCATE Oral Proficiency Standard 2006 2012

Sample Met Standard Did Not Meet Standard

n % n %

Combined 1,584 54.8% 1,306 45.2%Institution Known 1,126 52.2% 1,030 47.8%Institution Unknown 457 62.3% 277 37.7%

n 2,890. Includes ACTFL OPIs and Advanced Level Checks.

276 SUMMER 2013


14/26

TABLE 4

Percent of Sample Meeting the ACTFL/NCATE Oral Proficiency Standard byLanguage Category: 2006 2012

Sample Met Standard Did NotMeet Standard

Comparison toSpanish

n % n % p value

Arabic 1 11.1% 8 88.9% 0.047Chinese (Mandarin) 126 88.7% 16 11.3% < 0.001French 256 64% 144 36% < 0.001German 51 45.1% 62 54.9% 0.248Hebrew 0 0% 1 100%

Italian 81 77.1% 24 22.9%

oral proficiency standards & teacher candidate standards flan12030

Documents