research in the schools - fall 2012

Anthony J. Onwuegbuzie and John R. Slate, Editors

Volume 19, Number 2 Fall 2012 Editorial: Evidence-based Guidelines for Avoiding Reference List Errors in Manuscripts Submitted to Journals for Review for Publication: A Replication Case Study of Educational Researcher ............................................................................................................................... i - xvi

Anthony J. Onwuegbuzie, Eunjin Hwang, Julie P. Combs, and John R. Slate Foreign Language Immersion Programs and School Policy: Conflicting Agendas ................. 1-16

Heather K. Olson Beal, Michelle Haj-Broussard, and Nicole Boudreaux Factors Predicting Pre-service Teachers’ Adoption of Web 2.0 Technologies ....................... 17-29

Jongpil Cheon, Fanni Coward, Jaeki Song, and Sunho Lim The Revised School Culture Elements Questionnaire: Gender and Grade Level Invariant? .............................. 30-44

Thomas A. DeVaney, Nan B. Adams, Flo Hill-Winstead, and Mitzi P. Trahan Examining Variations in Programmatic Delivery on Teaching Candidates’ Sense of Efficacy .................................................................................................................................................. 45-61

S. Michael Putman Research Teaching Pedagogy: Lessons Learned From an Arabic Language Intervention Pilot Study ........................................................................................................................................ 62-74

Thomas W. Christ and Hosam Elmetaher Unpacking the Effects: Identifying School and Teacher Factors and Their Influence on Teachers’ Intentions to Stay or Leave the Profession ............................................................................. 75-89

Amy L. Sedivy-Benton and Carrie J. Boden-McGill

RESEARCH IN THE

SCHOOLS

A nationally refereed journal sponsored by the Mid-South Educational Research Association and

Sam Houston State University

MSERA

Research in the Schools Information for Authors

Statement of Purpose RESEARCH IN THE SCHOOLS (ISSN 1085-5300) is a nationally/internationally refereed journal co-sponsored by the Mid-

South Educational Research Association and Sam Houston State University. The journal publishes original contributions

in the following areas: (1) Research in Practice--empirical studies focusing on the results of applied educational research

including cross-cultural studies, (2) Topical Articles--scholarly reviews of research, perspectives on the use of research

findings, theoretical articles, and related articles, (3) Methods and Techniques--descriptions of technology applications in

the classroom, descriptions of innovative teaching strategies in research/measurement/statistics, evaluations of teaching

methods, and similar articles of interest to instructors of research-oriented courses, (4) Assessment--empirical studies of

norm-referenced, criterion-referenced, and informal tests in the areas of cognitive ability, academic achievement,

personality, vocational interests, neuropsychological functioning, and the like, (5) Educational Policy, Reform, and

Accountability–empirical studies or conceptual pieces focusing on educational policy, school and educator accountability,

politics of education, or educational responses to reform mandates, and (6) Other topics of interest to educational

researchers. RESEARCH IN THE SCHOOLS is devoted to research conducted in any educational setting from a conventional

elementary school or high school to a training program conducted within an industry. Likewise, there are no age

restrictions on the sample, since the educational settings may include preschools, continuing education classes for adults,

or adaptive skills courses in nursing homes. Studies conducted in settings such as clinics, hospitals, or prisons are

ordinarily inappropriate for RESEARCH IN THE SCHOOLS unless they involve an educational program within such a setting.

One goal of RESEARCH IN THE SCHOOLS is to provide a training ground for graduate students to learn effective reviewing

techniques. Consequently, the journal utilizes a Graduate Student Editorial Board composed mostly of students in

educational psychology and educational research. Members of this Editorial Board, each sponsored by a professor,

provide supplementary reviews for a selection of submitted articles, and receive both direct and indirect feedback of the

quality of these reviews.

Submission of Manuscripts Manuscripts should be submitted electronically to Anthony J. Onwuegbuzie, Co-Editor, RESEARCH IN THE

SCHOOLS. Please direct questions via email to [email protected]. All manuscripts will undergo a rigorous

internal review process to ensure that the manuscript is within the scope of RITS, is of sufficient quality, and conforms to

sixth edition APA style guidelines. Only manuscripts that satisfy all of the following requirements will be sent out for

external review:

Manuscripts that have been published elsewhere in part or in whole will not be considered for publication in

RITS. We will not consider any manuscripts that have been published in such arenas as conference

proceedings, yearbooks, online sources, journals, and book chapters. Only original unpublished works will be

considered for review in RITS. Manuscripts of which only the abstract has been published previously may be

submitted for review.

Length of the manuscripts, including references and tables, should ordinarily range from about 10 to 40 typed,

double-spaced, 8-1/2 X 11-inch pages, using 12-point type.

Abstracts are limited to 120 words.

The title page should provide title, names, and affiliation of all authors, and contact information of one

corresponding author.

Tables should be created using the autotable format (using the "Table" menu in Word) such that no column

lines (i.e., vertical lines) are used in the tables; and only row lines (i.e., horizontal lines) are used at the top,

bottom, and heading of the table (cf. APA, 2010, pp. 127-150).

Tables and figures should not be embedded within the body of the manuscript; rather, all tables should appear

(one table on each page) after the reference list, followed by all figures (one figure per page) (cf. APA, 2010,

pp. 127-150).

Brief reports of research are not encouraged.

All manuscripts submitted for review should be original material, not published in part or in whole in any other

publication. Moreover, no manuscripts submitted for review should be under review at any other journal.

Authors are encouraged to keep an electronic copy of the manuscript to guard against loss. The editorial team will "blind"

all manuscripts before sending them for external review. Review time is 3-4 months.

Copyright and Permissions Authors are granted permission to reproduce their own articles for personal use. Others must request permission to

reproduce tables, figures, or more than 500 words of text from the editors. Copyright © 2012 by the Mid-South

Educational Research Association.

mailto:[email protected]

mailto:[email protected]

RESEARCH IN THE SCHOOLS

Volume 19, Number 2 Fall 2012

EDITORS

Anthony J. Onwuegbuzie and John R. Slate


COPY EDITOR Gail H. Hughes

University of Arkansas, Little Rock

EDITORS EMERITI

James E. McLean

The University of Alabama

Alan S. Kaufman

Yale University School of Medicine

PRODUCTION EDITOR

Eunjin Hwang


EDITORIAL BOARD

Amy B. Dellinger, D.A.T.A., LLC

Angela Gibson, American Public University System

Bethany Bell, University of South Carolina

Bruce Thompson, Texas A & M University and Baylor

College of Medicine (Houston)

Cecil R. Reynolds, Texas A & M University

Charles M. Achilles, Seton Hall University and Eastern

Michigan University

Charles McLafferty, Purpose Research, LLC

Cindy Benge, Sam Houston State University

Claire Meljac, Hospital Sainte-Anne (France)

Daniel Krenzer, Mesa County Valley School District

David Segal, University of Central Florida

David T. Morse, Mississippi State University

Debbie Hahs-Vaughn, University of Central Florida

Dianne Swain, The University of West Alabama

Dimiter Dimitrov, George Mason University

James M. Ernest, University of Alabama at Birmingham

Jane Nell Luster, Louisiana State University

Janet C. Richards, University of South Florida

Jennifer Styron, University of South Alabama

Jennifer Purcell, Touro University

Jerrid Freeman, University of Arkansas

Jim Flaitz, University of Louisiana

Julie P. Combs, Sam Houston State University

Jwa K. Kim, Middle Tennessee State University

Kathleen Campbell, Southeastern Louisiana University

Kathleen M. T. Collins, University of Arkansas at

Fayetteville

Keith Hyatt, Western Washington University

Kevin Kieffer, Saint Leo University

Linda Zientek, Sam Houston State University

Lynn Loftin, Southern University

Margaret L. Rice, The University of Alabama

Mark Baron, University of South Dakota

Michelle Brown, Texas A&M, Kingsville

Michael D. Richardson, Southeastern Louisiana

University

Nancy L. Leech, University of Colorado Denver

Nataliya V. Ivankova, University of Alabama at

Birmingham

Paula Stanley, Radford University

Qaisar Sultana, Eastern Kentucky University

Randy Parker, Louisiana Tech University

R Burke Johnson, University of South Alabama

Robert Kennedy, University of Arkansas for Medical

Sciences

Ronald Styron, The University of South Alabama

Russell Mays, Georgia Southern University

Sandra Acosta, Texas A&M University

Shujie Liu, Northeast Normal University, China

Susan Troncoso Skidmore, Sam Houston State University

Thomas W. Christ, University of Bridgeport-Carlson

Tracy Goodson-Espy, Appalachian State University

William A. Spencer, Auburn University

GRADUATE STUDENT EDITORIAL BOARD Brandi Van Horn, University of Denver

Christina Hagerty, Sam Houston State University

Christy Wilmore, Sam Houston State University

Heather Elliot, University of Michigan

Megan Jones, Sam Houston State University

Robert Horton, Sam Houston State University

Copyright 2012 by the RESEARCH IN THE SCHOOLS

Mid-South Educational Research Association 2012, Vol.19, No. 2, i-xvi

Fall 2012 i RESEARCH IN THE SCHOOLS

Editorial: Evidence-based Guidelines for Avoiding Reference List Errors in Manuscripts

Submitted to Journals for Review for Publication: A Replication Case Study of Educational

Researcher

Anthony J. Onwuegbuzie, Eunjin Hwang, Julie P. Combs, and John R. Slate


In a previous editorial, Onwuegbuzie,

Hwang, Frels, and Slate (2011) reported the

findings of Onwuegbuzie and Hwang (2012) (see

also, Onwuegbuzie & Hwang, 2013), who

examined the frequency and characteristics of

violations to the American Psychological

Association (APA) style guide (APA, 2010) that

occurred in the reference lists of manuscripts

(hereafter referred to as reference list errors)

initially submitted (i.e., unpublished manuscripts)

to Research in the Schools (RITS), a

nationally/internationally refereed journal, as well

as the relationships between reference list errors

and selected manuscript variables (e.g., number of

authors, editor decision [e.g., reject, accept]).

Onwuegbuzie and Hwang (2012, 2013) used mixed

research techniques to examine 131 manuscripts

submitted to RITS over a 6-year period (i.e., 2004-

2010). These researchers identified a total of 466

unique reference list errors that were identified

across these 131 manuscripts, with the prevalence

of each of these reference list errors ranging from 1

(0.75%) to 102 (76.7%). Also, Onwuegbuzie and

Hwang (2012, 2013) documented an average of 12

reference list errors per manuscript (M = 12.83, SD

= 7.25).

Further, Onwuegbuzie and Hwang (2012,

2013) conducted a qualitative analysis (i.e.,

constant comparison analysis; Glaser, 1965; Glaser

& Strauss, 1967) of the identified 466 reference list

errors that yielded the following 14 reference list

error themes: (a) General errors; (b) Reference

heading, (c) Names of authors, (d) Publication

year/date, (e) Title of work, (f) Publisher

information, (g) Source of journal/periodical, (h)

Source of authored book, (i) Source of edited book,

(j) Source of website, (k) Source of paper

presentation, (l) Source of dissertation/thesis, (m)

Correspondence concerning this article should be

addressed to Anthony J. Onwuegbuzie, Department

of Educational Leadership and Counseling, Box

2119, Sam Houston State University, Huntsville,

Texas 77341-2119, or E-Mail:

[email protected]

Onwuegbuzie, Hwang, Frels, and Slate (2011) reported the findings of Onwuegbuzie and Hwang

(2012, 2013), who examined the frequency and characteristics of violations to the American

Psychological Association (APA) style guide (APA, 2010) that occurred in the reference lists of 131

manuscripts submitted to Research in the Schools over a 6-year period. Findings revealed that authors

committed more than 12 reference list errors per manuscript, on average (M = 12.83, SD = 7.25).

Further, a total of 466 unique reference list errors were identified, which yielded 14 reference list

error themes. However, it could be questioned whether the same reference list error rates would be

observed among manuscripts submitted to Tier I journals. Consequently, in the present editorial, we

replicate and extend their work by using mixed analysis techniques to examine the reference list error

rate of 83 manuscripts submitted to the highest ranked educational journal, Educational Researcher,

over a 3.5-year period. Findings revealed that Educational Researcher authors committed more than

14 reference list errors per manuscript, on average (M = 14.25, SD = 8.05). Also, a total of 324

unique reference list errors were identified, which yielded the same aforementioned 14 reference list

error themes. A multiple regression analysis revealed that every author of a manuscript was

associated with an increase of 2.40 reference list errors, on average. These findings support the

hypothesis that journals with the highest impact factors also have manuscripts submitted to their

journals that have high rates of reference list errors. Implications are discussed.

EDITORIAL: EVIDENCE-BASED GUIDELINES FOR AVOIDING REFERENCE LIST ERRORS IN

MANUSCRIPTS SUBMITTED TO JOURNALS FOR REVIEW FOR PUBLICATION: A REPLICATION

CASE STUDY OF EDUCATIONAL RESEARCHER

Fall 2012 ii RESEARCH IN THE SCHOOLS

Source of newspaper article, and (n) Source of

government document. An exploratory factor

analysis of these 14 reference list error themes led

to the identification of four meta-themes that

contained between two and five reference list error

themes.

Onwuegbuzie and Hwang (2012, 2013) also

conducted a latent class analysis of the six

reference list error themes that were committed by

the majority (i.e., > 50%) of the 131 authors, which

revealed two distinct clusters of manuscripts, with

one cluster (comprising 57.1% of the manuscripts)

being relatively high with respect to all six

reference list error themes, and the other cluster

(comprising 42.9% of manuscripts) being high on

two reference list error themes and low on the

remaining four reference list error themes.

Finally, Onwuegbuzie and Hwang (2012,

2013) conducted three sets of quantitative analyses,

namely, a correlation analysis, independent

samples t test, and canonical correlation analysis.

Specifically, with respect to the correlation

analysis, a series of nonparametric (i.e., Spearman)

correlations, after applying the Bonferroni

adjustment to control for the inflation of Type I

error, revealed that the number of reference list

errors was statistically significantly and positively

related to (a) the number of citation errors (rs[131]

= .39, p < .001), suggesting a moderate-to–large

relationship (Cohen, 1988); and (b) the length of

manuscript (rs[131] = .23, p < .001), suggesting a

small-to-moderate relationship (Cohen, 1988). The

independent samples t test revealed that

manuscripts that were not accepted by the editor

(M = 13.70, SD = 7.43)—that is, they were either

rejected or received a revise-and-resubmit

decision—contained statistically significantly more

reference list errors than did manuscripts that were

accepted (M = 9.44, SD = 4.95), with a large effect

size of 0.83 (Cohen, 1988).

A canonical correlation analysis revealed a

multivariate relationship between the 14 reference

list error themes and selected demographic

variables. Specifically, this multivariate

relationship was mainly characterized by the

relationship between reference list errors associated

with publisher information, source of

dissertation/thesis, and source of edited book on the

one side, and number of authors and length of

manuscript on the other side.

However, although some of the authors who

publish articles in RITS are among the most

prolific, it is likely that a much greater proportion

of prolific authors submit their manuscripts to

journals with the highest impact factors. And,

assuming that prolific authors tend to be the most

experienced authors, it is reasonable to hypothesize

that journals with the highest impact factors would

have manuscripts submitted to their journals that

have a significantly lower rate of reference list

errors. Yet, to date, this hypothesis has not been

tested empirically. Consequently, in this editorial,

we replicate and extend the works of Onwuegbuzie

and Hwang (2012, 2013) and Onwuegbuzie,

Hwang, et al. (2011) by examining the frequency

and characteristics of reference list errors among

manuscripts submitted to a top Tier I educational

journal.

Sources of Evidence

In order to examine the frequency and

characteristics of reference list errors among

manuscripts submitted initially to a top Tier I

educational journal, we conducted a mixed research

study of 83 manuscripts submitted to Educational

Researcher over a period of 3.5 years. The journal

Educational Researcher was selected because not

only does it represent the premier flagship journal

of the American Educational Research Association

but also it represents the educational journal with

the highest impact factor—specifically, its impact

factor of 3.774 makes it the highest ranking among

177 journals representing education and education

research. Two of the authors of this editorial were

part of the editor team (i.e., editor and associate

editor) of Educational Researcher (2006-2010) that

secured this extremely high impact factor. As such,

they had complete access to every manuscript

submitted to Educational Researcher during this

period. The 83 manuscripts selected for study

represented those manuscripts that were submitted

for the first time to the Research News and

Comment section of Educational Researcher—one

of two sections at that time (with the other section

being called Features that was co-edited by Drs.

Patricia B. Elmore and Gregory Camilli). Further,

these 83 manuscripts represented those manuscripts

that had not undergone what is referred to as a desk

reject or internal rejection during the internal

review process (i.e., before the manuscript is sent

out for external review, the manuscript was deemed

inappropriate for Educational Researcher because

it had a focus or content that was outside the scope

of the journal [e.g., the topic did not pertain to an

educational issue]; did not follow adequately the

stipulated format for manuscripts [e.g., the

manuscript resembled more of a traditional

empirical report rather than an essay]; or the

manuscript was written in a style that was not

adequately consistent with APA [e.g., the

manuscript followed Chicago Manual of style;

Chicago Manual, 2003; no discernible style]). In

other words, each of the 83 manuscripts had

satisfied the criteria for being sent out for external

review. These 83 manuscripts represented 49.11%

ANTHONY J. ONWUEGBUZIE, EUNJIN HWANG, JULIE P. COMBS, AND JOHN R. SLATE

Fall 2012 iii RESEARCH IN THE SCHOOLS

of all manuscripts submitted to the Research News

and Comment section of Educational Researcher

over this period, which rendered our findings

generalizable to the population of manuscripts

submitted to Educational Researcher—at least over

this period of time.

The two editors of the Research News and

Comment section of Educational Researcher

meticulously documented every APA error—

including reference list errors—committed by these

83 sets of authors over the 3.5-year period. Each

manuscript took up to 8 hours to identify all the

APA errors—representing as much as 664 hours of

coding. Alongside collecting information about

each APA error, these editors collected an array of

information corresponding to each of these

manuscripts, including the following: the length of

the manuscript (i.e., number of pages, number of

words), the length of the reference list (i.e., total

number of references), topic of the manuscript, and

the number of authors per manuscript. As such, the

data set created by these editors was extremely

rich, representing a data set that only journal

editors have the opportunity to develop.

Methodology

Dialectic pluralism was the philosophical

lens that drove our mixed research study. As

conceptualized by Johnson (2012), dialectic

pluralism represents a philosophical stance

wherein multiple epistemological perspectives are

combined by the researcher(s) within the same

investigation. With respect to the data analysis

step of our mixed research study, we utilized a

sequential mixed analysis (Onwuegbuzie &

Combs, 2010). Specifically, because our present

study represented a replication of the studies of

Onwuegbuzie and Hwang (2012, 2013) and

Onwuegbuzie, Hwang, et al. (2011), we used a

five-stage sequential mixed analysis procedure.

Each of these stages is described below.

Stage 1 Analysis

The first stage of the sequential mixed

analysis involved conducting a classical content

analysis (Berelson, 1952; see also Leech &

Onwuegbuzie, 2007, 2008, 2011) of the 83

manuscripts to determine the number of unique

reference list errors. In addition, the total number

of reference list errors per manuscript was

identified. Thus, the classical content analysis led

to the determination of prevalence rates.

Stage 2 Analysis

Determination of the reference list errors led

to the second stage. This stage involved conducting

a constant comparison analysis (Glaser, 1965;

Glaser & Strauss, 1967) of the reference list errors

to determine the number of themes that could be

extracted from them. These themes were extracted

iteratively (Constas, 1992). That is, these themes

were extracted using both a priori coding and a

posteriori coding. The a priori coding involved the

potential use of the aforementioned 14 themes that

stemmed from the study of Onwuegbuzie and

Hwang (2012, 2013). In addition, the researchers

allowed for the possibility of other themes

emerging.

Stage 3 Analysis

The third stage of the sequential mixed

analysis involved quantitizing the reference list

error themes (Sandelowski, Voils, & Knafl, 2009;

Tashakkori & Teddlie, 1998). This quantitizing

involved converting the themes that were extracted

in the second stage (i.e., qualitative data) to

numerical codes (i.e., quantitative data) for

statistical analyses. Specifically, each reference list

error theme was converted to a quantitative code by

assigning a “1” if the manuscript contained one or

more reference list error errors that were classified

under that theme and a “0” if the manuscript did

not contain any reference list error errors that were

classified under that theme (Onwuegbuzie, 2003;

Onwuegbuzie & Teddlie, 2003)—leading to the

creation of what Onwuegbuzie (2003) referred to as

an “inter-respondent matrix” (i.e., manuscript x

reference list error theme matrix) that comprised a

combination of 0s and 1s (p. 396). This inter-

respondent matrix was used to conduct a principal

component analysis (cf. Field, 2009) to determine

the underlying structure of the reference list error

themes by transforming it to a matrix of bivariate

associations that represented tetrachoric correlation

coefficients to take into account the fact that the

reference list error themes had been quantitized to

dichotomous data (i.e., “0” vs. “1”). As noted by

Onwuegbuzie et al. (2007), tetrachoric correlation

coefficients are appropriate to use when examining

the association between two (artificial)

dichotomous variables. In addition, a varimax (i.e.,

orthogonal) rotation was employed (Field, 2009),

using the following three procedures to determine

an appropriate number of factors to retain:

eigenvalue-greater-than-one rule (i.e., K1; Kaiser,

1958), scree test (representing a plot of eigenvalues

against the factors in descending order; Cattell,

1966; Zwick & Velicer, 1986), and a parallel

analysis (involving extracting eigenvalues from

random data sets that parallel the actual data set

with respect to the sample size and number of

variables; Thompson, 2004; Zwick & Velicer,

1982, 1986). These extracted factors yielded meta-

themes (Onwuegbuzie, 2003) such that each meta-

theme contained one or more of the reference list

error themes that emerged in the second stage of

the mixed analysis. As described by Onwuegbuzie

(2003), the proportion of variance explained by

each factor after rotation (i.e., the trace) served as




Fall 2012 iv RESEARCH IN THE SCHOOLS

an effect size index for each meta-theme. Using

Constas’s (1992) typology, by determining the

hierarchical relationship among the reference list

themes and identifying the meta-themes, the

verification component of categorization was

technical, empirical, and rational.

Stage 4 Analysis

The fourth stage of the sequential mixed

analysis process involved conducting a latent class

analysis to ascertain the number of clusters or

groups (i.e., latent classes) underlying the reference

list error themes. The latent class analysis was

conducted under the assumption that the 83

manuscripts could be classified into a small number

of distinct clusters known as latent classes based on

their profiles of reference list error themes, such

that each manuscript belonged to only one cluster.

This latent class analysis represented qualitizing of

the data, which involves converting numeric data

into (qualitative) narrative profiles (Tashakkori &

Teddlie, 1998).

Stage 5 Analysis

The fifth stage of the sequential mixed

analysis involved using the inter-respondent matrix

to examine the relationship between the reference

list error themes and selected variables.

Specifically, we conducted the following analyses:

(a) a series of correlation analyses to investigate the

relationships between the total number of reference

list errors and the number of citation errors, number

of authors, and length of manuscript; and (b) a

multiple linear regression analysis to identify an

optimal combination of independent variables (i.e.,

number of references in the reference list, number

of manuscript pages, number of manuscript words,

number of authors) that predicted the number of

reference list errors. Further, we were interested in

conducting a canonical correlation analysis to

examine the multivariate relationship between the

reference list error themes and selected

demographic variables (i.e., gender of the lead

author, number of authors, length of manuscript,

and size of institution of the lead author). However,

because of the relatively small case-to-variable

ratio (i.e., 83 cases to 14 reference list error themes

and 4 demographic variables), this analysis lacked

sufficient statistical power. Also of interest was to

examine whether the number of reference list errors

predicted the decision made by the editor regarding

the suitability of the manuscript for publication.

However, because the Educational Researcher

editors accepted only one of these 83 manuscripts

the first time, it was not possible to examine the

relationship between the number of reference list

errors and the decision made by the editors.

Results

Stage 1 Findings

The classical content analysis (Berelson,

1952) revealed a total of 1,183 reference list errors

across the 83 manuscripts, which represented more

than 14 reference list errors per manuscript, on

average (M = 14.25, SD = 8.05). The number of

unique reference list errors per manuscript ranged

from 3 to 35, with 83.1% of manuscripts containing

more than five unique reference list errors, 60.2%

of manuscripts containing more than 10 unique

reference list errors, and 24.1% of manuscripts

containing more than 20 unique reference list

errors. The classical content analysis also led to the

identification of a total of 324 unique reference list

errors that were identified across these 83

manuscripts. Additionally, this analysis revealed

that the prevalence of each of these reference list

errors ranged from 0 to 58.

Because of the number of unique reference

list errors identified (i.e., n = 324), we decided to

deem an error as being significantly common when

it occurred a minimum of five occasions. The cut-

point of five was used because it represented an

endorsement rate of 6%, which translated to a

moderate effect size, using Cohen’s (1988, pp. 180-

183) non-linear arcsine transformation criteria.

Interestingly, a total of 60 (18.5%) reference list

errors yielded moderate effect sizes (i.e.,

endorsement rates of five or greater). Table 1

presents the 50 most prevalent reference list errors

documented by Onwuegbuzie and Hwang (2012,

2013) and the ranks of these frequencies, as well as

the frequencies and ranks pertaining to these

reference list errors stemming from the present

study. As such, Table 1 provides a direct

comparison of the frequencies between

Onwuegbuzie and Hwang’s (2012, 2013)

investigation and the present study. It can be seen

from this table that the same two most common

reference list errors were the two most frequent

errors in both studies. In fact, three of the top four

reference list errors yielded identical ranks in both

studies, namely: Serial (issue) numbers presented

when the page numbers in each volume are

continuous; Comma not presented to separate two

authors; and Space not presented between initials

of each author.


Fall 2012 v RESEARCH IN THE SCHOOLS

Table 1

Stage 1 Findings: The 50 Most Prevalent Reference List Errors in Present Study and Onwuegbuzie and

Hwang’s (2012, 2013) Study and the Corresponding Rank in Present Study

Reference List Error Frequency1

Rank2 Frequency

3 Rank

4

Serial (issue) numbers presented when the

page numbers in each volume are continuous

102

1

58

1

Comma not presented to separate two authors

56 2 48 2

Superscripts inappropriately used when

providing edition number

53

3

33

6

Space not presented between initials of each

author

49

4

38

4

Period not presented after the author’s name

(when the author does not represent a person

but an organization) and before the publication

year

37

5

19

13

Website inappropriately underlined 34

6 23 9

Month not given for a paper presentation

31 7 10 30

"Publications” or “Publications Inc"

inappropriately presented when listing the

publisher

30

8

13

20

Reference list not double spaced 28

9 36 5

Citations not presented in alphabetical order

27 10 11 28

Title of journal article inappropriately

capitalized

27

10

39

3

Comma not presented after retrieval year of

Internal source

25

12

21

12

Volume number not italicized 24

13 26 7

"Inc" inappropriately presented when listing

the publisher

21

14

13

20

Title of book inappropriately capitalized

20 15 15 16

"&" not used to separate the last two authors

18 16 9 34

Reference heading is bolded 18

16 25 8

Retrieval date not provided for web-based

citations

18

16

13

20

First letter of the second-part of the title not

capitalized

18

16

14

19

Title of edited books inappropriately

capitalized

17

20

9

34




Fall 2012 vi RESEARCH IN THE SCHOOLS


Rank2 Frequency

3 Rank

4

Title of journal not italicized 16

21 3 89

Space not presented to separate initials of each

editor of an edited book

15

22

15

16

"And" instead of “&” to separate the last two

authors

14

23

13

20

City, state, and/or publisher not always

provided

14

23

11

28

Title of book not always italicized 14

23 5 55

Period inappropriately appears after the

numbers of ERIC

14

23

1

236

Page number of book chapters not presented

after the title of the book

12

27

23

9

Space not presented between pp and the page

number

12

27

4

69

Page number of journal articles not presented

12 27 7 37

Initials of all authors not presented 11

30 3 89

State pertaining to the publisher not

abbreviated

11

30

2

124

Reference heading represented by all

uppercase text

11

30

10

30

Serial number not presented when

discontinuous when the page numbers in each

volume are not continuous

11

30

6

38

Volume number of journal article not provided

11 30 12 25

Comma not presented to separate the last two

authors of a reference

(when references have more than two authors)

10

35

10

30

Period not presented after an author's initial

10 35 18 15

Volume number of journal (periodicals) not

italicized

10

35

22

11

Abbreviation (of authors) inappropriately

included

9 38 1 236

Citations not presented in chronological order

9

38

8

36

Title of paper presentation not italicized

9

38

0

323

Period inappropriately presented at the end of

the reference (e.g., when the reference ends

with a website address)

9

38

0

323


Fall 2012 vii RESEARCH IN THE SCHOOLS


Rank2 Frequency

3 Rank

4

Title of edited book not italicized 9

38 2 124

Title of journal article inappropriately

italicized

9

38

1

236

Period not presented at the end of reference

8 44 19 13

Reference list does not begin on a separate

page

8

44

12

25

Comma inappropriately appears between

initials of some authors

8

44

15

16

State of publisher not provided 8

44 12 25

"And" instead of “&” used to separate the

editors of edited books

8

44

10

29

Space inappropriately appears between six

numbers of an ERIC

8

44

1

236

Space inappropriately appears between volume

number and series number of a periodical

8

44

13

20

1 Represents data from Onwuegbuzie and Hwang’s (2012, 2013) study; Frequencies between 8 and 21 represent

moderate effect sizes; frequencies greater than 22 represent large effect sizes, using Cohen’s (1988, pp. 180-

183) non-linear arcsine transformation criteria. 2 Represents data from Onwuegbuzie and Hwang’s (2012, 2013) study

3 Represents data from the present study; Frequencies between 5 and 13 represent moderate effect sizes;

frequencies greater than 13 represent large effect sizes, using Cohen’s (1988, pp. 180-183) non-linear arcsine

transformation criteria. 4 Represents data from the present study

Stage 2 Findings

A constant comparison analysis of these 324

reference list errors yielded the following 14

reference list error themes: (a) General errors; (b)

Reference heading, (c) Names of authors, (d)

Publication year/date, (e) Title of work, (f)

Publisher information, (g) Source of

journal/periodical, (h) Source of authored book, (i)

Source of edited book, (j) Source of website, (k)

Source of paper presentation, (l) Source of

dissertation/thesis, (m) Source of newspaper article,

and (n) Source of government document. These

reference list error themes were identical to the

ones identified by Onwuegbuzie and Hwang (2012,

2013). Table 2 presents descriptive statistics

regarding the number of reference list errors for

each of the 14 citation error themes. It can be seen

from this table that reference list errors associated

with the Source of journal/periodical represented

the most prevalent errors, followed by reference list

errors associated with Names of authors.

Stage 3 Findings

With regard to the number of factors

underlying the 14 emergent reference list error

themes extracted in Stage 2, the Kaiser-Meyer-

Olkin (KMO) measure of sampling adequacy was

greater than .5 (i.e., KMO = .51) and Bartlett’s test

of sphericity was statistically significant (Χ2[91] =

128.45, df = 91, p = .006), thereby justifying

conducting a principal component analysis. Both

the eigenvalue-greater-than-one rule (i.e., K1;

Kaiser, 1958) and scree test indicated that four

factors (i.e., meta-themes) should be retained. A

parallel analysis verified the K1 and scree test

(Zwick & Velicer, 1982, 1986) for the current data

of 83 manuscripts and 14 variables (i.e., reference

list error themes). Specifically, a series of (i.e., n =

1,000) random data matrices of size 83 x 14 was

generated, and eigenvalues were computed for the

correlation matrices for the original data and for

each of the 1,000 random data sets. Next, the

eigenvalues derived from the actual data were

compared to the eigenvalues derived from the




Fall 2012 viii RESEARCH IN THE SCHOOLS

random data for the purpose of identifying the

number of components that accounted for more

variance than did the components obtained from

random data. Consistent with the K1 and scree test,

the parallel analysis suggested retaining four

factors.

Table 3 presents this four-factor principal

components solution. Using a cutoff correlation of

0.3 recommended by Lambert and Durand (1975)

as an acceptable lower bound for pattern/structure

coefficients, it can be seen from Table 3 that three

reference list error themes had pattern/structure

coefficients with large effect sizes on the first

factor: (a) Source of edited book, (b) Names of

authors, and (c) Source of authored book. It should

be noted that in addition to having a

pattern/structure coefficient with a large effect size

on Factor 1, Source of authored book also had a

significant but smaller pattern/structure coefficient

on Factor 4 (i.e., cross-loading). Further, the

following four reference list error themes had

pattern/structure coefficients with large effect sizes

on the second factor: (a) Source of website, (b)

Publication year/date, (c) Source of newspaper

article, and (d) Title of work. Interestingly, in

addition to having a pattern/structure coefficient

with a large effect size on Factor 2, Source of

newspaper article also had a significant but smaller

pattern/structure coefficient on Factor 4, and Title

of work also had a significant but smaller

pattern/structure coefficient on Factor 1 (i.e., cross-

loadings). Furthermore, the following four

reference list error themes had pattern/structure

coefficients with large effect sizes on the third

factor: (a) Source of dissertation/thesis, (b) Source

of government document, (c) Source of paper

presentation, and (d) Reference heading. In

addition to having a pattern/structure coefficient

with a large effect size on Factor 3, Source of paper

presentation also had a significant but smaller

pattern/structure coefficient on Factor 2 (i.e., cross-

loading). Finally, three reference list error themes

emerged that had pattern/structure coefficients with

large effect sizes on the fourth factor: (a) Publisher

information, (b) General errors, and (b) Source of

journal/periodical. In addition to having a

pattern/structure coefficient with a large effect size

on Factor 4, Reference heading also had a

significant but smaller pattern/structure coefficient

on Factor 3 (i.e., cross-loading).

Table 2

Stage 2 Findings: Prevalence Rates of Themes Emerging from Reference List Errors for Manuscripts Submitted

to Educational Researcher

Reference List Error Theme

Total Number of

Unique Reference

List Errors Contained

in Theme

Total Number

of Reference

List Errors

Contained in

Theme

Average incidence of

reference list errors per

manuscript (%)

Source of journal/periodical 65 233 96.4

Names of authors 39 209 83.1

General errors 19 92 68.7

Source of edited book 43 149 63.9

Source of authored book 27 85 62.7

Reference heading 13 78 62.7

Title of work 19 81 59.0

Source of website 23 86 51.8

Publisher information 24 69 48.2

Publication year/date 22 42 41.0

Source of paper presentation 22 48 27.7

Source of government document 5 5 6.0

Source of newspaper article 1 4 4.8

Source of dissertation/thesis 2 2 2.4


Fall 2012 ix RESEARCH IN THE SCHOOLS

Table 3

Stage 3 Findings: Summary of Themes and Factor Pattern/Structure Coefficients from Principal Component

Analysis (Varimax): Four-Factor Solution

Factor Coefficients1

Theme 1 2 3 4 Communality

Coefficient

Source of edited book .77 .05 .02 -.10 .61

Names of authors .68 .10 .10 -.20 .52

Source of authored book .64 -.04 .02 .43 .60

Source of website -.09 .63 -.09 .09 .42

Publication year/date .17 .62 .12 .19 .46

Source of newspaper article -.01 .61 -.12 -.37 .52

Title of work .40 .54 .20 .21 .54

Source of dissertation/thesis .07 .01 .73 .01 .54

Source of government document .06 -.11 .72 -.10 .54

Source of paper presentation -.02 -.31 .55 .28 .48

Reference heading -.03 .06 -.39 .34 .27

Publisher information .18 -.10 -.11 .75 .61

General errors -.20 .17 -.09 .40 .24

Source of journal/periodical -.11 .16 .14 .38 .20

Trace 1.76 1.63 1.63 1.52 6.54

% variance explained 12.57 11.65 11.65 10.82 46.70

1Coefficients in bold represent pattern/structure coefficients with the largest effect size within each theme using

a cut-off value of 0.3 recommended by Lambert and Durand (1975).

The first meta-theme (i.e., Factor 1) was

labeled Author and Book and explained 12.57% of

the total variance; the second meta-theme (i.e.,

Factor 2) was labeled: Website, Year, Newspaper,

and Title and explained 11.65% of the total

variance; the third meta-theme (i.e., Factor 3) was

labeled: Dissertation/Thesis, Government, Paper

Presentation, and Heading and explained 11.65 of

the total variance; and the fourth meta-theme (i.e.,

Factor 4) was labeled Publisher Information,

Miscellaneous Errors, and Journal and explained

10.82% of the total variance. These four meta-

themes combined accounted for 46.70% of the total

variance, which using the findings of Henson,

Capraro, and Capraro (2004) and Henson and

Roberts (2006), represents a large effect size.

The manifest effect size—(i.e., actual

reference list error rate per meta-theme) associated

with the four meta-themes was as follows: (a)

Author and Book (91.6%), (b) Website, Year,

Newspaper, and Title (79.5%), (c)

Dissertation/Thesis, Government, Paper

Presentation, and Heading (74.7%), and (d)

Publisher Information, Miscellaneous Errors, and

Journal (98.8%). Figure 1 displays the thematic

structure (i.e., relationships among the reference

list error themes and the reference list error meta-

themes), including the manifest effect sizes and

latent effect sizes. This figure represents a

crossover visual representation (Onwuegbuzie &

Dickinson, 2008), which depicts the integration of

both quantitative and qualitative findings within the

same display.

Stage 4 Findings

A latent class analysis was conducted to

determine the smallest number of clusters (i.e.,

latent classes) that explains all the relationships

among select reference list error themes under the

assumption that manuscripts could be classified

into a small number of distinct clusters known as

latent classes depending on their profiles of the

select reference list errors, such that each

manuscript belonged to only one cluster. We

decided to conduct the latent class analysis on the

six most common error themes because for all of

these themes, at least 60% of the authors made

reference list errors that were classified under these

themes—namely, Source of journal/periodical,

Names of authors, General errors, Source of edited




Fall 2012 x RESEARCH IN THE SCHOOLS

book, Source of authored book, and Reference

heading, respectively.

Our latent class analysis of the six reference

list error themes revealed a two-cluster solution (L2

= 39.07, df = 50, p = .87, Bootstrap p = .13). Figure

2 displays these two distinct groups of manuscripts.

It can be seen from Figure 2 that Cluster 1

(comprising 66.0% of manuscripts) was relatively

high with respect to all six reference list error

themes, whereas Cluster 2 (comprising 34.0% of

manuscripts) was high on General errors, Names of

authors, Reference heading, and Source of

journal/periodical, but relatively low on Source of

authored book and Source of edited book.

Figure Caption

Figure 1. Stage 3 Findings: Thematic structure pertaining to reference list error themes and meta-themes.

Author and Book

Latent Effect Size = 12.57%

Manifest Effect Size = 91.6%

Website, Year, Newspaper, and

Title



Source of

website

Title of work

Names of

authors

Source of edited

book

Source of authored

book

Source of

newspaper

article

Publication

year/date

Dissertation/Thesis, Government,

Paper Presentation, and

Heading



Government and Miscellaneous Errors



Source of

dissertation/

thesis

Source of

paper

presentation

Source of

government

document

Reference

heading

Publisher

information

Source of

journal/periodical

General errors


Fall 2012 xi RESEARCH IN THE SCHOOLS

Figure 2. Stage 4 Findings: Profiles of the manuscripts with respect to the reference list error themes.

Stage 5 Findings

Correlation analysis. A series (i.e., n = 6) of

correlations was conducted to assess the


errors and an array of reference list error variables,

namely, the total number of reference list errors,

and the following five components identified by

Onwuegbuzie, Frels, and Slate (2010): Not in

Reference List, Not Consistent with Reference List,

Not in Text, Incomplete or Incorrect Citation, and

Incomplete or Incorrect Reference. According to

Onwuegbuzie and Daniel (2002), variables for

which either the standardized skewness coefficient

(i.e., skewness coefficient divided by its standard

error) or the standardized kurtosis coefficient (i.e.,

kurtosis coefficient divided by its standard error),

or both, are outside the 3 range suggest extreme

departure from normality. An examination of the

skewness and kurtosis coefficients pertaining to the

six reference list error variables revealed a serious

departure from normality for all six reference list

error variables, with all these variables indicating

distributions that were positively skewed and

leptokurtic (i.e., overly peaked shape). Thus, a

nonparametric correlation, namely, Spearman’s

rank, was used to examine these six sets of

relationships. After applying the Bonferroni

adjustment to control for the inflation of Type I

error (i.e., adjusted α = .05/6 = .00833), the

Spearman’s rank correlations revealed that the

number of reference list errors was statistically

significantly and positively related to five of the six

reference list error variables, namely: total number

of reference list errors (rs[83] = .59, p < .001), Not

in Reference List (rs[83] = .39, p < .001), Not

Consistent with Reference List (rs[83] = .48, p <

.001), Not in Text (rs[83] = .42, p < .001), and

Incomplete or Incorrect Reference (rs[83] = .38, p <

.001). All these relationships were large (Cohen,

1988). The only statistically non-significant

Ge

ne

ra

l

0-1

Me

an

Na

me

_A

uth

or

0-1

Me

an

Re

f_h

ea

din

g

0-1

Me

an

So

urce

_A

uth

ore

dB

oo

k

0-1

Me

an

So

urce

_E

dite

dB

oo

k

0-1

Me

an

So

urce

_Jo

urn

al

0-1

Me

an

1 .0

0.8

0.6

0.4

0.2

0.0

Cluster1

Cluster2




Fall 2012 xii RESEARCH IN THE SCHOOLS

relationship was that between the number of

reference list errors and reference list errors

pertaining to Incomplete or Incorrect Citation

(rs[83] = .13, p = .266).

Further, a series (i.e., n = 4) of nonparametric

(i.e., Spearman) correlations, after applying the

Bonferroni adjustment to control for the inflation of

Type I error, revealed that the number of reference

list errors was statistically significantly related to

the number of references in the reference list

(rs[83] = .27, p = .016). Using Cohen’s (1988)

criteria, this relationship was moderate. However,

no statistically significant relationship was

observed between the number of reference list

errors and the number of manuscript pages (rs[83]

= .17, p = .12), the number of manuscript words

(rs[83] = .21, p = .06), and the number of authors

(rs[83] = .17, p = .19).

Multiple regression analysis. An all

possible subsets (APS) multiple linear regression

(Onwuegbuzie & Daniel, 2003; Thompson 1995)

was used to identify an optimal combination of the

four independent variables (i.e., number of

references in the reference list, number of

manuscript pages, number of manuscript words,

number of authors) that predicted the number of

reference list errors. This analysis, which has been

advocated by many statisticians (e.g.,

Onwuegbuzie & Daniel, 2003; Thompson 1995),

involved examining all possible models containing

some or all of the four independent variables.

When using this analytical technique, separate

regression analyses are conducted for the one

possible set of four independent variables, all four

possible trios of independent variables, all six

possible pairs of independent variables, and all four

independent variables singly—yielding the fitting

of 15 multiple regression models. These 15 models

then were compared to identify the best subset of

independent variables using the following two

criteria: (a) the maximum proportion of variance

explained (R2) and (b) Mallow’s Cp (Myers, 1986;

Sen & Srivastava, 1990). The APS multiple

regression analysis revealed that a model

containing one variable provided the best fit to

these data. In fact, the four-variable model (i.e.,

adding the remaining three independent variables)

only increased the proportion of variance explained

by 3.5%. In addition, Mallow’s Cp was closer in

value to the number of regressor variables (Myers,

1986; Sen & Srivastava, 1990) with the one-

variable solution than with any other variable

solution.

The selected model indicated that the

following variable contributed significantly (F[1,

82] = 4.53, p < .05) to the prediction of the number

of reference list errors: the number of authors. This

variable explained 6.8% of the variation in the

number of reference list errors (Adjusted R2 =

5.3%). Using Cohen’s (1988) criteria for assessing

the predictive power of a set of independent

variables in a multiple regression model, the

proportion of variance explained indicates a small

effect size, because it lay between 2% and 12.99%.

With respect to the assumptions for the

selected one-variable linear regression model, the

Durbin-Watson coefficient of 1.90 was sufficiently

close to 2.00 to suggest that for any two

observations, the residual terms were uncorrelated

(i.e., lack of autocorrelation), which was a desirable

outcome. In addition, an examination of the

standardized residuals pertaining to each of the

participants revealed that no manuscript had a

standardized residual that exceeded 2.00. Thus, in

summary, the selected final regression model

suggested that the manuscripts with the most

citation errors tended to have the highest number of

authors. The regression equation was as follows:

Number of Reference List Errors = 10.69 + 2.40 *

Number of Authors

This equation indicated that among manuscripts

submitted to Educational Researcher, every

additional author of a manuscript was associated

with an increase of 2.40 reference list errors, on

average.

Discussion of Findings

Our current editorial provides further

compelling evidence that APA-related reference

list errors are very common among manuscripts

initially submitted to journals for consideration for

publication. More specifically, the present findings

indicate that not only do reference list errors

permeate manuscripts that are submitted to RITS

but also they similarly pervade manuscripts that are

submitted to the foremost journal in education,

namely, Educational Researcher. And, our

experience as editors of Educational Researcher

led us to conclude that a high proportion of authors

who submit manuscripts to Educational Researcher

are among the most prolific of authors in the world.

Thus, our present editorial has led us to conclude

that reference list errors are not only committed by

beginning authors, but also they are being

committed by prolific authors—thereby supporting

the hypothesis that journals with the highest impact

factors also have manuscripts submitted to their

journals that have high rates of reference list errors.

In fact, the characteristics of reference list errors

for both the Educational Researcher manuscripts

and RITS manuscripts were very similar, including

the distributions of the 14 reference list error

themes. Other similarities in the findings pertaining


Fall 2012 xiii RESEARCH IN THE SCHOOLS

to the RITS authors and Educational Researcher

authors include the mean number of reference list

errors per manuscript (M = 12.83 vs. M = 14.25,

respectively) and the frequency rate of the

reference list errors. With respect to the latter, from

Table 1, the relationship between the frequencies of

the 50 themes relating to the two studies was

statistically significant and large for both the raw

frequencies (r = .80, p < .001) and the frequency

ranks (rs = .56, p < .001), with three of the top four

reference list errors yielding identical ranks in both

studies.

As was the case for the RITS authors, not a

single author had an error-free reference list, with

the smallest number of unique reference list errors

being three. Also, it should be noted that the mean

number of reference list errors of 14.25 among

Educational Researcher authors likely represents a

lower bound when one takes into account that these

manuscripts were submitted before the writers of

sixth edition of APA required authors to include

digital object identifiers (DOIs) whenever they are

available (cf. section 6.31). As explained by the

writers of the sixth edition of [APA] Publication

Manual, DOI numbers represent unique numbers

assigned by the publisher for electronic referencing

of published journal articles and other documents.

Accordingly, in reference lists, authors should

place the DOI at the end of the reference. Thus,

under the sixth edition of APA, failure to include

available DOI numbers represents a reference list

error—specifically, a reference list error pertaining

to Source of journal/periodical. Moreover, when

we take into account the fact that the 83

Educational Researcher manuscripts represented

those manuscripts that were sent out for external

review, it is reasonable to conclude that this 14.25

mean number of reference list errors is even more

of a lower bound. Further, this mean number of

reference list errors is even more of a lower bound

when we take into account the fact that we only

assessed reference list errors with respect to

violations to APA, and did not assess reference list

errors by comparing each reference contained in

the reference list to the original work, as has many

researchers across numerous fields and disciplines

(e.g., business, economics, social work,

psychology, medicine, library information science;

cf. Adhikari & Bhandari, 2011; de Lacey, Record,

& Wade, 1985; Doms, 1988; Eichorn & Yankauer,

1987; Faunce & Job, 2001; Gatten, 2010; Gosling,

Cameron, & Gibbons, 2004; Hernon & Metoyer-

Duran, 1992; Holt, Siebers, Suder, Loan, & Jeffery,

2000; Kristof, 1997; Ngan Kee, Roach, & Lau,

1997; Nishina, Asano, Mikawa, Maekawa, &

Obara, 1995; O’Connor, 2002; O'Connor &

Kristof, 2001; Roach, Lau, & Ngan Kee, 1997;

Siebers, 2000; Siebers & Holt, 2000; Spivey &

Wilks, 2004; White, 1987)—leading to reference

list errors that have ranged from 4.1% to 66.7%

(Onwuegbuzie, 2012).

The statistically significant and moderate


errors and the number of references in the reference

list, although not surprising, suggests that authors

who write manuscripts that contain many

citations—and hence contain longer reference

lists—should be especially careful when compiling

their reference list. An extremely disturbing finding

is that manuscripts submitted to Educational

Researcher that involve more co-authors tend to

exhibit more reference list error themes. This

finding, which replicates the finding of

Onwuegbuzie, Hwang, et al. (2011) and echoes the

positive relationship between the number of

authors and the number of citation errors

documented by Onwuegbuzie, Frels, et al. (2010)

and Onwuegbuzie, Combs, Frels, and Slate

(2011)—again suggests that the bystander effect

likely prevails (Darley & Latané, 1968; Hudson &

Bruckman, 2004; Levine & Thompson, 2004). That

is, when manuscripts are written by multiple

authors, some, if not all, authors assume that one or

more of their other co-authors will/have checked

the in-text citations and the reference list carefully

and/or they believe that one or more of their other

co-authors are more qualified to check the in-text

citations and the reference list and thus their

contribution in this area is not needed

(Onwuegbuzie, Frels, et al., 2010). Thus, our series

of editorials on citation errors and reference list

errors clearly indicate that for manuscripts that

involve multiple authors, all co-authors should be

aware of the potential pitfalls stemming from the

bystander effect and attempt to maximize the

communication channels among all members of the

authorship team.

Conclusions

Our series of articles and editorials in the area

of APA errors of omission and commission make it

clear that the vast majority of authors who submit

manuscripts to journals commit an unnecessarily

large number of APA errors at all components of

an article, including the abstract (Hahs-Vaughn,

Onwuegbuzie, Slate, & Frels, 2009) and body of

the manuscript (Onwuegbuzie & Combs, 2009;

Onwuegbuzie, Combs, Slate, & Frels, 2010). Thus,

in these works, we provided tools and strategies

that we hope will help authors minimize these APA

errors in the future. However, our most recent

articles and editorials on citation errors

(Onwuegbuzie, Combs, et al., 2011; Onwuegbuzie,

Frels, et al., 2010) and reference list errors

(Onwuegbuzie & Hwang, 2012, 2013;

Onwuegbuzie, Hwang, et al., 2011), including the

present editorial, have demonstrated that authors

also commit an unnecessarily large number of APA




Fall 2012 xiv RESEARCH IN THE SCHOOLS

errors in their reference lists. And, just as authors

who commit in-text APA errors are significantly

more likely to have their manuscripts rejected for

publication by editors, so too are authors who

commit reference list errors significantly more

likely to have their manuscripts rejected

(Onwuegbuzie, Hwang, et al., 2011). Yet, avoiding

reference list errors is not just merely a quality

issue, it is, even more importantly, also an ethical

one, wherein all authors should make every attempt

to document all their sources accurately and fully.

Various professionals at the college level (e.g.,

instructors, mentors, advisors, thesis/dissertation

committee members, chairs/supervisors) and

beyond (e.g., journal editors, publishers, and

writers of future editions of the APA Publication

Manuals) can play an important role in promoting

what Onwuegbuzie, Hwang, et al. (2011) referred

to as a “culture of reference lists that are minimally

error free” (p. xiv). Onwuegbuzie, Combs, et al.

(2010), Onwuegbuzie, Hwang, et al. (2011), and

Onwuegbuzie, Frels, et al. (2010) have discussed

several ways in which these personnel can assist, as

well as have provided tools and strategies (e.g.,

using Table 1 and Table 2 presented in the current

editorial as starting points by focusing on these

most common types of reference list errors and

reference list error themes, respectively) to help

students, researchers, and experienced scholars

prevent making reference list errors. Whatever

strategies are used, we hope that the efficacy of

these strategies is monitored, documented, and

disseminated by those persons using them—as was

undertaken in the aforementioned series of articles

and editorials—so that we can all learn from their

findings.

References

Adhikari, P., & Bhandari, S. (2011). Accuracy of

references in Internet Journal of Medical

Update. Internet Journal of Medical

Update, 6(2), 47-49. Retrieved from

http://www.akspublication.com/ijmu

American Psychological Association. (2010).

Publication manual of the American

Psychological Association (6th ed.).

Washington, DC: Author.

Berelson, B. (1952). Content analysis in

communicative research. New York, NY:

Free Press.

Cattell, R. B. (1966). The scree test for the number

of factors. Multivariate Behavioral Research,

1, 245-276. doi:10.1207/s15327906mbr01

02_10

Chicago Manual of Style. (15th ed.). (2003).

Chicago, IL: The University of Chicago

Press.

Cohen, J. (1988). Statistical power analysis for the

behavioral sciences (2nd ed.). Hillsdale, NJ:

Lawrence Erlbaum.

Constas, M. A. (1992). Qualitative data analysis as

a public event: The documentation of

category development procedures. American

Educational Research Journal, 29, 253-266.

doi:10.3102/00028312029002253

Darley, J. M., & Latané, B. (1968). Bystander

intervention in emergencies: Diffusion of

responsibility. Journal of Personality and

Social Psychology, 8, 377-383. Retrieved

from

http://www.wadsworth.com/psychology_d/te

mplates/student_resources/0155060678_rathu

s/ps/ps19.html

de Lacey, G., Record, C., & Wade, J. (1985). How

accurate are quotations and references

in medical journals? British Medical

Journal (Clinical Research ed.), 291(6499),

884-886. doi:10.1136/bmj.291.6499.884

Doms, C. A. (1988). A survey of reference

accuracy in five national dental journals.

Journal of Dental Research, 68, 442-444.

Eichorn, P., & Yankauer, A. (1987). Do authors

check their references? A survey of accuracy

of references in three public health journals.

American Journal of Public Health, 77,

1011-1012. doi:10.2105/AJPH.77.8.1011

Faunce, G. J., & Job, R. F. S. (2001). The accuracy

of reference lists in five experimental

psychology journals. American Psychologist,

56, 829-830.

doi:10.1037//0003066X.56.10.829

Field, A. (2009). Discovering statistics using SPSS

(3rd ed.). London, England: Sage.

Gatten, R. (2010). A case study in reference list

accuracy. New Library World, 111(1), 16-25.

doi:10.1108/03074801011015658

Glaser, B. G. (1965). The constant comparative

method of qualitative analysis. Social

Problems, 12, 436-445.

doi:10.1525/sp.1965.12.4.03a00070

Glaser, B. G., & Strauss, A. L. (1967). The

discovery of grounded theory: Strategies for

qualitative research. Chicago, IL: Aldine.

Gosling, C. M., Cameron, M., & Gibbons, P. F.

(2004). Referencing and quotation accuracy

in four manual therapy journals. Manual

Therapy, 9(1), 36-40.

doi:10.1016/S1356689X(03)00056-0

Hahs-Vaughn, D. L., Onwuegbuzie, A. J., Slate, J.

R., & Frels, R. K. (2009). Editorial: Bridging

research-to-practice: Enhancing knowledge

through abstracts. Research in the Schools,

16(2), xxxvii-xlv. Retrieved from

http://www.msera.org/download/RITS_16_2

_Abstracts.pdf

http://www.wadsworth.com/psychology_d/templates/student_resources/0155060678_rathus/ps/ps19.html



http://en.wikipedia.org/wiki/Journal_of_Personality_and_Social_Psychology

http://en.wikipedia.org/wiki/Journal_of_Personality_and_Social_Psychology





Fall 2012 xv RESEARCH IN THE SCHOOLS

Henson, R. K., Capraro, R. M., & Capraro, M. M.

(2004). Reporting practice and use of

exploratory factor analysis in educational

research journals: Errors and explanation.

Research in the Schools, 11(2), 61-72.

Henson, R. K., & Roberts, J. K. (2006). Use of

exploratory factor analysis in published

research. Educational and Psychological

Measurement, 66, 393-416.

doi:10.1177/0013164405282485

Hernon, P., & Metoyer-Duran, C. (1992).

Literature reviews and inaccurate

referencing: An exploratory study of

academic librarians. College & Research

Libraries, 53, 499-512.

Holt, S., Siebers, R., Suder, A., Loan, R., &

Jeffery, O. (2000). The accuracy of

references in Australian and New Zealand

Medical Journals. New Zealand Medical

Journal, 113, 416-417.

Hudson, J. M., & Bruckman, A. S. (2004). The

bystander effect: A lens for understanding

patterns of participation. Journal of the

Learning Sciences, 13, 165-195.

doi:10.1207/s15327809jls1302_2

Johnson, R. B. (2012). Dialectical pluralism and

mixed research. American Behavioral

Scientist, 56, 751-754.

doi:10.1177/0002764212442494

Kaiser, H. F. (1958). The varimax criterion for

analytic rotation in factor analysis.

Psychometrika, 23, 187-200. doi:10.1007/

BF02289233

Kristof, C. (1997). Accuracy of references in five

entomology journals. American

Entomology, 43, 246-251.

Lambert, Z. V., & Durand, R. M. (1975). Some

precautions in using canonical analysis.

Journal of Market Research, 12, 468-475.

doi:10.2307/3151100

Leech, N. L., & Onwuegbuzie, A. J. (2007). An

array of qualitative analysis tools: A call for

data analysis triangulation. School

Psychology Quarterly, 22, 557-584.

doi:10.1037/10453830.22.4.557

Leech, N. L., & Onwuegbuzie, A. J. (2008).

Qualitative data analysis: A compendium of

techniques for school psychology research

and beyond. School Psychology Quarterly,

23, 587-604. doi:10.1037/1045-

3830.23.4.587

Leech, N. L., & Onwuegbuzie, A. J. (2011).

Beyond constant comparison qualitative data

analysis: Using NVivo. School Psychology

Quarterly, 26, 70-84. doi:10.1037/a0022711

Levine, M., & Thompson, K. (2004). Identity,

place, and bystander intervention: Social

categories and helping after natural disasters.

Journal of Social Psychology, 144, 229-245.

doi:10.3200/SOCP.144.3.229-245

Myers, R. H. (1986). Classical and modern

regression with applications. Boston, MA:

Duxbury Press.

Ngan Kee, W. D., Roach, V. J., & Lau, T. K.

(1997). How accurate are references in the

Australian and New Zealand Journal of

Surgery? Australian and New Zealand

Journal of Surgery, 67, 417-419.

doi:10.1111/j.1445-2197.1997.tb02005.x

Nishina, K., Asano, M., Mikawa, K., Maekawa, N.,

& Obara, H. (1995). The accuracy of

reference lists in Acta Anaesthesiologica

Scandinavica. Acta

Anaesthesiologica Scandinavica, 39, 577-

578. doi:10.1111/j.1399-

6576.1995.tb04129.x

O’Connor, A. E. (2002). A review of the accuracy

of references in the journal. Emergency

Medicine. Emergency Medicine, 14, 139-141.

doi:10.1046/j.1442-2026.2002.00307.x

O'Connor, L. G., & Kristof, C. (2001). Verify your

citations: Accuracy of reference citations in

twelve business and economics journals.

Journal of Business & Finance

Librarianship, 6(4), 23-40.

doi:10.1300/J109v06n04_03

Onwuegbuzie, A. J. (2003). Effect sizes in

qualitative research: A prolegomenon.

Quality & Quantity: International Journal of

Methodology, 37, 393-409.

doi:10.1023/A:1027379223537

Onwuegbuzie, A. J. (2012). Meta-analysis of

studies of the accuracy of reference lists in

published articles. Unpublished manuscript,

Sam Houston State University, Huntsville,

TX.

Onwuegbuzie, A. J., & Combs, J. P. (2009).

Writing with discipline: A call for avoiding

APA style guide errors in manuscript

preparation. School Leadership Review, 4,

116-149.

Onwuegbuzie, A. J., & Combs, J. P. (2010).

Emergent data analysis techniques in mixed

methods research: A synthesis. In A.

Tashakkori & C. Teddlie (Eds.), Handbook of

mixed methods in social and behavioral

research (2nd ed., pp. 397-430). Thousand

Oaks, CA: Sage.

Onwuegbuzie, A. J., Combs, J. P., Frels, R. K., &

Slate, J. R. (2011). Editorial: Citation errors

revisited: The case for Educational

Researcher. Research in the Schools, 18(1),

i-xxxv.

Onwuegbuzie, A. J., Combs, J. P., Slate, J. R., &

Frels, R. K. (2010). Editorial: Evidence-

based guidelines for avoiding the most

common APA errors in journal article

submissions. Research in the Schools, 16(2),

ix-xxxvi. Retrieved from

http://en.wikipedia.org/wiki/Journal_of_the_Learning_Sciences

http://en.wikipedia.org/wiki/Journal_of_the_Learning_Sciences

http://en.wikipedia.org/wiki/Digital_object_identifier

http://dx.doi.org/10.1207%2Fs15327809jls1302_2

http://psycnet.apa.org/doi/10.1037/a0022711

http://en.wikipedia.org/wiki/Journal_of_Social_Psychology

http://en.wikipedia.org/wiki/Digital_object_identifier

http://dx.doi.org/10.3200%2FSOCP.144.3.229-245

http://dx.doi.org/10.1111%2Fj.1445-2197.1997.tb02005.x




Fall 2012 xvi RESEARCH IN THE SCHOOLS

http://msera.org/download/RITS_16_2_APA

Errors6th.pdf

Onwuegbuzie, A. J., & Daniel, L. G.

(2002). Uses and misuses of the

correlation coefficient. Research in

the Schools, 9(1), 73-90.

Onwuegbuzie, A. J., & Daniel, L. G. (2003,

February 12). Typology of analytical and

interpretational errors in quantitative and

qualitative educational research. Current

Issues in Education, 6(2). Retrieved from

http://cie.ed.asu.edu/volume6/number2/

Onwuegbuzie, A. J., & Dickinson, W. B. (2008).

Mixed methods analysis and information

visualization: Graphical display for effective

communication of research results. The

Qualitative Report, 13, 204-225. Retrieved

from http://www.nova.edu/ssss/QR/QR13-

2/onwuegbuzie.pdf

Onwuegbuzie, A. J., Frels, R. K., & Slate, J. R.

(2010). Editorial: Evidence-based guidelines

for avoiding the most prevalent and serious

APA error in journal article submissions—

The citation error. Research in the Schools,

17(2), i-xxiv. Retrieved from

http://www.msera.org/download/RITS_17_2

_Citations.pdf

Onwuegbuzie, A. J., & Hwang, E. (2012).

Reference list errors in manuscripts

submitted to a journal for review for

publication. Unpublished manuscript, Sam

Houston State University, Huntsville, TX.

Onwuegbuzie, A. J., & Hwang, E. (2013).

Reference list errors in manuscripts

submitted to a journal for review for

publication. International Journal of

Education, 5(1), 1-14.

doi:10.5296/ije.v5i2.2191. Retrieved from

http://www.macrothink.org/journal/index.php

/ije/article/view/2191

Onwuegbuzie, A. J., Hwang, E., Frels, R. K., &

Slate, J. R. (2011). Editorial: Evidence-based

guidelines for avoiding reference list errors in

manuscripts submitted to journals for review

for publication. Research in the Schools,

18(2), i-xli. Retrieved from

http://msera.org/rits.htm

Onwuegbuzie, A. J., & Teddlie, C. (2003). A

framework for analyzing data in mixed

methods research. In A. Tashakkori & C.

Teddlie (Eds.), Handbook of mixed methods

in social and behavioral research (pp. 351-

383). Thousand Oaks, CA: Sage.

Onwuegbuzie, A. J., Witcher, A. E., Collins, K. M.,

Filer, J. D., Wiedmaier, C. D., & Moore, C.

W. (2007). Students’ perceptions of

characteristics of effective college teachers:

A validity study of a teaching evaluation

form using a mixed-methods analysis.

American Educational Research Journal, 44,

113-160. doi 10.3102/0002831206298169

Roach, V. J., Lau, T. K., & Ngan Kee, W. D.

(1997). The quality of citations in major

international obstetrics and gynecology

journals. American Journal of Obstetrics and

Gynecology, 177, 973-975.

doi:10.1016/S0002-9378(97)70303-X

Sandelowski, M., Voils, C. I., & Knafl, G. (2009).

On quantitizing. Journal of Mixed Methods

Research, 3, 208-222.

doi:10.1177/1558689809334210

Sen, A. K., & Srivastava, M. (1990). Regression

analysis: Theory, methods, and applications.

New York, NY: Springer-Verlag.

Siebers, R. (2000). The accuracy of references of

three allergy journals. Journal of Allergy and

Clinical Immunology, 105, 837-838.

doi:10.1067/mai.2000.104935

Siebers, R., & Holt, S. (2000). Accuracy of

references in five leading medical journals.

Lancet, 356 (9239), 1445.

doi:10.1016/S0140-6736(05)74090-3

Spivey, C. A., & Wilks, S. E. (2004). Reference list

accuracy in social work journals. Research

on Social Work Practice, 14, 281-286.

doi:10.1177/1049731503262131

Tashakkori, A., & Teddlie, C. (1998). Mixed

methodology: Combining qualitative and

quantitative approaches. Applied Social

Research Methods Series (Vol. 46).

Thousand Oaks, CA: Sage.

Thompson, B. (1995). Stepwise regression and

stepwise discriminant analysis need not apply

here: A guidelines editorial. Educational and

Psychological Measurement, 55, 525-534.

doi:10.1177/0013164495055004001

Thompson, B. (2004). Exploratory and

confirmatory factor analysis: Understanding

concepts and applications. Washington, DC:

American Psychological Association.

White, A. (1987). Reference list inaccuracies: A

four-decade comparison. Journal of

Counseling & Development, 66, 195-196.

doi:10.1002/j.1556-6676.1987.tb00846.x

Zwick, W. R., & Velicer, W. F. (1982). Factors

influencing four rules for determining the

number of components to retain. Multivariate

Behavioral Research, 17, 253-269.

doi:10.1207/s15327906mbr1702_5

Zwick, W. R., & Velicer, W. F. (1986).

Comparison of five rules for determining the

number of components to retain.

Psychological Bulletin, 99, 432-442.

doi:10.1037//0033-2909.99.3.432

http://msera.org/download/RITS_16_2_APAErrors6th.pdf

http://msera.org/download/RITS_16_2_APAErrors6th.pdf


Mid-South Educational Research Association 2012, Vol.19, No. 2, 1-16

Fall 2012 1 RESEARCH IN THE SCHOOLS

Foreign Language Immersion Programs and School Policy: Conflicting Agendas

Heather K. Olson Beal

Stephen F. Austin State University

Michelle Haj-Broussard

McNeese State University

Nicole Boudreaux

Lafayette Parish School System

School districts and policymakers across the

United States are experimenting with various

reforms in an attempt to improve academic

achievement, close the achievement gap, satisfy

ongoing court-ordered desegregation mandates, and

enhance parent-school-community connections

(Aberger, Brown, Mantil, & Perkins, 2009; Caldas

& Bankston, 2005; Epstein et al., 2008; Rothstein,

2004). These reforms include curricular changes,

such as schools with a unified curricular theme, and

structural changes, such as single-sex classes, small

learning communities (SLCs), and early college

high schools (Conchas & Rodríguez, 2008; Hayes,

Pahlke, & Bigler, 2011; Hoffman & Vargas, 2005).

Early foreign language education models are

another type of promising educational reform.

Although research consistently shows that foreign

language study improves academic achievement

(Cobb, Vega, & Kronauge, 2009; Genesee, 2007;

Gómez, Freeman, & Freeman, 2005; Shneyderman

& Abella, 2009; Taylor & Lafayette, 2010), the

percentage of public elementary schools offering

foreign language instruction decreased from 24%

in 1997 to 15% in 2008 (Center for Applied

Linguistics, 2009a).

Despite the overall decrease in early foreign

language education programs, immersion programs

have grown significantly in the United States since

the first one began in 1971 to 448 programs in 38

states and Washington D.C. (Center for Applied

Linguistics, 2011). Twenty-two foreign

languages are represented, with Spanish (45%) and

French (21%) making up the majority of available

programs. There are three types of immersion

programs: total immersion (i.e., at least 90% of

instruction is delivered in the target language),

partial immersion (i.e., approximately 50% of

instruction is delivered in the target language), or

two-way or duali immersion (i.e., equal emphasis

is placed on English and a second language, with

content taught in both languages). The overarching

goal of immersion education is for students to

develop second language proficiency while

mastering subject content, which they learn through

instruction in the second language. In the United

States, immersion students usually are monolingual

English-speakers who participate in an immersion

program in order to acquire a second language.

Immersion programs vary with respect to the


addressed to Heather K. Olson Beal, Stephen F.

Austin State University, Secondary Education and

Educational Leadership Department, SFA Station

13018 Nacogdoches, TX 75962-3018

Email: [email protected]

In this position article, we explore what happens when school district policies regarding

desegregation, accountability, and foreign language immersion education collide. Specifically, we

contrast 2 immersion programs that experienced distinct outcomes as a result of the conflicting

agendas underlying these 3 policies. One program, originally created to satisfy court-ordered

desegregation objectives, grew and continues to thrive, whereas the other program was damaged and

eventually destroyed due to conflicts that emerged from the collision of the diverse objectives of these

educational policies and programs. We highlight significant implications for practice that emerge

from our analysis of conflicting policy agendas.

HEATHER K. OLSON BEAL, MICHELLE HAJ-BROUSSARD, AND NICOLE BOUDREAUX


objectives of the program, the percentage of

instruction in the target language, the

characteristics of participating students, the

teachers’ primary language, the age at which

second language instruction is initiated, and the

language used to teach basic subjects. However,

they all share three objectives: (a) linguistic:

developing second language proficiency (Collier &

Thomas, 2004; Genesee, 2004; Hammerly, 1987),

(b) academic: increasing content area academic

achievement (Cobb et al., 2009; Genesee, 2007;

Gómez et al., 2005; Shneyderman & Abella, 2009),

and (c) socio-cultural: promoting positive attitudes

towards self and others (Caldas & Boudreaux,

1999; Haj-Broussard, 2003; Ingram & O'Neill,

1999).

Immersion programs also have been used to

achieve other goals unrelated to second language

acquisition. This article focuses on two

educational policies that have been used in

conjunction with immersion programs in

Louisiana: policies designed to promote school- or

district-wide desegregation and policies designed

to raise standardized test scores (Olson Beal, 2008;

Olson Beal, Haj-Broussard, & Boudreaux, 2007).

When immersion programs are co-opted for

policy-related purposes, the objectives of

immersion education can become compromised.

What happens, for instance, when racial quotas

imposed for desegregation purposes become more

important than the immersion curriculum? What

happens when standardized test scores take

priority over target language proficiency? The

results of this complex relationship between

immersion education and educational policy

ranges from diminishing the linguistic proficiency

objective to dissolving immersion programs

altogether when they fail to achieve mandated

racial quotas or predetermined test score targets.

Despite a significant body of literature regarding

the impact of immersion programs on student

achievement (Genesee, Lindholm-Leary,

Saunders, & Christian, 2006; Gómez et al., 2005;

Shneyderman & Abella, 2009), one area that

remains largely unexamined in the literature is the

consequences of using immersion education to

address other policy objectives. Thus, this

position article explores the complexities that arise

when the goals of immersion education become

entangled with desegregation and standardized

testing policy objectives by contrasting two

immersion contexts in Louisiana. One context—

South Boulevard Foreign Language Academic

Immersion Magnetii (South Boulevard) in Baton

Rouge—was created explicitly to achieve

desegregation goals. The other context—an

immersion program at School Xiii—ultimately was

destroyed because compliance with desegregation

mandates and accountability measures took

precedence over maintaining the integrity of the

immersion program.

In the next section, we briefly review the

bodies of literature pertinent to this position article

and then describe the history and research upon

which Louisiana immersion programs are created.

We present a brief portrait of each school and then

explore what happened in two school contexts

when immersion education objectives were co-

opted to achieve other educational objectives. We

conclude with some policy implications.

Review of Relevant Literature

This section provides a brief overview of three

bodies of literature relevant to this position article:

the primary objectives of immersion education, the

primary objectives of school desegregation

policies, and the effects of accountability policy on

language education. These bodies of literature are

relevant to this position article because both

programs highlighted here have been shaped

largely by the convergence of immersion

objectives, school desegregation objectives, and

accountability policies.

Objectives of Immersion Education

Relative to the linguistic objective of

immersion education, results are mixed regarding

immersion students’ target language proficiency.

Although some research has focused on immersion

students’ near-native comprehension skills, high

levels of fluency, and confidence in speaking the

second language (Collier & Thomas, 2004; Swain,

2000), other research has focused on immersion

students’ less-than-native-like grammatical and

lexical accuracy (Hammerly, 1987; Lyster, 1998)

or their less-than-native-like productive language

skills (Genesee, 2004). In his seminal critique of

immersion education, Hammerly (1987) argued

that although immersion programs have been

culturally and politically successful, they fail

linguistically, resulting in students whose target

language proficiency is poor and underdeveloped.

Thus, one strand of immersion pedagogy research

has focused on identifying and implementing

strategies that promote greater accuracy while

simultaneously maintaining the emphasis on

communication.

Relative to the academic objective of

immersion education, much academic research

shows that immersion students experience no

negative effects on either their native language

proficiency or their achievement in other academic

subjects (Caldas & Boudreaux, 1999; Gilbert,

2001; Shneyderman & Abella, 2009). In fact,

findings from many studies show that immersion

students outperform their non-immersion peers

(Genesee, 2007; Howard, Christian, & Genesee,

2003; Howard, Sugarman, & Christian, 2003; Kirk-

FOREIGN LANGUAGE IMMERSION PROGRAMS AND SCHOOL POLICY: CONFLICTING AGENDAS


Senesac, 2002; Thomas & Collier, 2002). Caldas

and Boudreaux (1999) found that Louisiana

immersion students outperformed non-immersion

students on both the English Language Arts (ELA)

and the mathematics sections of the Louisiana

Educational Assessment Program (LEAP) iv —a

particularly important finding considering that the

mathematics instruction was conducted entirely in

French. Haj-Broussard (2003) concluded that the

French immersion education context appeared to

bridge the achievement gap between White non-

immersion and Blackv immersion students.

Regarding the sociocultural objective of

immersion education, foreign language education

advocates assert that language study should

broaden students’ horizons by promoting

appreciation, tolerance, and respect for other

cultures and peoples (National Standards in

Foreign Language Education Project, 1999).

Despite the difficulty in measuring attitudes and

beliefs, some researchers have studied the impact

of immersion programs on this important issue.

Several early studies of the relationship between

foreign language education and attitudes revealed

that immersion students made more favorable

assessments of the “Other” than did non-immersion

students (Cziko, Lambert, & Gutter, 1979; Lambert

& Tucker, 1972). Numerous other researchers

since similarly have reported that second language

education created an openness to cultural and

linguistic differences (Kirk-Senesac, 2002;

Lindholm-Leary & Borsato, 2006; Walsh, 2005).

Haj-Broussard (2003) also studied the affective

dimension of immersion education and found that

Black elementary French immersion students had a

higher collective self-esteem, particularly in

regards to how they viewed their schools, than did

Black non-immersion participants.

Objectives of School Desegregation Early school desegregation research (Coleman

et al., 1966; St. John, 1970) involved determining

the academic and social effects of school

desegregation. In terms of the academic effects,

early research indicated that Black students’

achievement increased as the proportion of White

students in a classroom or school increased

(Coleman et al., 1966; Mahard & Crain, 1983)—

findings that became a catalyst for the

implementation of busing policies to achieve racial

balance in public schools (Armor, 1972; Bradley &

Bradley, 1977; Gerard & Miller, 1975).

Additional research since then consistently has

indicated a positive relationship between

attendance at socioeconomically and racially

diverse schools and higher test scores, grades, and

high school and college completion rates (Condron,

2009; Goldsmith, 2011; Hanushek, Kain, & Rivkin,

2009; Harris, 2006; Lleras, 2008; Mickelson, 2003;

Southworth, 2010).

Numerous researchers have sought to

understand the social effects of desegregation

(Allport, 1954; Braddock II & Gonzales, 2010;

Carter, 2010; Feddes, Noack, & Rutland, 2009;

Schofield, 1991; Tatum, 2003). In a seminal

study, Allport (1954) posited that intergroup

contact would lead to reduced intergroup prejudice

if it occurred in a positive context, which he

defined as one characterized by equal status among

the groups, common goals, lack of competition

among the groups, and institutional sanction for the

contact. More recently, Carter (2010) studied the

role of racially integrated schools in the

development of “cultural flexibility” and found that

mixed-race schools promote social cohesion.

Feddes et al. (2009) and Aboud, Mendelson, and

Purdy (2003) similarly found that cross-ethnic

friendships in ethnically heterogeneous schools led

to positive out-group evaluations among majority

status children (but not minority status children).

Some studies indicate that early school racial

segregation can lead to continued segregation as

adults (Braddock II & Eitle, 2004; Braddock II &

Gonzales, 2010). Other researchers have focused

on the organizational characteristics of schools

(e.g., tracking, transportation methods,

extracurricular offerings) that promote (or inhibit)

“a sense of belonging” among students (Holland,

2012; Meier, 2002; Osterman, 2000).

Accountability Policy and Second Language

Education In addition to the academic and social

objectives of immersion education and

desegregation policies, schools also have to

contend with increased accountability measures

brought on by No Child Left Behind (NCLB) (U.S.

Department of Education, 2001). Many educators

and researchers argue that NCLB has not achieved

its primary goal of reducing the achievement gap

but rather has distorted how education is delivered

in the United States by narrowing curriculum to

include only what is tested (Boykin & Noguera,

2011; Popham, 2004; Rothstein, 2008). Another

significant body of researchers has sought to

identify the effects of increased accountability

measures (particularly high stakes testing) on the

educational experiences of English Language

Learners (Abedi, 2004; Menken, 2009; Solórzano,

2008; Wiley & Wright, 2004) and on minority

youth (McNeil, 2000; Sloan, 2007; Wiley &

Wright, 2004).

A small but growing body of research has

explored the effects of NCLB on second language

instruction and has indicated that despite the fact

that NCLB established foreign languages as a core

curricular content area, implementation of the law

has negatively affected second language programs.

According to Rosenbusch and Jensen (2005), for

instance, foreign language principals and teachers



identified numerous negative effects of NCLB.

More than one fifth of 165 respondents reported

that after passage of NCLB, their school or school

district had eliminated one or more grade levels in

their foreign language program; almost one fourth

of the respondents reported that foreign language

teaching positions had been eliminated and more

than one fifth of respondents reported that one or

more languages had been eliminated. When

asked to identify the cause of these cuts to their

foreign language programs, 43% of respondents

identified insufficient funds, 18% identified

administrator support, and 14% identified state

testing. A 2008 national survey similarly revealed

that more than one third of principals, foreign

language teachers, and department chairs from

more than 5,000 public and private schools stated

that NCLB’s focus on mathematics and reading test

scores had drawn attention and resources away

from foreign language programs (Center for

Applied Linguistics, 2009b).

Understanding these bodies of literature is

important when considering the academic and

social objectives of immersion programs that have

been established in areas (such as Louisiana) that

use immersion education in conjunction with

district desegregation policies and objectives.

Whereas other research focuses on only one or

perhaps two of these policies (Collier & Thomas,

2004; Holland, 2012; Lleras, 2008; Menken, 2009),

this article connects all three policies—immersion

education, school desegregation, and increased

accountability measures—and identifies some

recommendations for practice.

Immersion Education in Louisiana

Louisiana is an apt context through which to

explore this issue for two primary reasons. First,

Louisiana has been grappling with ongoing school

desegregation litigation since the first

desegregation case was filed by the National

Association for the Advancement of Colored

People (NAACP) in New Orleans in 1952 (Muller,

1976). Parish school districts across the state

continue to grapple with mandates to desegregate

their student populations (Bankston & Caldas,

2002). Second, Louisiana has the fourth highest

number of immersion programs in the United

States despite being only the 22nd most populous

state (Center for Applied Linguistics, 2011).

Louisiana has a rich cultural heritage, which

includes the original Native American inhabitants

and the descendants of enslaved persons from

Africa as well as French, Spanish, German, and

Acadian settlers. Louisiana culture has been

largely influenced by its French and Spanish

history. As evidence of that French legacy,

Caldas and Boudreaux (1999) asserted that as much

as 50% of the population of some parishes in

Acadiana still spoke French in the 1990s, although

many of these residents were more than 50 years of

age. Louisiana has renewed its commitment to the

revival of the French language by creating

numerous immersion programs throughout the

state.

In 1984, the Louisiana Board of Elementary

and Secondary Education (BESE) mandated that a

foreign language be taught to all academically able

students in fourth through eighth grades.vi The

result of this mandate has been the initiation and

growth of FLES (Foreign Language in Elementary

School) and immersion programs throughout the

state. As of the 2010-2011 school year, there

were 31,876 fourth- through eighth-grade students

enrolled in French, Spanish, and Latin FLES

programs and an additional 15,441 pre-K–3

students in French and Spanish FLES programs.

In Grades pre-K–8, 3,416 students participated in

French immersion programs, 1,061 in Spanish

immersion programs, and 22 in a new Chinese

immersion program, for a total of 4,499 students.

Thirty-one percent of eligible pre-K-8 Louisiana

public schools offer a foreign language. Both

programs described in this position article are

partial immersion programs, which means that

students spend approximately 60% of the

instructional day immersed in the second language

(in this case, either French or Spanish). Table 1

provides some basic information about both school

sites.

Recommendations in this position article are

based upon several data sources—namely, two

extensive case studies: a case study of South

Boulevard, a K-5 Spanish and French partial

immersion program in Baton Rouge, Louisiana

(Olson Beal, 2008); and a case study of School X, a

French partial immersion program in south central

Louisiana (Haj-Broussard, 2003). Thus, findings

from this article do not constitute original empirical

research; rather, they are the result of our

experience conducting research in two study sites,

as well as our combined lived experiences as

immersion teachers, researchers, administrators,

and parents (Patton, 1990). All three authors have

connections to the school settings that provided us

with unique vantage points from which to explore

these issues. One author was an immersion

teacher and is the current president of the Louisiana

Consortium of Immersion Schools, one was a

parent of two immersion students, and one is a

district immersion program coordinator in

Louisiana. All three authors have conducted

research on immersion programs in Louisiana. A

brief summary of the research design of the two

original studies follows.

Data and recommendations regarding South

Boulevard Elementary are based on an

ethnographic case study involving document

analysis, interviews, and participant observation.



In-depth interviews were conducted with 53

students, parents, school faculty, district

administrators, and school board members. Using

purposeful sampling, participants were selected

who represented diverse backgrounds and

perspectives. All interviews were recorded and

transcribed. On-site participant observation

(including classes, recess, lunch, PTO activities and

meetings, and school board meetings) was

conducted for 1 school year.

Table 1.

Immersion Contexts in Study

Data and recommendations regarding School X

stem from a mixed methodology study that examined

academic achievement and experiences of Louisiana

fourth-grade students and teachers in both the regular

education and the French immersion contexts. The

quantitative phase compared these students' LEAP

test scores using an analysis of covariance,

controlling for the Iowa Tests of Basic Skills (ITBS)

scores.vii The qualitative phase was a cross-case

comparison of four classrooms—an extreme class

(90% of the school population in poverty) and a

typical class (Black students of average academic

achievement) in each context using observation,

interviews, and questionnaires. School X’s

participating class—a fourth-grade class of students

in their fifth year of the French immersion

program—was the extreme French immersion case

study from the larger study. All 14 students in the

class were Black and 93% received free/reduced

lunch. Information on the demise of School X,

which occurred after the above study, was drawn

from authors’ lived experiences as officers in the

Louisiana Consortium of Immersion Schools and

from board minutes from the school district

meetings.

In both studies, all interviewees signed Informed

Consent forms and were promised confidentiality.

Thus, pseudonyms are used throughout this article to

protect the identities of study participants. In terms

of fieldwork, detailed field notes were kept along

with a contact summary sheet for each site visit

(Miles & Huberman, 1984). All the data (i.e.,

interview transcripts, field notes) were broken down

into units of meaning (Lincoln & Guba, 1985) and

then the data were organized into discrete categories

that represented events, ideas, or themes that

emerged from the data via an open coding process

(Strauss & Corbin, 1998). The categories then were

compared with one another via the method of

constant comparison (Glaser & Strauss, 1967). A

number of strategies were implemented in both

studies to enhance the credibility, transferability,

dependability, and confirmability of the findings,

including prolonged engagement at the study site

(i.e., 1 school year at South Boulevard and 4 months

at School X), data triangulation through the use of

multiple sources (i.e., follow-up interviews at both

sites, questionnaires at School X, observation,

archival research), peer debriefing, member checks,

and thick description (Geertz, 1973; Lincoln & Guba,

1985; Patton, 2002).

School Location Type of

Program

Admissions

Criteria

Target Student

Demographic

Quotas

Actual Student

Demographics

at time of study

Original

Research

Time

frame

Current

Status

South

Boulevard

Foreign

Language

Academic

Immersion

Magnet

Baton

Rouge

K-5 partial

immersion

in either

French or

Spanish

*Academic

screening

*must be

on-grade

level

1996-2007:

race-based

quota (60%

Black, 40%

non-Black)

2008-present:

family income

quota (55%

full-pay/45%

free or reduced

lunch)

2007: 58%

Black, 42%

non-Black

59%

free/reduced

lunch

08/2006–

08/2007

Program

still

thriving

School X

Acadiana

area

K-5

partial

immersion

in French

*No criteria

*Parents

must

request the

program

More than

90% free and

reduced lunch

or 90% Black

100% Black

93%

free/reduced

lunch

08/2001-

08/2002

Program

closed

in 2003



We conclude this section with a final word

regarding transferability or applicability (or the more

traditional term, generalizability). Although the

two cases examined in this article included a

relatively small number of participants, thereby

potentially limiting the generalizability of the

findings, the authors were able to probe deeply into

the stakeholders’ lived experiences in those contexts.

This thick description allows for a high level of

transferability and applicability. The potential

conflicts that arise when educational reforms have

multiple and sometimes competing objectives are not

unique to Louisiana. Therefore, the positions taken

in this article might be applicable to other

geographical areas and educational contexts and

likely would be of interest to multiple groups who

have an interest in the success of public education:

parents, teachers, administrators, politicians,

policymakers, and concerned citizens and community

members.

South Boulevard: Created for Desegregation

Purposes

The immersion program at South Boulevard, an

urban school located in a historically Black

neighborhood in downtown Baton Rouge, was

originally created in 1996 by a court-approved

Consent Decree (U. S. District Court Middle District

of Louisiana, 1996) to the desegregation litigation

that was originally filed in 1965 with the express

purpose of creating a racially desegregated student

population. Findings in the following section—in

which we explore the three primary objectives of

immersion education and how each was impacted by

the imposition of one or more additional educational

policies—are based on data derived from interviews

and participation observation from the original study

(Olson Beal, 2008).

The development of students’ linguistic skills is

a primary objective of immersion education

(Johnstone, 2002). Participant observation in

immersion classes confirmed that students made

numerous grammatical and syntax errors in speech—

particularly when they branched out to conversation

topics outside the school setting—yet, these mistakes

did not impede communication. Although no

official tests had been administered, South Boulevard

immersion teachers developed their own end-of-

school oral proficiency interview that teachers

conducted to assess students’ target language oral

proficiency. Students were interviewed in their

second language by a teacher (other than their regular

classroom teacher) who assessed their speech on a

scale from 1 (needs work) to 4 (very good) according

to fluency, comprehension, and vocabulary.

Because of confidentiality rules and norms, we did

not have access to these test scores. However,

during an interview, the head immersion teacher

recalled staff members’ reactions after administering

the test for the first time: “We realized that the

children could understand everything we were

saying, but they could only give us a little back. Or

they couldn’t explain themselves.” Staff members

used results from the yearly oral proficiency testing

to identify weaknesses and create campus-wide goals

to strengthen the students’ target language

proficiency. Because reading and writing in the

target language are not a focus of the curriculum,

students’ skills in these two areas are not as strong.

In general, the head immersion teacher reported that

South Boulevard students are “very good orally.

They need to work on their grammar skills.

Especially on written work.”

South Boulevard students spoke their target

language with confidence and ease. One Spanish

immersion teacher originally from Venezuela

concurred, noting the following in an interview:

“These kids, they have incredible Spanish. The way

they talk is incredible. Their pronunciation,

especially. They speak really good Spanish.”

They are willing to take risks with the language,

often speaking “Franglais” or “Spanglish”—as do

their native-speaking teachers. For example, during

fieldwork, a fourth-grade teacher asked her class to

explain what a “recurso natural” (natural resource)

was. A boy immediately raised his hand and

offered the following impromptu explanation: “Un

recurso natural es una cosa que una persona no build;

es de nature” (“A natural resource is something that

a person doesn’t build; it’s from nature”). He

neither stumbled nor hesitated. The teacher

enthusiastically accepted his response and continued

with the lesson. In sum, South Boulevard students’

target language skills are something of a mixed bag.

They make mistakes in oral and written

communication; yet, they also understand the target

language, speak it fluently, and are understood by

each other, by their teachers, and by native speakers

outside the South Boulevard community.

Increasing student achievement in academic

content areas is another objective of immersion

education. Although there are numerous ways to

judge a school’s success, because standardized test

scores are currently the most widely accepted

measure used to evaluate students and schools, they

warrant attention here. Findings from the larger

case study (Olson Beal, 2008) revealed a marked

emphasis on standardized testing throughout the

school year despite the principal’s concern expressed

during interviews that there was too much emphasis

on standardized testing. The LEAP is administered

in the fourth grade—the fifth year of students’

immersion education. Participant observation at the

school revealed a noticeable increase in the quantity

of English used in fourth-grade classrooms by both

teachers and students compared to the lower grades.

During interviews, teachers noted that this shift in

emphasis was due to pressure from both the principal

and the parents to prepare students for this test.



Parents of fourth graders are required to attend

an informational meeting about the LEAP test every

fall. During the meeting, the principal explains the

importance of the test—both to individual students

(who must receive a score of Basic or above on the

mathematics and ELA portions of the test in order to

be promoted to fifth grade) and to the school

(because test scores comprise 90% of the school

performance score and monetary awards are given

for performance). The fourth-grade teachers

described the kinds of questions students can expect

and explained how student responses are graded.

Parents also learned that students would receive a

LEAP packet every Monday that needed to be

completed and brought back to school every Friday.

All these LEAP practice questions were written in

English. Thus, although immersion students in

other grades completed homework exercises in the

second language, the fourth graders spent significant

amounts of time completing homework in English.

The teachers spent part of each day, also in English,

reviewing the practice questions. One Spanish

teacher even explained that they “had to tell the

students some of the words in English, because that’s

the way they will see it on the test.” Thus, in this

case, the linguistic proficiency objective was

compromised—although perhaps only temporarily—

in an attempt to reach the academic objective of

increased test scores.

Considering that students received 60% of their

instruction in a second language and that the tests

were in English, one might expect immersion

students to underperform compared to similar peers

on the standardized tests. Instead, analysis of

School Report Cards from 1998-2010 viii revealed

that South Boulevard fourth graders had consistently

scored better on the LEAP than did other students in

East Baton Rouge Parish (EBRP) and in the state of

Louisiana. Figures 1 and 2 belowix illustrate trends

in ELA and mathematics test scores at South

Boulevard compared to the district and the state.

These test scores make South Boulevard fifth out of

54 elementary schools in reading and fourth in

mathematics. Thus, the program is considered

successful because of its students’ standardized test

scores.

The promotion of positive attitudes towards

other cultures, languages, and peoples is a third

objective of immersion education. Although the

South Boulevard immersion curriculum did not

explicitly include multicultural education, teachers in

the sample believed that participation in the

immersion program made students more accepting

and more aware of cultural differences. The head

immersion teacher explained during interviews that

the immersion curriculum exposed students to

diversity to a greater degree than did a traditional

curriculum:

I do think it makes them more culturally aware.

We have Vietnamese children here. We have

Hispanic children here. We have East Indian

children here. So we have a mixed culture of

students and parent population that they get to be

exposed to and see that we’re all people. I

really think they get a much better sense of that.

Madame Rivet, a native Francophone teacher from

Belgium, noted that being in the immersion program

“opens the mind of the children that there is not just

their country, that not everyone is the same.”

Seňore Gonzalez, a Spanish immersion teacher

originally from Colombia, similarly commented that

the immersion curriculum, by definition, promotes

respect, explaining: “This is the idea. You are in

immersion, and immersion means that you need to

know other cultures. You have to respect.”

Cross-racial friendships were commonplace at

South Boulevard. Several parents in the larger

study (Olson Beal, 2008) commented that they liked

the diversity in which their children were immersed

at South Boulevard and identified diversity as a

primary factor that motivated them to choose the

school. David, for example, explained that he did

not think race impacts social relationships at South

Boulevard and described his son’s three best friends

via the following quotation: “Michael is Asian,

James is White, and Anthony is Black.” Ms.

Brown, a veteran South Boulevard ELA teacher,

explained that “for years there have been friendships

across racial lines.” The majority of on-site

observations confirmed positive, cross-racial social

interactions during recess and in classrooms.

Figure 1. Percentage Proficient (Basic or Above),

fourth-grade English Language Arts (ELA) LEAP

scores.

0

20

40

60

80

100

120

1999

2001

2003

2005

2007

2009

South

Boulevard

District

State



Figure 2. Percentage Proficient (Basic or Above),

fourth-grade Mathematics LEAP scores.

Furthermore, the immersion curriculum creates

a unique subculture that adds to the cultures and

identities the students bring with them to school and

creates a culture of integration that nurtures positive

social relationships. The immersion curriculum

causes students to expand upon or to multiply their

identities to include an identity of themselves as

French or Spanish speakers. They either adopt a

new name in Spanish or French or their teachers

pronounce their English names with Spanish or

French pronunciation. Students refer to their

teachers with Spanish and French titles. They refer

to each other as “Spanish kids” and “French kids”

rather than referring to themselves and others as

White, Black, Mexican, or Asian. This new role

allows students to see themselves as members of a

unique community of learners in which all students

are equal. The immersion curriculum is like a third

or an in-between space that belongs equally to all the

students because it is new and unrelated to students’

race, language, family socioeconomic status, or

gender.

In addition to these three objectives, the

immersion program at South Boulevard has another

unique objective that perhaps trumps the other three

in terms of continued administrative support of the

program: the achievement of specific racial and

socioeconomic status quotas. Between 1996 and

2006, the targeted racial quota for South Boulevard

was 55% Black students and 45% non-Black

students. Between 1996 and 2002, non-Black

student enrollment followed the same downward

trend that EBRP as a whole experienced because

many non-Black parents chose to send their children

to private schools and schools in outlying parishes

(Bankston & Caldas, 2002). Beginning in 2002,

however, non-Black enrollment began to increase.

Figure 3 compares student enrollment at South

Boulevard since the immersion program began in

1996 to EBRP as a whole.x

Figure 3. Percentage of non-Black enrollment at

South Boulevard and in the EBRP district, 1995-

2007.

In 2007, the parish moved from race-based

quotas to a family income quota (55% full-pay lunch,

45% free and/or reduced lunch). If and when the

school receives more applications than it has

available seats (which occurs every year), all student

names go into a lottery that admits students

according to the family income quota and an

academic screening that ensures that they are at least

on-grade level. The inclusion of a minimum

academic standard for admission is unusual for

immersion programs, which generally take all

interested students. In fact, at the time of the study,

South Boulevard’s immersion program was one of a

few in Louisiana to include a minimum academic

standard for admission. However, the inclusion of

the academic screening was not a district or a school-

based strategy; rather, it was implemented in

conjunction with ongoing parish desegregation

litigation. As part of the 4-year term of the Final

Settlement Agreement (U. S. District Court Middle

District of Louisiana, 2003), the EBRP school board

agreed to operate an academic theme strand of

dedicated magnet schools, which included South

Boulevard. Addition of the academic label meant

that all students would be screened to ensure that

they were at least on grade level and would have to

maintain a minimum 2.5 GPA to remain in the

program. One participant (a former school board

president) in the larger study explained that this

decision was made to appease the plaintiffs of the

original desegregation lawsuit, who wanted to see

students tested to get into magnet programs. Thus,

the immersion admission policy was dictated—at

least in part—by ongoing school desegregation

policy.

0

20

40

60

80

100

1999

2001

2003

2005

2007

2009

South

Boulevard

District

State

0

10

20

30

40

50

60

70

80

90

100

1995 1999 2002 2004 2007

Perc

en

tag

e o

f n

on

-bla

ck e

nro

llm

en

t

Year

South Boulevard



The success of the immersion magnet program at

South Boulevard has been measured principally

according to the degree to which it has achieved the

objectives of school desegregation: creating a more

racially desegregated student body and increasing

student achievement. South Boulevard has been

largely successful at both creating an integrated

student population and increasing student

achievement. The degree to which South Boulevard

has been successful at achieving one of the primary

goals of immersion education—second language

proficiency—has not been considered by district

administrators. During interviews, EBRP district

administrators were asked whether they considered

South Boulevard to be a successful school and, if so,

in what ways. All of the administrators stated that

South Boulevard was a successful school for a

variety of reasons, including “a positive trend in

improvement in student achievement by standard

measures,” “meets the needs of its clients,” and “the

parents are happy; the kids are happy.” None of the

administrators in the sample mentioned any aspect of

second language acquisition in their explanation of

why South Boulevard was a successful school. The

primary issues they mentioned were the racial

composition of the school, students’ test scores, and

parental satisfaction. Thus, desegregation

objectives take precedence over immersion

objectives.

School X: Destroyed by Desegregation and

Accountability Policies In contrast to South Boulevard, which was

created because of school desegregation objectives,

desegregation and accountability policies ultimately

destroyed School X’s immersion program. The

immersion program at School X was negatively

affected by conflicts among immersion education

objectives, testing policies brought about by

increased accountability measures, and school

desegregation policies. As mentioned earlier, a

primary objective of immersion education is the

development of students’ linguistic proficiency. In

the case of School X, the second language

development objective was compromised by policies

imposed by the school principal (who had no

background or training in immersion education) in

the hopes of increasing student test scores. For

instance, the principal required all teachers to

compartmentalize, with different teachers selecting

certain content areas to teach and students moving

from teacher to teacher rather than remaining with a

single teacher who would teach all the subject areas.

She believed that this compartmentalization would

ultimately lead to higher student achievement by

reducing the amount of planning time for teachers

and allowing them to focus on teaching a single

subject area.

This compartmentalization meant that

immersion students had one teacher for ELA, another

for science and mathematics, another for social

studies, and a fourth for French Language Arts—a

context that made it difficult for the immersion

teachers to create a cohesive immersion experience

for their students. The students started each day in

English, stayed in the same class for a French lesson,

and then moved down the hall to their French

mathematics class. For their French social studies

lesson, they went downstairs, outside and across

campus. Finally, for their French science lesson,

they once more went across campus and upstairs

until they were back in the same room as their

mathematics lesson. Having three separate teachers

teach them in French created a disarticulated system.

Rather than meeting the goal of the school, which

was to strengthen subject knowledge, the constant

movement resulted in the loss of valuable

instructional time and an inability to create a

cohesive French environment necessary for the

development of second language proficiency. Thus,

an instructional policy meant to strengthen test scores

actually weakened French immersion students’

content knowledge as well as their second language

proficiency.

Another school policy that weakened students’

French language skills was the principal’s insistence

that because the LEAP test was in English, the

students needed to have anything they did not

understand translated into English. This practice

kept the students from having to work in and

negotiate with French—an important process of

second language acquisition. The students simply

pulled out what the researcher referred to as their

LEAP Trump Card and told the teacher that s/he

must translate for them. The following is a

quotation from the field notes of an interaction

between a fourth-grade immersion student and her

French immersion social studies teacher, M. Kaiga:

Rashona says that she doesn't understand and

that the test will be in English. Kaiga says the

test will be "en françaisvendredi"(in French

Friday). Rashona says no "the LEAP test

will be in English" and she doesn't

understand. (FN, A3, January 24, 2002)

Due to the disarticulated schedule and the LEAP

trump card, the students rarely spoke in French in the

fourth grade at School X. Thus, in order to keep

class going as much as possible in French, the

teachers began to employ what the researcher

described as a language safety net—jumping in

whenever students stumbled with the language to

provide the language for them. Again, this practice

kept students from working and negotiating with the

language. The following is another quotation from

the field notes of an interaction between two fourth-

grade immersion students and M. Kaiga:

Rashona asks what something is. Brianca starts

to explain in French, but Madame Maurice takes

over and answers Rashona's question. Brianca



keeps explaining only quietly (to herself in

French) (FN, A4, January 28, 2002).

These issues stemmed from the principal’s and

parents’ inability to understand the immersion

paradox—that more time in the second language

leads to higher academic achievement in the core

content areas (Johnstone, 2002).

These differences in scheduling and instruction

had ramifications in terms of the students’ academic

achievement. Considering the immersion students’

strong performance in 2001 (with a typical

immersion setting in which students had a single

teacher for mathematics, science and social studies

who spoke in the target language the entire time),

there were some surprising findings in terms of

students’ academic achievement in 2002 (see Table

2). The highest achieving boy whose mathematics

ITBS score was 200 (above the national norm) in

2001 failed the mathematics portion of the LEAP test

in 2002. So although his first 4 years of immersion

(pre-LEAP test pressures and with a single

immersion teacher each year rather than numerous

teachers with compartmentalized subjects) were very

successful, the experiences in his fourth-grade year

negatively affected his test scores.

A descriptive examination of the data shows a

decline in academic achievement throughout the

class when comparing third-grade ITBS scores to

fourth-grade LEAP scores. Table 2 includes

students’ scores on both measures, labeling those

scores as At or Below level, and then marking with

an asterisk when those scores were deemed

unacceptable by the state rating system. Whereas

more than 70% of the students were at or above the

national average of 185 on the ITBS in the third

grade, only approximately 40% were at the basic

level after fourth grade. Furthermore, one half of

the students who failed the state accountability test in

the fourth grade were at or above the national

average in the third grade.

Table 2

School X ITBS and LEAP Achievement Scores

Student number

Mathematics

ITBS scores (185

= national norm)

On/above or

below the

national average

Mathematics

LEAP scores

On or below level

(315 is on

level/basic)

Gain

(+),

loss

(-) or

=

1 202 On 304 Below -

2 187 On 295 Below -

3 203 On 326 On =

4 209 On 356 On =

5 219 On 365 On =

6 188 On 323 On =

7 201 On 323 On =

10 170 Below 289 Below =

11 186 On 241 Below* -

12 200 On 267 Below* -

13 179 Below 283 Below =

14 155 Below 256 Below* =

In addition to these structural and pedagogical

challenges, the principal also struggled adequately

to train and to mentor the immersion teaching

corps. Although the three immersion teachers at

School X were native French speakers, they were

new to immersion education, to Louisiana, and to

the United States, and, thus, had professional

development needs that were unmet. For

example, during fieldwork, the French Language

Arts teacher, who also taught daily 30-minute

French classes to the non-immersion students,

informed the researcher that she did not even

realize that the immersion class was taught using

immersion methods; she informed the researcher

she thought they were the smart fourth-grade class.

Thus, the French immersion teachers—although

fluent French speakers and experienced

educators—needed professional development in the

grammar of U.S. schooling (Tyack & Cuban,

1995), as well as training in immersion pedagogy.

In the midst of School X’s teachers’ efforts to

increase student achievement and promote second

language proficiency, School X came under

scrutiny because its student population was racially

identifiable. xi The school board—in order to

comply with a federal desegregation order—



decided to reassign teachers to racially balance the

district’s teaching corps across its campuses.

Thus, the strongest immersion teachers from

School X—three African immersion teachers who

had developed bonds with the Black students at

School X—were transferred to a majority White

campus. In the following quotation, one of the

African immersion teachers described the

similarities they had discovered between

themselves and their Black students:

When I first came here they told me, OK,

your school is the majority Black students.

So, this was a big surprise for me. First, I

was expecting to have White children in my

class . . . I discovered that we have some

things in common that are related to culture . .

. and also some people speak French. A kind

of French, which is different from what I

studied at school, but it's French. I can talk

with them and communicate and this is also

amazing … for us the United States there is

no other language—just English. (INT, Kaiga,

January 15, 2002)

Unfortunately, the ethnic and cultural similarities

that M. Kaiga identified were considered a negative

under the desegregation order, and the teachers

were not given a chance to capitalize on those

similarities. Instead, several weaker White

teachers were transferred into School X—a

majority Black campus. One of the White

teachers who was transferred to School X had been

asked to leave a neighboring parish school the

previous year because she was unsuccessful at

teaching at the majority Black campus. Once this

unsuccessful teacher had joined School X, the

attrition rate in the immersion classroom

accelerated.

The immersion program—already weakened

by scheduling constraints and pedagogical

challenges that emerged due to the principal’s lack

of understanding of immersion education—

suffered further because of staffing decisions made

to satisfy a school desegregation mandate. In this

case, the pressures of accountability and the

implementation of the district’s school

desegregation policies resulted in the destruction of

one of the last majority Black French immersion

programs in the state.

Notes Toward a Conclusion

In this article, we contrasted two immersion

programs that have experienced unique outcomes

as a result of educational policies with conflicting

agendas. In the case of South Boulevard, school

desegregation policies largely have dictated the

particulars of the immersion program from its

inception. South Boulevard’s immersion program

is evaluated primarily according to two criteria:

(a) its success at creating a student body that meets

first the racial and now socioeconomic targets

dictated by federal school desegregation policy and

(b) its students’ test scores. South Boulevard’s

students’ Spanish and French language proficiency

is not a criterion by which the school is evaluated

either by the school principal or district

administration, who are largely unaware of the

unique programmatic and pedagogical concerns of

immersion education. Thus, a desire to increase

student test scores often dictates scheduling

concerns and other pedagogical issues, such as

homework and the use (or lack of use) of the

second language in the classroom. The need to

remain in compliance with desegregation mandates

also dictates admission policies. Despite these

potentially troubling issues, South Boulevard’s

immersion program is thriving. Its students are

achieving success on their state’s standardized test,

they have a racially and socioeconomically diverse

student population and teaching staff, and they

continue to make strides in terms of second

language proficiency.

In contrast, the stakeholders' reactions to the

challenges brought on by conflicting policies

ultimately led to the demise of the immersion

program at School X. School and district

administrators—unaware of the unique needs of

immersion education—implemented scheduling

and staffing decisions that compromised the needs

of the immersion program. Efforts to increase

students’ test scores conflicted with students’

second language needs and their second language

development was ultimately derailed. Students

suffered further when teachers were reassigned to

satisfy district desegregation mandates. Thus, the

immersion program was systematically weakened

due to the pressures of standardized tests and

misguided school administration that did not

understand immersion and, eventually, the program

was closed.

What lessons did we learn from this analysis of

the complex interactions among competing

educational policies at these two schools? First,

as immersion educators, we must increase our

awareness and knowledge of educational policies

that affect our programs. We learned that these

immersion programs—like other programs with

specialized curricula—do not exist in isolation;

rather, they have their own unique objectives that

must be understood in conjunction with non-

immersion objectives. In order for these

immersion programs to thrive, their stakeholders

(including administrators, teachers, parents, and

even students) must become adept at negotiating in

order to protect the interests of their program. If

we do not advocate for maintaining the integrity of

the immersion programs with which we work,

immersion objectives might be compromised and

the programs themselves might be eliminated so

that other programs or interests might be served.



Second, the experiences of the immersion

programs at South Boulevard and School X

highlight a need for multiple layers of professional

development. Our school- and district-level

administrators need professional development that

will further their knowledge of immersion

pedagogy and allow them to make decisions that

will enable the immersion programs under their

purview to be successful. School- and district-

level administrators in these two cases also need

professional development to help them understand

the socio-cultural issues that many international

teachers face when they come to Louisiana to teach

in our public schools. Lastly, immersion teachers

at these schools need professional development to

deepen their understanding of immersion

pedagogy, allowing them to become teacher-

leaders on their campuses who are able to advocate

for their students’ academic and linguistic needs

and contribute in positive ways to the dialogue and

negotiation necessary for unique educational

programs with sometimes conflicting objectives to

thrive.

Lastly, in this era of increasing competition

among traditional public, charter, and private

schools, all stakeholders in these programs need to

understand and be explicit about the goals of their

programs. School boards, administrators,

teachers, and parents might have different primary

objectives. Immersion programs—particularly in

the case of Louisiana—often are used to entice

White and higher income parents to choose public

over private education. Thus, one group of

stakeholders might want to create a program that

will be the best carrot to attract parents to public

schools with unique educational programs.

Another might want to create a program that will

help its students achieve the highest test scores

possible. Yet another might want a program that

will enable students to develop high levels of

second language proficiency and intercultural

competence. These goals might conflict with

each other, which might lead to misunderstandings

and disappointment across all parties. Thus,

transparency and diplomacy are key to creating

programs that can meet the diverse needs of all

invested stakeholders.

The lead editors for this article were Anthony J.

Onwuegbuzie and John R. Slate.

References

Abedi, J. (2004). The No Child Left Behind Act

and English language learners:

Assessment and accountability issues.

Educational Researcher, 33(1), 4-14.

doi:10.3102/0013189X033001004

Aberger, S., Brown, B., Mantil, A., & Perkins, A.

(2009). Closing the student achievement

gap: The overlooked strategy of

socioeconomic integration. Paper

presented to Prof. Jal Mahta, Harvard

Graduate School of Education. Retrieved

from

http://a100educationalpolicy.pbworks.com

/f/Closing+the+Achievement+Gap+-

+Socioeconomic+Integration.pdf

doi:10.1080/01650250244000164

Aboud, F. E., Mendelson, M. J., & Purdy, K. T.

(2003). Cross-race peer relations and

friendship quality. International Journal

of Behavioral Development, 27, 165-173.

doi: 10.1080/01650250244000164

Allport, G. W. (1954). The nature of prejudice.

Cambridge, MA: Addison-Wesley.

Armor, D. J. (1972). The evidence on busing.

Public Interest, 28, 90-126. Retrieved

from

http://www.nationalaffairs.com/public_int

erest/issues/summer-1972

Bankston, C. L., & Caldas, S. L. (2002). A troubled

dream: The promise and failure of school

desegregation in Louisiana. Nashville,

TN: Vanderbilt University Press.

Boykin, A. W., & Noguera, P. (2011). Creating the

opportunity to learn: Moving from

research to practice to close the

achievement gap. Alexandria, VA:

Association for Supervision and

Curriculum Development.

Braddock II, J. H., & Eitle, T. (2004). The effects

of school desegregation. In J. Banks & C.

M. Banks (Eds.), Handbook of research

on multicultural education (2nd ed., pp.

828-843). San Francisco, CA: Jossey-

Bass.

Braddock II, J. H., & Gonzales, A. D. C. (2010).

Social isolation and social cohesion: The

effects of K–12 neighborhood and school

segregation on intergroup orientations.

Teachers College Record, 112, 1631-

1653.

Bradley, L. A., & Bradley, G. W. (1977). The

academic achievement of black students in

desegregated schools: A critical review.

Review of Educational Research, 47, 399-

449. doi:10.3102/00346543047003399

Caldas, S. J., & Bankston, C. L. (2005). Forced to

fail: The paradox of school desegregation.

Westport, CT: Praeger.

Caldas, S. J., & Boudreaux, N. (1999). Poverty,

race, and foreign language immersion:

Predictors of academic achievement.

Learning Languages, 5(1), 4-15.

Carter, P. L. (2010). Race and cultural flexibility

among students in different multiracial



schools. Teachers College Record, 112,

1529-1574.

Center for Applied Linguistics. (2009a). Directory

of two-way bilingual immersion programs

in U.S. schools. Center for Applied

Linguistics. Retrieved from

http://www.cal.org/twi/directory/index.ht

ml

Center for Applied Linguistics. (2009b). World

language teaching in U.S. schools:

Preliminary results from the National K-

12 Foreign Language Survey. Retrieved

from

http://www.cal.org/projects/flsurvey.html

Center for Applied Linguistics. (2011). Directory

of partial immersion programs in U.S.

schools. Washington, DC: Center for

Applied Linguistics. Retrieved from

http://www.cal.org/resources/immersion/

Cobb, B., Vega, D., & Kronauge, C. (2009).

Effects of an elementary dual language

immersion school on junior high school

achievement. In D. L. Hough (Ed.),

Middle grades research: Exemplary

studies linking theory to practice (pp. 1-

20). Charlotte, NC: Information Age

Publishing.

Coleman, J. S., Campbell, E., Hobson, C.,

McPartland, J., Mood, A., Weinfeld, F., et

al. (1966). Equality of educational

opportunity. Washington, DC: U.S. Office

of Education.

doi:10.1080/0020486680060504

Collier, V. P., & Thomas, W. P. (2004). The

astounding effectiveness of dual language

education for all. NABE Journal of

Research and Practice, 2(1), 1-19.

Conchas, G. Q., & Rodríguez, L. F. (2008). Small

schools and urban youth: Using the power

of school culture to engage students.

Thousand Oaks, CA: Corwin Press.

Condron, D. (2009). Social class, school and non-

school environments, and Black/White

inequalities in children's learning.

American Sociological Review, 74, 683-

708. doi:10.1177/000312240907400501

Cziko, G. A., Lambert, W. E., & Gutter, R. (1979).

French immersion programs and students'

social attitudes: A multidimensional

investigation. Working Papers on

Bilingualism, 19, 13-28.

Epstein, J., Sanders, M. G., Sheldon, S., Simon, B.

S., Salinas, K. C., Jansorn, N. R., et al.

(2008). School, family, and community

partnerships: Your handbook for action

(3rd ed.). Thousand Oaks, CA: Corwin

Press.

Feddes, A. R., Noack, P., & Rutland, A. (2009).

Direct and extended friendship effects on

minority and majority children’s

interethnic attitudes: A longitudinal study.

Child Development, 80(2), 377-390.

doi:10.1111/j.1467-8624.2009.01266.x

Geertz, C. (1973). The interpretation of cultures.

New York, NY: Basic Books.

Genesee, F. (2004). What do we know about

bilingual education for majority language

students? In T. K. Bhatia & W. C. Ritchie

(Eds.), Handbook of Bilingualism and

Multiculturalism (pp. 547-576). Malden,

MA: Blackwell.

Genesee, F. (2007). French immersion and at-risk

students: A review of research evidence.

The Canadian Modern Language Review,

63, 655-688. doi:10.3138/cmlr.63.5.655

Genesee, F., Lindholm-Leary, K., Saunders, W.

M., & Christian, D. (Eds.). (2006).

Educating English language learners: A

synthesis of research evidence. New York,

NY: Cambridge University Press.

doi:10.1017/CBO9780511499913

Gerard, H. B., & Miller, N. (Eds.). (1975). School

desegregation: A long-term study. New

York, NY: Plenum Press.

Gilbert, S. M. (2001). The impact of two-way dual-

language programs on fourth-grade

students: Academic skills in reading and

math, language development, and self-

concept development. (Unpublished

doctoral dissertation). New Mexico State

University, Las Cruces, NM.

Glaser, B., & Strauss, A. (1967). Discovery of

grounded theory. Chicago, IL: Aldine.

Goldsmith, P. R. (2011). Coleman revisited: School

segregation, peers, and frog ponds.

American Educational Research Journal,

48, 508-535.

doi:10.3102/0002831210392019

Gómez, L., Freeman, Y., & Freeman, D. (2005).

Dual language education: A promising 50-

50 dual language model. Bilingual

Research Journal, 29, 145-164.

doi:10.1080/15235882.2005.10162828

Green v. School Board of New Kent County,

Virginia, 391 U.S. 430 C.F.R. (1968).

Haj-Broussard, M. (2003). Language, identity and

the achievement gap: Comparing

experiences of African-American students

in a French immersion and a regular

education context. (Unpublished doctoral

dissertation). Louisiana State University,

Baton Rouge, LA.

Hammerly, H. (1987). The immersion approach:

Litmus test of second language acquisition

through classroom communication.

Modern Language Journal, 71, 395-401.

doi:10.1111/j.1540-4781.1987.tb00378.x

Hanushek, E. A., Kain, J. F., & Rivkin, S. G.

(2009). New evidence about Brown v.

Board of Education: The complex effects



of school racial composition on

achievement. Journal of Labor

Economics, 27, 349-383.

doi:10.1086/600386

Harris, D. N. (2006). Lost learning, forgotten

promises: A national analysis of school

racial segregation, student achievement

and controlled choice plans. Washington,

DC: Center for American Progress.

Retrieved from

http://www.americanprogress.org/issues/2

006/11/pdf/lostlearning.pdf

Hayes, A. R., Pahlke, E. E., & Bigler, R. S. (2011).

The efficacy of single-sex education:

Testing for selection and peer quality

effects. Sex Roles, 65(9-10), 693-703.

doi:10.1007/s11199-010-9903-2

Hoffman, N., & Vargas, J. (2005). Early College

High School Initiative: Integrating grades

9 through 19: The Bill and Melinda Gates

Foundation. Retrieved from

http://eric.ed.gov/PDFS/ED486161.pdf

Holland, M. M. (2012). Only here for the day : The

social integration of minority students at a

majority white high school. Sociology of

Education, 85, 101-120.

doi:10.1177/0038040712440789

Howard, E. R., Christian, D., & Genesee, F.

(2003). The development of bilingualism

and biliteracy from grade 3 to 5: A

summary of findings from the

CAL/CREDE study of two way immersion

education. Washington, DC: Center for

Applied Linguistics. Retrieved from

http://www.cal.org/crede/pdfs/rr13.pdf.

Howard, E. R., Sugarman, J., & Christian, D.

(2003). Trends in two-way immersion

education: A review of the research.

Washington, DC: Center for Applied

Linguistics. Retrieved from

http://www.csos.jhu.edu/crespar/techRepo

rts/Report63.pdf

Ingram, D. E., & O'Neill, S. (1999). Cross-cultural

attitudes as a goal of language teaching in

the global context (No. ED430398).

Washington, DC: ERIC. Retrieved from

http://eric.ed.gov/PDFS/ED430398.pdf

Johnstone, R. (2002). Immersion in a second or

additional language at school: A review of

the international research: Retrieved from

the Scottish CILT website:

http://www.scilt.stir.ac.uk

Kirk-Senesac, B. V. (2002). Two-way bilingual

immersion: A portrait of quality

schooling. Bilingual Research Journal,

26(1), 85-102.

Lambert, W. E., & Tucker, R. (1972). Bilingual

education of children: The St. Lambert

experiment. Rowley, MA: Newbury

House.

Lincoln, Y., & Guba, E. G. (1985). Naturalistic

inquiry. Beverly Hills, CA: Sage.

doi:10.1016/0147-1767(85)90062-8

Lindholm-Leary, K. J., & Borsato, G. (2006).

Academic achievement. In F. Genesee, K.

Lindholm-Leary, W. Saunders & D.

Christian (Eds.), Educating English

language learners: A synthesis of research

evidence (pp. 176-222). New York, NY:

Cambridge University Press.

doi:10.1017/CBO9780511499913.008

Lleras, C. (2008). Race, racial concentration, and

the dynamics of educational inequality

across urban and suburban schools.

American Educational Research Journal,

45, 886-912.

doi:10.3102/0002831208316323

Lyster, R. (1998). Negotiation of form, recasts, and

explicit correction in relation to error

types and learner repair in immersion

classrooms. Language Learning, 48, 183-

218. doi:10.1111/1467-9922.00039

Mahard, R. E., & Crain, R. L. (1983). Research on

minority achievement in desegregated

schools. In C. H. Rossell & W. D. Hawley

(Eds.), The consequences of school

desegregation (pp. 103-125). Philadelphia,

PA: Temple University Press.

McNeil, L. M. (2000). Contradictions of school

reform: Educational costs of standardized

testing. New York, NY: Routledge.

Meier, D. (2002). In schools we trust: Creating

communities of learning in an era of

testing and standardization. Boston, MA:

Beacon Press.

Menken, K. (2009). No Child Left Behind and its

effect on language policy. Annual Review

of Applied Linguistics, 29, 103-117.

doi:10.1017/S0267190509090096

Mickelson, R. A. (2003). The academic

consequences of desegregation and

segregation: Evidence from the Charlotte-

Mecklenburg schools. North Carolina

Law Review Association, 81, 1513-1562.

Miles, M. B., & Huberman, A. M. (1984). Drawing

valid meaning from qualitative data:

Toward a shared craft. Educational

Researcher, 13(5), 20-30.

doi:10.3102/0013189X013005020

Muller, M. L. (1976). New Orleans public school

desegregation. Louisiana History: The

Journal of the Louisiana Historical

Association, 17(1), 69-88. Retrieved

from

http://cls.louisiana.edu/lha/journal/index.s

html

National Standards in Foreign Language Education

Project. (1999). Standards for foreign

language learning in the 21st century.

Yonkers, NY: Author.



Olson Beal, H. K. (2008). Speaking the language of

desegregation: A case study of South

Boulevard Foreign Language Academic

Immersion Magnet. (Unpublished doctoral

dissertation). Louisiana State University,

Baton Rouge, LA.

Olson Beal, H. K., Haj-Broussard, M., &

Boudreaux, N. (2007, November). Lived

experiences: Perspectives on immersion

and desegregation in the South. Paper

presented at the annual meeting of the

American Council of Teachers of Foreign

Languages.

Osterman, K. F. (2000). Students' need for

belonging in the school community.

Review of Educational Research, 70, 323-

367. doi:10.3102/00346543070003323

Patton, M. Q. (1990). Qualitative evaluation and

research methods (2nd ed.). Newbury

Park, CA: Sage.

Patton, M. Q. (2002). Qualitative research and

evaluation methods (3rd ed.). London,

England: Sage.

Popham, W. J. (2004). America's "failing" schools:

How parents and teachers can cope with

No Child Left Behind. New York, NY:

Routledge Falmer.

Rosenbusch, M. H., & Jensen, J. (2005). Status of

foreign language programs in the

NECTFL states. NECTFL Review, 56, 26-

37. Retrieved from

http://nectfl.net/review.html

Rossell, C., & Clark, R. C. (1990). The carrot or

the stick for school desegregation policy:

Magnet schools or forced busing.

Philadelphia, PA: Temple University

Press.

Rothstein, R. (2004). Class and schools: Using

social, economic, and educational reform

to close the black-white achievement gap.

New York, NY: Teachers College Press.

Rothstein, R. (2008). Grading education: Getting

accountability right. New York, NY:

Teachers College Press.

Schofield, J. W. (1991). School desegregation and

intergroup relations: A review of the

literature. Review of Research in

Education, 17, 335-409.

doi:10.3102/0091732X017001335

Shneyderman, A., & Abella, R. (2009). The effects

of the extended foreign language

programs on Spanish-language

proficiency and academic achievement in

English. Bilingual Research Journal, 32,

241–259.

doi:10.1080/15235880903372829

Sloan, K. (2007). High-stakes accountability,

minority youth, and ethnography:

Assessing the multiple effects.

Anthropology and Education Quarterly,

38(1), 24-41.

doi:10.1525/aeq.2007.38.1.24

Solórzano, R. W. (2008). High states testing:

Issues, implications, and remedies for

English language learners. Review of

Educational Research, 78, 260-329.

doi:10.3102/0034654308317845

Southworth, S. (2010). Examining the effects of

school composition on North Carolina

student achievement over time. Education

Policy Analysis Archives, 18(29), 1-45.

Retrieved from

http://epaa.asu.edu/ojs/article/view/848/87

2

St. John, N. H. (1970). Desegregation and minority

group performance. Review of

Educational Research, 40(1), 111-133.

doi:10.3102/00346543040001111

Strauss, A., & Corbin, J. (1998). Basics of

qualitative research. Thousand Oaks, CA:

Sage.

Swain, M. (2000). French immersion research in

Canada: Recent contributions to SLA and

applied linguistics. Annual Review of

Applied Linguistics, 20, 199-212.

doi:10.1017/S0267190500200123

Tatum, B. D. (2003). "Why are all the Black kids

sitting together in the cafeteria?" and

other conversations about race. New

York, NY: Basic Books.

Taylor, C., & Lafayette, R. (2010). Academic

achievement through FLES: A case for

promoting greater access to foreign

language study among young learners. The

Modern Language Journal, 94(1), 22-42.

doi:10.1111/j.1540-4781.2009.00981.x

Thomas, W. P., & Collier, V. P. (2002). A national

study of school effectiveness for language

minority students' long-term academic

achievement: Center for Research on

Education, Diversity and Excellence,

University of California-Santa Cruz.

Tyack, D., & Cuban, L. (1995). Tinkering towards

utopia: A century of public school reform.

Cambridge, MA: Harvard University

Press.

U.S. Department of Education. (2001). No Child

Left Behind Act of 2001 (Vol. Pub. L.

107-110, § 115, Stat. 1425 (2002).

U. S. District Court Middle District of Louisiana.

(1996). Consent Decree (No. Civil Action

No. 56-1662-A). Baton Rouge, LA.

U. S. District Court Middle District of Louisiana.

(2003). Final Settlement Agreement (No.

Civil Action No. 56-1662-D-M3). Baton

Rouge, LA.

Walsh, N. (2005). Collaborer pour s’ouvrir aux

autres: L’éveil aux langues dans une

classe d’immersion française

(Unpublished master’s thesis). Simon



Fraser University, British Columbia,

Canada.

Wiley, T. G., & Wright, W. E. (2004). Against the

undertow: Language-minority education

policy and politics in the "age of

accountability." Educational Policy,

18(1), 142-168.

doi:10.1177/0895904803260030

i Dual immersion or two-way immersion programs

are ideally composed of a balance of native

English-speakers and native speakers of the non-

English language, whereas total and partial

immersion programs are typically composed of

native English-speakers. ii South Boulevard, the actual name of the school,

was used in the larger study because a history of

the school from 1956-2007 was a component of the

study. Furthermore, at the time of the study,

South Boulevard was the only public immersion

program in Louisiana to offer both Spanish and

French immersion tracks. Thus, the school would

have been easily identifiable. However,

pseudonyms are used throughout this article to

protect the anonymity of the participants. iii School X is the pseudonym that was used in the

larger study to protect the anonymity of the school

and the study participants. Pseudonyms are used

throughout this article to protect the anonymity of

the participants. ivThe LEAP is a criterion-referenced (CRT) test

that measures how well a student has mastered the

state content standards. The LEAP is

administered in grades 4 and 8. v We use the term “Black” rather than “African-

American” throughout because it was used by

study participants and in demographic records

throughout the era of court-ordered desegregation. vi Details regarding this mandate can be found in

Bulletin 741: Louisiana Handbook for School

Administrators, which can be accessed at the

following URL:

http://doe.louisiana.gov/bese/policies.html vii This was possible because strong and

statistically significant correlations were found

between the covariates (ITBS scores) and the

dependent variables (the LEAP scores) in and

across the mathematics and language subject tests,

with correlation coefficients ranging from .58 to

.74. viii 1998-1999 was the first school year during

which the LEAP for Mathematics and English

Language Arts was administered to all Louisiana

public school students in Grades 4 and 8. ix Data obtained from annual School Report Cards

issued by the Louisiana State Department of

Education and are available online at

www.louisianaschools.net from 1996 to the

present.

x Data obtained from Louisiana School Directories

and Louisiana State Department of Education

Annual Financial and Statistical Reports. xi The phrase “racially identifiable” came from a

statement from the Commission on Civil Rights

that was cited in the Supreme Court Green vs.

County School Board of New Kent County (1968)

decision, which stated that New Kent County’s

freedom of choice plan was inadequately

addressing lingering racial segregation in the

district. Later, Rossell and Clark (1990) defined it

as a school with a racial composition that deviated

20 percentage points or more from the district

racial composition.




Factors Predicting Pre-service Teachers’ Adoption of Web 2.0 Technologies

Jongpil Cheon, Fanni Coward, Jaeki Song, and Sunho Lim

Texas Tech University

A decade ago, Prensky (2001), along with

others, called the new generation of students in K-12

classrooms digital natives because of the students’

familiarity with technologies such as computers,

video games, digital music players, and cell phones.

More than a decade later, young people have added

to their digital birthright the use of online spaces,

such as Google, YouTube, Pinterest, Facebook and

Twitter.

Web 2.0 is a term reflecting a family of online

environments in which users create, share and

combine information (O’Reilley, 2005). These new

environments afford opportunities for instructionally

driven, student-centered learning using social and

collaborative functionalities (Williams & Chinn,

2009). The adoption by school systems and teachers

of various web-enabled portable and hand-held

devices (e.g., laptops, iPods, and iPads) is helping

move K-12 teachers closer to a facilitative role (Li,

Liu, Li, Wang, & Chen, 2011; Russell, 2009). In

this context, however, teachers are the primary

curriculum designers and innovators, and their

readiness to apply new technologies represents a

significant milestone for use of these new

pedagogical approaches in the classroom.

As students think and process information in

ways that are different from the earlier generations

(Prensky, 2001), there is a corresponding pressure for

the role of teachers to shift away from being an

information provider toward becoming an

information facilitator. Recently, researchers have

begun to conceptualize the instructional use of

technology as an important dimension of teachers’

pedagogical content knowledge (Mishra & Koehler,

2006; Niess, 2005; Niess, Zee, & Gilow-Wiles,

2010). The factors that influence teachers’ adoption

of instructional technology is fairly well understood,

but much less is known about what influences the

instructional adoption of Web 2.0 technologies. The

current study employed the Theory of Planned

Behavior (TPB; Ajzen, 1991) as an analytical

framework to provide a more comprehensive

explanation of technology adoption behavior specific

to Web 2.0 technologies.

Technology Affordances and Teacher Beliefs

Web 2.0 Technologies

Web 2.0 refers to a read/write web in which

users actively participate in creating and sharing

information (e.g., Wikipedia, Facebook, Twitter,

blogs, or YouTube). Web 1.0, on the other hand,

refers to a read-only web in which users consume

available information (Murugesan, 2007). Web 2.0


addressed to Fanni Coward, College of Education,

Texas Tech University Lubbock, TX 79409-1071,

USA. Email: [email protected]

Classrooms full of “digital natives” represent the norm in U. S. schools, but like their predecessors, they

mostly inhabit spaces characterized by a traditional view of teaching and learning. Understanding

contributors to this mismatch, and especially teachers’ role, is especially critical as Web 2.0 technologies

enable greater learner autonomy. This study used Theory of Planned Behavior (TPB) to predict pre-

service teachers’ (n = 172) intention to use new web-based technologies for instruction. The participants

were from 7 sections of a technology application course, and all of them were selected via a convenience

sampling. Self-reported data were analyzed using a structural equation modeling method, Partial Least

Squares (PLS). Results revealed that the attitude and perceived control components of the theory

predicted intention to use Web 2.0 technology to a statistically and practically significant degree, but

subjective norm did not. Antecedent predictors of the TPB constructs were all predictive. Discussion

focuses on the utility of TPB to examine how pre-service and in-service preparation might use the model to

create situations where technologies might be more widely applied in the classroom.

JONGPIL CHEON, FANNI COWARD, JAEKI SONG, AND SUNHO LIM


technologies provide a variety of services for

collaboration, contribution, and communication that

enable users to produce and to share information with

others (Mills, 2007). These characteristics could

enhance teaching and learning by enabling various

student-centered learning activities requiring active

participation and collaboration (Cheon, Song, Jones,

& Nam, 2010; McLoughlin & Lee, 2008; Williams &

Chinn, 2009). Many researchers anticipate that use of

Web 2.0 resources has greater benefits including (a)

increasing students’ creativity when reasoning and

problem-solving, (b) enhancing interaction when

collaborating, (c) helping to sustain engagement, and

(d) developing communities for professional growth

(cf. International Society for Technology in

Education, 2008; McLoughlin & Lee, 2008;

Vaughan, Nickle, Silovs, & Zimmer, 2010). The

interactive affordances of Web 2.0 technologies

make them easily adopted tools in a student-centered

classroom (Grant & Mims, 2009).Learning that is

guided by Web 2.0 functions also is closely aligned

with a view of learning embraced by social

constructionism, which posits that learners construct

knowledge through social interaction. Researchers

are understandably eager to explore the potential of

Web 2.0 technologies to enhance student-centered

learning, and educators are eager to utilize the

technology to facilitate active participation.

Factors Related to Teachers’ Technology

Adoption Researchers have argued that educational

reforms often fail because teachers’ beliefs,

capabilities, and routine practices are not aligned

with the changes (Niederhauser & Stoddart, 2001).

The slow pace at which instructional technology, let

alone Web 2.0 resources, is adopted might be a case

in point. Thus, it is important to investigate how

technologies are being adopted and why the

technology adoption speed is slow. Currently,

there is a fair amount of consensus around factors

and barriers that directly or indirectly affect teachers’

adoption of instructional technology (e.g., Becker,

2000, 2001; Mumtaz, 2000; Sheingold & Hadley,

1990; Shiue, 2007). Factors that influence adoption

include: (a) individual perceptions (e.g., motivation,

commitment, or confidence); (b) extent of personal

experience; (c) technological issues (e.g., technical

supports, software quality of software and hardware);

and (d) environmental supports (e.g., access to

resources, incentives to change or school polices)

(Evans-Andris, 1995; Mumtaz, 2000). Among these

influences, positive attitudes towards technologies

seems to be the most widely recognized pre-

condition for teachers’ use of technologies for their

teaching (Shiue, 2007). Teachers with positive

attitudes are more apt to utilize technology for

instruction (e.g., Moseley & Higgins, 1999; Russell,

Bebell, O'Dwyer, & O'Connor, 2003; Smerdon et al.,

2000; Yildirim, 2000), and to have a higher

frequency of technical use in the classroom (e.g.,

Pedretti, Smith-Mayer, & Woodrow, 1999; Ravitz,

Wong, & Becker, 1999; Russell et al., 2003; van

Braak, Tondeur, & Valcke, 2004).

Technology and Teacher Preparation It is common for teacher educators to include

some degree of technology competency in their

teacher preparation programs (Shoffner, 2009).

Similar to the studies involving in-service teachers,

pre-service teachers’ self-efficacy or attitude toward

general technologies has been found to be a

significant predictor of technology use (e.g., Littrell,

Zagumny, & Zagummy, 2005; Wozney, Venkatesh,

& Abrami, 2006). Researchers have shown that

educational technology courses can improve pre-

service teachers’ computer self-efficacy (Abbitt &

Klett, 2007). Many researchers have pointed to the

need to integrate pedagogical methods and content

knowledge with technology skills (Abbitt, 2011;

Lambert & Cuper, 2009), but concerns often are

raised about the stand-alone nature of many

educational technology courses for pre-service

teachers. These courses often focus exclusively on

technical skill development in isolation instead of

fostering an understanding of technology as powerful

tools to increase student achievement and to improve

learning. The result of this divorce between technical

and learner-centered views is that integration of

technology in education is still lacking (Vockley,

2008).

At present, it is difficult to synthesize findings

across studies in a way that provides a systematic

picture of how the various beliefs and attitudes might

be related to intention and behavior. This lack of a

systematic view presents a particular challenge to

teacher education programs struggling to focus

technology preparation on improving student

achievement instead of building isolated skills. The

Theory of Planned Behavior offers a framework for

studying the relations among beliefs, attitudes, and

intentions to use Web 2.0.

Theory of Planned Behavior and Teaching

with Web 2.0

Theory and a Research Model The Theory of Planned Behavior (TPB) has been

applied in a wide variety of contexts (e.g.,

technology, health care, and political science) to

predict behavior (e.g., Cheon, Lee, Crooks, & Song,

2012; Conner & Armitage, 1998; Davis, 1989;

Sheppard, Hartwick, & Warshaw, 1988; Song &

Zahedi, 2005; Taylor & Todd, 1995). The theory

posits that individuals are inclined toward a behavior

based on three determinant intentions (Ajzen, 1991).

The first is an individual’s attitude toward the

behavior. Attitude refers to the degree to which a

person has a favorable or unfavorable feeling about

performing the particular behavior. The second is a

FACTORS PREDICTING PRE-SERVICE TEACHERS’ ADOPTION OF WEB 2.0 TECHNOLOGIES


subjective norm. This norm reflects perceived

social pressure to perform or not to perform the

behavior. Finally, perceived behavioral control

reflects the individual’s belief about how easy it is to

perform the behavior. Based on TPB, we

formulated our research model (see Figure 1) to

predict pre-service teachers’ intention to adopt Web

2.0 technologies. For this study, we used teachers’

behavioral intention to use Web 2.0 technologies as

the final dependent variable based on

recommendations from Ajzen (1985, 1991). The

research model proposes that external beliefs

reinforce the three attitudinal constructs. These

attitudinal constructs, in turn, positively enhance

behavioral intention. The antecedent variables of

each attitudinal construct (shown in the left stem) are

described in Figure 1. Specific hypotheses follow

from the model.

Figure 1. Conceptual model for pre-service teachers’ adoption of Web 2.0 technologies

Attitudinal Constructs and Behavioral Intention In the TPB, attitude toward a behavior,

expectations based on a subjective norm, and

perceived control are all predictors of individuals’

intention to act. Researchers have repeatedly

demonstrated the connection between attitude and

intention (Davis, 1989; Notani, 1998; Taylor &

Todd, 1995). Subjective norm, too, has been linked

to intentions to act. For example, Ajzen (1991)

reported that 15 out of 19 TPB-based studies

showed that subjective norm had significant effects

on behavioral intention. Lastly, perceived

behavioral control also has been associated with

intention to act. Previous literature has suggested

that perceived behavioral control functions as a

crucial determinant of behavior when the behavior

cannot be performed without certain knowledge,

skills, resources, cooperation from others, or

opportunities (Ajzen, 1985; Hartwick & Barki,

1994; Sheppard et al., 1988).

Based on this prior work, we hypothesized:

H1: Pre-service teachers’ attitudes toward

Web 2.0 technologies is positively related to

their intentions to adopt Web 2.0 technologies;

H2: Pre-service teachers’ subjective norm

of Web 2.0 technologies is positively related to

their intentions to adopt Web 2.0 technologies;

H3: Pre-service teachers’ perceived

behavioral control of Web 2.0 technologies is

positively related to their intentions to adopt

Web 2.0 technologies.

Beliefs

Attitudinal Constructs

Behavioral Intention

Attitudinal Beliefs

Perceived Ease of Use H5

Perceived Usefulness H6

Normative Beliefs

Perceived Pressure

Attitude

Subjective

Norm Intention

H1

H2

H3

Perceived Behavioral

Control

H7

H8

H9

H10

Control Beliefs

Perceived General

Self-efficacy

Perceived Web 2.0 Specific

Self-efficacy

H4



Antecedents of Attitude, Subjective Norm and

Perceived Control

Beliefs predict attitude. The antecedents of

the first attitudinal construct (i.e., attitude) are

attitudinal beliefs. Attitudes are developed from the

beliefs associated with attributes and outcomes

(Fishbein & Ajzen, 1975). For example, a person

will have a favorable attitude towards a new

technology when he/she believes that the

technology results in better job performance

(Taylor & Todd, 1995). In our research model,

variables for attitudinal beliefs were derived from

the Technology Acceptance Model (TAM; Davis,

1989). This model has been adapted by many

researchers to investigate the relation between

users’ perceptions and their actual use of a

technology system (Adams, Nelson, & Todd, 1992;

Benbasat & Barki, 2007; Segars & Grover, 1993).

Davis (1989) concluded that perceived ease of use

and perceived usefulness determine an individual's

intention to use a system.

Based on this work, we also included ease of

use and usefulness in our model and hypotheses.

H4: Pre-service teachers’ perceived ease

of use Web 2.0 technologies is positively

related to perceived usefulness of Web 2.0

technologies;

H5: Pre-service teachers’ perceived ease

of use of Web 2.0 technologies is positively

related to their attitudes toward Web 2.0

technologies;


usefulness of Web 2.0 technologies is

positively related to their attitudes toward Web

2.0 technologies.

Normative beliefs toward subjective norm.

The representation of how others will view a

behavior reflects the individual’s subjective norm.

Attitudes reflective of the subjective norm account

for the social pressure an individual feels when

contemplating a particular behavior. Individuals are

dependent on context, and they are socially

constructed beings (Shah, 1998). The influence

from referent groups (e.g., friends, superiors,

family members, and neighbors) affects

individuals’ subjective norm (Taylor & Todd,

1995). For pre-service teachers, the pressure to

adopt new technologies could come from their

classes, peers, and instructors. Therefore, we

hypothesized:


pressure of Web 2.0 technologies positively

influences their subjective norms with Web 2.0

technologies.

Control beliefs toward perceived behavioral

control. Perceived behavioral control is compatible

with the concept of self-efficacy. Self-efficacy

refers to individuals' beliefs about their ability and

motivation to perform specific tasks (Bandura,

1986, 1997). Previous researchers have

documented that higher levels of self-efficacy lead

to higher levels of behavioral intention and the

usage of information technology (Compeau &

Higgins, 1995; Gist, Schwoerer, & Rosen, 1989).

In the field of information technology,

researchers now distinguish between general

computer self-efficacy and task-specific self-

efficacy (Agarwal & Karahanna., 2000; Marakas,

Mun, & Johnson, 1998). General computer self-

efficacy is defined as an individual's judgment of

efficacy across multiple computer application

domains, whereas task-specific computer self-

efficacy is defined as perceptions of ability to

perform specific computer-related tasks. In the

context of Web 2.0 technologies, this study

employed these discrete concepts of self-efficacy

as control beliefs that influence perceived

behavioral control.

Based on these research findings, we

hypothesized:


general self-efficacy of Web 2.0 technologies

is positively related to their perceived Web 2.0

specific self-efficacy.


general self-efficacy is positively related to

their perceived behavioral control.

H10: Pre-service teachers’ perceived Web

2.0 specific self-efficacy is positively related to

their perceived behavioral control.

In summary, we hypothesized that TPB would

explain pre-service teachers’ intention to adopt

Web 2.0.

Method

Participants A total of 172 pre-service teachers at a large

university in the southwestern United States

participated in this study. The pre-service teachers

were required to take a technology application

course, and all of them in seven sections in the

course were selected as a convenience sample. The

participants were mostly in their junior or senior

years of post-secondary education. Their average

age was 23, and 82% were female. Their major

courses of study were varied and included early

childhood, mathematics education, history, language

arts, and special education. The Web 2.0

technologies that they had used mostly represented

social networking services (e.g., Facebook) and

video sharing service (e.g., YouTube), and 15

participants had used Google Docs. However, the

participants had not used any other Web 2.0

technologies presented in this study.



Procedures

The participants took a Web 2.0 special

training session, “Web 2.0 technologies for the

Classroom,” led by one of the authors. Four

categories were selected for the training among the

eight categories of Web 2.0 technologies (Franklin

& van Harmelen, 2007), because these services had

the potential to be a learning tool in the classroom

for pre-service teachers. In addition, they had more

collaborative functions than did other categories,

and they were relatively new to the participants.

One site for each category was selected based on

popularity and free functionality, and the last site

(Voicethread) was presented in addition. This site

does not belong to any Web 2.0 category, but it

provides options for group conversation that would

be useful for collaborative learning. The

participants had heard of Web 2.0, but none of

them knew these websites except Google Docs. A

total of five sites were introduced in the training

session as follows;

• Wikis – Pbworks (http://pbworks.com)

• Social bookmarking – Diigo

(http://diigo.com)

• Media-sharing services -Slideshare

(http://www.slideshare.net)

• Collaborative editing tools – Google Docs

(http://doc.google.com)

• Group conversation - Voicethread

(http://voicethread.com)

The training session consisted of three phases.

In the first phase, the trainer introduced the

dynamic nature of Web 2.0 technologies and the

potential benefits of using them for teaching and

learning. For the second phase, the five sites above

were introduced to the participants. The trainer

showed an introductory video clip for each site, and

basic functions were demonstrated. In the last

phase, the participants were asked to complete the

survey described below.

Instrument Development

We compiled a 27-item measure (three items

for nine constructs) from already-existing

questionnaire items found in published literature

(see Table 1). Participants responded using a 7-

point Likert-type scale, ranging from totally

disagree to totally agree. The first five constructs,

in the first column in Figure 1, are related to the

pre-service teachers' salient beliefs. The three

constructs in the second column are the attitudinal

constructs. The last construct is behavioral

intention to adopt Web 2.0 technologies.

To assess the score-validity of our instrument,

we administered a voluntary pilot test with 10

participants. Each of the 10 participants was asked

to (a) complete each item on the instrument to

indicate how they rated the importance of each

belief about Web 2.0 technologies; and (b) provide

comments on the wording, instruction, and clarity

of the items on the instrument. Based on feedback

from participants in the pilot test, we made minor

modifications to the survey items.

Data Analysis and Results

Partial Least Squares (PLS) method was used

to analyze the data. PLS is “a structured equation

modeling estimation technique which generates

estimation of item loadings (i.e., structure

coefficients) and path coefficients simultaneously”

(Gefen & Straub, 2005, p. 109). The PLS method is

component-based and formative analysis for

prediction rather than fit, whereas other structural

equation modeling (SEM) methods, such as

LISREL or AMOS, focus on testing the degree of

consistency between a model and data (Bontis,

Crossan, & Hulland, 2002; Chin, 1998). Further,

the PLS method can model latent constructs under

conditions of non-normality and small to medium

sample sizes (Chin, Marcolin, & Newsted, 1996,

Hulland, 1999). We completed data analysis with

SmartPLS in two steps: a measurement model

estimate, followed by a structural model evaluation

(Jarvenpaa, Shaw, & Staples, 2004).

Measurement Model Estimate

In order to estimate our measurement model,

we conducted a confirmatory factor analysis with

SmartPLS. Table 2 presents the relevant statistics.

For all constructs, the item score reliability and

convergent validity were evaluated by examining

the individual item loading (i.e., structure

coefficients), composite reliability (CR), and

average variance extracted (AVE). As shown in

Table 2, all structure coefficients on the respective

construct were higher than 0.7, with a range from

0.786 to 0.958 (Chin, 1998), and t values for the

items were statistically significant. The composite

reliability for each construct was greater than 0.7

(Nunnally & Bernstein, 1994). The values of

AVE were all above 0.50 (Fornell & Larcker,

1981). Thus, convergent validity was adequately

established.

The next step was to evaluate the discriminant

validity of the constructs. The square root of the

average variance extracted (AVE) for a given

construct was compared with the correlations

between the construct and other constructs (Fornell

& Larcker, 1981). If the square root of the AVE of

a construct is larger than its correlations with any

other construct, a construct is more closely related

with its indicators than with the other constructs

(Barclay, Thompson, & Higgins, 1995; Chin, 1998;

Fornell & Larcker, 1981; Hulland, 1999). Because

the square root of the AVE for each construct

exceeded the correlations between that construct

and other constructs, discriminant validity appeared

to be satisfactory for all constructs.



Table 1

Survey Instrument and References

Constructs Items Survey items References

Perceived

Ease of Use

ease1 I believe that the services given by Web 2.0 are easy to use.

Davis, 1989;

Venkatesh & Davis, 2000 ease2

I believe that the services given by Web 2.0 are easy to get to

do what I want them to do.

ease3 I believe that the services given by Web 2.0 are easy to

operate.

Perceived

Usefulness

useful1 I believe that the services given by Web 2.0 allow me to get

my work done more quickly.

Davis, 1989;

Venkatesh & Davis, 2000 useful2

I believe that using the services given by Web 2.0 enhances

my effectiveness on my teaching.

useful3 I believe that the services given by Web 2.0 are useful for

my teaching.

Perceived

Pressure

pres1 I believe that the preparation courses for being a teacher

require me to adopt the services given by Web 2.0. Karahanna, Straub, &

Chervany, 1999;

Iacovou Benbasat, &

Dexter, 1995;

Premkumar & Roberts,

1999

pres2 Instructors of the courses I have taken think I should

integrate the services given by Web 2.0 into the classroom.

pres3 My peers in this course I am taking think I should use the

services given by Web 2.0 for teaching.

General

Self -Efficacy

eff1 I would be able to manage the services given by Web 2.0.

Compeau & Higgins,

1995 eff2

I am confident about managing the services given by Web

2.0.

eff3 Learning the services given by Web 2.0 is straightforward.

Web2.0

Specific Self-

Efficacy

web1 I would be able to enhance my teaching through student

collaboration with the services given by Web 2.0.

O’Reilley, 2005;

Mills, 2007 web2

I am confident that I can increase students’ participation with

the services given by Web 2.0.

web3 I would enrich communications with student and between

students with the services given by Web 2.0.

Attitude

att1 Adopting the services given by Web 2.0 would make me feel

good. Taylor & Todd, 1995;

Song & Zahedi, 2005

att2

If I adopt the services given by Web 2.0 in my future

teaching, I would feel that I am getting a good deal.

att3 Adopting the services given by Web 2.0 would be pleasant.

Subjective

Norm

snorm1 Other people have provided supporting evidence in favor of

adopting the services given by Web 2.0. Taylor & Todd, 1995;

Song & Zahedi, 2005

snorm2 Most people who are important to me think that it is fine to

adopt the services given by Web 2.0.

snorm3 Most people who are important to me would adopt the

service given by Web 2.0.

Perceived

Behavioral

Control

Becon1 The extent of knowledge that I feel I have in adopting Web

2.0 technologies is sufficient. Taylor & Todd, 1995;

Song & Zahedi, 2005

Becon2 The extent of control that I feel I have in making my

adopting decision is sufficient.

Becon3 The extent of self-confidence that I feel I have in making my

adopting decision is sufficient.

Behavioral

Intention

Beint1 I plan to use the services given by Web 2.0 sometime after

becoming a teacher. Song & Zahedi 2005;

Venkatesh, Brown,

Maruping, & Bala, 2008

Beint2 My use of the services given by Web 2.0 after becoming a

teacher would be probable.

Beint3 The likelihood that I would adopt the services given by Web

2.0 is high.



Table 2

Statistics of the Measurement Model

Construct Item Mean Standard

Deviation

Item

Loadinga

Standard

Error t value

Composite

Reliability AVE

Perceived

Ease of Use

ease1 5.895 1.054 0.935 0.024 15.709

.957 0.882 ease2 5.727 1.015 0.942 0.025 13.607

ease3 5.733 0.978 0.941 0.026 13.494

Perceived

Usefulness

useful1 5.407 1.159 0.786 0.034 9.236

.910 0.773 useful2 5.587 1.270 0.929 0.026 15.398

useful3 5.820 1.091 0.915 0.026 15.855

Perceived

Pressure

pres1 5.378 1.281 0.814 0.044 7.822

.896 0.742 pres2 5.250 1.415 0.885 0.037 10.430

pres3 5.128 1.300 0.884 0.049 8.792

General

Self -Efficacy

eff1 5.913 1.042 0.936 0.022 16.890

.938 0.835 eff2 5.785 1.068 0.915 0.023 15.448

eff3 5.756 1.159 0.890 0.032 11.738

Web2.0

Specific Self-

Efficacy

web1 5.581 0.973 0.881 0.022 15.753

.935 0.828 web2 5.738 0.983 0.916 0.019 19.430

web3 5.750 1.009 0.931 0.024 15.912

Attitude

att1 5.407 1.183 0.893 0.018 19.312

.928 0.811 att2 5.605 1.101 0.917 0.019 20.995

att3 5.587 1.059 0.892 0.018 20.483

Subjective

Norm

snorm1 5.233 1.151 0.791 0.038 9.349

.903 0.756 snorm2 4.727 1.294 0.923 0.025 15.569

snorm3 4.837 1.306 0.889 0.033 12.075

Behavioral

Control

Becon1 5.390 1.152 0.908 0.019 18.918

.937 0.833 Becon2 5.570 1.032 0.916 0.016 23.031

Becon3 5.698 0.974 0.914 0.019 20.446

Behavioral

Intention

Beint1 5.983 1.142 0.958 0.020 18.218

.960 0.889 Beint2 5.890 1.182 0.916 0.022 14.941

Beint3 5.849 1.160 0.954 0.022 16.725 a Structure coefficient

Structural Model Evaluation

The structural model analyzed the relations

among the various latent variables. Figure 2

presents the standardized path coefficients and the

explained construct variances. The results show

that nine of the 10 hypotheses were supported.

Hypotheses 1 and 3 were supported (H1,

coefficient of 0.50, t = 4.52, p < 0.01; H3,

coefficient of 0.37, t = 3.83, p < 0.01), whereas the

second hypothesis was not supported. In other

words, behavioral intention was positively related

to attitude and perceived behavioral control but not

to subjective norm, although the relation between

the two constructs was positive (t = 0.34).

All hypotheses regarding the relations between

the three attitudinal constructs and the antecedent

variables for each construct were supported. First,

both perceived ease of use (H5, coefficient of 0.25,

t = 2.97, p < 0.01) and perceived usefulness (H6,

coefficient of 0.56, t = 7.08, p < 0.01) were

statistically significantly related to attitude. Also,

perceived usefulness was related to a statistically

significant degree to perceived ease of use (H4,

coefficient of 0.61, t = 7.89). Second, the

assumption of the positive relation between

perceived pressure and subjective norm was met

(H7, coefficient of 0.62, t = 8.10, p < 0.01).

Last, we found evidence to support

Hypotheses 8 to 10. For example, perceived

general Web 2.0 self-efficacy was positively

related to perceived Web 2.0 specific self-efficacy

(H8, coefficient of 0.58, t = 8.34, p < 0.01). Both

self-efficacy perceptions were positively related to

perceived behavioral control (H9, coefficient of

0.19, t = 2.41, p < 0.05; H10, coefficient of 0.73, t

= 9.92, p < 0.01), although the relationship

involving Web 2.0 specific self-efficacy was much

greater than was the relationship involving general

self-efficacy.



*: p < 0.05, **: p < 0.01, N.S.: Not significant

Figure 2. Results of the structural model with PLS.

Discussion

Our research model shows how pre-service

teachers’ perceptions are related to their behavioral

intention to adopt Web 2.0 technologies. We

believe that pre-service teachers who have strong

intention likely will adopt Web 2.0 technologies in

their future classrooms because the behavioral

intention is assumed to be the immediate

antecedent of actual behavior (Ajzen, 2002).

The results suggested that pre-service teachers’

attitudes and perceived behavioral control

positively influenced their intention to adopt Web

2.0 technologies, whereas the impact of their

subjective norm on behavioral intention was not as

strong. These findings have important theoretical

and practical implications, and suggest directions

for future research.

Support for the TPB and Implications for

Teacher Preparation

Usefulness of technology influences attitude.

The relations among beliefs (i.e., perceived ease of

use and usefulness), attitude and behavioral

intention confirmed the findings from Cheon et

al.’s (2010) study that employed the Expectancy-

Value Model and Technology Acceptance Model

(TAM). In line with that study, our findings

demonstrated that perceived ease of use and

usefulness are significant predictors of new

technology (i.e., Web 2.0 technologies) acceptance.

In other words, pre-service teachers who believe

that Web 2.0 services are easy to use and useful are

Attitudinal Beliefs

Perceived Ease of Use

Perceived Usefulness

Normative Beliefs

Perceived Pressure

Attitude

Subjective

Norm Intention

Perceived Behavioral

Control

Control Beliefs

Perceived General

Self-efficacy

Perceived Web 2.0 Specific

Self-efficacy

0.61(7.89) **

0.25(2.97) **

0.56(7.08) **

0.62(8.10) **

0.19(2.41) *

0.73(9.92) **

0.58(8.34) **

0.50(4.52) **

0.03(0.34)

0.37(3.83)**

N.S.



more likely to adopt Web 2.0 service for teaching.

Because the coefficient value of perceived

usefulness (β = 0.57) was higher than was the value

of perceived ease of use (β = 0.25), it can be

concluded that the perception toward usefulness is

a stronger predictor than is attitude. We assume

that the difference might be caused by the

affordance of Web 2.0 technologies.

Most Web 2.0 technologies focus on a specific

function that requires a low learning curve (Grant

& Mims, 2009). For example, users can be easily

familiar with the core functions via a short

introduction video clip. However, how to apply

these services to teaching and learning is not

simple.

As shown by the higher coefficient value of

perceived usefulness, teacher educators should

provide successful learning experiences using a

Web 2.0 site in a wider variety of teacher

preparation courses in addition to a few courses

about technology application (Ertmer, 2005;

McRobbie, Ginns & Stein, 2000; Pope, Hare &

Howard, 2002). In order for pre-service teachers to

believe that Web 2.0 service provides a valuable

teaching tool, teacher educators should strengthen

their beliefs about its usefulness through learner-

centered activities and rich resources. Doing so

might reduce uncertainty or hesitation in pre-

service teachers’ perceptions about technology

(Cheon et al., 2010).

Efficacy, perceived control and intention to

act. We found that pre-service teachers’ behavioral

control is another important predictor of their

intention to adopt Web 2.0 technologies. Both

general self-efficacy and specific self-efficacy were

positively related to their perceived behavior

control and, in turn, the behavioral control

predicted positive intention. Pre-service teachers

who perceived more control over Web 2.0

technologies indicated higher intention to adopt the

technologies.

Perceptions of control over the technology

were related to both self-efficacy levels.

Consistent with the findings of prior research,

general self-efficacy was positively related to the

specific task-specific (i.e., Web 2.0 technologies)

self-efficacy. Because the results confirmed the

assumption in TPB, this finding implies that

enhancing pre-service teachers’ confidence in using

Web 2.0 technologies would increase the likelihood

of technology adoption. More specifically, teaching

strategies or training mechanism to increase self-

efficacy likely would be beneficial to pre-service

teachers who have lower technology self-efficacy.

Instead of using the same tool or software in a

course, for example, the freedom of selecting a tool

based on individual preference or capacity likely

would enhance learning autonomy and decrease the

level of anxiety in teacher preparation courses.

Subjective norm and intention to act. This

study did not find support for the relation between

subjective norm and behavioral intention, although

perceived pressure served as a significant

antecedent of subjective norm. We conjecture that

the finding is due to the environments of pre-

service teachers. Because they were not in real

practice, they might not feel any pressure to

integrate technology including Web 2.0

technologies into curricula. This finding is

consistent with what Shiue (2007) reported about

subjective environments weakly influencing the

actual use of technology. Subjective norm,

however, might be more important for in-service

teachers. The norm in a real school setting would

be influenced by more determinants such as

administrative support or pressure and peer use of

technology (e.g., Shiue, 2007).

As teacher educators, we can create a positive

social norm within pre-service teachers’ society.

Previous researchers documented that in-service

teachers tend to use the computer as a tool to

engage students in cognitively challenging tasks if

they regularly participate in professional

interactions beyond their classrooms (Becker &

Riel, 2000). Accordingly, if pre-service teachers

have experiences of producing and sharing

meaningful information with others, their sense of

belonging and kinship could create a more positive

social norm. For example, creating an online

community of pre-service teachers in a university

could be the first step.

General Implications for Teacher Preparation

The proposed adoption model describes a set

of factors related to pre-service teachers’ use of

Web 2.0 technologies. Teacher education programs

could manipulate those factors that facilitate

teacher candidates’ adoption of the use of Web 2.0

technologies. The findings about the importance of

attitude and perceived behavioral control suggest

that teacher education curricula should include

opportunities to learn and to utilize various

functions of Web 2.0 technologies. With more Web

2.0 integration experiences, teacher candidates

should have positive attitudes and higher levels of

confidence to use technology tools.

Limitations and Future Research Directions

This study has at least two limitations that

circumscribe our interpretation and implications.

First, from a methodological standpoint, the

findings should be interpreted with caution due to

the fact that our results were obtained from a

single, self-report questionnaire, and the sample

size was relatively small. In terms of the

intervention, the training session might have been

too short to provide pre-service teachers with

adequate hands-on experiences; thus, it might limit



the generalizability of the results to other

situations.

We propose several exciting directions for

future research. First, although we included several

antecedent variables, there are others in the

research literature that should be included in future

studies. For example, social pressure as a

determinant of subjective norm might have only

negative connotations, and positive variables such

as support from instructors or colleagues could be

added to the research model. In addition, the

general self-efficacy variables focused on only

Web 2.0 technologies. More general computer self-

efficacy also should be considered to develop a

more systematic and robust conceptual model.

Next, we find it very intriguing to consider a

longitudinal perspective and imagine how the TPB

might help in understanding the emergence of

beliefs, attitudes, and intentions among pre-service

teachers. In addition, this perspective could be

applied to in-service teachers in order to determine

significant predictors of their intentions to integrate

technology. Shiue (2007) discovered other

significant factors (e.g., technical support,

computer access, and professional development)

for technology integration using a decomposed

theory of planned behavior. Further researchers

could produce a revised model for technology

integration that extends the original theory.

Conclusion

The teaching profession is in need of new

teachers with positive attitudes about technology

and the skills to apply them in the classroom

(Russell et al., 2003). This study provides an

analysis of what might motivate new teachers’

technology use via the Theory of Planned

Behavior. Our empirical analysis of pre-service

teachers’ various beliefs provide a picture of what

might influence their attitudes and intentions to

adopt Web 2.0 technologies. Further studies are

needed to explore the pedagogical ways to increase

teacher candidates’ levels of self-efficacy and to

create a supportive social norm.

The lead editors for this article were Anthony J.

Onwuegbuzie and John R. Slate.

References

Abbitt, J. T. (2011). An investigation of

relationship between self-eficacy beliefs

about technology integration and

technological pedagogical content

knwoledge (TPACK) among preservice

teachers. Journal of Digital Learning in

Teacher Education, 27(4), 134-143.

Abbitt, J. T., & Klett, M. D. (2007). Identifying

influences on attitudes and self-efficacy

beliefs towards technology integration

among preservice educators. Electronic

Journal for the Integration of Technology

in Education, 6, 28-42.

Adams, D. A., Nelson, R. R., & Todd, P. A. (1992).

Perceived usefulness, ease of use, and

usage of information technology: A

replication. MIS Quarterly, 16, 227-247.

doi:10.2307/249577

Agarwal, R., & Karahanna, E. (2000). Time flies

when you’re having fun: Cognitive

absorption and beliefs about information

technology usage. MIS Quarterly, 24, 665-

694. doi:10.2307/3250951

Ajzen, I. (1985). From intentions to action: A

theory of planned behavior. In J. Huhl &

J. Beckmann (Eds.), Action-control: From

cognition to behavior (pp. 11-39). Berlin,

Germany: Springer-Verlag.

Ajzen, I. (1991). The theory of planned behavior.

Organizational Behavior and Human

Decision Processes, 50, 179-211.

doi:10.1016/0749-5978 (91)90020-T

Ajzen, I. (2002). Perceived behavioral control, self-

Efficacy, locus of control, and the theory

of planned behavior. Journal of Applied

Social Psychology, 32, 665-683.

doi:10.1111/j.1559-1816.2002.tb00236.x

Bandura, A. (1986). Social foundations of thought

and action: A social cognitive theory.

Englewood Cliffs, NJ: Prentice-Hall.

Bandura, A. (1997). Self-efficacy: The exercise of

control. New York, NY: W. H. Freeman.

Barclay, D. W., Thompson, R., & Higgins, C.

(1995). The partial least squares (PLS)

approach to causal modeling: Personal

computer adoption and use an illustration.

Technology Studies, 2, 285-309.

Becker, H. J. (2000). Findings from the teaching,

learning and computing survey: Is Larry

Cuban right? Report from a National

Study. Irvine, CA: Center for Research on

Information Technology and

Organizations. Retrieved from

http://www.crito.uci.edu/tlc/findings/ccsso

.pdf

Becker, H. J. (2001, April). How are teachers using

computers in instruction? Paper presented

at the annual meeting of the American

Educational Research Association, Seattle,

WA.

Becker, H. J., & Riel, M. M. (2000). Teacher

professional engagement and

constructivist-compatible computer use

[on-line]. Centre for Research on

Information Technology and

Organisations, University of California,



Irvine. Retrieved from

http://www.crito.uci.edu/tic/findings.html

Benbasat, I., & Barki, H. (2007). Quo vadis, TAM?

Journal of the Association for Information

Systems, 8, 211-218.

Bontis, N., Crossan, M., & Hulland, J. (2002),

Managing an organizational learning

system by aligning stocks and flows.

Journal of Management Studies, 39, 437-

469. doi:10.1111/1467-6486.t01-1-00299

Cheon, J., Lee, S., Crooks, S. M., & Song, J.

(2012). An investigation of mobile

learning readiness in higher education

based on theory of planned behavior.

Computers & Education, 59, 1054-1064.

doi:10.1016/j.compedu.2012.04.015

Cheon, J., Song, J., Jones, D. R., & Nam, K.

(2010). Influencing pre-service teachers’

intention to adopt Web 2.0 services.

Journal of Digital Learning in Teacher

Education, 27, 53-64.

Chin, W. W. (1998). Issues and opinion on

structural equation modeling. MIS

Quarterly, 22, 7-16.

Chin, W. W., Marcolin, B. L., & Newsted, P. R.

(1996, December). A partial least squares

latent variable modeling approach for

measuring interaction effects: Results

from a Monte Carlo simulation study and

voice mail emotion/adoption study. Paper

presented at the 17th International

Conference on Information Systems,

Cleveland, OH.

Compeau, D., & Higgins, C. A. (1995). Computer

self-efficacy: Development of a measure

and initial test, MIS Quarterly, 19, 189-

211. doi:10.2307/249688

Conner, M., & Armitage, C. J. (1998). Extending

the theory of planned behavior: A review

and avenues for further research. Journal

of Applied Social Psychology, 28, 1429-

1464. doi:10.1111/j.1559-

1816.1998.tb01685.x

Davis, F. D. (1989). Perceived usefulness,

perceived ease of use, and user acceptance

of information technology. MIS Quarterly,

13, 319-340. doi:10.2307/249008

Ertmer, P. A. (2005). Teacher pedagogical beliefs:

The final frontier in our quest for

technology integration? Educational

Technology Research and Development,

53, 25-39.

Evans-Andris, M. (1995) Barrier to computer

integration: micro-interaction among

computer co-ordinators and classroom

teachers in elementary schools, Journal of

Research on Computing in Education, 28,

29-45.

Fishbein, M., & Ajzen, I. (1975). Belief, attitude,

intention, and behavior: An introduction

to theory and research. Reading, MA:

Addison-Wesley

Fornell, C., & Larcker, D. (1981). Evaluating

structural equation models with

unobservable variables and measurement

error. Journal of Marketing Research, 18,

39-50. doi:10.237/3151312

Franklin, T., & van Harmelen, M. (2007). Web 2.0

for content for learning and teaching in

higher Education. Retrieved from

http://www.jisc.ac.uk/media/documents/pr

ogrammes/digitalrepositories/web2-

content-learning-and-teaching.pdf

Gefen, D., & Straub, D.W. (2005). A practical

guide to factorial validity using PLS-

graph: Tutorial and annotated example.

Communications of AIS, 16(1), 91-109.

Gist, M. E., Schwoerer, C. E., & Rosen, B. (1989).

Effects of alternative training methods on

self-efficacy and performance in computer

software training. Journal of Applied

Psychology, 74, 884-891.

doi:10.1037/0021-9010.74.6.884

Grant, M. M., & Mims, C. (2009). Web 2.0 in

teacher education: Characteristics,

implications, and limitations. In T. Kidd &

I. Chen (Eds.), Wired for learning: An

educators guide to Web 2.0 (pp. 343-360).

Charlotte, NC: Information Age

Publishing.

doi:10.4018/9781605667829.ch007

Hartwick, J., & Barki, H. (1994). Explaining the

role of use participation in information

system use. Management Science, 40,

440-465. doi:10.1287/mnsc.40.4.440

Hulland, J. (1999). Use of partial least squares

(PLS) in strategic management research:

A review of four recent studies. Strategic

Management Journal, 20, 195-204.

doi:10.1002/(SICI)1097-

0266(199902)20:2<195::AID-

SMJ13>3.0.CO;2-7

Iacovou, C. L., Benbasat, I., & Dexter, A. S.

(1995). Electronic data interchange and

small organizations: Adoption and impact

of technology, MIS Quarterly, 19, 465-

485. doi:10.2307/249629

International Society for Technology in Education.

(2008). National Educational Technology

Standards and Performance Indicators for

Teachers. Eugene, OR. Retrieved from

http://www.iste.org/standards/nets-for-

teachers.aspx

Jarvenpaa, S. L., Shaw, T. R., & Staples. D. S.

(2004). Toward contextualized theories of

trust: The role of trust in global virtual

teams. Inform. Systems Research, 15, 250-

267. doi:10.1287/isre.1040.0028

Karahanna, E., Straub, D. W., & Chervany, N. L.

(1999). Information technology adoption

http://portal.acm.org/citation.cfm?id=210999&CFID=9245978&CFTOKEN=43080846




http://dx.doi.org/10.2307/249629



across time: A cross-sectional comparison

of pre-adoption and post-adoption beliefs.

MIS Quarterly, 23, 183-213.

doi:10.2307/249751

Lambert, J., & Cuper, P. (2009). Multimedia

technologies and familiar space: 21st

century teaching for 21st century learners.

Contemporary Issues in Technology and

Teacher Education, 8, 264-276.

Li, G., Liu, M., Li, G., Wang, Z., & Chen, W.

(2011). Approach of Web 2.0 application

pattern applied to the information

teaching. Communications in Computer

and Information Science, 175, 123-127.

doi:10.1007/978-3-642-21783-8_20

Littrell, A. B., Zagumny, M. J., & Zagumny, L. L.

(2005). Contextual and psychological

predictors of instructional technology use

in rural classroom. Educational Research

Quarterly, 29(2), 37-47.

Marakas, G. M., Mun, Y. Y., & Johnson, R. D.

(1998) The multilevel and multifaceted

character of computer self-efficacy:

Toward clarification of the construct and

an integrative framework for research.

Journal Information Systems Research, 9,

126-163. doi:10.1287/isre.9.2.126

McLoughlin, C., & Lee, M. (2008, December).

Mapping the digital terrain: New media

and social software as catalysts for

pedagogical change. Paper presented at

the ASCILITE Melbourne 2008, Deakin

University, Melbourne, Australia.

McRobbie, C., Ginns, I., & Stein, S. (2000). Pre-

service primary teachers’ thinking about

technology and technology education.

International Journal of Technology and

Design Education, 10(1), 81-101.

doi:10.1023/A:1008941520152

Mills, L. B. (2007). The next wave now: Web 2.0.

Education Digest: Essential Readings

Condensed for Quick Review, 73(4), 4-5.

Mishra, P., & Koehler, M. J. (2006). Technological

pedagogical content knowledge: A

framework for integrating technology in

teacher knowledge. Teachers College

Record, 108, 1017-1054.

doi:10.1111/j.1467-9620.2006.00684.x

Moseley, D., & Higgins, S. (1999). Ways forward

with ICT: effective pedagogy using

information and communications

technology for literacy and numeracy in

primary schools. London, England:

Teacher Training Agency.

Mumtaz, S. (2000). Factors affecting teachers’ use

of information and communications

technology: a review of the literature.

Journal of Information Technology for


doi:10.1080/14759390000200096

Murugesan, S. (2007). Understanding Web 2.0. IT

Professional, 9(4), 34-41.

doi:10.1109/MITP.2007.78

Niederhauser, D. S., & Stoddart, T. (2001).

Teachers’ instructional perspectives and

use of educational software. Teaching and


doi:10.1016/SO742-051X(00)00036-6

Niess, M. L. (2005). Preparing teachers to teach

science and mathematics with technology:

Developing a technology pedagogical

content knowledge. Teaching and Teacher

Education, 21, 509-523.

doi:10.1016/j.tate.2005.03.006

Niess, M. L., Zee, E. H., & Gilow-Wiles, H.

(2010). Knowledge growth in teaching

mathematics/science with spreadsheets:

moving PCK to TPAC through online

professional development. Journal of

Digital Learning in Teacher Education,

27(2), 42-52.

Notani, A. S. (1998). Moderators of perceived

behavioral control’s predictiveness in the

theory of planned behavior: A meta-

analysis. Journal of Consumer

Psychology, 7, 247-271.

Nunnally, J. C., & Bernstein, I. H. (1994).

Psychometric theory (3rd ed.). New York,

NY: McGraw-Hill.

O’Reilley, T. (2005). What is Web 2.0: Design

patterns and business models for the next

generation of software. Retrieved from

http://www.oreillynet.com/pub/a/oreilly/ti

m/news/2005/09/30/what-is-web-20.html

Pedretti, E., Smith-Mayer, J., & Woodrow, J.

(1999). Teaming technology enhanced

instruction in the science classroom and

teacher professional development, Journal

of Technology and Teacher Education, 7,

131-143.

Pope, M., Hare, D., & Howard, E. (2002).

Technology integration: Closing the gap

between what pre-service teachers are

taught to do and what they can do. Journal

of Technology and Teacher Education, 10,

191-203.

Premkumar, G., & Roberts, M. (1999). Adoption of

new information technologies in rural

small business. Omega International

Journal of Management Science, 27, 467-

484. doi:10.1016/S0305-0483(98)00071-1

Prensky, M. (2001). Digital natives, digital

immigrants. On The Horizon, 9(5), 1-6.

Ravitz, J., Wong, Y., & Becker, H. (1999). Report

to participants. Irvine, CA: Center for

Research on Information Technology and

Organizations. Retrieved from

http://www.crito.uci.edu/tlc/findings/speci

al_report/index.htm

http://dx.doi.org/10.2307/249751



Russell, M., Bebell, D., O'Dwyer, L., & O'Connor,

K. (2003). Examining teacher technology

use: Implications for pre-service and in-

service teacher preparation. Journal of


doi:10.1177/0022487103255985

Russell, S. (2009). Taking care of business:

Authentic use of Web 2.0 in schools. In

M. Orey, V.J. McClendon & R.M. Branch

(Eds.), Educational media and technology

year book (Vol. 34, pp. 199-209). Springer

Science & Business Media, LLC.

doi:10.1007/978-0-387-09675-9_14.

Segars, A. H., & Grover, V. (1993). Re-examining

perceived ease of use and usefulness: A

confirmatory factor analysis. MIS

Quarterly, 17, 517-525.

doi:10.2307/249590

Shah, P. P. (1998). Who are employees' social

referents? Using a network perspective to

determine referent others. The Academy of

Management Journal, 41, 249-268.

doi:10.2307/256906

Sheingold, K., & Hadley, M. (1990). Accomplished

teachers: Integrating computers into

classroom practice. New York, NY: Bank

Street College of Education, Center for

Technology in Education.

Sheppard, B. H., Hartwick, J., & Warshaw, P. R.

(1988). The Theory of reasoned action: A

meta-analysis of past research with

recommendations for modifications and

future research. Journal of Consumer

Research, 15, 325-343.

doi:10.1086/209170

Shiue, Y. (2007). Investigating the sources of

teachers’ instructional technology use

through the decomposed theory of planned

behavior. Journal of Educational

Computing Research, 36, 425-453.

doi:10.2190/A407-22RR-50X6-2830

Shoffner, M. (2009). Personal attitudes and

technology: Implications for preservice

teacher reflective practice. Teacher

Education Quarterly, 36, 143-161.

doi:10.1016/j.tate.2008.11.012

Smerdon, B., Cronen, S., Lanahan, L., Anderson,

J., Iannotti, N., & Angeles, J. (2000).

Teachers’ tools for the 21st century: A

report on teachers’ use of technology

(NCES 2000-102). Retrieved from

National Center for Education Statistics

website:

http://nces.ed.gov/pubs2000/2000102.pdf

Song, J., & Zahedi, F. M. (2005). A theoretical

approach to Web design in e-commerce: A

belief reinforcement model, Management

Science, 51, 1219-1235.

doi:10.1287/mnsc.1050.0427

Taylor S., & Todd, P. (1995). Decomposition and

crossover effects in the theory of planned

behavior: A study of consumer adoption

intentions. International Journal of

Research in Marketing, 12, 137-156.

doi:10.1016/0167-8116(94)00019-K

van Braak, J., Tondeur, J., & Valcke, M. (2004).

Explaining different types of computer use

among primary school teachers. European

Journal of Psychology of Education, 19,

407-422. doi:10.1007/BF03173218

Vaughan, N., Nickle, T., Silovs, J., & Zimmer, J.

(2010). Moving to their own beat:

Exploring how students use Web 2.0

technologies to support group work

outside of class time. Journal of

Interactive Online Learning, 10, 113-127.

Venkatesh, V., & Davis, F. D. (2000). A theoretical

extension of the technology acceptance

model: Four longitudinal field studies.

Management Science, 46(2), 186-204.

Venkatesh, V., Brown, S. A., Maruping, L. M., &

Bala, H. (2008). Predicting different

conceptualizations of system use: The

competing roles of behavioral intention,

facilitating conditions, and behavioral

expectation. MIS Quarterly, 32, 483-502.

Vockley, M. (2008). Maximizing the impact: Te

pivotal role of technology in 21st century

education system. Retrieved from

http://www.setda.org/web/guest/maximizi

ngimpactreport

Williams, J., & Chinn, S. J. (2009). Using Web

2.0 to support the active learning

experience. Journal of Information System

Education, 20, 165-174.

Wozney, L., Venkatesh, V., & Abrami, P. C.

(2006). Implementing computer

technologies: Teachers’ perceptions and

practices. Journal of Technology and


Yildirim, S. (2000). Effects of an educational

computing course on pre-service and in

service teachers: A discussion and

analysis of attitudes and use. Journal of

Research on Computing in Education, 32,

479-495.




The Revised School Culture Elements Questionnaire: Gender and Grade Level Invariant?

Thomas A. DeVaney, Nan B. Adams, and Flo Hill-Winstead

Southeastern Louisiana University

Mitzi P. Trahan

University of Louisiana at Lafayette

School culture represents the assumptions,

beliefs, values, and habits that constitute the norm

for a school and shapes how the staff thinks, feels,

and acts. Culture also can be defined as “how we

do things around here” and the stories we tell

ourselves (DuFour, 1998). Deal and Peterson

(1990) referred to culture as deep patterns of

values, beliefs, and traditions that have been

formed over the course of the school's history,

consisting of “stable, underlying social meanings

that shape beliefs and behavior over time” (p. 3).

School cultures are complex webs of

traditions and rituals that have evolved over time

(Deal & Peterson, 1990; Schein, 1985). Senge et

al. (2000) stressed that a school's culture is not

static but a continual process in which attitudes,

values, and skills continually reinforce each other.

The concept of school culture provides school

leaders with a holistic way to look at the school and

school reform. The holistic view of school culture

was also espoused by Schoen (2005) who defined

school culture as “the holistic activities and ‘ways

of being and doing’ of those who work in or

participate on a regular basis within a school” (p.

13). Stolp and Smith (1995) believed that a

deepened understanding of school culture would

allow leaders to influence the values, beliefs, and

underlying assumptions held by school community

members.

The importance of school culture was

illustrated in the 1996 Interstate School Leaders

Licensure Consortium (ISLLC) standards and the

revised standards that were adopted in 2008 by the

National Policy Board for Educational

Administration (Council of Chief State School

Officers, 2008). In Standard 2 of the 1996 ISLLC

Standards, an educational leader is described as an

individual “who promotes the success of all

students by advocating, nurturing, and sustaining a

school culture and instructional program conducive

to student learning and staff professional growth”

(p. 18). The 2008 Standards continue to

recognize the importance of school culture by

stating, “An education leader promotes the success

of every student by advocating, nurturing, and

sustaining a school culture and instructional

program conducive to student learning and staff

professional growth” (p. 14).

The role of culture in the preparation of

educational leaders also was illustrated through

standards developed by the National Policy Board

for Educational Administration and adopted by the

National Council for Accreditation of Teacher

Education for review of Educational Leadership

preparation programs. Stated in the standards that

were published in 2002 was that candidates who

completed Educational Leadership preparation


addressed to Thomas A. DeVaney, SLU 10549

Hammond, LA 70402. Email: [email protected]

The purpose of this research was to examine the psychometric properties of the RSCEQ with respect to

invariance across gender and grade level, using a sample of 901 teachers from 44 schools in southeast

Louisiana. Reliability estimates were consistent with previous research and ranged from .81to .90 on the

actual and .83 to .92 on the preferred scales, respectively. Confirmatory factor analyses supported the 3-

factor structure of both scales and the theoretical assumption that the factors are correlated. Analyses

also provided evidence that the RSCEQ is invariant across gender and grade level. Additional evidence

that supports the use of the RSCEQ as a tool that educational leaders can use to assess and improve

school culture is provided by the results of this investigation.

THOMAS DeVANEY, NAN ADAMS, FLO HILL-WINSTEAD, AND MITZI TRAHAN


programs should have the knowledge and ability to

promote a positive school culture. Additionally,

completers are expected to have the ability to

assess school culture in order to improve school

programs and culture (National Policy Board for

Educational Administration, 2002). These

standards were revised in 2009 and continue to

illustrate the importance of school culture by

stating that an educational leader should “nurture

and sustain a culture of collaboration, trust,

learning, and high expectations” (National Policy

Board for Educational Administration, 2009, p.

12). The standards continue by stating that an

educational leader must have knowledge of “the

elements of school culture and ways it can be

influenced to ensure student success” (p. 12).

Although the importance of school culture has

been illustrated through professional standards

since 1996, its importance also has been recognized

through research activity since the release of the

1996 ISLLC Standards. A search of the

Education Resources Information Center (ERIC)

database for entries related to school culture in

elementary and secondary education between 1997

and 2012 revealed 525 journal articles and 711

ERIC documents, an average of 77.3 entries per

year.

The importance of school culture, as illustrated

through professional standards and publications,

raises an important question for practitioners and

researchers, “How can school culture be

measured?” Maslowski (2006), in a review of six

quantitative, multi-dimensional inventories

designed to measure school culture, concluded that

the instruments contained a common core of scales

but also contained notable differences. One of the

reviewed instruments was the School Culture

Elements Questionnaire, which contained the

unique approach of assessing perceptions of actual

and preferred school culture. The actual scale

requires individuals to respond with respect to how

things actually occur at their school; whereas on

the preferred scale, individuals respond with

respect to how they would like things to be at their

school.

The original 64-item School Culture Elements

Questionnaire (SCEQ) developed by Cavanagh

(1997) was based on the Cultural Elements Model

of School Culture and designed to assess the

following eight domains of school culture: (a)

professional values, (b) teachers as learners, (c)

collegiality, (d) mutual empowerment, (e)

collaboration, (f) shared vision, (g) school wide

planning, and (h) transformational leadership.

Cavanagh (1997) used a sample of 422 high school

and primary school teachers in Western Australia

to examine the psychometric properties of the

actual scale of the SCEQ. Reliability estimates

for the proposed scales ranged from .62 to .70.

Although the development of the SCEQ was based

on eight domains of culture, Cavanagh’s factor

analysis results supported a 6-factor structure

instead of the proposed 8-factor structure.

Other researchers examined the properties of

the SCEQ with samples of teachers in the

southeastern United States. Dellar, Cavanagh,

and Ellett (1998) adapted the original SCEQ by

adding 14 statements and editing language for

appropriateness. Dellar et al.(1988) used samples

of 3,279 and 1,389 teachers to examine the actual

scale of the Revised SCEQ (RSCEQ). They

showed the RSCEQ to be a useful measure of

school culture and identified three predominant

factors: (a) Shared Leadership and Vision, (b)

Collegial Teaching and Learning, and (c)

Professional Commitment. In addition to analyses

of the total sample, Dellar et al. (1998) examined

the factor structure of the RSCEQ’s actual scale

based on grade level (i.e., elementary, middle, and

high school). Although they identified a common

core of factors, differences among grade levels

were identified with respect to the proportions of

variance explained.

In 2001, Olivier continued the refinement of

the original SCEQ, leading to the creation of a

20-item RSCEQ used to measure professional

school culture. Olivier examined the structure of

the RSCEQ actual scale using a sample of 1,437

elementary level teachers from 12 school districts

in Louisiana. Factor analysis results supported

the 3-factor structure identified by Dellar et al.

(1998) and Ellet, Rugutt, and Cavanagh (2000).

Lorraine (2012) also used factor analysis based on

a sample of 356 teachers to support the 3-factor

structure of the actual scale.

Overall, the validation studies for the SCEQ

and RSCEQ indicate that these instruments

measure constructs representative of general

professionalism and perceived elements of school

culture. However, several limitations have been

identified in previous examinations of the

psychometric properties of these instruments.

First, neither Cavanagh (1997), Olivier (2001), nor

Lorraine (2012) examined the factor structure for

subgroups. It appears that the only researchers to

examine the factor structure for subgroups were

Dellar et al. (1998). Although their results

supported a common factor structure for the

subgroups based on grade level, their research also

contained the same limitation as did that of

Cavanagh, Olivier, and Lorraine. Specifically, the

researchers did not examine the structure of the

preferred scale. Therefore, the purpose of the

current research was to continue the analysis of the

20-item RSCEQ by examining the psychometric

properties of both scales, actual and preferred, with

respect the 3-factor structure and invariance across

gender and grade level.

THE REVISED SCHOOL CULTURE ELEMENTS QUESTIONNAIRE


Method

Procedure

The Revised School Culture Elements

Questionnaire (RSCEQ) was distributed to faculty

in PreK-12 schools in southeast Louisiana as part

of a class project for students enrolled in graduate

courses in Educational Leadership. Because the

distribution of the surveys was a required activity

for the students and different professors were

involved, the distribution involved traditional

paper-and-pencil delivery and electronic delivery.

The method of delivery was determined by the

professor in order to meet the needs of the class.

Regardless of delivery, students were required to

obtain consent from their school to administer the

surveys. For the paper-and-pencil delivery method,

a copy of the survey and cover letter was sent to the

students. They were then required to make the

necessary copies and arrange for the distribution

and collection of the surveys. For students who

used the electronic distribution, an email that

contained a link to the cover letter was sent to the

students. They were then required to distribute the

link to their faculty using an appropriate delivery

method.

After the paper-and-pencil surveys were

returned, they were entered into an Excel

spreadsheet, and random checks were conducted to

determine the accuracy of the data processing.

For the electronic submissions, the data were

exported from www. surveymonkey.com in an

Excel format. Prior to processing the paper-and-

pencil surveys, a sample survey was submitted

electronically, and the results were downloaded in

an Excel format. The downloaded file was used as

the template into which the responses from paper-

and-pencil surveys were entered. The use of the

template ensured an exact match with respect to the

number and order of the columns contained in the

downloaded response files and hand processed

response files. Because the formats of the two types

of files were identical, the files were easily merged

into one file for analysis. The final Excel files

were imported into SPSS 17.0 for the final

analyses. All computed scores were obtained using

the guidelines provided by the RSCEQ developers.

Sample Surveys were completed by 901 teachers in 44

schools throughout parishes in southeast Louisiana.

The number of respondents per school ranged from

a minimum of 1 to a maximum of 46, and 33

schools (75.0%) had at least 10 faculty responses.

In Louisiana, the school performance labels for

schools ranged from Academically Unacceptable to

Five Stars. The schools that are included in the

current analyses ranged from One Star to Four

Stars with 75% of the schools identified as Two or

Three Star schools based on the School Report

Cards available from the Louisiana Department of

Education. The most frequent gender was female

(81.1%), and the most frequently reported highest

degree earned was a Bachelor’s degree (59.3%).

Respondents also reported being certified in the

field in which they taught (89.3%). Table results

also illustrate uniform distributions with respect to

age and number of years at the current school.

Finally, demographics show that the sample

included teachers at the elementary, middle, and

high school levels.

An examination of the data indicated that 98

individuals had one or more missing responses on

the actual scale and 77 individuals had one or more

missing responses on the preferred scale. To

minimize the impact of missing values, individuals

with missing values were omitted from the

analysis. Therefore, the sample sizes were 803 and

824 for the actual and preferred scale analyses,

respectively. An examination of the respondent

characteristics for these samples showed that the

distributions were similar to the overall sample,

which suggests that a systematic pattern to explain

the missing values was not present.

Instrument

The RSCEQ measures perceptions related to

three dimensions: (a) shared leadership and vision,

(b) collegial teaching and learning, and (c)

professional commitment (see Table 1 for

conceptual definitions of the dimensions). For each

dimension, the RSCEQ requires respondents to rate

their actual school culture and how they would

prefer the culture to be at their school. The actual

scale addresses "how I and my school actually are"

and requires participants to respond to the

statements according to their perceptions of how

things actually occur in their schools. Conversely,

the preferred scale indicates participants' opinions

about the school in which they wish to work,

thereby detailing their preferences for

characteristics of an ideal school. Actual and

preferred perceptions are assessed by responding to

20 statements using a 4-point Likert-type scale

ranging from 1 (Strongly Disagree) to 4 (Strongly

Agree).

The questionnaire was formatted into three

columns. The center column contained the 20

statements while the left and right columns

contained the response options for the actual and

preferred scales, respectively (see Appendix A).



Table 1

Conceptual Definitions of RSCEQ Dimensions

Dimension Definition

Shared Leadership/Vision An ongoing process to accomplish school goals that reflect interpersonal

roles and relationships among organizational members grounded in the

norms, values, and beliefs of cooperation, sharing, support, and

encouragement in work tasks and sensitivity to the problems and

difficulties expressed by colleagues.

Collegial Teaching and Learning A continual growth process in which teachers prioritize the need to

continue to learn as an organizational member for the purpose of

enhancing learning through collaborative efforts in order to personally

and collectively benefit all students and staff; characterized by

collaborative work, shared planning, personal and group reflection,

dialogue among teachers, and incorporation of educational research.

Professional Commitment A continuous process that provides opportunities to enhance the

professional effectiveness of teachers through a commitment to the

continuous improvement of the learning process and a commitment

among teachers to serve as sources of help and support for colleagues

within the organization.

(Olivier, 2001).

Statistical Analyses

To examine the properties of the actual and

preferred RSCEQ scales, reliability estimates were

calculated for the total sample and subgroups using

SPSS 17.0, and confirmatory factor analyses

(CFAs) were conducted using AMOS 17.0. The

model used for the factor analyses was based on the

results of previous studies and specified a 3-factor

structure with independent factors (see Figure 1).

Based on reporting guidelines recommended by

Meyers, Gamst, and Guarino (2006), the following

fit measures were reported for each total group

CFA: (a) chi square, (b) normed fit index (NFI), (c)

comparative fit index (CFI), and (d) root mean

square error or approximation (RMSEA). The

following criteria were used to evaluate the

goodness of fit for the specified model: (a) chi

square, p ≥ .05, (b) NFI > .90, (c) CFI > .95, and

(d) RMSEA < .10 (Meyers et al., 2006).After the

scales were examined using the total sample,

subgroup analyses were conducted to test for three

types of invariance: (a) metric, (b) factor

covariance, and (c) error variance and covariance.

Metric invariance implies that the unstandardized

factor loadings for each indicator are equal across

groups. Factor covariance invariance focuses on

the inter-relationships between the factors across

groups. Error variance and covariance invariance

examines the equality of error variances across

groups (Blunch, 2008).

Results

Reliability Analyses

Actual scale. The results of the reliability

analyses for the actual scale are contained in Table

2. The reliability estimates for the total sample

ranged from .83 for Professional Commitment to

.90 for Shared Leadership and Vision and were

consistent with values reported by Olivier (2001)

and Lorraine (2012) for the 20-item RSCEQ. For

the subgroup analyses, the results showed that the

reliability estimates for subgroups based on gender

and grade level remained consistent with the values

for the total sample. The estimates ranged from

.83 to .89 for females and .83 to .90 for males.

Finally, the reliability estimates for elementary

respondents ranged from .82 to .90, and the

estimates for high school respondents ranged from

.83 to .88.



Table 2

Reliability Estimates for Total Sample and Subgroups on RSCEQ Actual Scale

RSCEQ Dimension

Sample

n

Shared

Leadership/Vision

Collegial Teaching

and Learning

Professional

Commitment

Total 803 .90 .85 .83

Gender

Female 655 .89 .86 .83

Male 115 .90 .81 .83

School Level

Elementary 474 .90 .85 .82

Secondary 227 .88 .84 .83

Note. Subgroup sample sizes may not equal total sample size due to missing values on the subgroup indicator.

Figure 1. Confirmatory factor analysis model for the RSCEQ assuming uncorrelated factors. Leadership =

Shared leadership and vision. Professionalism = Professional commitment. Collegiality = Collegial teaching and

learning.



Preferred scale. The results of the reliability

analyses for the preferred scale are presented in

Table 3. For the total sample, the reliability

estimates ranged from a low of .87 for Professional

Commitment to a high of .91 for Collegial

Teaching and Learning. For the subgroup analyses,

the results showed that the reliability estimates for

subgroups based on gender and grade level

remained consistent with the values for the total

sample. Reliability estimates ranged from .88 to .91

for females and .83 to .89 for males. Finally, for

the reliability estimates based on grade level, the

values ranged from .87 to .90 for elementary

respondents and .88 to .92 for high school

respondents.

Table 3

Reliability Estimates for Total Sample and Subgroups on RSCEQ Preferred Scale

RSCEQ Dimension

Sample

n

Shared

Leadership/Vision

Collegial Teaching

and Learning

Professional

Commitment

Total 824 .90 .91 .87

Gender

Female 672 .91 .91 .88

Male 121 .86 .89 .83

School Level

Elementary 493 .90 .90 .87

Secondary 226 .91 .92 .88

Note. Subgroup sample sizes may not equal total sample size due to missing values on the subgroup indicator.

Confirmatory Factor Analyses

Actual scale. In order to examine the

underlying structure of the actual scale, a

confirmatory factor analysis was conducted using a

3-factor model with uncorrelated factors. The

results of the analysis are presented in Table 4.

An examination of the standardized regression

weights showed that all weights were greater than

.60 and supported the proposed item to factor

pattern for the 3-factor model. The chi-square

value of 1,586.8 (df = 170, p < .001) indicated that

the specified model was not an acceptable fit to the

data. The analysis also resulted in values of .795

for NFI, .812 for CFI, and .102 for RMSEA.

Similar to the chi-square result, these values

suggested that the specified model did not provide

a good fit for the data. These results suggested

that an alternative model may provide a better fit

for the data.

Although previous researchers treated the

factors as independent or uncorrelated, Cavanagh

(1997) acknowledged that, theoretically, the factors

should be correlated. Therefore, in order to

improve the original model, it was modified so that

the factors were correlated, and the analysis was

repeated (See Figure 2). The correlations among

the factors ranged from .60 to .86 and supported

Cavanagh’s belief that the factors are related. As

shown in Table 5, the analysis produced a chi-

square value of 722.58 (df = 167, p < .001).

Although the chi-square value was statistically

significant and did not support the model as a good

fit, by allowing the factors to be correlated, the chi-

square value was reduced by 54.5%.

Additionally, the values for two of the fit indices

suggested that the model provided a good fit. The

NFI (.914) exceeded the criterion of .90, and the

RMSEA (.06) was below the criterion of .10.

These results indicated that allowing the factors to

be correlated provided a better fitting model.

Consequently, this model was used for the

remaining CFAs.



Table 4

Confirmatory Factor Analysis Results for Model with Uncorrelated Factors: Actual Scale

Item Leadership Collegial

Teaching

Professional

Commitment

Administrators provide visible, ongoing support for new

school programs and ideas. .681

Administrators are sympathetic with problems and

difficulties encountered by teachers in their work. .771

Administrators work to ensure the cooperation of

teachers. .812

Administrators visibly encourage teachers to be the best

that they can be in the classroom. .725

Teachers and administrators work cooperatively in

developing new school programs and policies. .728

Teachers receive the assistance they need from

administrators and colleagues to enhance the quality of

teaching and learning in their classroom. .795

Leadership roles are equally shared by teachers and

administrators. .694

Teachers are willing to help each other when problems

arise. .606

Teachers share classroom experiences with each other to

improve their understanding of students learning. .709

Teachers openly share problems with each other. .649

Teachers professionally share and learn from one

another. .786

Teachers encourage each other to use professional

judgment when making decisions. .658

Teachers feel comfortable in providing suggestions to

colleagues about ways in which to improve teaching and

learning in their classrooms. .680

Teachers spend time together to informally discuss ways

to improve the school. .620

Teachers give priority to helping their students develop

higher order thinking skills.

.677

Teachers incorporate the findings of educational research

into their own teaching and learning practices.

.622

Teachers believe that all students can learn. .645

Teachers are committed to professional growth to

improve teaching and learning.

.731

Teachers adequately plan teaching and learning activities

to accommodate individual differences among students.

.714

Teachers spend time in professional reflection about

their work.

.651

Model Fit Indices

χ2 NFI CFI RMSEA

1586.80* .795 .812 .102

*p < .001



Table 5

Confirmatory Factor Analysis Results for Model with Correlated Factors: Actual Scale


Teaching

Professional

Commitment






teachers. .802











arise. .614




Teachers professionally share and learn from one another. .783










.656



.623




.758



.702

Teachers spend time in professional reflection about their

work.

.655

Model Fit Indices

χ2 NFI CFI RMSEA

722.58* .914 .932 .061

*p < .001



Subgroup analyses by gender. To test for

model invariance across gender, a CFA was

conducted with gender as the grouping variable.

The results of the analysis are presented in Table 6

and suggest that the model was factor invariant

with respect to the measurement weights (metric

invariant), structural covariances (factor covariance

invariant), and measurement residuals (error

variance invariant). These results indicate that not

only did the model fit both genders but also the

measurement weights were equal for males and

females. Further, the factor covariance and error

variance invariances suggest that the factors were

inter-related in the same way for male and female

respondents and the error variances were equal

across gender.

Subgroup analysis by grade level. To test for

model invariance across grade level, a CFA was

conducted with grade level (elementary or high

school) as the grouping variable. The results of

the analysis are presented in Table 7 and suggest

that the model was factor invariant with respect to

the measurement weights (metric invariant),

structural covariances (factor covariance invariant),

and measurement residuals (error variance

invariant). These results indicate that not only did

the model fit both grade levels but also the

measurement weights were equal across groups.

Further, the factor covariance and error variance

invariances suggest that the factors were inter-

related in the same way for both grade levels and

the error variances were equal across grade levels.

Table 6

Chi Square Tests for Invariance of Actual Scale Across Gender: Actual Scale

Model df χ2 χ

2 change df change p

Unconstrained 334 887.74

Equal measurement weightsa 351 901.33 13.59 17 .696

Equal structural covariancesb 357 906.79 5.46 6 .486

Equal error variancesc 377 937.98 31.19 20 .053

a results based on the assumption that the unconstrained model was correct.

b results based on the assumption that the model with equal measurement weights was correct.

c results based on the assumption that the model with equal measurement weights and structural covariances was

correct.

Table 7

Chi Square Tests for Invariance of Actual Scale Across Grade Level: Actual Scale

Model df χ2 χ





Equal error variancesc 377 873.02 9.37 20 .978




correct.

Preferred scale. Because prior researchers

did not examine the properties of the preferred

scale, the best fitting model (see Figure 2) from the

actual scale analysis was used. Results of the

analysis provided support for the identified model.

The values of the correlation coefficients among

the factors ranged from .894 to .969, and

standardized regression weights were greater than

.60 and supported the 3-factor model (see Table 8).

Although the chi-square value (859.54, df = 167)

indicated a statistically significant difference was

present between the model and observed data, the

fit measures suggested that the model provided a

good fit for the data. The NFI (.925) and RMSEA

(.071) exceeded the respective criterion for

considering the model a good fit for the data.

Additionally, the CFI value (.939) approached the

criterion (.95) for classifying the model as a good

fit.



Table 8

Confirmatory Factor Analysis Results for Model with Correlated Factors: Preferred Scale


Teaching

Professional

Commitment






teachers. .786











arise. .706




Teachers professionally share and learn from one another. .834










.688



.707




.769



.797


work.

.754

Model Fit Indices

χ2 NFI CFI RMSEA

859.54* .925 .939 .071

*p < .001



Figure 2. Revised confirmatory factor analysis model for the RSCEQ assuming correlated factors. Leadership =

Shared leadership and vision. Professionalism = Professional commitment. Collegiality = Collegial teaching and

learning.

Subgroup analysis by gender. To test for

model invariance across gender, a confirmatory

factor analysis was conducted with gender as the

grouping variable. The results of the analysis are

presented in Table 9 and suggested that the model

was invariant with respect to the measurement

weights (metric invariant) and structural

covariances (factor covariance invariant). These

results indicated that not only did the model fit both

subgroups but also the measurement weights were

equal across gender and the factors were inter-

related in the same way for male and female

respondents. However, the results also indicated

that statistically significant differences were

present between the error variances associated with

the scale items for male and female respondents.

Subgroup analysis by grade level. To test for

model invariance across grade level, a confirmatory

factor analysis was conducted with grade level

(elementary or high school) as the grouping

variable. The results of the analysis are presented in

Table 10 and suggested that the model was

invariant with respect to the measurement weights

(metric invariant). However, the results also

indicated that the model was not invariant with

respect to the structural covariances or error

variances. These results suggested that not only did

the model provide a good fit at both grade levels

but also the measurement weights were equal

across grade level. However, these results also

suggested that a statistically significant difference

was present in the way the factors were related and

the error variances associated with the scale items

for the two grade levels.



Table 9

Chi Square Tests for Invariance of Preferred Scale Across Gender : Preferred Scale

Model Df χ2 χ





Equal error variancesc 377 1454.50 306.26 20 <.001




correct.

Table 10

Chi Square Tests for Invariance of Preferred Scale Across Grade Level: Preferred Scale

Model df χ2 χ





Equal error variancesc 377 1281.14 227.96 20 <.001




correct.

Discussion

In 2006, Maslowski noted an increase in the

prevalence of research related to the topic of school

culture. One possible stimulus for this increase

was the development and release of the 1996

Interstate School Leaders Licensure Consortium

(ISLLC) standards in which the need for

educational leaders to understand the role of school

culture was explicitly acknowledged. Of

particular interest for the current study was the

expectation that school culture and climate be

assessed on a regular basis. The increased

attention that has been given to school culture, as

evidenced by the increase in research activity and

its continued inclusion in professional standards for

educational leaders (e.g., revised ISLLC standards

and original and revised ELCC standards), has

resulted in the need for instruments to assess school

culture. One such instrument is the RSCEQ

(Olivier, 2001).

The RSCEQ was derived from the original

SCEQ developed by Cavanagh (1997) and provides

a unique feature among school culture instruments

by collecting data that reflect the perceptions of

faculty concerning not only the current school

culture but also the preferred culture. Researchers

who have reported the results of developmental

work leading to the current RSCEQ (Cavanagh,

1997; Dellar et al., 1998; Olivier, 2001) have only

focused on the factor structure and properties of the

actual scale. Further, the original examination by

Cavanagh (1997) and later examination of the

current 20-item version by Olivier (2001)

addressed the properties only for the total sample.

Therefore, the purpose of the current study was to

examine the psychometric properties of the actual

and preferred scales of the RSCEQ for the total



sample and subgroups based on gender and grade

level (i.e., elementary or high school).

The results of this study are based on a sample

of 901 elementary through high school teachers

across 44 schools in southeast Louisiana. The

psychometric properties of the RSCEQ were

examined through the calculation of score

reliabilities and use of confirmatory factor

analyses. The reliability analyses produced total

sample estimates that ranged from .83 to .90 on the

actual scale and .87 to .91 for the preferred scale.

Estimates for subgroups based on gender and grade

level (i.e., elementary or high school) were

consistent with the total sample estimates.

The original model for the confirmatory factor

analyses was based on findings by previous

researchers who examined the actual scale and

assumed independent factors. However, the

measures of fit suggested that the model shown in

Figure 1 did not provide a good fit for the data.

Consequently, the model was modified to align

with the theoretical assumption made by Cavanagh

(1997) that the factors were related. The results

for this model showed a substantial decrease in the

chi square value, and multiple measures of fit

satisfied the respective criterion for identifying a

good fitting model. These results provide

evidence to support not only the three-factor

structure of the data but also the theoretical

assumption that the factors are correlated.

The multi-group analyses provided evidence

that the RSCEQ scales were invariant across

gender and grade level. For the actual scale, the

scale was invariant with respect to measurement

weights, factor covariances, and error variances

across gender and grade level. Additionally, the

results of the preferred scale analyses indicated that

the scale was invariant with respect to the

measurement weights and factor covariance across

gender. However, across grade level, the

preferred scale was only invariant with respect to

the measurement weights. Therefore, although

the results indicate that statistically significant

differences were not present between

corresponding measurement weights, statistically

significant differences were related to the

relationships among the three factors at the

elementary and high school levels.

Because the current findings support the three-

factor structure of the RSCEQ for males and

females and elementary and high school teachers,

this instrument provides a tool that educational

leaders can use to assess and describe the climate

and culture among these subgroups. Further,

because the actual scale was invariant across

gender and grade level, the RSCEQ can be used by

educational leaders and researchers to compare

responses across these subgroups. However, the

preferred scale was not fully invariant across

genders or grade level. Although Byrne (2004)

noted that testing for full invariance (e.g., testing

for invariance of error parameters) is an overly

restrictive analysis, comparing results obtained

from the preferred scale across gender and grade

level should be made with caution.

Recommendations for Further Research

The results of this study provide additional

evidence that supports the theoretically based

structure of the RSCEQ that includes three

correlated factors. However, continued research

that examines its properties should be conducted.

Because the samples for the current study and

previous validation study of the 20-item RSCEQ

(Olivier, 2001) were based on teachers in

Louisiana, future research should utilize teachers

from other geographic locations in order to

strengthen the population validity of the

instrument. Also, as noted in the description of

the sample, variation was present in the number of

responses per school. Because of differences

among schools regarding the size of faculties and

the lack of available data regarding total faculty

size, response rates per school could not be

determined. However, it can be assumed that the

voluntary nature of participation resulted in

variations in per school response rates. Using the

teacher as the unit of analysis, instead of

aggregating data to the school level, should have

minimized the impact of per school response rate;

however, future research should attempt to utilize

techniques that will maximize the response rates

within schools.

The lead editors for this article were John R. Slate

and Anthony J. Onwuegbuzie.

References

Blunch, N. J. (2008). Introduction to structural

equation modeling using SPSS and

AMOS. Thousand Oaks, CA: Sage.

Byrne, B. M. (2004). Testing for multigroup

invariance using AMOS Graphics: A road

less traveled. Structural Equation

Modeling, 11, 272-300.

doi:10.1207/s15328007sem1102_8

Cavanagh, R. F. (1997). The culture and

improvement of Western Australia’s

senior secondary schools (Unpublished

doctoral dissertation). Curtain University,

Western Australia.

Council of Chief State School Officers. (2008).

Educational leadership policy standards:

ISLLC 2008. Washington, DC: Author.

Retrieved from http://www.npbea.org



Deal, T. E., & Peterson, K. D. (1990). The

principal's role in shaping school culture.

Retrieved from ERIC database.

(ED325914)

Dellar, G., Cavanagh, R., & Ellett, C. D. (1998,

April). Changing the context of learning

environment research. Paper presented at

the annual meeting of the American

Educational Research Association, San

Diego, CA.

DuFour, R. (1998). Schools and learning

communities: Learning-centered schools

grow from strong cultures. Journal of Staff

Development, 19(1), 58-59.

Ellett, C. C., Rugutt, J. K., & Cavanagh, R. (2000,

April). Conceptualizing and measuring

school environments from cultural

perspective: Phase three of the

development of a teacher perceptions

measure. Paper presented at the annual

meeting of the American Educational

Research Association, New Orleans, LA.

Lorraine, R. R. D. O. (2012). A study of the

relationship between teachers’ perception

of principal’s leadership practices and

school culture to student achievement

(Doctoral dissertation). Available from

ProQuest Dissertations and Theses

database. (UMI No. 3482494)

Maslowski, R. (2006). A review of inventories for

diagnosing school culture. Journal of

Educational Administration, 44(1), 6-35.

doi:10.1108/09578230610642638

Meyers, L. S., Gamst, G., & Guarino, A. J. (2006).

Applied multivariate research design and

interpretation. Thousand Oaks, CA: Sage.


Administration. (2002). Standards for

advanced programs in educational

leadership for principals, superintendents,

curriculum directors, and supervisors.

Austin, TX: Author. Retrieved from

http://www.npbea.org


Administration. (2009). Educational

leadership program standards: ELCC

revised standards – First draft for

advanced programs that prepare

principals, superintendents, curriculum

directors, and supervisors. Austin, TX:

Author. Retrieved from

http://www.npbea.org

Olivier, D. F. (2001). Teacher personal and school

culture characteristics in effective

schools: Toward a model of a professional

learning community. (Doctoral

dissertation). Available from ProQuest

Dissertations and Theses database. (UMI

No. 3016566)

Schein, E. H. (1985). Organizational culture and

leadership. San Francisco, CA: Jossey-

Bass.

Schoen, L. (2005). Conceptualizing, describing,

and comparing school cultures: A

comparative case study of school

improvement processes. (Doctoral

dissertation). Available from ProQuest

Dissertations and Theses database. (UMI

No. 3167154)

Senge, P., Cambron-McCabe, N., Lucas, T., Smith,

B., Dutton, J., & Kleiner, A. (2000).

Schools that learn: A fifth discipline

fieldbook for educators, parents, and

everyone who cares about education. New

York, NY: Doubleday.

Stolp, S., & Smith S. C. (1995). Transforming

school culture: Stories, symbols, values, &

the leaders role. University of Oregon:

ERIC Clearinghouse on Educational

Management.



Appendix A

Revised School Culture Elements Questionnaire

This questionnaire contains a number of statements about things which occur in some schools. After reading

each of the statements carefully, you are asked to judge each response according to two criteria: (1) “how you

and your school actually are…” and (2) “you would prefer that you or your school would be…” You are to

indicate the extent to which you agree or disagree with each of the statements. The “actual scale” applies to

how “YOU AND/OR YOUR SCHOOL ACTUALLY ARE” and the “prefer scale” describes what you

would “PREFER TO BE OR WOULD PREFER YOUR SCHOOL TO BE LIKE.”

SCALE: 1=Strongly Disagree (SD) 2=Disagree (D) 3=Agree (A) 4=Strongly Agree (SA)

“ACTUAL” SCALE

SD D A SA

STATEMENTS

“PREFER” SCALE

SD D A SA


school programs and ideas.


arise.




difficulties encountered by teachers in their work.


improve their understanding of students learning.



Administrators work to ensure the cooperation of teachers.

Teachers openly share problems with each other.

Teachers believe that all students can learn.


that they can be in the classroom.

Teachers professionally share and learn from one another.

Teachers are committed to professional growth to improve

teaching and learning.


developing new school programs and policies.


judgment when making decisions.





teaching and learning in their classroom.



learning in their classrooms.


work.


administrators.


to improve the school.

Copyright 2012 by the RESEARCH IN THE SCHOOLS Mid-South Educational Research Association 2012, Vol.19, No. 2, 45-61


Examining Variations in Programmatic Delivery on Teaching Candidates’ Sense of Efficacy

S. Michael Putman The University of North Carolina at Charlotte

The current climate in the United States,

markedly fixed on teacher effectiveness and student learning, has resulted in considerable pressure at multiple levels in the educational hierarchy, from elementary schools through institutions of higher education (Ball & Forzani, 2009; Darling-Hammond, 2010). The demands have been especially impactful on colleges of education as reformers request evidence of programmatic rigor while simultaneously calling for the preparation of teachers who are content-area experts able to work with diverse populations (Darling-Hammond, 2010). Numerous measures have been explored by coordinators of teacher education programs to address these challenges (see Darling-Hammond, 2010); yet, a common assertion is that attention must be directed towards designing authentic teaching-learning environments that will empower preservice teaching candidates to translate theory presented within university courses into their practices more effectively

(Moore, 2003). Field experiences have been cited as representing a mechanism to facilitate this process because they allow candidates with limited training to apply pedagogical skills and strategies in controlled settings under the guidance of a mentor (Berliner, 1985; Zeichner, 2010). However, recent research examining the impact of differing field-based experiences has been characterized as sparse at best (Shanahan, 2008). Although generally recognizing the paradigm that more is better, there are multiple considerations that must be examined to maximize the benefits associated with field experiences, including duration, context, opportunities for practice, and guidance (Gurvitch & Metzler, 2009; Hammerness, Darling-Hammond, & Bransford, 2005; Putman & Borko, 2000).

Acknowledging that field experiences represent an important vehicle for developing practical knowledge and impacting teaching behaviors, another very influential variable on practices is teaching candidates’ beliefs about their abilities (Woolfolk Hoy & Spero, 2005). Research has implicated teacher preparation as an important mechanism for understanding the impact of beliefs on practices and behaviors (Hammerness et al., 2005; Pajares, 1992). However, as a result of the complex, multifaceted nature of the construct and

Correspondence concerning this article should be addressed to S. Michael Putman, Charlotte, 9201 University City Boulevard, Charlotte, NC 28223 E-Mail: [email protected]

Researchers have recommended that teacher education candidates receive multiple opportunities for systematic implementation of practices within authentic contexts. These experiences form an important foundation for candidates’ beliefs about their abilities in classrooms. Yet, few researchers have examined the relationship between the context for delivery of the experiences and the development of teaching efficacy. This study involved investigation of the impact of variations in 2 teacher education courses along several dimensions of teaching efficacy, consisting of 43 teaching candidates enrolled in an undergraduate elementary education program, who were selected via convenience sampling. These candidates comprised two groups: (a) those enrolled in the blocked semester (n = 18) that involved both a pedagogical foundations course (i.e., included objectives focused on developing and extending knowledge of instructional planning and investigating theories and principles of effective classroom management) and a course that extended the content of the pedagogical foundations course within a focused practicum experience; and (b) those enrolled in a traditional course delivery format (n = 25), wherein teaching candidates enrolled in each course separately and completed the requirements in separate semesters. Participants in both groups were administered an instrument that assessed efficacy towards the beginning of the course and during the last class of the practicum course. Findings indicated that participation in an intensive, field-based placement (i.e., blocked group) yielded growth in several related areas of teaching efficacy. Implications are discussed as related considerations for teacher education programs as they seek to structure field experiences to maximize growth in candidates’

S. MICHAEL PUTMAN


variations in contexts where teaching experiences occur, previous investigations examining the relationship between efficacy and experiences have not resulted in definitive conclusions (Knoblauch & Woolfolk Hoy, 2008; Woolfolk Hoy & Spero, 2005). Researchers have called for additional studies that investigate features of pathways to teaching that are powerful for preparing teachers and demonstrate a related effect on efficacy for teaching (e.g. Grossman & McDonald, 2008; Zeichner & Conklin, 2005). This research was undertaken to answer such calls. The following research question was used to frame the investigation: What is the impact of variations in programmatic delivery on teaching candidates’ general and domain-specific sense of teaching efficacy?

Review of Literature

Teacher Preparation Preparing teaching candidates to enter the complex and challenging environment of today’s schools is a daunting task. Programs, whether traditional or alternative, must attend to multiple elements, including where, when, and how teacher preparation is structured, to ensure candidates are prepared for the rigors of the classroom (Zeichner, 2010). Regardless of the challenges inherent with the preparation of future teachers, however, the current accountability movement in the United States has resulted in a focus on the influence and impact of teacher quality on student achievement and educational outcomes (Hanushek & Rivkin, 2012; Henry, Kershaw, Zulli, & Smith, 2012). As a result, university-based programs are under increased scrutiny and criticism has mounted regarding an apparent lack of evidence of their effectiveness (Hanushek & Rivkin, 2012).

Although such debates regarding teacher preparation are highly politicized and charged, teacher education programs themselves might have contributed to the current state of affairs. Traditionally, teacher preparation programs have been characterized as focusing on preparation of candidates within a theoretical paradigm as opposed to facilitating development of skills through opportunities to practice them with authentic classroom contexts (Ball, Sleep, Boerst, & Bass, 2009). The emphasis behind this approach is to develop a sound understanding of pedagogy in combination with content knowledge prior to the application of these elements within a school setting (Korthagen & Kessels, 1999).

Programs operating within this paradigm have provided teaching candidates multiple opportunities to engage in “simulated planning” (Grossman & McDonald, 2008, p. 190) and reflection, but fewer opportunities to practice teaching in authentic settings under normal

teaching conditions (Grossman & McDonald, 2008). This likely contributes to a lack of connection between information presented in university coursework and practices observed within field experiences. Numerous researchers have cited the theory-practice gap discussed by candidates in teacher education programs (Allsopp, DeMarie, Alvarez-McHatton, & Doone, 2006; Korthagen, Loughran, & Russell, 2006). As Grossman, Hammerness, and McDonald (2009) point out,

Principles developed in the absence of assisted practice lack the depth required … to enact such principles in practice. At the same time, learning to enact instructional routines in the absence of a developing sense of the principles underlying such routines reinforces a view of teaching as a set of techniques. (p. 278)

When theory-practice connections are lacking or when there is dissonance between theory presented in coursework and actual practice observed in the classroom, teaching candidates are more likely to adopt the practices they observe in the classroom, regardless of whether it conflicts with information presented in teacher education courses (see Moore, 2003; Putman, 2009).

Perhaps exacerbating the problems associated with the lack of coherence between teacher education and the experiences of teaching candidates is the fact that there is often too little communication between classroom supervisors and university faculty. It is not uncommon for cooperating teachers to have limited information regarding the content presented within the campus-based courses and university instructors often know very little about the practices taking place in the P-12 classroom where candidates are placed (Darling-Hammond, 2010). As a result, opportunities to implement practices described within coursework vary greatly, even within the context of partnerships established with professional development schools (Moore, 2003; Zeichner, 2007).

To create effective teachers, research suggested that the learning process of teaching candidates should include multiple opportunities that engage teaching candidates as learners and build from their prior knowledge and understandings within the environments that they will teach (Borko & Putman, 1996; Hammerness et al., 2005). As described by Schön (1983), the emphasis should be placed on “theory-in-action” (p. 50). Candidates should engage in "work that is done as part of action or practice, like that done in the actual riding of a bicycle or the actual making of a medical diagnosis” (Cook & Brown, 1999, p. 387). This can be accomplished through carefully structuring and scaffolding candidate learning in sustained and intense field experiences that occur within programs.

EXAMINING VARIATIONS IN PROGRAMMATIC DELIVERY ON TEACHING CANDIDATES’ SENSE OF EFFICACY


Field Experiences Field experiences are “school-based

opportunities in which candidates may observe, assist, tutor, instruct, participate in service learning projects, or conduct applied research” (National Council for Accreditation of Teacher Education, 2008, p. 32). In short, they represent contexts where teaching candidates have the opportunity to explore the relationship between theory and action. The importance of the field experience within the development of teacher proficiency has been emphasized for more than a century. John Dewey (1904/1964) advanced this notion as he advocated for the use of learning to teach through extensive practice in real classrooms. Dewey believed that this facilitated opportunities for experimentation with methods and practices, similar to the apprenticeship models that proliferated in many industries during the same time period.

Contemporary educational theorists still support the importance of practical experiences in the classroom to advance teaching candidates’ knowledge, skills, and dispositions as they offer what Grossman and McDonald (2008) referred to as “approximations of practice” (p. 190). These approximations are the result of being able to develop teaching strategies in settings of reduced complexity where candidates do not have to attend to all elements within a teaching situation. Field experiences have the greatest impact on the development of the skills and practices of candidates when they are explicitly linked to content presented in methods courses (Darling-Hammond, 2006). This has led some universities to combine coursework with extended internships or residencies that occur in structured school-based settings. Such activities allow teaching candidates to benefit from the guidance of university faculty and staff as well as school personnel (see Castle, Fox, & O’Hanlan Souder, 2006; Fraser, 2007). It also allows teacher educators to coordinate information provided within their methods courses to the specific practices and expertise of teachers in schools (Zeichner, 2010). The coordination of practices as well as the joint support and guidance for teaching candidates is crucial for an understanding of their development and impact.

However, teacher education has not reached a juncture where there is consistent synchronization between university coursework and practices demonstrated in the field. Reinforcing the traditional paradigm, theory may be introduced in coursework, but field experiences do not provide specific opportunities to enact the principles advanced within the theory presented as part of the coursework (Korthagen & Kessels, 1999). Instead, Montecinos et al. (2011) found that during the early phases of field experiences, candidates’ primary tasks were focused on “observing, gathering, and analyzing information” (p. 285) as opposed to

engaging in teaching practices that would advance their pedagogical skills. The current challenge is that research on methods, structures, and conditions for field experiences has not produced definitive conclusions for maximizing the links between theory and practice (see Cochran-Smith & Zeichner, 2005; Wilson, Floden, & Ferrini-Mundy, 2002). Allsopp et al. (2006) provided several suggestions to foster these connections, including increasing the involvement of the supervising teacher, creating course assignments that demonstrate greater applicability to the field experience, and modifying the way courses are scheduled. Additional approaches for connecting the theory taught within teacher education programs and field-based experiences include:

• Developmentally sequenced field experiences aligned with theory-based and practical teaching assignments (Capraro, Capraro, & Helfeldt, 2010);

• Clearly articulated purposes that are shared amongst all stakeholders, including classroom supervisors (Dodds, 1989); and

• Altering the quantity and duration of the field experiences to match the needs of teaching candidates (Burant & Kirby, 2002).

When teacher preparation programs focus on these critical variables, they can link them with experiences focused upon reflection to strengthen connections between personal experiences and classroom pedagogy. The result of such activities will be teaching candidates who are better prepared to engage in the complex behaviors necessary of effective teachers (see Darling-Hammond, Hammerness, Grossman, Rust, & Shulman, 2005; Zeichner & Conklin, 2005).

Teachers’ Sense of Efficacy

Although field experiences represent a significant factor in developing skills and pedagogy, teachers’ beliefs about their abilities represent another important influence on their classroom behaviors as they mediate the relationship between knowledge and action. Teaching efficacy—to which this construct often is referred—was derived from Bandura’s (1986) social cognitive theory (SCT). Definitions of teachers’ sense of efficacy vary; yet, most share the common characteristic of being focused upon a teacher’s belief in her or his ability to affect student learning and behavior. In their seminal work, Tschannen-Moran and Woolfolk Hoy (2001) stated that the construct represented a teacher’s “judgment of his or her capabilities to bring about desired outcomes of student engagement and learning” (p. 783).

Teachers who demonstrate high levels of efficacy for teaching are more likely to implement effective methods of instruction (Ashton & Webb,

S. MICHAEL PUTMAN


1986; Bandura, 1997), to persist during difficult teaching situations (Dembo & Gibson, 1985; Woolfolk, Rosoff, & Hoy, 1990), and to be successful at maintaining student engagement (Ashton & Webb, 1986). Efficacious teachers also are more likely to seek ways to improve their teaching methods through alternative methods of instruction and experiment with innovative instructional materials (Allinder, 1994; Ashton & Webb, 1986). All of these practices are associated with improvements in student achievement. On the other hand, a low sense of efficacy has been associated with non-differentiation of instruction, lack of interest in collaboration with other teachers, and negative views toward inclusion (Soodak, Podell, & Lehman, 1998).

Teaching efficacy is complex and multifaceted, especially in regard to preservice teaching candidates. Posnanski (2007) noted that teaching candidates’ efficacy is enhanced through participation in university courses that employed a constructivist paradigm and included both modeling and opportunities to apply knowledge in contextually relevant settings. Similarly, Clift and Brady (2005) observed a positive impact on efficacy when teaching candidates were provided a combination of observation, simulation, and opportunities to work with small groups of students. However, questions remain regarding the best method to combine these elements within a teacher education program to maximize the impact on teaching efficacy (Erawan, 2011). The lack of certainty in how effectively to merge the three elements likely is exacerbated by the fact that teaching candidates generally perceive themselves as being effective at implementing a variety of instructional practices and management strategies prior to field experiences (Rimm-Kaufman & Sawyer, 2004). In fact, given the malleability of the beliefs of preservice teachers, precise measurement of teaching efficacy might be difficult due to the subtle changes that occur as candidates complete other programmatic requirements, including field experiences and coursework prior to student teaching (see Chambers, 2003; Oh, Ankers, Llamas, & Tomjoy, 2005; Wagler & Moseley, 2005). Field Experiences and Teaching Efficacy

There is little doubt that teaching candidates need field experiences that offer safe opportunities to teach and to receive feedback while simultaneously being able to observe and to implement effective teaching strategies (Morrell & Carroll, 2003). From a social cognitive perspective (Bandura, 1997), mastery experiences, such as those that occur within practica and student teaching, provide the greatest impact on efficacy. Research has established that when teaching candidates faced and succeeded in challenging

situations within field experiences, stronger efficacy beliefs were developed (Gurvitch & Metzler, 2009). Accordingly, the earlier such experiences can be introduced in the teacher education program, the more durable teaching candidates’ sense of efficacy becomes as the beliefs are based on actual experiences (Woolfolk Hoy & Spero, 2005). When teaching candidates are not provided sufficient experiences linked to practices present in a real classroom, they are more likely to experience the previously described positive, yet unrealistic, sense of efficacy (Rimm-Kaufman & Sawyer, 2004). For example, receiving high scores on classroom management plans that are not implemented within a classroom might inflate views regarding successful implementation of strategies and teaching candidates might be easily dismayed in the context of a true teaching experience (see Putman, 2009).

The majority of research on the impact of field experiences on efficacy has occurred within student teaching experiences. Results have not been found to be definitive in regard to the specific impact of field experiences and might be dependent on the form of efficacy that is examined. For example, Hoy and Woolfolk (1990) indicated that general teaching efficacy improved as a result of participation in field experiences that occurred in conjunction with coursework, but declined during student teaching. However, in studies focused on personal teaching efficacy, which is more domain specific, researchers have noted improvements in efficacy within student teaching (Knoblauch & Woolfolk Hoy, 2008; Woolfolk Hoy & Spero, 2005). In fact, Knoblauch and Woolfolk Hoy (2008) found that teaching candidates’ efficacy increased regardless of the setting (e.g., rural, suburban, and urban) in which student teaching occurred. Still other researchers have found no differences or declines in efficacy in preservice candidates with regard to the completion of student teaching (Oh et al., 2005; Wagler & Moseley, 2005). Variations might be attributable to differences within preparation as candidates engage in a range of field experiences prior to student teaching (Hammerness et al., 2005). Some early field experiences offer a variety of support and feedback mechanisms, whereas student teaching immerses candidates in a complex atmosphere, which might not always match their prior expectations of the day-to-day realities of the classroom (Charalambous, Philippou, & Kyriakides, 2008; Zeichner & Conklin, 2005).

The aforementioned support might come in the form of assistance and guidance within structured feedback from classroom supervisors and university personnel. Tschannen-Moran and Woolfolk Hoy (2007) noted that when feedback and guidance were present, they contributed significantly to efficacy development. Zeichner



(1992) found that undergraduate teaching candidates became more confident and successful when provided with adequate feedback within field experiences. Improvements in candidates’ sense of efficacy also can occur within the context of self-reflective activities. Graham (2006) determined that when candidates were given multiple opportunities to reflect upon and to engage in dialogue about their practices and the overall experiences of teaching and observing within field experiences, it had a significant influence on efficacy. In short, candidates need to believe that they are guided and supported by mentors, such as the classroom supervisor and university faculty member. On the other hand, when this support is withdrawn, efficacy levels are likely to drop (Tschannen-Moran, Woolfolk Hoy, & Hoy, 1998).

Recent research findings have expanded the knowledge-base regarding field experiences as well as teaching efficacy; yet, examinations of preservice candidates’ efficacy for teaching have yielded inconsistencies and raised multiple questions about the development of beliefs as candidates participate in various field experiences (see Chambers & Hardy, 2005; Knoblauch & Woolfolk Hoy, 2008; Oh et al. 2005; Woolfolk Hoy & Spero, 2005). As a result, it has been proposed that additional investigations of methods to strengthen and to develop efficacy in teaching candidates are necessary as programs seek to support candidates in their development (Henson, 2003). This research was undertaken to answer such calls; yet, unlike many of the other investigations wherein researchers have attempted to develop conclusions relative to field experiences, in this investigation, it was sought to examine the impact of variations in programmatic delivery, including coursework and field experiences, that occurred prior to student teaching on teaching efficacy. It was hoped that such an examination would provide researchers and developers of teacher education programs insights on how efficacy changes over time within a supportive environment that maximized connections between theory and practice, as well as on potential methods to alleviate the malleability of efficacy beliefs shown within previous investigations (see Pajares, 1992; Woolfolk Hoy & Spero, 2005).

Method

Context

The context for this research was a 4-year, undergraduate elementary education program in a Midwestern public university with a total enrollment of approximately 20,000 students. The research was conducted within a required two-course sequence typically completed in Year 3 of the program, prior to eligibility for student teaching. The first class in this sequence, which

will be referred to as the pedagogical foundations course, includes objectives focused on developing and extending knowledge of instructional planning and investigating theories and principles of effective classroom management, as applied to instructional contexts. As part of the course requirements, candidates develop two lesson plans central to a topic or theme that require the inclusion of effective instructional strategies identified by Marzano, Pickering, and Pollock (2004). Candidates also create a classroom management plan that introduces their ideas regarding the development of rules, procedures, and classroom organization, citing relevant theories of experts within the field. The pedagogical foundations course does not include a field experience; thus, preservice candidates are not provided with the opportunity to teach their lessons, nor do they implement any aspects of the classroom management plan as part of typical delivery format for the course. Instead, candidates are required to complete 10 hours of observation within an elementary classroom of their choosing. Upon completion of the observations, candidates are required to write a reflective paper that summarizes the experiences, citing relevant course content and documenting their growth as future educators.

The second course in the sequence extends the content of the pedagogical foundations course within a focused practicum experience. The objectives for candidates enrolled in the practicum experience include: demonstrating proficiency in planning and instructional practice, investigating the various perspectives on student achievement and the factors that affect it, and applying theories and strategies associated with classroom management strategies to teaching situations. Candidates experience 4 to 6 hours of observation, conduct 1 to 6 hours of whole class instruction, and conduct 24 to 28 hours of small group instruction within the practicum under the guidance of the university instructor and cooperating teacher at a field site. Lessons for the classroom instruction component are drawn from a 10-lesson unit plan developed by the candidates that focuses upon a specific topic as dictated by the curricular topics of the cooperating teaching. Feedback is provided to the candidate on lesson plans as well as teaching experiences by the university instructor in collaboration with the cooperating teacher.

This investigation focused on two variations of the delivery of the aforementioned classes. The first variation represented an intense blocked semester wherein candidates enrolled in both of the previously described courses simultaneously. All content and programmatic requirements for the pedagogical foundations course as well as the focused practicum experience were included and assigned within what will henceforth be referred to as the blocked semester or blocked experience. All

S. MICHAEL PUTMAN


activities for the blocked semester, including seminars, occurred in a local elementary school. In the second variation, which will be referred to as the traditional format or traditional experience, candidates enrolled in each course separately and completed the requirements described previously in separate semesters.

Participants

Non-probability, convenience sampling (Cohen, Manion, & Morrison, 2000; Onwuegbuzie & Collins, 2007) was used to select 43 participants enrolled in an undergraduate elementary education program. All participants had been accepted into the teaching curriculum associated with the major and had achieved junior-level status and, thus, were enrolled in the pedagogical foundations course, which was previously described. The participants comprised two groups: those enrolled in the blocked semester and those participating in the traditional course delivery format. Self-selection sampling was used to identify participants within the blocked semester because each teaching candidate had chosen to enroll in this method of delivery for the two courses. Eighteen candidates were enrolled in the blocked semester, 16 females and two males. All participants were White and the sample had a mean age of 20.17 years (SD = 0.38). Random purposeful sampling (see Onwuegbuzie & Collins, 2007) was used to select the participants from the traditional course delivery format. A list of all candidates (n = 81) enrolled in the traditional sections of the pedagogical foundations course who granted consent to participate in the study was compiled. Every third name from the list was selected until 25 names were generated. The selection process resulted in 22 females and three males. The participants’ self-identified race-ethnicity was White (92%) and African-American (8%). The mean age of the participants in the traditional format was 21.28 years (SD = 3.47).

Instrument

Tschannen-Moran and Woolfolk Hoy’s (2001) Teachers’ Sense of Efficacy Scale (TSES) was the instrument utilized for the assessment of efficacy of all participants. The TSES was selected due to its recognized acceptance within the field and its demonstrated use with preservice teachers (see O’Neill & Stephenson, 2012; Putman, 2012; Tschannen-Moran & Woolfolk Hoy, 2001). There are two versions of the TSES—a long form, which consists of 24 items, and a short form, which comprises 12 items from the long form. For this research, the short form was selected because the author and instructors collaboratively determined that the items in this form were more aligned than were the items in the long form, with participants’ previous experiences in the program as well as with the content of the courses included within the

research. The TSES also was designed to include domain-specific subscales that measure three related constructs of efficacy. Each subscale consisted of four items relating to one of the following three constructs: (a) student engagement (e.g., How much can you do to motivate students who show low interest in school work?), (b) instructional strategies (e.g., To what extent can you craft good questions for your students?), and (c) classroom management (e.g., How much can you do to control disruptive behavior in the classroom?). Calculation of the teachers’ sense of efficacy score as well as subscale scores is based upon the sum of most positive responses of the items, which are written along a 9-point continuum that ranges from 1 (nothing) to 9 (a great deal), with additional anchors at 3 (very little), 5 (some influence), and 7 (quite a bit).

The short form of the instrument has been shown to yield a high overall score reliability (α = .90), as has each sub-scale: α = .86 for student engagement, α = .81 for instructional strategies, and α = .86 for management (see Tschannen-Moran & Woolfolk Hoy, 2001). Reliability measurements for the current data were .91, .80, .77, and .88 for the overall score and each subscale, respectively. The construct-related validity of TSES was measured through its correlation with other existing scales of teaching efficacy and suggested that the TSES effectively assesses the construct of efficacy (see Tschannen-Moran & Woolfolk Hoy, 2001).

Procedure

The research design represented a quasi-experimental design, namely, a nonequivalent control group design. Prior to beginning the activities associated with this research, the author met with the two instructors, referred to as Instructor #1 and Instructor #2, who were scheduled to teach the sections of the pedagogical foundations and focused practicum courses offered within the traditional format for the 2009-2010 academic year. Instructor #1, a tenure-track assistant professor and former elementary school teacher, had previous experience teaching each of the classes and was scheduled to teach three sections of the foundations course during the fall semester when the research was scheduled to begin. Instructor #2, an adjunct faculty member and former elementary school teacher and principal, also had taught both classes multiple times. She was scheduled to teach the courses associated with the blocked experience as well as one section of the foundations course and one section of the focused practicum experience during the fall semester. Instructor #2 had previously agreed to participate in the research based on her teaching assignment within the blocked experience and both instructors agreed to allow the author to recruit students from



their respective classes within the traditional program.

Recruitment activities consisted of the author visiting all sections of the pedagogical foundations course and presenting information about the research during the first class of the semester. Students were provided with consent forms to complete immediately or to return by the second class, during which the initial administration of the survey was to be completed. All students enrolled in the blocked experience granted consent to participate. Within the foundations courses associated with the traditional program, 84% (n = 81) granted consent to participate.

All participants who granted consent, regardless of participation in the blocked or traditional experience, completed the TSES by the beginning of the third class of the fall semester in which they were enrolled in the pedagogical foundations course. The instrument was administered electronically, with the instructors of the respective courses reading the directions for administration provided within the instrument. To ensure the alignment of course content and delivery for the two experiences, the two instructors engaged in collaborative planning, using the department’s master syllabi for each of the two courses to guide their discussions. The focused practicum courses for both instructors occurred in the same elementary school, which is a professional development school for the university, and followed established, departmental guidelines for teaching and observing. The cooperating teachers had multiple years of experience supervising and working with preservice teachers within the context of both courses. Using the same process for delivery, all participants in the traditional experience completed the TSES a second time, referred to as the middle administration, at the beginning of the spring semester when they enrolled in the focused practicum experience. Instructor #1 and Instructor #2 taught all sections of the focused practicum experience and, thus, administered the instrument using the provided directions.

Within the blocked experience, Instructor #1 was the instructor of record for both the pedagogical foundations and practicum courses. To ensure continuity and coherence between coursework and the field experience, the instructor met multiple times with the cooperating teachers, all of whom had extensive experience working with preservice teachers within the context of a professional development school associated with the university, prior to the start of the course. The instructor and teachers jointly established a weekly observation and instruction schedule that facilitated the development of topics for seminars on various aspects of classroom management and instructional theory. Also, they developed a schedule to provide

candidates with opportunities to meet the departmental teaching and observation requirements for the practicum experience. There was a deliberate effort made by the instructor and cooperating teachers to connect academic and practitioner knowledge in support of student teachers’ learning how to enact specific teaching practices. To ensure that multiple opportunities for teaching and observing at different grade levels were present, teaching candidates equally split the semester into one primary and one upper elementary classroom. Of note, participants in the blocked experience did not complete the middle administration due to the shortened time frame to note changes in scores.

The final administration of the TSES was completed by all participants again during the last class of the practicum course. As with previous administrations, the instrument was administered electronically with directions provided by Instructor #1 or Instructor #2. For those in the blocked group, this represented the same semester as the first administration. It is important to note that for the traditional group, the final administration occurred the next semester after the first administration due to the extended time frame for the completion of both courses. To ensure parsimony and coherence within the analysis, only the administrations of the TSES that occurred at the beginning of the foundations course and end of the focused practicum course were used for the statistical analyses associated with this research.

Data Analysis

Scores for total efficacy and for each of domain-specific areas of efficacy (student engagement, instructional strategies, and classroom management) were computed for each participant and for each test administration. Descriptive statistics, including means and standard deviations, were calculated for the total efficacy scores as well as for each of the subscale scores for each administration of the surveys in each contextual variation for delivery (blocked, traditional). To ensure equality between the groups, an independent samples t test was utilized to compare scores on the first administration of the TSES. Accordingly, a Bonferroni adjustment was utilized to diminish the risk of a Type I error, yielding a statistical significance level of .0125. The same procedure was used within the statistical analysis to measure differences in the means for the final administration of the TSES. A paired samples t test with a Bonferroni adjustment then was performed to examine changes from initial administration within the pedagogical foundations course to the final administration at the end of the practicum course for each of the experiences. Finally, effect sizes were calculated using Cohen’s d to note the magnitude of the effects on efficacy scores.

S. MICHAEL PUTMAN


Results Descriptive statistics for all variables for the

first and final administrations of the TSES are shown in Table 1. Figures 1-4 graphically depict changes from the first administration of the TSES to the final administration Within the figures, the middle administration of the TSES completed by participants in the traditional experience also is reflected to provide additional insights into the changes in efficacy that were experienced by this group. Figure 1 illustrates that both the blocked and traditional groups experienced growth on the overall scores of the TSES between the first and final administration of the TSES. However, although the rate of growth appears initially to be very similar for the two groups, this was not maintained while the traditional group was enrolled in the practicum course, as indicated by the increasing gap between the two groups’ scores. A similar pattern was shown in Figure 2 with respect to overall change between the first and final administration of the instrument; but, in this case, the sense of efficacy for student engagement for the traditional group demonstrated a downward trend during their practica, thereby increasing the differences noted between the two groups’ scores.

As shown in Figure 3, the traditional group began the two-course sequence with a more efficacious view of their capabilities to implement instructional strategies. During the two-course sequence, the group’s scores increased slightly between each administration of the TSES, but the overall growth in the scores remained relatively flat. On the other hand, the blocked group demonstrated an increased sense of efficacy on this subscale, surpassing the traditional group and demonstrating what turned out to be a statistically significant change from the first to final administration of the TSES (see Table 1). Figure 4 depicts scores on the efficacy for classroom management subscale. In this case, each group displayed the largest change in scores from first administration to final administration among the three subscales contained in the instrument. Similar to what was observed in Figures 1 and 2, the slope of the growth was nearly parallel initially and the blocked group started and finished with a higher sense of efficacy for the construct. However, unlike the outcomes depicted in those figures, the scores of the traditional group demonstrated only a slight deterioration during the practicum.

To examine the potential effects of self-selection sampling bias associated with participation in the blocked experience, an independent samples t test was computed comparing the scores of each group on the first administration of the TSES on the primary scale as well as each of the subscales. To minimize the likelihood of a Type I error associated with

multiple tests, a Bonferroni adjustment was completed and a statistical significance level of .0125 (.05/4) was used in evaluating the results of each test. Although the participants in the blocked semester showed slightly higher efficacy at the time of the first administration of TSES in three of the four areas measured, no statistically significant differences were noted between the groups for the overall score on the TSES (t = 0.37, p > .05), student engagement (t = 1.43, p > .05), instructional strategies (t = -1.501, p > .05), and classroom management (t = 0.60, p > .05). Thus, additional comparisons were deemed acceptable to compare the effects of participation within the two experiences and the risk of self-selection sampling bias was deemed to be minimal.

Results of the t tests with the Bonferroni adjustments to compare scores on the first and final administration of the TSES and each subscale also are shown in Table 1. The participants in the blocked experience demonstrated statistically significant growth for the overall score as well as for each subscale, respectively, with the greatest difference in classroom management. Effect sizes also were large, using Cohen’s (1988) criteria. The largest effect size was noted for efficacy for instructional strategies. Examining scores for the participants in the traditional format revealed the largest gain within the subscale for classroom management, which was statistically significant. The participant’s total TSES score increased as did the scores pertaining to the remaining subscales; however, growth did not reach statistical significance. Employing Cohen’s (1988) criteria, the effect sizes were noted to be small for instructional strategies, moderate for the total TSES score and student engagement, and high for classroom management.

The independent samples t test with a Bonferroni adjustment to compare the means of the blocked group to those of the traditional group on the final administration of the TSES indicated statistically significant differences favoring the blocked group for general teaching efficacy, effective strategies, instructional strategies, and classroom management (see Table 2). To facilitate further examinations of these differences, the effect sizes were calculated for each variable using Cohen’s (1988) d criteria. Using these criteria, effects were noted to be large for the general scale as well as each for subscale (see Table 2).



Figure 1. Graphic representation of the mean TSES scores of the blocked and traditional groups for each administration of the instrument.

Figure 2. Graphic representation of the mean scores on the TSES student engagement subscale for the blocked and traditional groups for each administration of the instrument.

Figure 3. Graphic representation of the mean scores on the TSES instructional strategy subscale for the blocked and traditional groups for each administration of the instrument.

Figure 4. Graphic representation of the mean scores on the TSES classroom management subscale for the blocked and traditional groups for each administration of the instrument.

S. MICHAEL PUTMAN


Table 1 Descriptive Statistics and Paired Samples t Tests

1st administration Final administration

TSES M SD M SD Cohen’s

d ta

Blocked (n = 18)

Total 67.06 8.32 83.63 8.83 1.93 5.46**

SE 25.56 4.08 29.81 3.76 1.08 3.06** IS 17.50 2.73 23.25 2.35 2.26 6.38** CM 24.00 3.83 30.56 3.75 1.73 4.89**

Traditional (n = 25)

Total 65.92 10.48 72.44 8.13 0.68 2.46

SE 23.80 3.72 25.64 4.05 0.46 1.67 IS 19.00 3.34 19.68 2.34 0.23 0.83 CM 23.12 4.94 27.12 3.10 0.95 3.43**

Note. SE = Student Engagement; IS = Instructional Strategies; CM = Classroom Management aBlocked, df = 35; Traditional, df = 49 **p < .01 Table 2 Comparison Final Administration Scores of Traditional and Blocked Groups

Traditional (n = 25)

Blocked (n = 18)

TSES M SD

M SD Mean Score Differences

t (df = 41)

Cohen’s d

Total Score 72.44 8.13 83.63 8.83 11.19 4.15** 1.12 SE 25.64 4.05 29.81 3.76 4.17 3.31** 0.95 IS 19.68 2.34 23.25 2.35 3.57 4.76** 1.23 CM 27.12 3.10 30.56 3.75 3.44 3.19** 0.92 Note. SE = Student Engagement; IS = Instructional Strategies; CM = Classroom Management **p < .01

Discussion

Acknowledging efficacy for teaching as a potentially important factor in helping prospective teachers improve their teaching and impact student learning, researchers in teacher education have recommended that candidates receive multiple opportunities for systematic experimentation with teaching practices in authentic contexts (Grossman & McDonald, 2008). Early teaching experiences within a preparation program have the potential for long-term implications. It is, therefore, important that we examine the impact of programs that include field experiences on teachers’ beliefs and knowledge, as

well as examine structures within field experiences that maximize their influence (Sleeter, 2005).

This investigation demonstrated that efficacy was positively impacted through participation in both the traditional and blocked experiences as demonstrated by the overall increases in the means of each group for the total score on the TSES as well as those for each domain-specific area. These findings were consistent with those of Posnanski (2007) and Redmon (2007) and reinforced several tenets of Bandura’s (1997) theories inherent within the development of general efficacy. In essence, all candidates were provided with opportunities to plan lessons as well as to observe and to teach in



classrooms; thus, mastery and vicarious experiences likely contributed to increases in efficacy scores for candidates, regardless of the specific experience. However, it is notable that differential rates of growth were observed between candidates that showed no statistically significant differences at the outset of the research. This will be explored further with probable explanations linked to specific characteristics attributable to the design and implementation of blocked experience.

Although the middle administration of the TSES was not utilized in statistical analyses, it is important to engage in a visual examination of Figures 1-4, which include this administration. With the exception of the efficacy relative to the implementation on instructional strategies (i.e., Figure 3), the participants in the traditional format initially experienced a rate of growth in efficacy similar to those in the blocked group while completing the foundations course. This increase in efficacy occurred within a semester that was theory driven and lacked any field experience, although the candidates did engage in 10 hours of observation. Clift and Brady (2005) found that combinations of observations and simulations, which also were utilized to develop strategies for classroom management in the course, had a positive impact on efficacy. Thus, this might account for some of the initial growth comparable to the participants in the blocked semester. It could be surmised that the growth also might represent an instance of inflated views of personal capabilities, which has been associated with preservice teaching candidates (see Rimm-Kaufman & Sawyer, 2004). The teaching candidates might have identified with the teacher or university instructor during observations of instruction, yet lacked an adequate measure of their own capabilities because there was no actual teaching experience. As a result, when these candidates entered classrooms during the field experience course, the rate of growth of their efficacy levels became flatter or, in the case of student engagement, decreased, thereby confirming that candidates need a combination of opportunities to see and to work with children when engaged in completing coursework within their teacher education programs (Clift & Brady, 2005).

On the other hand, the blocked experience incorporated several principles that have been shown to have a positive impact on teaching efficacy. The most significant difference between this experience and the traditional one is represented by the incorporation of the pedagogical foundations course within a field experience and the use of the latter as the foundation from which content could emanate. Structuring the experience within this paradigm facilitated the planning of the format and schedule for the blocked semester as a collaborative effort between the university faculty member and classroom teachers who would supervise the teaching

candidates. There was a deliberate effort to connect academic and practitioner knowledge in support of candidates learning how to enact specific teaching practices advocated within coursework. The latter facet is important because the faculty member could trust the competent and skillful teachers to demonstrate and to model positive attitudes and effective instructional behaviors in classrooms. It ensured also candidates were able to benefit from mutual guidance from the university faculty member and cooperating teachers (see Castle et al., 2006).

Within the blocked experience, candidates received multiple opportunities to observe and to implement instructional and management-related strategies described within coursework immediately in an authentic context because of the coherence in planning the blocked experience. The capacity to make connections between theory and practice was immediate, specific, and tied to events that occurred within the context of the classroom (Allsopp et al., 2006; Darling-Hammond, 2006). Essential content discussed one afternoon in class could be observed or implemented to meet the needs of specific students the next morning. These experiences then could be deconstructed within the context of the seminar in the blocked experience. The demonstration of a skill or practice in conjunction with discussions as part of university coursework has been shown to be important in such situations because the modeling likely helped the teaching candidate not only understand the pedagogical technique, but also understand her/his own practices in relation to the demonstration (Bandura, 1997; Carlson & Gooden, 1999; Hodson & Hodson, 1998). In essence, candidates in the blocked experience were able to experience a constant state of connection and reference, both externally focused as they watched teachers and individually as they engaged in teaching practices, maximizing their growth in each area of efficacy. The traditional experience, on the other hand, was extended over two or more semesters. As a result, information introduced as part of the pedagogical foundations course might not have been observed or utilized until a period of time had elapsed, limiting candidates’ ability to connect practice and theory. One participant from the traditional experience who had only completed the foundations course noted, “While I am starting to understand some techniques for classroom management, I will need a lot more practice utilizing those techniques for me to become confident in my ability to effectively use them” (personal communication, January 23, 2010).

As students in the blocked experience engaged in lesson planning, course assignments were directly applicable to teaching practices within the context (see Allsopp et al., 2006). Students wrote plans that incorporated specific instructional strategies that they knew would be taught to various groups of students with whom they were familiar and, in many cases,

S. MICHAEL PUTMAN


had developed personal knowledge through day-to-day interactions. These formally planned teaching tasks were integrated into the learning experiences of the candidates and feedback on performance and remediation was immediately provided by the cooperating teacher and/or university instructor. Examining the overall impact of these practices with regard to instructional strategies (see Figure 3), the slope of the growth for this area of efficacy is much steeper for the participants in the blocked group than for those in the traditional experience. Immersion in the blocked experience facilitated continuous growth because students could make connections between the alignment of their instruction completed in the classroom and skills taught as part of coursework (see Allsopp et al., 2006; Capraro et al., 2010).

The feedback provided by the instructor or cooperating teacher also represented an important component in field experiences as Bandura (1997) suggested that it can modify individuals’ beliefs about their abilities (i.e., self-efficacy). Within the blocked experiences, the immediacy and specificity of feedback was a likely cause for the significant improvements demonstrated by the candidates. Verbal persuasion could occur directly in the context of the teaching experience or shortly following in conjunction with daily seminars. Noting the impact of assistance and structured feedback from supervisors and faculty in general (see Tschannen-Moran & Woolfolk Hoy, 2007), the ability to relate the feedback directly to experiences and course content while fresh in the candidates’ memory added direct relevance due to the distinct frame of reference. The daily seminars also offered the opportunity for candidates to debrief in a setting that included other teacher candidates. Teaching candidates could connect common problems or issues related to implementation of course principles amongst the group, facilitating the realization that any problems might not be individual specific, but instead were shared. In essence, the candidates gained greater proficiency in recognizing their own instructional capabilities and teaching competence due to the daily shared experiences and feedback of the group.

The latter notion of group experiences is significant because Tschannen-Moran and Woolfolk Hoy (2007) found verbal persuasion in the form of collegial and community support to be impactful on self-efficacy beliefs. Although collective efficacy was not measured as part of this research, Instructor #2 (personal communication, February 2, 2010) described a sense of shared purpose that developed amongst participants. These candidate-to-candidate relationships, coupled with sustained instructor-to-candidate and cooperating teacher-to-candidate relationships, supported the development of a strong professional community. Researchers have noted that such communities enhance efficacy through the rapport that develops amongst members, including

the candidates, classroom supervisors, and faculty members (Bandura, 1997; Clifford & Green, 2004; Tschannen-Moran, Woolfolk Hoy, & Hoy, 1998). Perhaps inhibiting this relationship for the traditional group was the extended time period between experiences as well as the traditional 3-day meeting schedule for coursework. In essence, students were unable to establish these relationships because class composition might have consisted of other students with whom they had little familiarity.

Limitations

This research study has several limitations that must be acknowledged. A small sample size and purposive sampling might have influenced the findings. In particular, participants from the blocked experience might not be representative of the general population of teaching candidates enrolled in the program due to their decision to participate in the program. Further, no methods, beyond comparisons of initial TSES scores, were employed to verify or to ensure through data collection that participants were similar. Additionally, Tschannen-Moran and Woolfolk Hoy (2001) noted that efficacy might vary across contexts, disciplines, and student populations. This would indicate an additional limitation for this study because no specific measures were in place to control the content areas in which the teaching candidates planned and taught lessons. For example, an individual who believed they were efficacious in mathematics might have planned lessons in this content area, which might have led to improvements in efficacy that were greater than if the student had been required to plan a lesson for social studies. Similar conclusions might be noted in relation to the grade levels in which the experiences occurred. Finally, although the author served as a course advisor for both Instructor #1 and Instructor #2, no external fidelity checks of instructional practices were conducted; thus, this must be recognized as a limitation.

Implications for Research

The findings of the current study can be used to inform examinations of teacher education programs as they seek to introduce prospective teachers to effective strategies and techniques and to reinforce these practices in such a way that they will be maintained during field and novice teaching experiences. Additional insights are presented that will address potential research activities for colleges of education to extend their capability to facilitate the development of a positive sense of efficacy within their candidates.

This research study did not include specific observations of practice, but teacher educators and researchers likely would gain valuable information by correlating changes in efficacy with actual



observations of practice. Increases in efficacy scores demonstrated by the candidates in this study showed perceptions of improvement, but observational data would have provided validity of the effectiveness of the blocked approach and further established the relationship between efficacy and practice. To facilitate this process, teacher preparation programs should consider correlating information gained through observation protocols among multiple measures to triangulate the impact of efficacy and to verify the relationship between efficacy and behaviors. Ideally, the same measures would be used throughout the program to ensure the ability to examine changes in both scores and practices longitudinally. The long-term impact of this is represented by improvements in candidates’ abilities more accurately to differentiate among different aspects of practices (i.e., implementing effective strategies or classroom management techniques), which Edwards, Higley, Zeruth, and Murphy (2007) noted was a skill that many candidates lacked.

Self-reflection on practices remains an important component within the development of personal efficacy when engaged in field experiences. One challenge associated with the intensity of the blocked semester described by Instructor #1 (personal communication, February 2, 2010) was that taking a methods course concurrently with a field experience limited candidates’ ability to engage in critical reflection outside of the seminar. Although connections could be made and discussed in the context, because the experience was so intensive due to the requirements of being in the classroom daily as well as engaging in lesson planning and maintaining a rigorous academic schedule of other university classes, candidates had difficulty reflecting on the big picture. To alleviate this, videos might serve as an important self-reflective tool that would help candidates feel less overwhelmed and create more time for reflection. Van Es and Sherin (2002) found that using video within instruction slowed down what is often perceived as a fast pace within the classroom and allowed preservice teachers to notice and to recall particular, specific aspects of the classroom that were not evident to them during teaching. Thus, the use of videos within field experiences might facilitate candidates’ ability to examine practices and to reflect upon their actions, creating more accurate beliefs about or perceptions of ability. It is recommended that teacher educators explore this form of technology for its potential to help preservice teachers study and learn from their experiences to improve their practices and beliefs (Rosean, Lundenberg, Cooper, Fritzen, & Terpstra, 2008; van Es & Sherin, 2002).

Finally, the daily seminars conducted in the blocked format offered the opportunity for candidates to debrief in a setting that included other teacher candidates. Teaching candidates could connect common problems or issues related to

implementation of course principles amongst the group, facilitating the realization that any problems might not be individual specific, but instead were shared. In essence, the candidates gained greater proficiency in recognizing their own instructional capabilities and teaching competence due to the daily shared experiences and feedback of the group. The latter notion of group experiences is significant because Tschannen-Moran and Woolfolk Hoy (2007) found collegial and community support to be impactful on efficacy. This investigation indicated a sense of shared purpose among participants in the blocked experience. Thus, additional research on candidate-to-candidate relationships, coupled with sustained instructor-to-candidate and cooperating teacher-to-candidate relationships, could help educators further understand the conditions that support the development of a strong professional community among preservice teaching candidates and its related impact on efficacy.

Experts continue to agree that the beliefs of the teacher impact her/his success in the classroom and that these beliefs have a long-term impact on effective teaching, classroom management and, subsequently, student learning (Woolfolk Hoy & Spero, 2005). As a result, educational stakeholders at all levels, but especially those in teacher education, must direct specific attention towards those variables that are likely to demonstrate the most significant impact on the formation of positive beliefs. This includes carefully structuring and scaffolding learning experiences that will enable teaching candidates to connect the theoretical knowledge gained with university classrooms to the practical knowledge enacted in the authentic classroom contexts. Doing so should create the effective teaching force that is sought as well as ensure that this teaching force remains in the profession and impacts the children they serve.

The lead editors for this article were Anthony J. Onwuegbuzie and John R. Slate.

References Allinder, R. M. (1994). The relationship between

efficacy and the instructional practices of special education teachers and consultants. Teacher Education and Special Education, 17, 86-95. doi:10.1177/088840649401700203

Allsopp, D. H., DeMarie, D., Alvarez-McHatton, P., & Doone, E. (2006). Bridging the gap between theory and practice: Connecting courses with field experiences. Teacher Education Quarterly, 33(1), 19-35. Retrieved from http://www.teqjournal.org/Back%20Issues/

S. MICHAEL PUTMAN


Volume%2033/VOL33%20PDFS/33_1/07allsoppetal-33_1.pdf

Ashton, P. T., & Webb, R. B. (1986). Making a difference: Teachers’ sense of efficacy and student achievement. New York, NY: Longman.

Ball, D. L., & Forzani, F. M. (2009). The work of teaching and the challenge for teacher education. Journal of Teacher Education, 60, 497- 511. doi:10.1177/0022487109348479

Ball, D. L., Sleep, L., Boerst, T., & Bass, H. (2009). Combining the development of practice and the practice of development in teacher education. Elementary School Journal, 109, 458-474. doi:10.1086/596996

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, CA: Prentice-Hall.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman and Company.

Berliner, D. (1985). Laboratory settings and the study of teacher education. Journal of Teacher Education, 36, 2-8. doi:10.1177/002248718503600601

Borko, H., & Putman, R. (1996). Learning to teach. In D. Berliner & R. Calfee (Eds.), Handbook of educational psychology (pp. 673-708). New York, NY: MacMillan.

Burant, T. J., & Kirby, D. (2002). Beyond classroom-based early field experiences: Understanding an “educative practicum” in an urban school and community. Teaching and Teacher Education 18, 561-575. doi:10.1016/S0742-051X(02)00016-1

Capraro, M. M., Capraro, R. M., & Helfeldt, J. (2010). Do differing types of field experiences make a difference in teacher candidates’ perceived level of confidence? Teacher Education Quarterly, 37, 131-154. Retrieved from http://aggiestem.tamu.edu/papers/articles/DrRCapraro/Do%20DifferingTypes.pdf

Carlson, R. D., & Gooden, J. S. (1999). Mentoring pre-service teachers for technology skills acquisition. In J. Price, J. Willis, D. A. Willis, M. Jost, & S. Boger-Mehall (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 1999 (pp. 1313-1318). Chesapeake, VA: AACE.

Castle, S., Fox, R. K., & O’Hanlan Souder, K. (2006). Do professional development schools (PDSs) make a difference? A comparative study of PDS and non-PDS teacher candidates. Journal of Teacher Education, 57, 65-80. doi:10.1177/0022487105284211

Chambers, S. M. (2003, February). The impact of length of student teaching on the self-efficacy and classroom orientation of preservice teachers. Paper presented at the Annual Meeting of the Southwest Educational Research Association, San Antonio, TX.

Chambers, S., & Hardy, J. (2005). Length of time in student teaching: Effects on classroom control orientation and self-efficacy beliefs. Educational Research Quarterly, 28(3), 3-9. Retrieved from http://www.eric.ed.gov/PDFS/EJ718127.pdf

Charalambous, C. Y., Philippou, G. N., & Kyriakides, L. (2008). Tracing the development of preservice teachers’ efficacy beliefs in teaching mathematics during fieldwork. Educational Studies in Mathematics, 67, 125-142.

Clifford, G., & Green, H. (2004). Teacher efficacy and beliefs. The Journal of Educational Research, 22(1), 30-41.

Clift, R. T., & Brady, P. (2005). Research on methods courses and field experiences. In M. Cochran-Smith & K. M. Zeichner (Eds.), Studying teacher education: The report of the AERA panel on research and teacher education (pp. 309–424). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.

Cochran-Smith, M., & Zeichner, K. (2005). Executive summary. In M. Cochran-Smith & K. Zeichner (Eds.), Studying teacher education: The report of the AERA panel on research and teacher education (pp. 1-37). New York, NY: Routledge.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Cohen, L., Manion, L., & Morrison, K. (2000). Research methods in education (5th ed.). New York, NY: Routledge.

Cook, J., & Brown, J. S. (1999). Bridging epistemologies: The generative dance between organizational knowledge and organizational knowing. Organization Science, 10, 381-400. doi:10.1287/orsc.10.4.381

Darling-Hammond, L. (2006). Powerful teacher education. San Francisco, CA: Jossey-Bass.

Darling-Hammond, L. (2010). Teacher education and the American future. Journal of Teacher Education, 61(1-2), 35-47. doi:10.1177/0022487109348024

Darling-Hammond, L., Hammerness, K., Grossman, P., Rust, F., & Shulman, L. (2005). The design of teacher education programs. In L. Darling-Hammond & J. Bransford (Eds.), Preparing teachers for a changing world (pp. 390-441). San Francisco, CA: Jossey Bass.



Dembo, M. H., & Gibson, S. (1985). Teachers’ sense of efficacy: An important factor in school improvement. The Elementary School Journal, 86, 173–184. doi:10.1086/461441

Dewey, J. (1904/1964). The relation of theory to practice in education. In R. D. Archambault (Ed.), John Dewey on education: Selected writings (pp. 314-388). Chicago, IL: University of Chicago Press.

Dodds, P. (1989). Trainees, field experiences, and socialization into teaching. In T. J. Templin & P. G. Schempp (Eds.), Socialization into physical education: Learning to teach (pp. 81-105). Indianapolis, IN: Benchmark.

Edwards, M. N., Higley, K., Zeruth, J. A., & Murphy, P. K. (2007). Pedagogical practices: Examining preservice teachers’ perceptions of their abilities. Instructional Science, 35, 443-465. doi:10.1007/s11251-006-9014-1

Erawan, P. (2011). A path analysis for factors affecting pre-service teachers’ teaching efficacy. American Journal of Scientific Research, 13, 47-58. Retrieved from http://www.eurojournals.com/AJSR_13_04.pdf

Fraser, J. W. (2007). Preparing America’s teachers: A history. New York, NY: Teachers College Press.

Graham, B. (2006). Conditions for successful field experiences: Perceptions of cooperating teachers. Teaching and Teacher Education, 22, 1118-1129. doi:10.1016/j.tate.2006.07.007

Grossman, P., Hammerness, K., & McDonald, M. (2009). Redefining teaching, re-imagining teacher education. Teachers and Teaching: Theory and Practice, 15, 273-289. doi:10.1080/13540600902875340

Grossman, P., & McDonald, M. (2008). Back to the future: Directions for research in teaching and teacher education. American Educational Research Journal, 45, 184-205. doi:10.3102/0002831207312906

Gurvitch, R., & Metzler, M. W. (2009). The effects of laboratory-based and field-based practicum experience on pre-service teachers’ self-efficacy. Teaching and Teacher Education, 25, 437–443. doi:10.1016/j.tate.2008.08.006

Hammerness, K., Darling-Hammond, L., & Bransford, J. (2005). How teachers learn and develop. In L. Darling-Hammond & J. Bransford (Eds.), Preparing teachers for a changing world (pp. 358-389). San Francisco, CA: Jossey-Bass.

Hanushek, E. A., & Rivkin, S. G. (2012). The distribution of teaching quality and implications for policy. Annual Review of Economics, 4, 131-157.

doi:10.11.46/annurev-economics-080511-111001

Henry, G. T., Kershaw, D. C., Zulli, R. A., & Smith, A. A. (2012). Incorporating teacher effectiveness into teacher preparation program evaluation. Journal of Teacher Education, 63, 335-355. doi:10.1177/0022487112454437

Henson, R. K. (2003). Relationships among preservice teachers’ self-efficacy, task analysis, and classroom management beliefs. Research in the Schools, 10(1), 53-62.

Hodson, D., & Hodson, J. (1998). Science education as enculturation: Some implications for practice. School Science Review, 80(290), 17-24.

Hoy, W. K., & Woolfolk, A. E. (1990). Socialization of student teachers. American Educational Research Journal, 27, 279–300. doi:10.3102/00028312027002279

Knoblauch, D., & Woolfolk Hoy, A. (2008). “Maybe I can teach those kids.” The influence of contextual factors on student teachers’ efficacy beliefs. Teaching and Teacher Education, 24, 166-179. doi:10.1016/j.tate.2007.05.005

Korthagen, F., & Kessels, J. (1999). Linking theory and practice: Changing the pedagogy of teacher education. Educational Researcher, 28, 4-17. doi:10.2307/1176444

Korthagen, F. A., Loughran, J. J., & Russell, T. (2006). Developing fundamental principles for teacher education programs and practices. Teaching and Teacher Education, 22, 1020-1041. doi:10.1016/j.tate.2006.04.022

Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2004). Classroom instruction that works: Research-based strategies for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum Development.

Montecinos, C., Walker, H., Rittershaussen, S., Nunez, C., Contreras, I., & Solis, M. C. (2011). Defining content for field-based coursework: Contrasting perspectives of secondary preservice teachers and their teacher preparation curricula. Teaching and Teacher Education, 27, 278-288. doi:10.1016/j.tate.2010.09.001

Moore, R. (2003). Reexamining the field experiences of preservice teachers. Journal of Teacher Education, 54, 31-42. doi:10.1177/0022487102238656

Morrell, P., & Carroll, J. (2003). An extended examination of preservice elementary teachers’ science teaching self-efficacy. School Science and Mathematics, 103, 246-

S. MICHAEL PUTMAN


252. doi:10.1111/j.1949-8594.2003.tb18205.x

National Council for Accreditation of Teacher Education. (2008). Professional standards for the accreditation of teacher preparation institutions. Washington, DC: Author.

Oh, D. M., Ankers, A. M., Llamas, J. M., & Tomjoy, C. (2005). Impact of pre-service student teaching experience on urban school teachers. Journal of Instructional Psychology, 32(1), 82-98. Retrieved from http://www.freepatentsonline.com/article/Journal-Instructional-Psychology/132048075.html

O’Neill, S., & Stephenson, J. (2012). Exploring Australian pre-service teachers sense of efficacy, its sources, and some possible influences. Teaching and Teacher Education, 28, 535-545. doi:10.1016/j.tate.2012.01.008

Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12, 281-316. Retrieved from http://www.eric.ed.gov/PDFS/EJ800183.pdf

Pajares, F. (1992). Teachers' beliefs and educational research: Cleaning up a messy construct. Review of Educational Research, 62, 307-332. doi:10.2307/1170741

Posnanski, T. J. (2007). A redesigned Geoscience content course’s impact on science teaching self-efficacy beliefs. Journal of Geoscience Education, 55, 152-157. Retrieved from http://nagt.org/files/nagt/jge/abstracts/posnanski-v55p152.pdf

Putman, R., & Borko, H. (2000). What do new views of knowledge and thinking have to say about research on teachers’ learning? Educational Researcher, 29, 4-15. doi:10.3102/0013189X029001004

Putman, S. M. (2009). Grappling with classroom management: The orientations of preservice teachers and impact of student teaching. The Teacher Educator, 44, 232-247. doi:10.1080/08878730903180226

Putman, S. M. (2012). Investigating teacher efficacy: Comparing preservice and inservice teachers with different levels of experience. Action in Teacher Education, 34, 26-40.

doi:10.1080/01626620.2012.642285 Redmon, J. R. (2007, October). Impact of teacher

preparation upon teacher self-efficacy. Paper presented at the Annual Meeting of the American Association for Teaching and Curriculum, Cleveland, Ohio. Retrieved from http://eric.ed.gov/PDFS/ED500013.pdf

Rimm-Kaufman, S. E., & Sawyer, B. E. (2004). Primary-grade teachers’ self-efficacy beliefs, attitudes toward teaching, and discipline and teaching practice priorities in relation to the responsive classroom approach. Elementary School Journal, 104, 321–341. doi:10.1086/499756

Rosean, C. L., Lundenberg, M., Cooper, M., Fritzen, A., & Terpstra, M. (2008). Noticing noticing: How does investigation of video records change how teachers reflect on their experiences? Journal of Teacher Education, 59, 347-360. doi:10.1177/0022487108322128

Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books.

Shanahan, C. H. (2008). Essential fieldwork for the preparation of teachers of reading for urban settings. In L. C. Wilkinson, L. M. Morrow, & V. Chou (Eds.), Improving literacy achievement in urban schools: Critical elements in teacher preparation (pp. 105-120). Newark, DE: International Reading Association.

Sleeter, C. E. (2005). Un-standardizing curriculum: Multicultural teaching in the standards-based classroom. New York, NY: Teachers College Press.

Soodak, L. C., Podell, D. M., & Lehman, L. R. (1998). Teacher, student and school attributes as predictors of teachers’ responses to inclusion. Journal of Special Education, 31, 480-497. doi:10.1177/002246699803100405

Tschannen-Moran, M., & Woolfolk Hoy, A. (2001). Teacher efficacy: Capturing and elusive construct. Teaching and Teacher Education, 17, 783-805. doi:10.1016/S0742-051X(01)00036-1

Tschannen-Moran, M., & Woolfolk Hoy, A. (2007). The differential antecedents of self-efficacy beliefs of novice and experience teachers. Teaching and Teacher Education, 23, 944-956. doi:10.1016/j.tate.2006.05.003

Tschannen-Moran, M., Woolfolk Hoy, A., & Hoy, W. K. (1998). Teacher efficacy: Its meaning and measure. Review of Educational Research, 68, 202-248. doi:10.2307/1170754

van Es, E. A., & Sherin, M. G. (2002). Learning to notice: Scaffolding new teachers’ interpretations of classroom interactions. Journal of Technology and Teacher Education, 10, 571-596. Retrieved from http://thorndike.tc.columbia.edu/~david/MTSU4083 /Readings/vanes.pdf

Wagler, R., & Moseley, C. (2005). Preservice teacher efficacy: Effects of a secondary education



methods course and student teaching. Teacher Education & Practice, 18, 442-457.

Wilson, A., Floden, R., & Ferrini-Mundy, J. (2002). Teacher preparation research: An insider's view from the outside. Journal of Teacher Education, 53, 190-204. doi:10.1177/0022487102053003002

Woolfolk, A. E., Rosoff, B., & Hoy, W. K. (1990). Teachers’ sense of efficacy and their beliefs about managing students. Teaching and Teacher Education, 6, 137-148 doi:10.1016/j.tate.2005.01.007

Woolfolk Hoy, A., & Spero, R. B. (2005). Changes in teacher efficacy during the early years of teaching: A comparison of four measures. Teaching and Teacher Education, 21, 343-356. doi:10.1016/j.tate.2005.01.007

Zeichner, K. (1992). Rethinking the practicum in the professional development school partnership. Journal of Teacher Education, 43, 296-307. doi:10.1177/0022487192043004009

Zeichner, K. (2007). Professional development schools in a culture of evidence and accountability. School-University Partnerships: The Journal of the National Association of Professional Development Schools, 1(1), 9-17.

Zeichner, K. (2010). Rethinking the connections between campus courses and field experiences in college- and university-based teacher education. Journal of Teacher Education, 61, 89-99. doi:10.1177/0022487109347671

Zeichner, K., & Conklin, H. (2005). Teacher education programs. In M. Cochran-Smith & K. Zeichner (Eds.), Studying teacher education: The report of the AERA panel on research and teacher education (pp. 645-735). New York, NY: Routledge.



Research Teaching Pedagogy: Lessons Learned From an Arabic Language Intervention Pilot Study

Thomas W. Christ University of Bridgeport

Hosam Elmetaher

Ubon Ratchathani University, Thailand

Teaching mixed methods and action research

to graduate students has great advantages but also poses interesting challenges. Many students who take research methodology courses at institutions such as the University of Hawaii are English Second Language learners with scholarships from a variety of sources including the Ford Foundation and the East-West research center. These students are under stringent timelines: 2 years for a masters or 4 years for a doctoral degree. Time becomes a significant constraint when conducting their research, and often they take their first research methodology course in the second year of the program after they have already collected data for their projects. Many students do not have a clear understanding of research processes or the importance of clearly articulating the purpose, research questions, methodology, and procedures (Christ, 2007, 2010; Creswell, 2010; Creswell & Plano Clark, 2010; Greene, 2007, 2008; Mertens,

2009; Teddlie & Tashakkori, 2009). Scholarship recipients at the master’s level must collect data during the first summer of their programs regardless of knowledge, or lack thereof, about research methodology. This often results in backward research planning (i.e., the process of collecting data before fully developing the methodology and procedures, including redesigning research questions, changing typology classification, modifying the methodology and methods; cf. Christ, 2009) out of necessity due to restrictions in available data and time constraints. Furthermore, many students indicate they are conducting mixed methods research without knowledge of this methodology or the training necessary to understand applicable procedures.

Students who take an introductory mixed methods research course in their second year after data are collected have fewer research design options from which to choose. Supporting these students becomes more complex because the methodology and data analysis procedures are restricted by the available data set. Backward research planning a thesis or pilot project clearly is not ideal but is commonly the challenge faced by

Correspondence concerning this article should be addressed to Thomas W. Christ, University of Bridgeport Carlson Hall, Room 116, 126 Park Ave Bridgeport, CT 06604. Email: [email protected]

Teaching mixed methods and action research to graduate students has great advantages but also poses interesting challenges. The purpose of this article is to make transparent the process by which a mixed methods research instructor helped inform a student-researcher of the limitations in a pilot study that was compromised by methodology, including sampling conditions and analysis procedures. The student-researchers’ Communicative Language Teaching (CLT) intervention was proposed initially to be examined via a pretest-posttest design intended to determine whether video clips were an effective intervention to teach Arabic language and Egyptian customs to non-native speakers. Although data included videotaped teaching sessions, interviews, observations, expert testimonials, and pre- and post-surveys, the research design, sample size, and analysis procedures were insufficient to assess causality. At best, the data collected before the student-researcher had taken an introductory mixed methods research course were useful to understand the participants’ perceptions about the effectiveness of CLT techniques for learning. This article provides specific suggestions with applicable references about how the CLT pilot study could be improved and presents a rationale for classifying the study as an exploratory multimethod study, or an arrested action research study, rather than as a mixed methods effectiveness study as originally proposed. Advice for novice researchers concerning paradigms, stages within the research process, interactive research elements, typologies, and issues of credibility are provided along with a 7-step planning process designed to improve novice researchers’ projects.

THOMAS W. CHRIST AND HOSAM ELMETAHER


those teaching research and mentoring students (Christ, 2009). This was the case in the Communicative Language Teaching (CLT) Arabic language intervention study. By the time the student-researcher entered the introductory mixed methods research class, the intervention designed by the student and an interim advisor had been administered and data including video, surveys, interviews, and expert testimonial had been collected. Opportunities to redesign and to improve the project were severely constricted as a result of time limitations.

When working with students who have begun their projects before having a clear understanding of methodological options, the research-teacher often becomes a mentor who advises students how to refine the research design and procedures. In the case of the CLT pilot study, rich data were collected that logically connected to the topic under investigation, but the research design precluded the opportunity to demonstrate causality and the limited sample size adversely affected both statistical power and the generalizability of the findings. Thus, it became necessary to provide the student-researcher curricular advice and written feedback as a way to support choosing the most applicable methodology and procedures. Although the student first defined the methodology as representing mixed methods research, other approaches including multimethod research (Brewer & Hunter, 1989), action research (Mertler, 2008), and case study (Yin, 2003) needed to be considered.

Christ (2010, 2013), Frels, Onwuegbuzie, Leech, and Collins (2012), Ivankova (2010), and Mertens (2010) all have documented pedagogical challenges and strategies to teaching mixed methods research, but no published works were found that specify what can be undertaken when the focus of the research project changes as a result of data limitations. Thus, the purpose of this article is to present a case study that illustrates a common mistake made by beginning researchers: collecting data that do not adequately reflect what the research questions/hypotheses intended or that could not possibly be analyzed in the ways in which the researcher assumed. Often, researchers end up having to reconsider the analytic approach and/or research questions/hypotheses; however, this should be a rare event and certainly discouraged. This article presents the challenge faced by the research-teacher in the CLT pilot study, to provide information to the student-researcher about why demonstrating a causal relationship between the intervention and speaking and listening skills as proposed by the student-researcher under advisement from an interim advisor was impossible due to improper sampling procedures and limited measures and available data. Determining the strength of the relationship

as originally conceived via the limited data set would have resulted in meaningless conclusions. However, other methodological options were available for analyzing descriptively the numerical data, interview data, expert testimonials, and student-researchers’ self-reflections. Methodological Options

According to Brewer and Hunter (2006), multimethod research entails different styles of research combined in the same research project and is not restricted to the use of both quantitative and qualitative data. For example, qualitative participant observations could be combined with qualitative in-depth interviewing, or quantitative surveys and norm-referenced measures could be included in a quasi-experimental research design. In the case of the CLT study, the student-researcher hoped to demonstrate causal inferences using a quasi-experimental design with survey and observation data presented as descriptive statistics to examine a possible post-intervention difference. Limitations of the research design would have made causal inferences invalid. As an alternative, the student-researcher could have conducted qualitative research techniques to assess causality (cf. Maxwell, 2004) but unfortunately did not have knowledge of this approach when the data were collected. Once the constraints of the available data sets and various methodological options were considered, the causal line of reasoning was abandoned.

During the introductory mixed methods course, the student-researcher was provided with specific readings about various research typologies (cf. Christ, 2009). Emphasis was placed on determining how to analyze and to merge the quantitative and qualitative data strands (Christ, 2008a, 2008b, 2010) when considering methodological options. After reading the provided curricula and creating methodological maps, the student-researcher focused on four methodological approaches: mixed, multi, case study, and arrested action research (i.e., a single research cycle; cf. Mertler, 2008). The most difficult distinction for the student-researcher was the difference between multi- and mixed methodologies. After reviewing Pat Bazeley’s definition as quoted in Johnson, Onwuegbuzie, and Turner’s (2007) article, the distinction between multimethod and mixed methods research was further clarified:

Multimethods research is when different approaches or methods are used in parallel or sequence but are not integrated until inferences are made. Mixed methods research involves the use of more than one approach to or method of design, data collection, data analysis within a single program of study, with integration of the different approaches or methods occurring during the program of

RESEARCH TEACHING PEDAGOGY: LESSONS LEARNED FROM AN ARABIC LANGAUGE INTERVENTION PILOT STUDY


study, and not just at its concluding point. (p. 119)

This definition helped the student-researcher to make the distinction between multi- and mixed methods. For the CLT intervention, considering multi- versus mixed research promoted a healthy class discussion about the value of different forms of data when merged to give meaning and what can be concluded when quantitative data are used for descriptive purposes in an intervention study. The CLT Pilot Study

The second language learning literature reveals that Grammar Translation Methods and rote memorization are the hallmark of current language instruction techniques. For learning to occur, foreign language learners need receptively to hear, to process, to comprehend, and to express in conversation words, phrases, and sentences for language skills to maintain (Gordon, 2001; Littlewood, 1981; Medd & Whitmore, 2001). Second language acquisition literature indicates that the Communicative Language Teaching approach is a viable alternative to Audiolingualism and Grammar Translation Methods. Because no literature was found specific to CLT techniques used when teaching the Arabic language, the strength of the pilot study is in exploring participant perceptions about the intervention. Although the original intent of the study was to examine whether CLT was an effective intervention for increasing Arabic communication skills with non native speakers, constraints in data and time limitations precluded a causal research design, although exploring the phenomenon and describing the results were possible.

Video recordings were used in three unique ways in this study: first as a source of information about how the Arabic language and culture was introduced as part of the intervention; second, to establish whether CLT techniques were used when teaching students; and third, if the CLT techniques were used, to determine whether they showed signs of helping, engaging, and improving the participants’ Arabic language proficiency. Two research questions were initially proposed to the student-researcher’s interim advisor before data were collected.

(a) To what extent does the CLT curriculum improve and develop the foreign language learners’ listening and speaking skills?

(b) What are the perceptions of the participants concerning their experiences with CLT strategies as a method of increasing Arabic speaking and listening skills?

After collecting the data and reviewing curricula assigned in the introductory mixed methods research course, the student-researcher realized that the first question, posed to the interim

advisor about generating a causal research hypothesis, could not be answered reliably given the limited data, time remaining in program, and restricted sample size. Demonstrating a causal relationship with any degree of credibility was less than ideal because the student-researcher also was the Arabic language teacher in this CLT project. After extensive discussions, review of the literature, and written feedback, the causal hypothesis was abandoned and the participants’ experience question remained, a somewhat weaker guiding question, but the best question that could be addressed given the available data. The student-researcher determined that multimethod exploratory case study was more applicable than was a quasi-experimental or a mixed methods research design (Creswell, 2008).

To clarify why this decision was made, it is important to present to the readers how the existing data restricted methodological options. In the CLT study, pre- and post-speaking and listening tasks were conducted to examine whether the participants’ skills improved as a result of the intervention. The independent variable was defined by the student-researcher as the “CLT curriculum and strategies,” whereas the dependent variable was considered to represent “ability in Arabic language, listening, and speaking skills.” The student-researcher’s first indication that a quasi-experimental research design was less than ideal occurred when attempting to define both the independent and dependent variables. In this case, after reviewing the literature, the student-researcher recognized that CLT is a construct with numerous definitions and teaching strategies that vary considerably. Although the intervention was operationally defined for the study, determining the dependent variables used to measure increases in performance turned out to be more challenging. As originally designed, the only quantitative measures were scores from a 6-item learning task (used as pre- and post-measures) and a 6-item rating scale provided to the study participants. Coded interviews and testimonies from experts who observed the videotaped teaching sessions were indicated by the student-researcher to be the qualitative markers of change. The CLT Intervention

Communicative Language Teaching is "an approach to foreign or second language teaching which emphasizes that the goal of language learning is communicative competence which seeks to make meaningful communication and language use a focus of all classroom activities" (Richards & Schmidt, 2002, p. 90). The CLT curriculum objectives designed for Arabic foreign language learners were taught over six 1-hour classes (see Figure 1). The pre-interview provided information about why they were interested in learning Arabic



and if they knew any Arabic writing and reading skills. Pre-intervention speaking and listening tasks were administered and video-recorded to establish a baseline. Post-intervention speaking and listening tasks then were administrated and video-recorded as a way to identify change in the participants’ levels of language proficiency. Data collection procedures are visually represented for clarity in Figure 2.

The task involved eliciting the following information via six items that were administered pre- and post-intervention: the study participant’s name, country, and university, followed by questions about food. Three professors and three language instructors then observed videos of CLT being taught followed by listening and speaking measures being re-administrated. This procedure provided the opportunity for determining whether CLT strategies were being used and whether changes in speaking and listening skills could be gauged by the panel of experts as well as by the Arabic language student-researcher-teacher. Post-surveys and interviews allowed for the Arabic language students to offer their opinions about the intervention and their perceptions of the class in general. Specifically, the participants were asked to

comment on whether they believed the intervention changed their expressive and receptive Arabic communication skills. According to the student-researcher, the main goal of the intervention was not to assess the degree to which the participants mastered the course content, but to measure the degree to which the target language skills (speaking and listening) had been improved and developed.

The insurmountable flaw in the original research design that necessitated the causal research hypothesis to be abandoned and a different research design to be chosen was the notion that a causal relationship could be measured between the CLT intervention and levels of Arabic language proficiency. Although results in terms of descriptive statistics were presented as a way to demonstrate that the intervention improved the Arabic language skills of the students, inferences regarding the causal nature of the relationship were not made by the student-researcher due to the limitations stemming from use of a quasi-experimental research design, small sample size, non-random sampling scheme, short duration of the intervention, and restricted type and amount of data collected.

Main goal: Improving and developing speaking and listening skills for the foreign language learners

(Through)

Culture

Content objectives (CBI) Language Objectives Expressions/idioms

About Customs and traditions

Grammar Values Gestures The restaurant

Words, structures, and sentences Intonations Oral language skills

Figure 1. Course objectives.

Six CLT class sessions

Teaching with focus on speaking and listening skills

CLT Curriculum Pre- and post-tasks (Confirmatory Quantitative) Participants (Intervention) Bounded by Pre- and post-interview and survey (Exploratory Qualitative) Bounded by

Language experts’ observations Numbers (Professors and language instructors) Arabic as foreign language

Literature Review Familiarity with Arabic reading and writing Difficulties in Arabic oral communication

Experienced the old teaching methods

Figure 2. Data collection procedures.



Research Methodology

The student-researcher first proposed a

concurrent mixed methods design (Creswell, 2008) for the CLT study based on the assumption that quantitative pre- and post-task data, interviews, observations, and open-ended survey items could demonstrate whether participants’ listening and speaking skills improved as a result of the intervention. After reviewing the existing data set and project timeline, it was clear to both the student-researcher and mixed methods research-teacher that answering the first research question with confidence due to the small number of participants and limited data was not possible. Because research projects are culminating activities designed for learning, it was more important for the student-researchers in the introductory class logically to deduce applicable methodologies and procedures for their projects. Assigned literature in the course (cf. Christ, 2009 for a detailed description of curricula and activities) helped the student-researcher to consider several methodological alternatives for the CLT pilot study. Helpful tools for determining methodological typologies included a simple decision-making matrix (see Table 1) and methodological maps. The methodological typology for the CLT study was determined by bounding the case considering: (a) the topic: CLT strategies; (b) time: six classes; (c) participants: college students, teacher, expert observers; (d) location: university setting; and (e) data: surveys, interviews, videotaped teaching sessions, and pre-post measures (Christ, 2010). The student-researcher created several methodological maps as a way to help define potential research designs (see Figure 3).

The student-researcher who had a very basic understanding of paradigms indicated that pragmatism (of the middle philosophy; cf. Johnson & Onwuegbuzie, 2004) was applicable for this study because the quantitative (pre- and post-tasks) and qualitative (interview, observation, survey) data were collected, analyzed, and merged to understand better the research problem. Although pragmatism was specified by the student-researcher, other philosophical assumptions that are emerging in the mixed methods literature such as critical realism (Christ, 2010; Maxwell & Mitappalli, 2010) and the transformative (Greene, 2007) stance were not considered because they were optional readings in the introductory research course. Although a philosophical worldview (ontology, epistemology, axiology) was requested

to be included in the CLT study, it was not articulated. Christ (2013), Creswell and Plano Clark (2010), and Teddlie and Tashakkori (2009) all specify that when designing research, it is important to specify one’s own worldview and the purpose of the study to help clarify that the research questions, methodology, and procedures are all consistent. Most helpful was the opportunity for the student-researcher to delineate the purpose of the study and to determine how the case was bound as a way to determine an appropriate methodology. If the purpose and parameters of the study were defined from the onset, the predisposition towards seeking to examine a causal relationship could have been avoided.

Data included in the study were self-reflections, pre- and post-speaking tasks, pre-intervention interview responses, and post-intervention responses. The pre- and post-speaking task raw scores were compared by the student-researcher with the intent of assessing changes in language skills. If the student-researcher had wanted to test for differences in the population by assessing whether differences found in the sample are statistically significant, an expected effect size would have had to be determined. With such a small convenience sample, even if applicable procedures had been followed, the results could not demonstrate with validity changes in language skills due to inadequate statistical power. The responses from the pre- and post-participant interviews and surveys, and the feedback from experts who reviewed the videotaped teaching sessions to determine how CLT teaching strategies were used were analyzed for themes using a simplified form of hand analysis (color marking themes in transcripts for quotable passages transferred to tables for review). In essence, the analysis of the qualitative data lacked sufficient rigor. Given the time constraints of 16 weeks in the introductory mixed methods research course, students with prior qualitative research training (e.g., method of constant comparison, discourse analysis, componential analysis, keywords-in-context, analytic induction, word count, domain analysis, taxonomic analysis; cf. Leech & Onwuegbuzie, 2007, 2008, 2011) were at an advantage to their peers because these techniques were only assigned as optional readings in the course. The student-researcher indicated that the three data strands (i.e., speaking task responses, interview responses, video responses) allowed for data triangulation as a way to increase credibility of the findings (Maxwell, 2005; Patton, 1999).



Table 1 Mixed Method, Multimethod, and Action Research Criteria

Figure 3. Methodological map.

Methodology Options Mixed Method Multimethod Action Research Purpose

Confirmatory Explanatory Exploratory

Exploratory Explanatory

Confirmatory

Self Reflective

Sequencing Concurrent Sequential

Concurrent Sequential

Sampling Purposeful Convenience Convenience

Generalization Possible Possible Not Possible



Pilot Study Results

According to the student-researcher’s analysis,

the CLT strategies offered participants the chance to talk, to interact, and to negotiate meaning using activities designed to enhance practicing communication. Strategies, including web-links to videos, were introduced to the participants as a way of enhancing their interest in the language through exposure to a different culture, traditions, and values. Students learned by studying, speaking, and searching for video materials, which is a hands-on approach promoted in the CLT literature (Lee & VanPatten, 2003; Littlewood, 1981). Searching links, role playing, and storytelling, according to the participants, allowed them to interact, to communicate, and to learn from each other.

Excerpts of the six sessions were observed by three professors and three English language instructors familiar with CLT teaching techniques. According to these six language experts, video as a source of curricula emerged as a particularly effective teaching strategy. One English language instructor approved of using “media and sounds in the class.” Another indicated the power of “video to gain attention of students.” Another comment by a professor included “Beginning the class with a short video on Egypt was a great idea.” The expert observers stated that many of the prominent CLT teaching strategies as indicated in the literature were observed (Liao, 2000a, 2000b; Littlewood, 1981).

Clearly, the data collected and analyzed by the CLT student-researcher were descriptive in nature and useful for understanding how the strategies were viewed by participants, the CLT teacher, and language experts, but could not be used to assess statistical significance (i.e., p values) or practical significance (i.e., effect size) (cf. Onwuegbuzie & Daniel, 2005) due to the convenience sample and small number of participants in the study. The six pre- and post-speaking and listening tasks conducted by the 10 participants in the study were presented simply in tables as a visual display demonstrating increased language performance due to the intervention.

As an alternative to presenting raw data in tables, the student-researcher could have used the test of multiple group differences devised by Onwuegbuzie and Levin (2005), who conceptualized three methods for testing the trend of multiple group differences. Onwuegbuzie and Levin indicated that the Binomial Test of Trend is the most notable, in which the binomial distribution is used to determine whether the number of group differences in the same direction (positive/negative) should be regarded as either a statistically real or a statistically chance finding. Specifically, as conceptualized by Onwuegbuzie,

Levin, and Ferron (2011), for a 5% level of statistical significance,

when a study includes at least five outcome measures (for directional alternatives) or at least six outcome measures (for nondirectional alternatives), the binomial formula can be used to determine the probability of obtaining the observed proportion of findings in the same direction, under the null hypothesis specification that p (the success probability) and q (the failure probability) are each equal to .5 (i.e., assuming that a difference in one direction is as likely as obtaining a difference in the opposite direction). If the observed proportion differs statistically from .5, an effect-size index (typically the observed proportion itself, which serves as an unbiased estimator of p), along with a corresponding confidence interval, could be reported and interpreted. (p. 130) Thus, using this procedure in the case of the

CLT study, the probability that all six post-test scores were higher than was the pretest scores is less than .05, which indicates a statistically significant finding could have been used to address the first research question. Unfortunately, the Binomial Test of Trend was not used for statistical analysis in the case of the CLT pilot study because several of the post-test items were modified, precluding a meaningful comparison of the results.

Participants asked to comment on their experience with CLT strategies and their views about Arabic communication skills reported self-perceived improvements as a result of learning CLT strategies. One participant indicated “During the class I felt my listening and speaking were improved. I can distinguish some words in Arabic conversation now. I learned some necessary words and sentences in Arabic conversation.” Another participant stated the following:

I knew almost nothing when I began this course. Now I know some useful phrases and the numbers. Although I made many mistakes and still have not mastered everything presented in the class, I came to understand a great deal of what the teacher spoke, I was able to produce language independently, and my answers were understandable to both the teacher and fellow students. Analysis of the post-surveys, according to the

student-researcher, also revealed that the participants found communicative activities, opportunities to practice, and visual aids helpful to improve their Arabic language skills. A series of quotations from the 10 participants was presented as evidence to support these conclusions, such as the following: “language has been presented very vividly”; “I think I will remember and also be more capable of productive use”; “great interactions



between cultures, between individuals…Yes, definitely, there were lots of activities which enabled everyone to participate”; and “I most enjoyed the group activities where I spoke with other students. Knowing that other students were also experiencing difficulty with pronunciation I felt less pressure.”

Although the selected quotations present some evidence that support categories that the student-researcher presented in the conclusions section of the study, it was not clear how these categories were created. Credibility of the categories and themes could have been much improved if four steps suggested by Constas (1992) had been made explicit: (a) where the responsibility or authority for the creation of categories resided (i.e., participants, programs, investigative, literature, or interpretive); (b) what the grounds were on which one could justify the existence of a given set of categories (i.e., external, rational, referential, empirical, technical, or participative); (c) what was the source of the name used to identify a given category (i.e., participants, programs, investigative, literature, or interpretive); and (d) at what point during the research process the categories were specified (i.e., a priori, a posteriori, or iterative).

Pre- and post-interviews used in the CLT study according to the student-researcher “revealed that motivation to learn Arabic differed for all the participants.” Although the student-researcher had been instructed by his interim advisor to use a pretest-posttest (pre-experimental) design, it was not until after data were collected for the study that readings assigned in the mixed methods class revealed that pre-experimental designs are considered to be extremely weak because it is impossible to make trustworthy causal statements. Gay and Airasian (2003) indicated that pre-experimental designs are so weak that they simply should be avoided because findings would be so questionable that they could not be useful for most purposes. Clearly, the CLT study, which was a one-group pretest-posttest design, could not control for the countless threats to internal validity (e.g., history, maturation, testing, instrumentation, regression) and external validity (e.g., pretest-treatment interaction).

Post-videotaped interview results, according to the student-researcher, support the use of communication activities and technology aids as an alternative to memorization. Quotations provided to support this conclusion included the following: “I think the activities were very important for us to learn…because during those activities we had to communicate with each other in Arabic what we learned in class”; “It is very comfortable engaging in communication and interaction among people”; and “we learned to communicate and to interact by the class activities.”

The student-researcher concluded that CLT strategies appeared to improve students’ abilities to communicate as seen in the higher post-speaking and listening scores. With inferential tests being invalid due to the small sample size (cf. Onwuegbuzie & Levin, 2005), this conclusion was unwarranted; yet, the qualitative data including student reports of their language performance, and observations of use, suggested improved Arabic speaking skills. Experts who viewed the videotapes also supported this conclusion. The student-researcher’s qualitative findings did align with prior research about CLT (Galloway, 1993; Gatbonton & Segalowitz, 2005; Jones & Wang, 2001; Lee & VanPatten, 1995, 2003; Liao, 2000a, 2000b) adding a level of credibility to the student-centered theme that emerged.

The most important process of conducting a pilot study and taking an introductory mixed methods research course was not the knowledge gained from the pilot study itself, but the ability to understand a variety of research methodologies, articulate numerous limitations inherent in the initial proposal (i.e., pre-experimental design, convenience sample, small sample size, no control group, threats to verification and trustworthiness of the qualitative interview data analyses techniques), and what could be undertaken to improve the study in the future.

Conclusions

Backward research planning, a term used to

describe how the student-researcher collected data before a clearly articulated research design was concluded, is common. The CLT example demonstrates the limitations faced by the student-researcher and why researchers should consider available time and specify the purpose of the study when designing the research questions and accompanying methodology. Because mixed and action research have increased in popularity over the past 10 years, many students indicate that their studies are mixed without understanding the qualities of these research designs (Christ, 2010; Teddlie & Tashakkori, 2009). Fortunately, the number of experts with knowledge about multi-, mixed-, and action research also has increased, offering opportunities for appropriate guidance. As Tashakkori and Creswell (2008) indicated, it is the responsibility of “stewards of the social-behavioral research enterprise” (p. 291) to help those new to the arena who are struggling with terminology, methods, and methodology. The CLT pilot project highlights several points that stewards of mixed and action research should consider, including a reasonable level of rigor, criteria for establishing legitimacy (Collins, Johnson, & Onwuegbuzie, 2012) and authenticity (Guba & Lincoln, 1989), in terms of stated methodology, paradigmatic



consistency, and credible evidence. For example, the CLT study demonstrates how numerical data increased the knowledge gained about the phenomenon, but the data that were collected could not be used to demonstrate a causal relationship. The CLT pilot study highlights several shortcomings that can inform those new to the field of research: the importance of discussing threats to verification, trustworthiness, authenticity, credibility, transferability, and dependability of qualitative data (e.g., Creswell, 2007; Guba & Lincoln, 1989; Lather, 1993; Lincoln, 1995; Lincoln & Guba, 1985; Maxwell, 1997, 2004, 2005; Miles & Huberman, 1994). Many of these threats could have been countered via one or more of the following 18 qualitative procedures: (a) prolonged engagement; (b) persistent observations; (c) triangulation of data sources or participants; (d) contextualization of observations; (e) method of constant comparison; (f) checking for representativeness of sources of data; (g) checking for researcher effects; (h) weighing the evidence; (i) examining extreme cases; (j) checking for spurious relations; (k) examining rival explanations; (l) looking for negative evidence; (m) obtaining feedback from informants; (n) leaving an audit trail; (o) thick description; (p) assessing structural relationships; (q) use of referential material; and (r) theoretical sampling (for a complete explanation, see Onwuegbuzie & Leech, 2007a, 2007b).

This article is not meant to limit new and freely developed typological combinations or procedures that are emerging in the mixed and action research literature; rather, it is an appeal to those who teach methodology to maintain a high standard for logical and sound empirically based research. The challenge for student-researchers is to look holistically at study conditions and the limitations inherent in their research skills, available data, and time when determining the most appropriate methodological design (e.g., case, mixed, action research design) that they plan to use.

Four steps can help those considering the most appropriate methodological classification. First, researchers should become familiar with the underpinnings of a minimum of five paradigmatic views: constructivism, critical, transformative, pragmatism, and postpositivism (Christ, 2010). These five paradigmatic views have multiple definitions and it is critical for student-researchers to understand at least the basics of each. For example, there are at least two prominent types of constructivism that should be reviewed (a) radical constructivism (cf. Glasersfeld, 1996), which focuses more on the individual knower and acts of cognition; and (b), social constructionism (cf. Berger & Luckmann [1966] or Schwandt [1994, 2001]), which focuses more on social processes

and interaction (e.g., symbolic interactionism) and ethnomethodology that emphasize the participant’s interpretation of the situation; how social people identify, produce, and reproduce social actions; and how they come to share an intersubjective understanding of specific life circumstances. Key to both forms of constructivism is the rejection of scientific realism (i.e., the view that our theories map out real features of the world) and scientific objectivity (i.e., the view that it is possible to obtain accurate and reliable interpretations of the way the world is). Similarly, there are several forms of critical realism, with distinctions between U.S. and British versions (cf. Danermark, Ekström, Jakobsen, & Karlsson, 2002; Maxwell & Mittappali, 2010; Sayer, 1992); yet, most forms of critical realism are closely associated with Bhaskar’s view who denied there is any objective or certain knowledge of the world and that there is the possibility of alternative valid accounts of any phenomenon. Maxwell and Mittappali (2010) summarized an important tenet in the definition of critical realism: “All theories about the world are grounded in a particular perspective and worldview, and all knowledge is partial, incomplete, and fallible” (p.150).

Second, four levels of the research process should be considered by researchers as they construct their projects: (a) the epistemological level, (b) theoretical perspectives, (c) methodology, and (d) methods. Crotty (1998) summarized these four levels in his excellent text, which should be strongly considered as mandatory reading for students in introductory mixed methods research courses because it is easily understood and clarifies these complex concepts.

Third, the problem, purpose, and research questions should be understood as representing interactive elements in a research study (Christ, 2009; Maxwell, 2005). Maxwell (1997) originally published his interactive qualitative research model, which included five components: goals, conceptual framework, validity, methods, and the research question. Christ (2010) expanded Maxwell’s methodological graphic display more readily to fit mixed and action research studies: (a) Goals: personal, practitioner, policy, and intellectual; (b) Methodology: research design, typologies; (c) Methods/procedures: sampling, data collection, analysis, merging, bounding the case; (d) Validity: reliability, triangulation (data sources, paradigms, methods, participants), credibility, member check, extended observations, thick descriptions, expert audit; (e) Conceptual Framework: existing theories, researchers’ worldviews, paradigms; and (f) Research Questions: overarching and subsumed.

Fourth, there are numerous typologies that should be considered. Teddlie and Tashakkori (2009, p. 141) developed a useful tool that lists



seven criteria for determining potential typologies, number of approaches, strands, implementation process, integration stages, priority, purpose, and theoretical/ideological perspectives. In terms of potential methodological typologies, the CLT study could have been classified as a mixed method, multimethod, arrested action research, or a case study design. Defining these four typologies can help the reader to determine which methodology was most appropriate and why. For example, mixed methods research, according to Johnson et al. (2007), is research in which

a researcher or team of researchers combines elements of qualitative and quantitative research approaches (viewpoints, data, inferences) for the broad purposes of breadth and depth of understanding and corroboration. A mixed methods study would involve mixing within a single study; a mixed method program would involve mixing within a program of research, and the mixing might occur across a closely related set of studies. (p. 123) Creswell’s (2006) definition incorporates

paradigms and philosophical assumptions, and theoretical perspectives, as well as research questions and interpretations. In short, mixed methods encompass the totality of all phases of research, not just the methods. Brewer and Hunter’s (1989) definition of multimethod research indicates that both quantitative and qualitative research approaches are used, but they remain relatively independent until the interpretation stage.

Reason and Bradbury (2008) defined action research as a “participatory process concerned with developing practical knowing in the pursuit of worthwhile human purposes. It seeks to bring together action and reflection, theory and practice, in participation with others, in pursuit of practical solutions to issues of pressing concern to people.” (p. 4). Gorard (2010, pp. 241-242) simply defined research as “a spiral which has no clear beginning or end, in which activities (phases) overlap, can take place simultaneously, and iterate.” which is an appropriate description of the central tenet in action research.

In the case of the CLT study, numerical and verbal data briefly were analyzed separately, resulting in exploratory descriptions that were compared. Available data could not confirm if the CLT strategies improved listening and speaking skills, precluding a causal mixed methods classification; but the self-reflective activities were useful for improving student-researcher’s own teaching practices, which is critical in action research. Where the CLT study fell short of being classified as action research was the study did not follow a reiterative process of planning, acting, developing, and reflecting (Bradbury, 2007; Greenwood & Levin, 2001). Without repeated cycles of reflection and intervention modification

(Mertler, 2008), including a plan for improvements to the intervention and expressed modifications to teaching praxis (Christ, 2010), the project only could be classified as arrested action research.

Implications for Research Teachers For many student-researchers interested in

creating and evaluating an intervention, action research is an ideal form of confirmatory mixed methods research in practice (e.g., intervention improves performance, self reflection improves praxis), but it typically requires following several steps: (a) topic and intervention identification, (b) intervention administration and data collection, (c) data analysis and reflection leading to intervention modifications, (d) repeated iterations, and (e) dissemination to stakeholders. Action research is an ideal medium for learning how to conduct research because it is exploratory, confirmatory, and explanatory requiring data collection, analysis, and reflexive action (Reason & Torbert, 2001). The first step of an action research study often represents an exploratory design asking a how question with focus upon improving a situation, then moves through cycles of data collection, analysis, and reflection, becoming an explanatory and/or confirmatory design that determines if the actions have an intended effect (Christ, 2009). The action research cycles are sequential, which is one form of mixed methods design (Creswell & Plano Clark, 2007; Tashakkori &Teddlie, 1998; Teddlie & Tashakkori, 2009).

Scholars who advise students conducting a thesis or dissertation should encourage the pilot testing process to determine if a study is logical, representative of the proposed typology, and demonstrates credible and valid findings. They should provide specific written feedback to highlight any flaws and suggest readings the student-researchers can use to improve their projects in the future. In the case of the CLT study, the descriptive data presented in tables and themes that emerged from interviews and expert opinions demonstrate knowledge. Yet, the findings clearly cannot show causation between the intervention and the acquisition of speaking and listening skills. The numerical results, expert testimonials, and student interviews combined only could provide the student-researcher insight about her or his own teaching habits, the value of CLT strategies, and whether the intervention appeared to improved students’ expressive and receptive language skills. Most meaningful were themes that emerged about the use of video as a medium for increasing student engagement and the acquisition of language skills. Clearly, the use of backward research planning is far from ideal, but it is quite common at the thesis level. Most critical when first



engaging in research is to learn the basics, conduct the research, and learn from the shortcomings.

Strategies including following a project outline with headers for each section and creating methodological diagrams (Christ, 2009) help student-researchers to learn about the strengths and challenges associated with various forms of mixed-, multi-, and action research. By introducing paradigmatic views, terminology, methodological maps, and rubrics for evaluating research at the proposal stage (Christ, 2010), the likelihood is that the final research project will be rigorous, logical, and, perhaps, even publishable.

The lead editors for this article were Anthony J. Onwuegbuzie and John R. Slate.

References Berger, P., & Luckmann, T. (1966). The social

construction of reality: A treatise in the sociology of knowledge. Garden City, NY: Anchor Books.

Bradbury, H. (2007). Quality and actionability: What action researchers offer from the tradition of pragmatism. In A. B. (Rami) Shani, S. A. Mohrman, W. Pasmore, B. Stymne, & N. Adler (Eds.), Handbook of collaborative management research (pp. 583–600). Thousand Oaks, CA: Sage.

Brewer, J., & Hunter, A. (1989). Multimethod research: A synthesis of styles. Newbury Park, CA: Sage.

Brewer, J., & Hunter, A. (2006). Foundations of multimethod research: Synthesizing styles (2nd ed.). Thousand Oaks, CA: Sage.

Christ, T. W. (2007). A recursive approach to mixed methods research in a longitudinal study of postsecondary education disability support services. Journal of Mixed Methods Research, 1, 226–241. doi:10.1177/1558689807301101

Christ, T. W. (2008a). A cross-case analysis of leadership qualities in three postsecondary disability support centers. Journal of Ethnographic & Qualitative Research, 2, 223-230.

Christ, T. W. (2008b). Technology support services in postsecondary education. A mixed methods study. Technology and Disability, 20(1), 25-35.

Christ, T. W. (2009). Designing, teaching, and evaluating two complementary mixed methods research courses. Journal of Mixed Methods Research, 3, 292–325. doi:10.1177/1558689809341796

Christ, T. W. (2010). Teaching mixed methods and action research: Pedagogical, practical, and evaluative considerations. In A.

Tashakkori & E. Teddlie (Eds.), Sage handbook of mixed methods in social & behavioral research (2nd ed., pp. 643-676). Thousand Oaks, CA: Sage.

Christ, T. W. (2013). The worldview matrix as a strategy when designing mixed methods research. International Journal of Multiple Research Approaches, 7, 110-118.

Collins, K. M. T., Onwuegbuzie, A. J., & Johnson, R. B. (2012). Securing a place at the table: A review and extension of legitimation criteria for the conduct of mixed research. American Behavioral Scientist, 56, 849-865.

Constas, M. A. (1992). Qualitative analysis as a public event: The documentation of category development procedures. American Educational Research Journal, 29, 253-266.

Creswell, J. (2006, April). Continuing the discourse: Advocates for and challengers to mixed methods research. Symposium conducted at the American Education Research Association Mixed Methods SIG Business Meeting, San Francisco.

Creswell, J. (2007). Qualitative inquiry and research design: Choosing among five approaches. (2nd ed.).Thousand Oaks, CA: Sage.

Creswell, J. (2008). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Upper Saddle River, N.J.: Pearson/Merrill Prentice Hall.

Creswell, J. (2010). Mapping the developing landscape of mixed methods research. In A. Tashakkori & E. Teddlie (Eds.), Sage handbook of mixed methods in social and behavioral research (2nd ed., pp.45-68). Thousand Oaks, CA: Sage.

Creswell, J., & Plano Clark, V. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.

Creswell, J., & Plano Clark, V. (2010) Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

Crotty, M. (1998). Foundations of social research: Meaning and perspective in the research process. Thousand Oaks, CA: Sage.

Danermark, B., Ekström, M., Jakobsen, L., & Karlsson, J. C. (2002). Explaining society: Critical realism in the social sciences. London, England: Routledge.

Frels, R. K., Onwuegbuzie, A. J., Leech, N., & Collins, K. M. T. (2012). Challenges to teaching mixed research courses. The Journal of Effective Teaching, 12, 23-44.

Galloway, A. (1993). Communicative language teaching: An introduction and sample



activities. In, ERIC Digests (pp. 1-8). Washington DC: ERIC Clearinghouse on languages and Linguistics . (Reprinted from www.eric.ed.gov)

Gatbonton, E., & Segalowitz, N. (2005). Rethinking communicative language teaching: A focus on access to fluency. The Canadian Modern Language Review, 61, 325 353. doi:10.1353/cml.2005.0016

Gay, L. R., & Airasian, P. (2003). Educational research: Competencies for analysis and application (7th ed.). Upper Saddle River, NJ: Pearson Education.

Glasersfeld, E. V. (1996). Radical constructivism: A way of knowing and learning. London, England: Falmer Press.

Gordon, D. (2001). Practical suggestions for supporting speaking and listening in classrooms. In P. G. Smith (Ed.), Talking classrooms: Shaping children's learning through oral language instruction (pp. 57-73). Newark, DE: International Reading Association.

Gorard, S. (2010). Research design as independent of methods. In A. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social and behavioral research (2nd ed., pp. 237-252). Thousand Oaks, CA: Sage.

Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.

Greene, J. C. (2008). Is mixed methods social inquiry a distinctive methodology? Journal of Mixed Methods Research, 2, 7-22. doi:10.1177/1558689807309969

Greenwood, D., & Levin, M. (2001). Pragmatic action research and the struggle to transform universities into learning communities. In P. Reason & H. Bradbury (Eds.), Handbook of action research (pp.103-113). Thousand Oaks, CA: Sage.

Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.

Ivankova, N. (2010). Teaching and learning mixed methods in computer-mediated environment: Educational gains and challenges. International Journal of Multiple Research Approaches, 4, 49-65. doi:10.5172/mra.2010.4.1.049

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26. doi:10.3102/0013189X033007014

Johnson, R., Onwuegbuzie, A., & Turner, L. (2007). Towards a definition of mixed methods research. Journal of Mixed Methods Research, 1, 112–133. doi:10.1177/1558689806298224

Jones, N. B., & Wang, S. H. (2001). Applying communicative language teaching in a Taiwanese elementary school. Retrieved from http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/19/bf/31.pdf

Lather, P. (1993). Fertile obsession: validity after poststructuralism. The Sociological Quarterly, 34, 673–693. doi:10.1111/j.1533-8525.1993.tb00112.x

Lee, J. F., & VanPatten, B. (1995). Making communicative language teaching happen. The McGraw-Hill foreign language professional series, vol 1. New York, NY: McGraw-Hill.

Lee, J. F., & VanPatten, B. (2003). Making communicative language teaching happen. Boston, MA: McGraw-Hill.

Leech, N. L., & Onwuegbuzie, A. J. (2007). An array of qualitative analysis tools: A call for data analysis triangulation. School Psychology Quarterly, 22, 557-584. doi:10.1037/1045-3830/22/4/557

Leech, N. L., & Onwuegbuzie, A. J. (2008). Qualitative data analysis: A compendium of techniques for school psychology research and beyond. School Psychology Quarterly, 23, 587-604. doi:10.1037/1045-3830.23.4.587

Leech, N. L., & Onwuegbuzie, A. J. (2011). Beyond constant comparison qualitative data analysis: Using NVivo. School Psychology Quarterly, 26, 70-84. doi:10.1037/a0022711

Liao, X. Q. (2000a). Communicative language teaching innovation in China: Difficulties and solutions. Retrieved from http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/16/52/0b.pdf

Liao, X. Q. (2000b). Communicative language teaching: Approach, design and procedure. Retrieved from http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/16/67/40.pdf

Lincoln, Y. S. (1995) Emerging criteria for qualitative and interpretive research. Qualitative Inquiry, 3, 275–289. doi:10.1177/107780049500100301

Lincoln, Y. S., & Guba, E. G. (1985) Naturalistic inquiry. Thousand Oaks, CA: Sage.

Littlewood, W. (1981). Communicative language teaching: An introduction. Cambridge, England: Cambridge University Press.

Maxwell, J. A. (1997). Designing a qualitative study. In L. Bickman & D. J. Rog (Eds.), Handbook of applied social research



methods (pp. 69–100). Thousand Oaks, CA: Sage.

Maxwell, J. A. (2004). Causal explanation, qualitative research, and scientific inquiry. Educational Researcher, 33(2), 3-11. doi:10.3102/0013189X033002003

Maxwell, J. A. (2005). Qualitative research design: An interactive approach. Applied Social Research Methods Series, no. 41. Thousand Oaks, CA: Sage.

Maxwell, J. A., & Mitappalli, K. (2010). Realism as a stance for mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social and behavioral research (2nd ed., pp. 145-167). Thousand Oaks, CA: Sage.

Medd, S. K., & Whitmore, K. F. (2001). What’s in your backpack? Exchanging funds for language knowledge in an ESL classroom. In P. G. Smith (Ed.), Talking classrooms: Shaping children's learning through oral language instruction (pp. 42-65). Newark, DE: International Reading Association.

Mertens, D. M. (2009). Transformative research and evaluation. New York, NY: Guilford Press.

Mertens, D. M. (2010). Philosophy in mixed methods teaching: The transformative paradigm as illustration. International Journal of Multiple Research Approaches, 4, 9-18. doi:10.5172/mra.2010.4.1.009

Mertler, C. (2008). Action research: Teachers as researchers in the classroom (2nd ed.). Thousand Oaks, CA: Sage.

Miles, M., & Huberman, M. (1994). Qualitative data analysis: An expanded sourcebook. (2nd ed.) Thousand Oaks, CA: Sage.

Onwuegbuzie, A. J., & Daniel, L. G. (2005). Editorial: Evidence-based guidelines for publishing articles in Research in the Schools and beyond. Research in the Schools, 12(2), 1-11. Retrieved from www.msera.org/download/Rits_editorial_12_2.pdf

Onwuegbuzie, A. J., & Leech, N. L. (2007a). A call for qualitative power analyses. Quality & Quantity: International Journal of Methodology, 41, 105-121.

Onwuegbuzie, A. J., & Leech, N. L. (2007b). Validity and qualitative research: An oxymoron? Quality & Quantity: International Journal of Methodology, 41, 233-249. doi:10.1007/s11135-006-9000-3

Onwuegbuzie, A. J., & Levin, J. R. (2005). Strategies for aggregating the statistically nonsignificant outcomes of a single study. Research in the Schools, 12(1), 10-19.

Onwuegbuzie, A. J., Levin, J. R., & Ferron, J. M. (2011). A binomial test of group

differences with correlated outcome measures. The Journal of Experimental Education, 79, 127-142. doi:10.1080/00220970903352761

Patton, M. Q. (1999). Enhancing the quality and creditability of qualitative analysis. HSR: Health Services Research, 34, 1189-1208.

Reason, P., & Bradbury, H. (2008). Introduction. In P. Reason & H. Bradbury (Eds.), Sage handbook of action research (pp. 1-10). Thousand Oaks, CA: Sage.

Reason, P., & Torbert, W.R., (2001). The action turn: Towards a transformative social science. Concepts and Transformation, 6(1), 1-37.

Richards, J. C., & Schmidt, R. W. (2002). Longman dictionary of language teaching and applied linguistics. London, England: Longman.

Sayer, A. (1992). Method in social science: A realist approach, London, England: Routledge.

Schwandt, T. (1994). Constructivist, interpretivist approaches to human inquiry. In N. K. Denzin, & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 118-137). Thousand Oaks: Sage.

Schwandt, T. A. (2001). Dictionary of qualitative inquiry (2nd ed.). Thousand Oaks, CA: Sage.

Tashakkori, A., & Creswell, J. (2008). Envisioning the future stewards of the social-behavioral research enterprise. Journal of Mixed Methods Research, 2, 291-295. doi:10.1177/1558689808322946

Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining the qualitative and quantitative approaches. Thousand Oaks, CA: Sage.

Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage.

Yin, R. (2003). Case study research: Design and methods (3rd ed.). Thousand Oaks, CA: Sage.



Unpacking the Effects: Identifying School and Teacher Factors and Their Influence on Teachers’ Intentions to Stay or Leave the Profession

Amy L. Sedivy-Benton University of Arkansas at Little Rock

Carrie J. Boden-McGill Texas State University

As we move full speed into the 21st Century, many things are changing, ranging from how we communicate and work to how we learn (Schleicher, 2010). These changes are major factors in how the United States will have to shape and adjust in order to compete in the global marketplace. One reoccurring issue that has not seemed to cease in the last decade is the ever-growing focus on the condition of education in the United States (Aud et al., 2012). As the globalization of jobs, politics, and economics become commonplace in the everyday lives of individuals in the United States, schools, teachers, and students are compared to their counterparts across the world. On a consistent basis, the United States scores at the bottom for both student achievement and teacher status (Organisation for Economic Co-operation and Development, 2012). Implementation of change in the United States school system tends to be incredibly slow, and the results from these actions cannot realistically be seen in the near future, as students need to

matriculate for results to be measured. For example, A Nation at Risk was published in 1983 by National Commission on Excellence in Education, and no substantial changes were noted in education until the implementation of No Child Left Behind in 2001 (U.S. Department of Education, 2005). This lack of change could be because schools needed time to determine what was not working and how to adjust. However, the expectation of the federal government seems to be contrary, and results are expected in a much shorter timeframe as further mandates are being passed in a shorter period. It has taken less than 10 years for the federal government to implement Race to the Top as an attempt to increase the standing and academic achievement of United States schools (U.S. Department of Education, 2010).

Since the passage of the No Child Left Behind Act of 2001 (U.S. Department of Education, 2010), which was also revisited again in 2010, districts, schools, and teachers have keenly focused on the performance of students and the movement and gains that have been made regarding test scores. The largest push in this movement has been to focus on the accountability of individuals involved with education. The U. S. Department of Education has just issued its fourth round of Teacher Incentive Fund grants. A focus of these

Correspondence concerning this article should be addressed to Amy L. Sedivy-Benton, 2801 South University Avenue DKSN 314, Little Rock, AR 72204. Email: [email protected]

Teacher turnover is costly in its financial implications and negative impact on student learning. Documented in many recent studies many teachers choose to transfer to a more preferable work setting or leave the field. For this study, the researchers conducted an analysis of the most recent School and Staffing Survey (SASS) data from the National Center for Education Statistics. Analysis methods entailed implementing a nested design using Hierarchal Linear Modeling and yielded results that indicated school level contextual factors such as teacher influence on school, teacher perception of control, and teacher perceived support played a significant role in teachers’ intentions to leave or remain in the field. An understanding of these factors allows policymakers and administrators to implement practices to improve work environments for teachers. Retaining good teachers is a key to improve teacher and school quality nationwide.

UNPACKING THE EFFECTS: IDENTIFYING SCHOOL AND TEACHER FACTORS AND THEIR INFLUENCE ON TEACHERS’ INTENTIONS TO STAY OR LEAVE THE PROFESSION


grants is to entice teachers with pay based upon their performance, or rather that of their students; the notion of paying for performance is most commonly called performance-based compensation. The current iteration of education reform, Our Future, Our Teachers: The Obama Administration’s Plan for Teacher Education Reform and Improvement, focuses largely on accountability measures (U.S. Department of Education, 2011). Characteristics of this plan, such as using performance based-measures to indicate improvement, as well as how federal funds will be appropriated to schools, like the NCLB Act before it, may do little to improve working conditions for teachers. This initiative remains focused on student achievement rather than on the environment in which teachers work and on efforts to retain those teachers who are effective.

Much discussion exists around student learning and student improvement, and student learning is certainly a crucial issue. However, the focus should be on the learning environment and teacher retention. Increased focus on accountability might not yield a working environment conducive to retaining teachers and fostering commitment to teaching (Feng, 2009). To address how these issues impact teachers’ intention to remain in the profession, we examined the data from the most recent complete Schools and Staffing Survey 2007-2008 (SASS) provided by the National Center for Education Statistics (NCES). These data were analyzed by using a nested design with Hierarchal Linear Modeling (HLM) to determine if contextual factors in the work environment of teachers play a significant role in teachers’ intentions to leave or to remain in the field. What we discovered is that environmental factors are strong when a teacher is deciding whether to remain or to leave the profession.

Thus, despite the current interest in teacher quality and supply, some clear gaps are evident in the published literature as indicated in the meta-analysis conducted by Borman and Dowling in 2008. Studies of teacher attrition seem to lack at least one of the following characteristics: use of a nationally representative sample or explicit linkages to a conceptual or theoretical framework within which to situate data analysis. Isolated in these works are the teacher individual characteristics from the context of the organizations in which they work, rather than taking them into consideration jointly. Singer and Willett (1994) proposed a plan for the NCES that would allow a true longitudinal study to be conducted on teachers and the directions that their careers are going. However, until a dataset like that proposed by Singer and Willett is executed, it is necessary to use the best and most comprehensive data about teachers available, the SASS dataset.

Building on previous research, we explored the contextual factors in the work environment as well as teacher characteristics that influence teachers’ intentions to remain in or to leave the profession. Rather than focus on factors that policymakers and administrators cannot control or that might take a long time to address, we elected to focus on those factors in the teaching environment that administrators might influence in a relatively short period of time. Using the most recent complete data set from SASS by NCES regarding the teachers and their perceptions, we discovered that teacher influence on school, teacher perception of control, and teacher’s perceived support, are factors in teachers’ intentions to leave or to remain in the field. In the discussion that follows, we briefly review prior research used to structure our study and present the methods, data, and results. The final section includes a discussion of the implications of the results, limitations of the study, and possibilities for future research.

Review of the Relevant Literature

Frequent and perpetual teacher turnover is

costly. When teacher turnover is continuous, the financial consequences are potentially dire for the school, district, and nation. Additionally, teacher turnover has a negative impact on student learning and the environmental conditions at the school. According to the National Commission on Teaching and America’s Future (NCTAF) (2003), estimates are that it costs between $4,300 in a small rural district to almost $20,000 in a large urban district for each teacher not retained (NCTAF, 2007; Shakrani, 2008). Regarding teacher turnover, 84% is due to teachers transferring between schools or leaving the profession (Alliance for Excellent Education, 2006), and the annual national cost of teacher turnover in public schools is more than several billion dollars a year (NCTAF, 2007). Virtually everyone in the educational system, including students, parents, and taxpayers, are affected by teacher turnover. This turnover makes it difficult to measure intangible effects of teacher turnover on a school, such as loss of morale, cohesion, and instructional quality in schools (Boyd et al., 2011; Luther & Richman, 2009).

Leading researchers, such as Ingersoll, Cochran-Smith, and Darling-Hammond, have conducted studies on teacher retention and attrition and have documented current trends: often, when given the opportunity, many teachers choose to move to a more preferable school or leave the profession (Boyd, Lankford, Loeb, & Wyckoff, 2005; Hanushek & Rivkin, 2004; Ingersoll, 2001, 2002; Ingersoll & Smith, 2003; Scafidi, Sjoquist, & Stinebrickner, 2007). Keigher (2010) indicated an increase from 6.6% attrition in the mid-1990s to

AMY L. SEDIVY-BENTON AND CARRIE J. BODEN-MCGILL


8% in 2008-2009 (O’Donovan, 2011; Snyder & Dillow, 2011). Individuals who chose teaching as a career often leave in the first years of teaching (Ingersoll, 2001), and teachers with more education (e.g., a master’s degree) are more likely to leave the profession than are their less-educated counterparts (Grissmer & Kirby, 1997; Scafidi et al., 2007). The problem is not a shortage of teachers; rather, the issue is keeping teachers in the field (Darling-Hammond, 2003; Ingersoll & Smith, 2004). According to the U.S. Department of Education’s examination of departures, in 2003-2004, 56% of those teachers who left teaching cited job dissatisfaction as the reason and indicated a desire to find an entirely new career (U.S. Department of Education, National Center for Educational Statistics, 2004). Results were that in this country, almost four million individuals could teach based on their training but choose not to do so (Curran, Abrahams, & Manual, 2000). The environment that these teachers work in could potentially explain this phenomenon. According to Farber (2012):

We have an anti-teacher climate that has only worsened since 2010…The situation has only gotten worse, with layoffs, pay cuts, anti-union sentiment, program cuts and strict mandates that are part of federal education laws. If we are to make any reform or new initiative work in education, we have to create schools that are supportive, humane, dynamic, and creative. (para 4) Several researchers identified working

conditions to be a significant factor in teachers’ decisions to leave the field (Bronfenbrenner, 1976; Donaldson & Johnson, 2010; Hanushek & Rivkin, 2004). A myriad of factors contributing to teachers’ decisions to leave include too little support for teachers, student discipline problems, low funding for schools and low pay for teachers, lack of teacher influence, low respect for the profession, teacher self-efficacy, job stress, staff relations, effective school leadership, and safety (Darling-Hammond, 2003; Hirsch & Emerick, 2007; Ingersoll, 2007; Klassen & Chiu, 2011; Kopkowski, 2008; Loeb, Darling-Hammond, & Luczak, 2005). Working conditions include administrative support, integrity of the classroom, resources for teaching, and the amount of input that teachers feel they have when making decisions (Ingersoll, 2001, 2002). Workplace conditions also include items such as class size, student achievement, discipline problems, and parental involvement (Bronfenbrenner, 1976). Administrative support, in particular, can greatly affect the rate of teacher turnover in a school setting (Greiner & Smith, 2009; McLaughlin, Pfiefer, Swanson-Owens, & Yee, 1986). Ferriter and Norton (2004) pointed out that “administrators have a great deal of influence over school climate

and teacher efficacy,” (p. 19) and the most effective administrators are supportive. Darling-Hammond (2003) added that school leadership may have a magnetic effect, and teachers will search for and stay in the work environments where they are supported and are most able to excel. Conversely, McLaughlin et al. (1986) established that often teachers feel that they have inadequate opportunities for personal success, and a lack of administrative support exists. This lack of support may negatively influence a teacher’s intention to stay in or leave the teaching profession.

Little and McLaughlin (1993) examined a mathematics collaborative as well as other subjects and identified factors that primarily focus on support and collaboration within the school with teachers and their peers. Based upon interviews and surveys, they noted that teachers work better with support, and they need to feel that they fit into the organization. Similarly, Kalleberg and Mastekaasa (1998) discovered that employees who felt to be “part” of the organization were more likely to remain. Not only does the feeling of involvement influence the teacher’s career decision, it can also have positive impact on student achievement. Teachers who are more vested and involved with their schools tend to pass this investment on to the students (Huberman, 1989, 1995). When teachers feel a part of the process, they are often more willing to stay.

Researchers also have provided evidence that student characteristics can influence teachers’ decisions (Boe, Bobbitt, Cook, Barkanic, & Maislin, 1998; Fowler, 2004; Grissmer & Kirby, 1997; Sclan, 1993). Indicated in these works was that teachers who leave the profession are working in schools that have high proportions of low-achieving, low-income students. Loeb et al. (2005) indicated that a school’s racial make-up as well as low-income student population could predict teacher turnover.

National and local personnel policies can contribute to teachers’ decisions. In 2001, the implementation of the NCLB Act held states accountable for student outcomes. No Child Left Behind became a federal law, reauthorized in 2010, with revisions to adjust for shortcomings in the original plan. However, the focus remains on accountability. In 2009, the Obama Administration initiated Race to the Top; this $4.35 billion fund is in place to serve as an initiative for states to use innovative ways to show significant improvements in student outcomes. In addition to the race to the top, the Elementary and Secondary Education Act (ESEA) also offered school districts Teacher Incentive Fund (TIF) grants for implementing performance based programs for high need schools (U.S. Department of Education, 2009; 2012). With the implementation of NCLB, the number of alternatively certified teachers



entering the classroom has risen. The NCLB requirement of highly qualified indicates that teachers must hold a degree, pass a required state- mandated exam, as well as teach in their subject area (U.S. Department of Education, 2005). Retention Strategies With the current climate of the 2012 Presidential administration, an intense focus has been placed on the premise that money will be a significant factor in the improvement of student achievement. The thinking seems to be that with TIF incentives, teachers will stay in the profession longer. However, alluded to in the literature was that incentive pay does not influence the decision to stay in the profession. Rather, salary is significant only when it is not attached to any type of student achievement. The monetary factor seems to be more important during the beginning of their teaching career (Brewer, 1996; Darling-Hammond, 2003; Hanushek, Kain, & Rivkin, 1999), and researchers indicated that when teachers are paid a lower salary, they are more likely to move on to opportunities that provide more from a monetary standpoint (Fresko & Alhija, 2009; Hung & Smith, 2008). In addition to researchers documenting the effects of teacher salary, authors of other studies primarily focus on monetary policies and methods for teacher retention (Clotfelter, Glennie, Ladd & Vigdor, 2006).

A second, less-than-effective retention strategy has been to offer bonuses to teachers to work in poorly performing schools. In South Carolina, districts offered teachers an $18,000 bonus to work in the weakest schools. Only 20% of the teachers who were offered the bonus lasted for the length of the contact (Kopkowski, 2008). Similarly, in Massachusetts, districts offered teachers a $20,000 bonus over a 4-year period. Most teachers decided that the net extra income was not worth moving from a preferred school to a less desirable one (Kopkowski, 2008). According to the Education Commission of the States, other retention strategies include scholarship programs, loan forgiveness programs, tax credit or mortgage assistance, and stipends (Glennie, Coble, & Allen, 2004). Unfortunately, few of the programs have been effective in retaining teachers (Hammer, Hughes, McClure, Reeves, & Salgado, 2005). It appears in the case of teacher retention, environmental factors outweigh the incentive of extra income. Conceptual Model

The conceptual model of teacher retention and attrition used in this research is grounded in Bronfenbrenner’s (1979) Ecological Theory. The model follows common characteristics that have been associated with teachers leaving the field at both the teacher and organizational levels.

Bronfenbrenner suggested that individuals are not only affected by their immediate surroundings, but also their mesosystem, connections with their microsystem such as other teachers working in school, and the exosystem in which the teacher does not function directly, such as the federal government. In the past, Darling-Hammond (2000, 2003) and others have primarily focused only on the contribution of individual teacher characteristics and how they contribute to attrition. In turn, little information exists about how teachers and their workplaces interact and contribute to their decision to leave the classroom. The organizational factors or school characteristics need to be considered because they encompass the school level at which the teacher works, as does locale, poverty level, and control and influence that teachers have within the school.

Teachers in various districts might perform similar work; however, they do this work in different workplaces. These schools vary in both location and student demographics, and the schools also differ in what types of management or administration that are present. Based upon this framework and indication that schools are different from one another, examined in this research were both the teachers and their environments. By studying the teachers and their environments, we considered how the differences of teachers within a school, the environments across schools, and individual and environmental factors interact to affect a teacher’s intention to remain in the profession. Research Question

The question guiding this research was: Which individual and environmental factors most heavily influence teachers’ intentions to remain in or leave the profession? We addressed this question by employing HLM and allowing for the separation of individual and environmental factors.

Method

Research Design

To examine the factors in the work environment that influence teachers’ intentions to remain in or to leave the profession, we examined the most recent dataset from 2007-2008 SASS by the NCES. This comprehensive dataset contains information regarding teachers and their perceptions. We employed a hierarchical linear model in which two levels of data were prepared: level-1 contained individual information about the teacher, characteristics, demographics and their perceptions of the work environment and level-2 contained information about the characteristics of where they work, such as autonomy and control. This nested design allows for the variance between and within school to be extracted. The



combination of these variables described the variation between teachers and their choices. Teachers’ intention to remain in the profession was used as the dependent variable; the continuous nature of this variable allows a range to be established and allowed for better understanding of the impact of the independent variables. Equations presented represent hypothesized relationships between the predictive variables, such as contextual work variables–influence, support, and control–on the outcome of a teacher’s intention to leave the profession. Data

The Schools and Staffing Survey restricted dataset was secured from the NCES to allow us to conduct this analysis. This dataset is the largest, most extensive survey of K-12 school districts, schools, teachers, and administrators in the United States today. Each cycle of SASS administers survey questionnaires to a random sample of approximately 53,000 teachers, 12,000 principals, and 4,500 districts, representing all types of teachers, schools, and districts in all 50 states. These data, nationally representative at the school level, are based on a complex survey design that was administered by the NCES to national probability samples. Schools are the primary sampling unit in SASS. The public school sampling frame was based on Common Core of Data, a file of information collected annually by NCES from all state education agencies and believed to be the most complete public school listing available at the time of sample selection. Criteria for inclusion in the survey at the school level were set at least three teachers per school in order to protect the participants. During the 2007-2008 school year, close to 51,000 public school teachers completed the survey. These data contain a substantive subsection targeting those individuals who are involved with any school-level aspect of education, such as principals and teachers. The dataset includes training experience, mentoring programs, school structure, and subject taught for participants. The teaching-focused sections of the survey include questions about employment history, job changes, and school climate. For the purposes of this study, only those teachers who indicated they would either remain or leave the profession were kept in the sample. Teachers who were undecided or did not provide sufficient data were removed. To adhere to the SASS definition of a teacher, this sample includes any individual who was a full-time K-12 teacher. Part-time, long- and short-term substitutes, student teachers, teacher aids, and non-teaching employees were not included. Based on these criteria, a sample of 20,324 public school teachers were identified in this dataset and included in this analysis.

Teachers’ perception of workplace: Created variables. The SASS survey contains several items that are directed at the amount of involvement that a teacher has within the school, how much involvement a teacher perceives s/he has within the school, and the teacher’s perception of support. Items that addressed this issue were plentiful, and 20 items addressed how much control, support, or influence teachers had within their schools. These specific items were removed from the survey because they indicated how the teachers viewed their workplace environments. It would be inappropriate to leave all of these items as individual variables as they would provide inaccurate results due to multicollinearity. The items that are directed at these issues were analyzed via a factor analysis with a Primary Components Factoring. This method was used because it reflects both the common and unique variance of the variables. A Varimax rotation was used because results make it as easy as possible to identify each variable with a single factor (Gorsuch, 1983). The factor scores, as presented in Table 1, then were taken from the factor analysis and used to create a composite variable for teacher influence within the school as perceived by the teachers and a composite variable influence and support as perceived by the teachers.

This factor analysis on the teacher survey variables yielded three factors. Questions that were reported by the teacher regarding influence clustered together under Factor 1, which was renamed teacher influence on school. Factor 2 displayed factor coefficients of items pertaining to teachers’ perception of control and was renamed teacher perception of control. For Factor 3, items regarding the teachers’ perception of management and support clustered together, and this factor was renamed teacher’s perceived support. The factor scores from these items then were used to create a composite variable for each of the factors, which in turn were used in the analysis. These newly created composite variables were tested for score reliability. The newly created variables produced high reliability scores, including α=.84 for teacher influence on school, α=.76 for teacher perception of control, and α=.76 for teacher’s perceived support. Because these newly created variables provided high reliability scores, these variables were used in the analysis. Data Analysis

A hierarchical linear regression model was used on the SASS data to examine teachers’ intentions to leave the profession. Individual characteristics that were included in level-1 as control variables were: gender, race/ethnicity, new to teaching, highest level of education, and salary. To explore the variables that contribute to a teacher’s intention to leave the profession due to



contextual factors or workplace conditions, level-2 variables were included. Level-2 variables consist of: school level, locale, poverty level, violence in the school, teacher’s perception for and control in the classroom, teacher’s perception of support in

the school, teacher’s perception of influence at school, and the perception of control that teachers feel they have in their working environment.

Table 1 Rotated Component Factor Matrix for School Influence/Teacher Perceptions Component

Factor 1 Teacher Influence at

School

Factor 2 Teacher Perception of

Control

Factor 3 Teacher Perceived

Support Influence-Performance Standards .625 .429 .320

Influence-Curriculum .596 .419 .066

Influence-School budget .531 .325 .058

Influence-Discipline .686 .487 .182

Influence-Teacher Hiring .568 .268 .087

Influence-Teacher Evaluation .571 .054 .207

Influence-Prof Dev Content .605 .105 .115

Control-Selecting Materials .111 .734 .328

Control-Selecting Content .095 .664 .296

Control-Selecting Techniques .158 .5657 .142

Control-Evaluating and Grading .286 .739 .144

Control -Disciplining Students .113 .642 .238

Control-Homework to be Assigned .086 .570 .201

Teacher- Staff Expectations .240 .394 .731

Teacher-Admin Supportive and Encouraging

.403 .143 .867

Teacher-Principal Support .184 .073 .723

Note: Rotated Factor Matrix Extraction Method: Principal Component Factoring. Three factors extracted. Five iterations required.



Results Hierarchical Regression

To determine what factors contribute to teachers leaving the profession, a three-step hierarchical model was tested. The first block of variables included demographic information about the teachers such as age, gender, ethnicity, degree level, education, and years in the profession. The second block included information about their workplace, such as wage. These variables were tested at level-1 to determine statistical significance. Upon adding the third block school-level characteristics were entered at level-2. Variables in the third block included mean responses from teachers and principals on perceptions of school violence, influence on decision-making, control over the environment, effectiveness of school management, support, salary, school-level (elementary or secondary), locale, and poverty level (measured by number of students receiving free and reduced lunch). Teacher Turnover

To address if teachers will decide to leave the profession or not, the outcome variable addresses the question of how long these teachers intend to remain in teaching. When asked if they would remain in teaching, their possible response options were: 1= As long as I am able, 2 = Until I am eligible for retirement, 3 = Will probably continue unless something better comes along, 4 = Definitely plan to leave teaching as soon as I can, and 5 = Undecided at this time. Response number 5 did not provide a definitive decision; thus, teachers with this response were removed from the data. Hierarchical linear models were constructed and tested to determine the effects individual and school characteristics have on teachers leaving the profession. The sample used in these analyses comprised 20,794 full-time teachers and 5,964 schools.

The first model created in HLM contains no level-1 or level-2 predictors, commonly called the unconditional model or a one-way analysis of variance (ANOVA) with random effects because it separates variability into within-school and between-school components (Raudenbush & Byrk, 2002). This preliminary step provides information regarding whether enough variance exists to indicate the presence of statistically significant differences among schools in teachers’ perceptions and their intentions on leaving the profession. The equations for the one-way ANOVA model are:

(2.1) Level 1 Yij = β0j + rij Level 2 β0j = γ00 + u0j

In this equation, Yij is the outcome for teacher i in school j; β0j is the intercept or school mean teacher response for whether they intend to leave the teaching profession; rij is the teacher-level residual or the difference between the response of teacher i from the average response of school j; γ00 is the grand mean response for the outcome across all schools; and u0j is the level-2 residual or the difference between the mean response of school j and the grand mean (Raudenbush & Byrk, 2002). Results from this unconditional model are presented in Table 2.

For the fixed effects, the weighted least squares estimate for the grand mean of teachers choosing to leave the profession was = 1.79. The grand mean indicated, that on average, teachers responded that they intended on remaining in the teaching profession. The within-school variance was σ2 = 0.63, and the between-school variance was τ00 = 0.02. Because τ00 was statistically significant from zero, the average teacher’s intention of leaving or remaining in the profession differed from the different school means. Based on the information in Table 3, this null model shows that the estimate for between-school differences on the outcome is statistically different from zero (χ2 = 6584.13, p < 0.001). Thus, schools differed in their average teacher response for their intention to leave the profession.

The plausible range of values for the average teacher intention of leaving the profession, at the 95% confidence interval, was 1.51 to 2.08. On the scale, a response of “1” represented the teacher’s intention to stay in the profession as long as possible, whereas a response of “4” represented the teacher’s intention to leave the field as soon as possible. Consequently, on average, the teachers indicated that they would remain in the profession.

The interclass correlation represents the proportion of variance in the outcome that is between schools (Raudenbush & Byrk, 2002). The equation is:

(3.1)

(3.2) This analysis produced an intraclass

correlation coefficient of .032, which indicated that 3% of the variance in teacher’s responses was between-schools and 97% was within-schools. From this outcome, it can be inferred that much more within school variation was present than between school variation on a teacher’s intention to leave the profession.

ρ = τ 0 0 τ 0 0 + σ 2

ρ = . 0 2 1 . 0 2 1 + . 6 2 4

= . 0 3 2



Table 2 Results from the One-Way ANOVA Model Fixed Effect Coefficient SE Average school mean, γ00 1.797 .005 Random Effect

Variance Component df χ2 p-value

School mean, u0j

.021 5963 6584.13 < .0001

Level-1 effect, rij .623 Table 3 Effects of Personal Characteristics on a Teachers Decision to Remain in the Profession

Model 0 Model 1 Model 2

Constant 1.797 (p < .0001) 1.776 (p < .0001) 1.816 (p < .0001)

Teacher Demographics

Male 0.101* 0.104*

New teacher 0.109* 0.139*

Black 0.004 0.008

American Indian -0.028 -0.022

Asian -0.009 -0.020

Hispanic -0.025 -0.034

Teacher Education

Alt Cert -0.018

Degree 0.019

Earnings -0.003*

Variance Component

σ2

.623 τ2

.021

σ2

.622 τ2

.019

σ2

.605 τ2

.028

* statistically significant at p < .05



Due to the statistical significant variation both

within and between schools, variables were entered at level-1 and level-2. The null model showed a large percentage of the variation lies within the schools. To determine which teacher variables contribute statistically significantly to a teacher’s intention to leave the profession, teacher variables were entered at level-1. Proceeding with the analysis, variables were added in blocks starting with personal characteristics of the teachers and then education of the teachers. Variables that were not statistically significant were deleted from the model. This resulting model was:

(4.1) Level-1 Yij = β0j + β1jX1ij+ β2jX2ij +…..+βQjXQij+rij (4.2) Level-2 βqj=γq0+γq1W1j+γq2W2j+ ….γqSqWSqj+uqj

In Model 1, teacher characteristics were

added to the unconditional model to determine if a decrease was present in the variance, σ2, at level-1, the teacher level. The proportion of variance explained at level-1 was found by using the following formula:

(5.1) (5.2) = .04

Table 3 presents the results of Model 1.

Gender and teaching for less than 3 years accounted for 4.7% of the variance in teachers’ intentions of leaving the profession at the teacher level, leaving 95.3% of the within-school variance as unexplained. This model also suggests that males’ intention of leaving the profession was greater than females’ intentions, and the intentions of staying in the profession of new teachers were greater than were those of teachers who had been in the profession for longer than 3 years.

In Model 2, teacher characteristics also were combined with teacher education. Male teachers had a greater intention of leaving the profession than did females, and new teachers were less likely to remain in the profession than were teachers who had been teaching for longer than 3 years. This finding was consistent with Darling-Hammond (2000), who concluded that beginning teachers leave the profession at a higher rate than do those teachers who have been teaching for more than 5 years. Another finding was that teachers who earned a higher salary had greater intentions of remaining in the profession. By including these

additional teacher education characteristics to the model, the variance accounted for within-school was

(6.1) (6.2) = .028

The addition of teacher education

characteristics accounted for an additional 2.8% of the within-school variance when compared to the null model, leaving 97.2% of the within-school variation as unexplained.

Upon examining both of these models, we discovered that the race/ethnicity of a teacher was not statistically significant and thus did not impact their intentions to leave the profession. Alternative certification also was not statistically significant in this model, suggesting that teachers’ intentions to leave the profession are not influenced by their route of certification. This finding regarding alternative certification is inconsistent with the literature in which teachers who do not pursue traditional routes of certification leave the profession at a higher rate than do those teachers who receive their certification via a traditional route such as a 5-year preparation program (Darling-Hammond, 2001). Overall, each step of the model of within-school variation explained slightly more of the variance with Model 3 explaining approximately 3% more variation than the unconditional model did on its own.

All variables were fixed in level-2 except for earnings because of the lack of between school variation and the fact that these variables were dummy coded as 0 or 1. Due to the lack of significant between-school differences, teacher earnings were kept as a fixed effect in the full model.

Full Hierarchal Linear Model

The personal and educational variables that had the strongest effect on a teacher’s intention of leaving the profession at level-1 were selected to provide a preliminary model. Workplace conditions and school context variables were entered at level-2. An exploratory model was implemented that examined the variability of the teachers’ decision to leave the profession not only within the school but also between the schools. The personal and educational variables that were statistically significant in the prior models were kept at level-1, and workplace conditions and school context variables were entered at level-2. These two models are placed together to give a full HLM.

ˆ σ 2 A N O V A − ˆ σ 2 r a n d o m c o e f f i c i e n t ˆ σ 2 A N O V A

ˆ σ 2 A N O V A − ˆ σ 2 r a n d o m c o e f f i c i e n t ˆ σ 2 A N O V A



The Male and New to teaching variables were entered at level-1 uncentered. The teachers’ income that they earn at the school (earnings) was group mean centered. In this full model, only the intercept of the outcome variable has random effects. Earnings were entered with random effects but it was determined to be not statistically significant. The remaining variables at level-1 were dummy coded as 0 or 1 and thus were treated as fixed.

To explain the school-level variations in the intercept slope, level-2 school characteristics were entered. These variables were selected based upon past study findings, such as Smith and Ingersoll (2004), who examined organizations and characteristics of those organizations that lead to a teacher’s decision to leave the profession. Free and reduced lunch (lunch) was entered grand mean centered; school level (level-2) was entered uncentered and Influence in school (INFSCH_1), support of management (support), student delinquency (SDLQ_T_1), salary (SALARY), influence on school (influence), and control in school (control) all were entered grand centered. The final model was as follows:

(7.1) Level-1 Model Y = β0 + β1(Male) + β2(NEWTCH) + β3(earnings) + r (7.2) Level-2 Model β0 = γ00 + γ01(Lunch) + γ02(School level) +

γ03(Student Delinquency)+γ04(Support) + γ05(School Influence) + γ06(Salary) + γ07(Influence) + γ08(Control) + u0

β1 = γ10 β2 = γ20 β3 = γ30 β4 = γ40 β5 = γ50

Table 4 contains the following teacher-level variables that were determined to be statistically significant predictors: gender, and if they were a new teacher or not. For school-level variables predicting the variation in school means on teachers’ intention to leave the profession, the school level (elementary, secondary), poverty of the school, reported teacher influence in school, mean perception of school support, mean student delinquency, mean salary, mean teacher control, and mean teacher influence statistically significantly predicted the teacher-level intercept. The proportion of variance that is explained by the model for the intercept term was determined using the formula suggested by Raudenbush and Byrk (2002):

(8.1) Proportion Variation Explained in

βq =

(8.2) Proportion Variation Explained in βo = = .42

The proportion of variance explained, in β0

was 43%. Forty three percent of the original random variation in the intercept was explained by the teacher’s perception of the school, influence, control, support, the student delinquency in the school, poverty of the school, school level, and the salary offered. These results suggest that the mean intention for all teachers in schools was related to school-level predictors. Summary of Results

It is difficult to pinpoint one factor that influences a teacher’s decision to remain in the profession; however, a few key variables present themselves as contributors to this decision. Although holding everything else constant, the following conclusions were made: Based on the analysis, the teachers in the sample seemed to have greater intentions of remaining in the profession than to leave. More than 80% of the teachers included in the sample responded that they would remain the profession for as long as they were able, or until they were eligible for retirement. Gender was a statistically significant variable (β = .084), indicating that females were more likely to intend to remain in the profession; new teachers, those teachers who have been in the field for 3 years or less, also indicated they intended to remain in the profession, as did elementary school teachers. The level of the school was statistically significant with a coefficient of β = -.074, indicating that elementary school teachers had greater intentions of remaining in the profession. In addition, salary (β = -.006) was a statistically significant indicator regarding whether or not a teacher had intentions of remaining in the profession. The more that teachers were paid, the more were their intentions to remain in the profession increased. The data also indicated that schools with a high student population that received free and reduced lunch had teachers with lower intentions of leaving the profession (β = .056).

It is worthwhile to recognize the variables that are listed above. It is equally important to recognize that all of these statistically significant variables were either those characteristics over which school administrators have no control, such as whether the teacher is new to the teaching field, or the poverty level of the school, or they are factors that will take a significant amount of



planning and time to initiate a change, such as salary. However, the variables that are of interest are those items that the district and school administrators have both control over and can initiate change in a short amount of time, because that is what is desired in the current climate of education. The factors over which both school and district administrators have direct control are factors that exist in the context in which they work. When examining these contextual factors, as identified by the created variables that address teachers’ perceptions of support, influence, and

control at the school and in their classrooms, we determined that when teachers believed that they had a supportive environment in which to work, they had greater intentions of remaining in the profession (β = -.013). Similarly, when teachers’ perception of influence that they had within their schools was high (β = -.007), they were more likely to remain in the profession, and teachers who believed they had control in their schools (β = -.056) had greater intentions of remaining in the profession.

Table 4 Results from the Intercepts- and Slopes-as-Outcomes Model

Fixed Effect Coefficient SE Model for School Means, β0 Intercept, γ00 1.847 0.024 Percent of Free Lunch, γ01 -0.056* 0.025 School Level, γ02 -0.074* 0.013 Mid-size City Local, γ03 -0.022 0.027 Fringe of Large City Local, γ04 -0.029 0.026 Fringe of Mid-size City Local, γ05 -0.016 0.028 Large Town, γ06 -0.053 0.049 Small Town, γ07 -0.018 0.026 Rural, γ08 -0.010 0.026 Student Delinquency, γ09 0.018* 0.003 Salary, γ10 -0.006* 0.001 Influence in School, γ11 -0.007* 0.002 Support by Management, γ012 0.013* 0.002 Influence by Teachers, γ013 -0.022* 0.004 Control by Teachers, γ014 -0.056* 0.006 Model for Male – Choice Slope, β1 Intercept, γ10 0.084* 0.013 Model for New Teacher – Choice Slope, β2 Intercept, γ20 0.145* 0.022 Model for Earnings – Choice Slope, β3 Intercept, γ30 -0.003* .000 Intercept, γ50 -0.044* .016

Random Effect Variance Component df χ2 p-value School Mean, u0j 0.012 5950 6209.001 0.010

Level-1 Effect, rij 0.619

Discussion

These findings provide functional results when discussing potential reasons why teachers might choose to leave the profession. Not only do these results provide some basic information to policymakers, but this information is helpful to school administrators as well. When hiring, principals might more specifically consider their work environments and how well their candidates will fit within these environments to avoid potential turnover. The consideration of the work

environment can assist in their budget planning and how their funds may be used in the future. These significant contextual factors are factors over which school administrators have control, and they can initiate immediate change without having to go through many administrative channels and wait for approval to implement measures that might produce change. For example, principals in a high poverty school might want to hire new teachers and then give those teachers support and influence within the school environment. This additional level of support might not seem like a



simple task, but the notion of control and support can go a long way in providing a stable teaching force in the school environment (Huang & Waxman, 2009; Johnson, 2006; Sclan, 1993).

One finding that is inconsistent with the literature is that of poverty. Cited in previous works (e.g., Ingersoll, 2001), authors have implicated poverty level of the school as a contributing factor to teachers choosing to leave the profession. In this study, the opposite effect was present; teachers in schools with a high level of poverty were more likely to stay. These inconsistencies might be because the schools that fit the poverty profile are more likely to be in urban areas. It is plausible that the teachers who work at these schools do not have the mobility that a teacher at a low poverty suburban school might have. The relationships between the teachers’ personal characteristics and the characteristics within the school context and how they interact seem to provide the most information regarding why a teacher intends to remain or to leave the profession.

Implications

One final question that remains is: what practical and policy implications can be taken from this study that would reduce the amount of turnover in teachers? The results from this analysis indicate that contributing factors associated with individual teacher information (e.g., age, gender, education) are predictive of teachers’ intentions, and these factors cannot be manipulated. However, relationships between where the teachers worked and their intention of leaving the field were influenced by the experiences that they had within the school.

The focus of these findings indicates that attention should be paid to the attitudes and the perceptions of teachers’ working environments, how teachers perceive their role in the school when it comes to influence and control, and the support that teachers receive in their teaching roles. This work establishes that the amount of control and influence that teachers have within their environments can affect their decision to stay or to leave the field (Mastekaasa, 1998; McLaughlin et al., 1986). Our research also supports the argument that schools potentially can reduce teachers leaving the profession by allowing them maximum involvement in school decisions as well as by providing them with some control over their classrooms and curricula. These findings also are relevant to federal mandates that are currently on the rise, because these mandates primarily focus on student scores and financial rewards rather than focusing on those factors that will keep good teachers in the field, such as their workplace environment.

Limitations and Suggestions for Future Research

In our study, we made an effort to create a

research design that allowed us to capture the complex dynamics regarding reasons that teachers leave the profession. However, several limitations were present. First, it is important to note that throughout this study, teacher intentions of leaving the profession rather than the actual act of leaving the profession were discussed. Thus, it should be stated that intentions of leaving the profession might not necessarily be inclusive of those teachers who actually leave the profession, and direct inference is not warranted. Second, because differences were present among the schools in which these teachers teach and this influences their level of commitment to the field, it would be worthwhile to investigate what these schools are doing differently. The variables in this study could be more precisely defined and measured.

Conclusion

The focus on the status of education in the

United States is not something that will be going to the wayside anytime soon. The NCLB Act and Race to the Top (U.S. Department of Education, 2012) focus more on monetary rewards, and less focus is on those environments in which teachers work. Meaningful and lasting educational reform is a lengthy process, and many factors that lead to teachers leaving the field are out of the immediate control of policymakers and school administrators, who struggle with abysmal state budgets, policy implementation lag, and other constraints. It is unlikely that factors such as facilities and pay will be improved in the near future. Consequently, rather than focusing solely on student learning, policymakers and administrators also need to consider contextual factors to improve teachers’ work environments. Creating an environment that supports teachers and allows a commitment to teaching is crucial to student success (Feng, 2009).

With tight budgets developing and the exorbitant cost of teacher turnover, this research provides policymakers with a better understanding of the environmental factors contributing to the loss of teachers. Armed with the understanding that teacher influence on school, perception of control, and perceived support are vital factors to retention, policymakers and administrators may develop practices to increase teacher retention, improve teacher quality and, thus, improve the quality of our nation’s schools The lead editors for this article were John R. Slate and Anthony J. Onwuegbuzie.



References Alliance for Excellent Education. (2006). Tapping

the potential: Retaining and developing high-quality new teachers. Washington, DC: Author. Retrieved from www.all4ed.org/publications/

Aud, S., Hussar, W., Johnson, F., Kena, G., Roth, E., Manning, E., Wang, X., & Zhang, J. (2012). The Condition of Education 2012 (NCES 2012-045). U.S. Department of Education, National Center for Education Statistics. Washington, DC. Retrieved from http://nces.ed.gov/pubsearch

Boe, E., Bobbitt, S., Cook, L., Barkanic, G., & Maislin, G. (1998). Teacher turnover in eight cognate areas: National trends and predictors. Philadelphia, PA: University of Pennsylvania, Center for Research and Evaluation in Social Policy.

Borman, G. D., & Dowling, N. M. (2008). Teacher attrition and retention: A meta-analytic and narrative review of the research. Review of Educational Research, 78, 367-409. doi:10.3102/0034654308321455

Boyd, D., Grossman, P., Ing, M., Lankford, H., Loeb, S., & Wyckoff, J. (2011). The influence of school administrators on teacher retention decisions. American Educational Research Journal, 48, 303-333. doi:10.3102/0002831210380788

Boyd, D., Lankford, H., Loeb, S., & Wyckoff, J. (2005). Explaining the short careers of high-achieving teachers in school with low-performing students. The American Economic Review, 95, 166-171. doi:10.1257/000282805774669628

Brewer, D. J. (1996). Career paths and quit decisions: Evidence from teaching. Journal of Labor Economics, 14, 313-339. doi:10.1086/209813

Bronfenbrenner, U. (1976). The experimental ecology of education. Educational Researcher, 5(9), 5-15. doi:10.2307/1174755

Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press.

Clotfelter, C. T. Glennie, E. , Ladd, H. F.& Vigdor, J. L. (2006) Would higher salaries keep teachers in high-poverty schools? Evidence from a policy intervention in North Carolina. (NBER Working Paper No. 12285). Cambridge, MA National Bureau of Economic Research.

Curran, B., Abrahams, C., & Manual, J. (2000, January 25). Teacher supply and demand: Is there a shortage? Washington, DC: National Governor’s Association, Center

for Best Practices, Education Policy Studies Division.

Darling-Hammond, L. (2000). Futures of teaching in American education. Journal of Educational Change, 1, 353-373.

Darling-Hammond, L. (2001). Standard setting in teaching: Changes in licensing, certification, and assessment. In V. Richardson (Ed.), Handbook of research on teaching (4th ed., pp. 751-776). Washington, DC: American Educational Research Association.

Darling-Hammond, L. (2003). Keeping good teachers: Why it matters what leaders can do. Educational Leadership, 60(8), 6-13.

Donaldson, M., & Johnson, S. M. (2010). The price of misassignment: The role of teaching assignments in Teach for America teachers’ exit from low-income schools and the teaching profession. Educational Evaluation and Policy Analysis, 32, 299-323. doi:10.3102/0162373710367680

Farber, K. (2012). My view: Six ways to retain great teachers. Retrieved from http://schoolsofthought.blogs.cnn.com/2012/06/19/my-view-six-ways-to-retain-great-teachers/?iphoneemail

Feng, L. (2009). Opportunity wages, classroom characteristics, and teacher mobility. Southern Economic Journal, 75, 1165-1190.

Ferriter, W., & Norton, J. (2004). Creating a culture of excellence. Threshold, 2(1), 18-21.

Fowler, W. J., Jr. (Ed.). (2004). Developments in school finance, 2003: Fiscal proceeding from the Annual State Data Conference of July 2003. (Report No. NCES 2004-325). Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Fresko, B., & Alhija, F. (2009). When intentions and reality clash: Inherent implementation difficulties of an induction program for new teachers. Teaching and Teacher Education, 25, 278-284. doi:10.1016/j.tate.2008.12.001

Glennie, E., Coble, C. R., & Allen, M. (2004, November). Teacher perceptions of the work environment in hard-to-staff schools. Denver, CO: Education Commission of the States. Retrieved from http://www.ecs.org/clearinghouse/55/87/5587.doc

Greiner, C. S., & Smith, B. (2009). Analyses of selected specific variables and teacher attrition. Education, 129, 579-583.

Grissmer, D., & Kirby, S. N. (1997). Teacher turnover and teacher quality. Teachers College Record, 99(1), 45-56.



Gorsuch, R. L. (1983). Factor analysis. Hillsdale, NJ: Lawrence Erlbaum.

Hammer, P. C., Hughes, G., McClure, C., Reeves, C., & Salgado, D. (2005). Rural teacher recruitment and retention practices: A review of the research literature. National Survey of Rural Superintendents and Case Studies of Programs in Virginia. Edvantia, VA: Appalachia Educational Laboratory.

Hanushek, E. A., & Rivkin, S. G. (2004). How to improve the supply of high quality teachers. Brookings Papers on Educational Policy 2004 (pp. 7-44). Washington, DC: Brookings Institution Press.

Hanushek, E. A., Kain, J. F., & Rivkin, S. G. (1999). Do higher salaries buy better teachers? (Working paper #7082). Cambridge, MA: National Bureau of Economic Research.

Hirsch, E., & Emerick, S. (2007). Teacher working conditions are student learning conditions: A report on the 2006 North Carolina teacher working conditions survey. Center for Teaching Quality. Retrieved from http://www.eric.ed.gov/PDFS/ED498770.pdf

Huberman, M. (1989). The professional life cycle of teachers. Teachers’ College Record, 91, 31-57.

Huberman, M. (1995). Networks that alter teaching. Teachers and Teaching, 1, 193-221. doi:10.1080/1354060950010204

Huang, S., & Waxman, H. (2009). The association of school environment to student teachers’ satisfaction and teaching. Teaching and Teacher Education, 25, 235-243. doi:10.1016/j.tate.2008.07.015

Hung, L., & Smith, C. (2008). Why do teachers leave the teaching profession?: Can alternative certificate programs eliminate the U.S. teacher shortage issue? Saarbrücken, Germany: VDM Publishing.

Ingersoll, R. M. (2001). Teacher turnover and teacher shortages: An organizational analysis. American Educational Research Journal, 38, 499-534. http://dx.doi.org/10.3102/00028312038003499

Ingersoll, R. M. (2002). Out of field teaching, educational inequity, and the organization of schools: An exploratory analysis. Seattle, WA: Center for the Study of Teaching and Policy, University of Washington.

Ingersoll, R. M. (2007). The science and mathematics teacher shortage: Fact and myth. NSTA Reports, 18(9), 6-7.

Ingersoll, R. M., & Smith T. (2004). Do teacher induction and mentoring matter? NAASP Bulletin, 88(638), 28-40. http://dx.doi.org/10.1177/019263650408863803

Ingersoll, R. M., & Smith, T. M. (2003). The wrong solution to the teacher shortage. Educational Leadership, 60(8), 30-33.

Johnson, S. M. (2006, Summer). Why new teachers stay? American Federation of Teachers, 9-45.

Kalleberg, A., & Mastekaasa, A. (1998). Organizational size, layoffs and quits in Norway. Social Forces, 76, 1243-1273. http://dx.doi.org/10.2307/3005834

Keigher, A. (2010). Teacher attrition and mobility: Results from the 2008-09 Teacher Follow-up Survey. (NCES 2010 – 353). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved from http://nces.ed.gov/pubsearch

Klassen, R. M., & Chiu, M. M. (2011). The occupational commitment and intention to quit of practicing and pre-service teachers: Influence of self-efficacy, job stress, and teaching context. Contemporary Educational Psychology, 36, 114-129. doi:10.1016/j.cedpsych.2011.01.002

Kopkowski, C. (2008). Why they leave. NEA Today, 26(7), 21-25.

Little, J. W., & McLaughlin, M. W. (1993). Teachers’ work: Individuals, colleagues and contexts. New York, NY: Teachers College Press.

Loeb, S., Darling-Hammond, L., & Luczak J. (2005). How teaching conditions predict teacher turnover in California schools. Peabody Journal of Education, 80(3), 44-70. doi:10.1207/s15327930pje8003_4

Luther, V., & Richman, L. J. (2009). Teacher attrition: Listening to teachers to find a solution. Academic Leadership, 7(4), 29.

McLaughlin, M. W., Pfiefer, S., Swanson-Owens, D., & Yee, S. (1986). Why teachers won’t teach. Phi Delta Kappan, 67, 420-426.

Mastekaasa, A. (1998). Organizational size, layoffs, and quits in Norway. Social Forces, 76(4), 1243-1273.

National Commission on Excellence in Education, (1983). A Nation at Risk: The imperative for educational reform. Washington, D.C.: U.S. Department of Education.

National Commission on Teaching and America’s Future. (2003). No dream denied: A pledge to America’s children. New York, NY: Author.

National Commission on Teaching and America’s Future. (2007). The cost of teacher



turnover in five school districts: A pilot study. New York, NY: Author.

O’Donovan, E. (2011). Is there a teacher shortage on the horizon? District Administration, 47(3), 74-76.

Organisation for Economic Co-operation and Development. (2012). Education at a glance 2012: OECD Indicators. Retrieved from http://dx.doi:.org/10.1787/eag-20120en

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.) Newbury Park, CA: Sage.

Scafidi, B., Sjoquist, D., & Stinebrickner, T. (2007). Race, poverty, and teacher mobility. Economics of Education Review, 26, 145-159. doi:10.1016/j.econedurev.2005.08.006

Schleicher, A. (2010). The case for 21st century learning. Retrieved from http://www.oecd.org/general/thecasefor21st-centurylearning.htm

Sclan, E. (1993). The effect of perceived workplace conditions on beginning teacher’s work commitment, career choice commitment, and planned retention. Ann Arbor, MI: University Microfilms International.

Shakrani, S. (2008). Teacher turnover: Costly crisis, solvable problem. Education Policy Center, Michigan State University. Retrieved from http://0search.ebscohost.com.iiiserver.ualr.edu/login.aspx?direct=true&db=eric&AN=ED502130&site=ehost-live&scope=site

Singer, J. D., & Willett, J. B. (1994). Modeling duration and the timing of events: Using survival analysis to long-term follow-up Studies. In S. L. Friedman & H. C. Heywood (Eds.), Developmental follow-up: Concepts, genres, domains, and methods (pp. 315-330). Orlando, FL: Academic Press.

Smith, T. M., & Ingersoll, R. M. (2004). What are the effects of induction and mentoring on beginning teacher turnover? American Educational Research Journal, 41, 681-714. http://dx.doi.org/10.3102/00028312041003681

Snyder, T. D., & Dillow, S. A. (2011). Digest of Education Statistics 2010 (NCES 2011-015). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.

U.S. Department of Education, National Center for Education Statistics. (2004). The Condition of Education 2004 (NCES 2004–077). Washington, DC: U.S.

Government Printing Office. U.S. Department of Education. (2005). New No

Child Left Behind flexibility: Highly qualified teachers. Retrieved from http://www2.ed.gov/nclb/methods/teachers/hqtflexibility.html

U.S. Department of Education. (2009). Executive summary Race to The Top program. Retrieved from http://www2.ed.gov/programs/racetothetop/index.html

U.S. Department of Education. (2010). ESEA reauthorization: A blueprint for reform. Retrieved from http://www2.ed.gov/policy/elsec/leg/blueprint/index.html

U.S. Department of Education. (2011). Our future, our teachers: The Obama Administration’s plan for teacher education reform and improvement. Retrieved from http://www.ed.gov/sites/default/files/our-future-our-teachers.pdf

U.S. Department of Education. (2012). Teacher incentive fund. Retrieved from http://www2.ed.gov/programs/teacherincentive/index.html

MID-SOUTH EDUCATIONAL RESEARCH ASSOCIATION

The Mid-South Educational Research Association (MSERA) was founded in order to encourage quality

educational research in the Mid-South and to promote the application of quality educational research in

schools. Members of MSERA share interests in educational research, development, and evaluation.

While most members are from institutions of higher education, many others represent state departments

of education, public and private agencies, and public school systems. Graduate students comprise a

significant portion of the membership. A majority of MSERA members are from the six states

represented by the organization, but others are from many other states and several foreign countries. The

MSERA is the largest regional educational research organization in the country.

The organization provides several services for its members. The annual meeting, held every November,

offers many formal and informal opportunities for professional development through special training

courses, sharing of research findings, and programmatic interests with colleagues. Members receive a

subscription to RESEARCH IN THE SCHOOLS and the Mid-South Educational Researcher. The

MSERA also provides recognition and cash rewards for its outstanding paper, an outstanding

dissertation, and professional service.

For access to the index of articles published in Research in the Schools past issues, please

visit our website at:

http://msera.org/rits.htm

In addition, editorials are presented at the website for download and classroom use.

RESEARCH IN THE SCHOOLS Mid-South Educational Research Association

College of Education


Box 2119

Huntsville, Texas 77341-2119

BULK RATE U.S. POSTAGE

PAID PERMIT NO. 3029

BIRMINGHAM, AL

research in the schools - fall 2012

Documents

instructors of research

educational policy

use of research findings

bodenmcgill research

scholarly reviews of

school policy

related articles

topical articles