Using a standardized video-based assessment in a university teacher education program to examine preservice teachers knowledge related to effective teaching
Post on 20-Dec-2016
a b s t r a c t
teaching strategies to their own use is a key inuence to their futuresuccess in the classroom (Cochran-Smith, 2004).
Examining preservice teachers ability to recognize and inter-nalize effective teaching strategies allows teacher education
ipation in the program.This study seeks to understand the effectiveness of imple-
menting a standardized, video-based assessment of teaching acrossteacher education programs to determine preservice teachersability to recognize effective teaching practices. In particular, welooked at the Video Assessment of Interactions of Learning (VAIL,2010), a standardized measure that has been used reliably in alarge study of in-service teachers (Hamre et al., 2012). In the UnitedStates, multiplemeasures of preservice teacher growth are required
* Corresponding author. Curry School of Education, Department of Curriculum,Instruction, and Special Education, University of Virginia, 405 Emmett St., Char-lottesville, VA 22904, USA. Tel.: 1 540 820 0030.
Contents lists available at
Teaching and Tea
Teaching and Teacher Education 33 (2013) 24e33E-mail address: firstname.lastname@example.org (P.D. Wiens).1. Introduction
Preservice teacher education programs emphasize the develop-ment of effective instructional strategies, through pedagogymethods courses, eld experiences, and instructional-technologytraining (Adler, 2008). Recent research highlights that an importantstep in being able to implement effective instructional strategies istheability to recognize thoseeffective teaching inother teachers (vanEs & Sherin, 2002, 2006; Pianta & Hamre, 2009). The extent towhichpreservice teachers are able to internalize and adapt effective
programs to understand the learning progress of its preserviceteachers while also providing an indication of the effectiveness ofthe education program. However, there is a lack of standardizedmeasures examining the learning of teachers in teacher educationprograms across contexts (Zeichner, 2005) and common tools thatcan connect data for different teacher education settings(Grossman & McDonald, 2008; Wiens, 2012). Designing andimplementing measures that assess preservice teacher growththrough a teacher education program is an important aspect ofunderstanding and directing the learning that occurs from partic-a r t i c l e i n f o
Article history:Received 30 January 2012Received in revised form6 November 2012Accepted 24 January 2013
Keywords:Preservice teacher educationPreservice teachersVideo technologyProgram evaluationTeacher effectiveness0742-051X/$ e see front matter 2013 Elsevier Ltd.http://dx.doi.org/10.1016/j.tate.2013.01.010The Video Assessment of Interactions and Learning (VAIL), a video-based assessment of teacher un-derstanding of effective teaching strategies and behaviors, was administered to preservice teachers.Descriptive and regression analyzes were conducted to examine trends among participants and identifypredictors at the individual level and program level. Results from this study demonstrate that a stan-dardized assessment used previously with in-service teachers can be implemented in a teacher educa-tion program. Analysis shows variability in preservice abilities to detect effective teaching strategies andbehaviors that is partially explained by teacher education program factors.
2013 Elsevier Ltd. All rights reserved.< Assessment detects variance in abilit< Part of the variance in performance intify teaching strategies and behaviors.ned by teacher education program factors.< Preservice teachers completed Video Assessment of Interactions and Learning (VAIL).y to ideUsing a standardized video-based assesprogram to examine preservice teachers
Peter D. Wiens*, Kevin Hessberg, Jennifer LoCasaleUniversity of Virginia, USA
h i g h l i g h t s
journal homepage: wwwAll rights reserved.ent in a university teacher educationnowledge related to effective teaching
rouch, Jamie DeCoster
lsevier .com/locate/ tate
Teacfor accreditation of teacher education programs (NCATE, 2008) andthe VAIL may be considered a viable option.
We begin by exploring the need for standardized measures ofeffective teaching and what practices constitute effective practice.We will then examine the implementation of a standardizedmeasure of preservice teacher knowledge of effective teaching witha cross-sectional sample of participants. Finally, we will discuss theresults and viability of the measure for documenting learning in ateacher education program.
1.1. Need for standardized measures in teacher education
Teaching is a nuanced and dynamic exercise that is difcult tocategorically evaluate (Brophy, 1999; Brophy & Good, 1986). Effec-tive teaching includes developing the cognitive skills of studentswhile also being sensitive to the emotional and social needs of thelearner (National Research Council, 2000). A good measure ofteaching identies a teachers ability to engage in instruction thatmeets all of these needs. Such measures, if implemented in astandardized way, provide valuable information for teacherlearning and improvement (Pianta & Hamre, 2009).
A goodmeasure of preservice teacher learning will determine towhat degree a preservice teacher understands the cognitive, social,and emotional aspects of quality instruction. Typically, assessmentsof student learning are collected and analyzed by the teacher ed-ucation program itself. The National Council for the Accreditation ofTeacher Education (NCATE), the largest teacher accreditation bodyin the United States, endorses multiple measures of student growthand performance (NCATE, 2008). These measures include lessonplans, evaluations of content knowledge, assessments of studentteaching, and evaluations of portfolios among others. Takentogether, these measures theoretically demonstrate whether apreservice teacher has developed a minimal level of skills andknowledge of effective teaching in order to implement thisknowledge in a classroom setting. These evaluations are oftenconducted at the end of the teacher education program and providea picture of what the preservice teacher knows.
The problems with these commonly used measures of preser-vice learning are three-fold. First, they are largely subjective. Pre-service teacher-created artifacts such as portfolios and lesson plansare evaluated by teacher educators and often lack rigorous controlsfor variation between assessors. Even evaluation of studentteaching is generally done in a subjective format with universitypersonnel and cooperating teachers judging the performance of thepreservice teacher based on the evaluators own values and per-spectives (Greenberg, Pomerance, & Walsh, 2011). The secondproblem with using end-of-program measures is that they do notprovide any understanding of preservice teacher growth inknowledge and abilities vis a vis understanding the domains ofinstructional quality. Finally, these measures are site-specic anddo not allow for comparison across programs and contexts.
Standardized evaluation measures provide a tool to measurepreservice teachers knowledge and skills within and across set-tings. However, not all standardized measures of teaching knowl-edge or ability can capture growth in these areas throughout ateacher education program. Capturing preservice teachers growthin understanding effective teaching is a difcult task because of thelimited time preservice teachers spend in the classroom. Thismakes it impossible to use year after year value-added models(Glazerman et al., 2010) to measure preservice teacher growth.Additionally, most traditional teacher education programs do notput preservice teachers in a lead position in a real classroom at thebeginning of their program. Without a measure of teaching per-formance at the beginning of the teacher education program
P.D. Wiens et al. / Teaching andobservational protocols (Pianta & Hamre, 2009) will not showpreservice teacher growth. Therefore, the standardized measuresused for studying effective teaching in in-service situations, do nottranslate directly tomeasuring growth in preservice teachers. Thereis need for a standardized measure that builds on the domains ofcurrent observational measures, but can measure growth in pre-service teacher knowledge.
1.2. A standardized measure of effective classroom interactions ethe Classroom Assessment Scoring System (CLASS)
In a 2008 review of teacher education, Adler notes, teachersand researchers have responded to criticism of schools, teachers,and teacher education by looking for the empirical links betweenteacher practices and teacher education and student outcomes (p.332). From this line of empirical research it has become clear thateffective teachers create a positive classroom climate, effectivelymanage classroom time, and deliver high quality instruction andfeedback (La Paro, Pianta, & Stuhlman, 2004). It is possible toexamine these three categories of effective instruction by analyzingthe classroom interactions between students and teachers (Hamre,Pianta, Mashburn, & Downer, 2007). The Classroom AssessmentScoring System (CLASS) was developed as a standardized, reliableand valid measure to capture these components of effective class-room teaching and learning by measuring the quality of in-teractions between teachers and students (Pianta & Hamre, 2009;Pianta, La Paro, & Hamre, 2008).
Pianta and Hamre (2009) conceptualize the CLASS framework asan observation tool that assesses those teacherestudent in-teractions that contribute to student development as a result of theclassroom experience and environment. The CLASS framework(Pianta, La Paro, et al., 2008) divides classroom interactions intothree major domains (see Fig. 1)dEmotional Supports, ClassroomOrganization, and Instructional Supports. Hamre et al. (2012) detailthe empirically- and theoretically-based research behind each ofthese domains. Each of the three domains represents a set of tenspecic dimensions of academic and social supports that are linkedto student development (Hamre et al., 2007; Pianta & Hamre, 2009,see list in Fig. 1). Finally, each of the dimensions is supported byindicators that are demonstrable to the observer (see Fig. 1). Forexample, an observer who sees a teacher provide repetitive andscaffolded feedback to students during instruction would beassessed as appropriate within the Instructional Support domain,the Quality of Feedback dimension, and the Feedback Loopindicator.
The CLASS framework is supported by research in both educa-tion and psychology (Hamre & Pianta, 2007), and is designed to be auseful metric for the systematic research of classroom effects inteacher education (Hamre et al., 2007; Pianta & Hamre, 2009). Asdescribed in Hamre et al. (2012), observers are initially trained toreliability on the tool through a rigorous two day training sessionwhere they learn the CLASS framework and conduct multiplepractice tests. Next, observers must then pass a reliability test, us-ing the tool successfully across multiple classroom situations.Finally, observers complete regular reliability tests to ensure thatthey do not drift from established CLASS coding protocols.
CLASS has been utilized by various researchers as an effectivemeasurement in elementary and secondary classrooms both in theUnited States (Graue, Rauscher, & Shenski, 2009; La Paro et al.,2009; LoCasale-Crouch et al., 2007; Malmberg & Hagger, 2009)and internationally (Cadima, Leal, & Burchinal, 2010; Pakarinenet al., 2010). CLASS-based studies consistently nd associationsbetween observable classroom behaviors outlined in the CLASSprotocol and student development and learning. For example, in alongitudinal study of 147 kindergartners through rst grade, Curby,
her Education 33 (2013) 24e33 25Rimm-Kauman, and Ponitz (2009) found that teachers high in
Teacemotional support had students that demonstrated faster growthin phonological awareness. In another study, Pianta, Belsky,Vandergrift, Houts, and Morrison (2008), and Pianta, La Paro,et al. (2008) examined nearly 800 students in various elementaryclassrooms and found a link between emotional support andreading achievement. Additionally, the Measures of EffectiveTeaching (MET) Project assessed nearly 3000 teachers and found apositive relationship between teachers ratings on CLASS and study
Fig. 1. Classroom Assessment Scoring System (CLASS
P.D. Wiens et al. / Teaching and26value-added estimates (Gates Foundation, 2012). Consequently,several recognized educational research agencies such as The GatesFoundation, Educational Testing Service, and the United StatesNational Institute of Child Health and Human Development haveincluded CLASS as part of their in-service teacher studies (Ewing,2008; Gates Foundation, 2010).
While the use of a standardized observation measure can behelpful in evaluating preservice teaching performance it cannottypically be used to judge teacher growth. Given that preserviceteachers typically do not have the opportunity to take on teachingresponsibilities until the very end of the teacher education expe-rience, an observational measure like CLASS cannot always be usedto document preservice teacher growth. There is a need to examinestudents progressing skills that will translate into effective teach-ing practice. The question remains whether this can be done using astandardized lens? One standardized measure that could poten-tially be used to measure teacher learning is the Video Assessmentof Interactions and Learning.
1.3. Video Assessment of Interactions and Learning (VAIL)
The eld currently lacks standardized measures for analyzingteachers understanding of the domains of effective instruction(Zeichner, 2005). Empirical research is beginning to make a strongcase that the ability to notice effective teaching is foundational tothe ability to conduct effective teaching (Hamre, Downer, Jamil, &Pianta, 2012). Using the Classroom Assessment Scoring System(CLASS) dimensions as a conceptual framework, researchersdeveloped the Video Assessment of Interactions and Learning(VAIL) protocol to assess participants ability to identify effectiveteaching strategies and examples as specied within the CLASSdomains (Hamre et al., 2012). The VAIL builds upon the under-standing that the ability to see and notice good teaching is animportant component of being an effective teacher (Jamil, Sabol,Hamre, & Pianta, 2011). The VAIL uses the ability to identify effec-tive teaching strategies and actions as a measure of a participantsknowledge of effective teaching.
nceptual framework (Pianta & Hamre, 2009, p. 111).
her Education 33 (2013) 24e33Hamre et al. (2012) rst used VAIL as a component of a largerstudy in which participants in a treatment course were evaluatedonwhether they had improved skills (compared to a control group)to detect effective teacherestudent interactions through video. Inthe VAIL component of the Hamre et al. study, CLASS was used asthe treatment measure in the form of a CLASS-based cours...