reflection as a visible outcome for preservice teachers

ARTICLE IN PRESS

Teaching and Teacher Education 20 (2004) 243–257

*Correspondin

E-mail addre

suzanne.mccotte

0742-051X/$ - see

doi:10.1016/j.tat

Reflection as a visible outcome for preservice teachers

John R. Warda,*, Suzanne S. McCotterb

aMillersville University, 107 Stayer, Millersville, PA 17551, USAbMillersville University, 173 Stayer, Millersville, PA 17551, USA

Abstract

As the standards movement progresses, efforts to encourage reflection by student teachers are often undermined. In

this piece, we analyze exemplars of student teacher reflection coming from two very different approaches to outcomes-

based teacher preparation. We use these exemplars to develop a rubric that illuminates the dimensions and qualities of

reflection. This rubric helps clarify how meaningful reflection and an emphasis on learning are not incompatible if the

focus is placed on the process of learning, rather than on outcomes alone. Finally, we contend that engagement in the

process of reflection and reflection on the moral enterprise of teaching can be considered as important outcomes in their

own right.

r 2004 Elsevier Ltd. All rights reserved.

Keywords: Critical thinking; Reflective teaching; Teacher effectiveness; Work sample tasks; Evaluation methods; Student teacher

evaluation; State standards; Academic achievement; Preservice teacher education; Scoring rubrics

1. Introduction

Like many teacher educators, we have hadfrequent conversations with novice teachers aboutthe lessons they have taught in field experiencesand student teaching. Too frequently, thesediscussions turn into interrogations that soundsomething like this:

Teachereducator(TE):

Why was this important for studentsto learn about?

Preserviceteacher(PT):

I needed to cover this material for thestate test in two weeks. The wholeschool is getting ready for the tests.

g author. Tel.: +1-717-872-3835.

sses: [email protected] (J.R. Ward),

[email protected] (S.S. McCotter).

front matter r 2004 Elsevier Ltd. All rights reserv

e.2004.02.004

TE:

ed.

Do you think they learned what youwanted them to?

PT:
Yes, definitely. TE: How do you know? PT: Well, they haven’t taken the test yet,
but they didn’t have any questions.
TE: What would you change if you could
teach it again?
PT: Well I would like to have developed
this lesson further and teach more forunderstanding, but I don’t think Icould have changed my approachbecause I have to cover this materialto get students ready for the test.

Our well-intentioned questions designed toprovoke deliberate thought about student learningand facilitate self-improvement fall flat. Pressingconcerns about standardized tests replace andundermine thoughtful questions about teaching

ARTICLE IN PRESS

J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257244

and learning. We are perpetually challenged to getour students to reflect on their practice in mean-ingful ways, to consider the effect their teachinghas on student learning, and develop habits thatwill stay with them.The increasing prevalence of standards,

high-stakes testing, and outcomes assessmentobscure the value of reflection, and muchelse, from fields of vision. In many states,district-wide performance on standardized testsis linked to school funding and governance.The Bush administration’s ‘‘No Child LeftBehind’’ Act mandates annual testing bythe 2005–2006 school year, and states that‘‘If the district or school continually fails tomake adequate progress toward the standards,then they will be held accountable.’’ (www.nochildleftbehind.gov). Knowing that testscores will be a visible and public indicator ofperceived quality makes it difficult to see beyondthe graphs and tables on the front pages ofnewspapers.The standards which are so widespread in basic

education have been extended to teacher educationprograms (Cochran-Smith, 2000). A fundamentalshift from an input to an output model ofevaluation is taking place in the field. It is nolonger enough for teacher preparation programsto demonstrate that their candidates have theknowledge, skills, and dispositions associated witheffective teachers; teacher preparation programsmust now demonstrate that their candidates makea positive impact on student learning (NCATE,1999). We are all being asked to critically analyzestudent work in terms of how it is meetingstandards.The standards movement raises both challenges

and opportunities for the fields of teaching andteacher education. Beyond the oft-cited disadvan-tages of academic standards leading to standardi-zation of teaching, and the danger of relying tooheavily on standardized tests as the sole measureof mastery, there is a risk that the value of teacherreflection will be diminished and overwhelmed bystandards. The value of reflection to the develop-ment of teachers has a growing consensus amongteacher educators, although there is still somedebate over the way in which reflection is defined.

For this study, we find the definition positedby Hatton and Smith (1995) to be the moststraightforward and useful: ‘‘[Reflection is] delib-erate thinking about action with a view to itsimprovement’’.As teacher educators, we place a high value

on reflective thinking and practice that sup-ports preservice teachers in broadening perspectiveand developing concern for others. We wantpreservice teachers to think about the moralenterprise of teaching and what is ‘‘taken-forgranted’’. Unfortunately, standards are oftenviewed as closing the door on the need toask questions about the curriculum, questionsthat may raise those moral concerns. Further,the process of dialogue and questioning thatis at the heart of reflection is often perceivedas conflicting with the ‘‘coverage’’ mentalityof a standardized environment (Ketter & Pool,2001). This is particularly true for preserviceteachers. When they join the profession as firstyear teachers they will be immersed in thepressures of standards-driven curriculum andclosely examined student outcomes. How will thehabits of reflection and questioning survive underthese conditions?The first specific challenge posed by the

standards movement focuses on the purposeof reflection. Do standards and, more specifically,the tests that measure standards define andlimit what is meant by improvement of practice?Is reflection nothing more than a tool forhelping teachers increase student test scoresor is there value in reflection on broaderthemes such as social justice? Another challengerelates to the heightened value placed on out-comes that are easily measured. As teachereducators we are facing calls to demonstratethe ‘‘value-added’’ by our work. Will thehabits of reflection that we seek to develop infuture teachers become devalued simply becausethey are difficult to evaluate, summarize, andreport?The emphasis on student learning related to

standards presents an opportunity, however, whenit becomes the very fabric of reflection, rather thanthe barrier that precludes it. In fact, teacherexamination of student work and student learning

http://www.nochildleftbehind.gov

http://www.nochildleftbehind.gov

ARTICLE IN PRESS

J.R. Ward, S.S. McCotter / Teaching and Teacher Education 20 (2004) 243–257 245

can be an excellent vehicle for reflection. Whilesingle-minded attention to high-stakes tests mayundermine the reflective process, a rigorousemphasis on student learning offers an opportu-nity for broadening the reflective focus of pre-service teachers. Expecting preservice teachers torelate their teaching behaviors to student learningrather than to their own performance representsan important shift from a focus on self to a focuson others. This shift is reminiscent of Fuller’s(1969) conceptualization of the stages that pre-service teachers go through in transitioning fromcollege students to classroom teachers. In fact,examining the nature of student learning is thevery form of professional development that caneffectively increase student learning (Meier, 2000;Mitchell, 1996; National Commission on Teaching& America’s Future, 1996). Accomplishing this,however, means resisting the temptation to onlyask the simplistic question, ‘‘Did students do wellon the test?’’One way we have begun to answer these

challenges is by development of a rubric forevaluating teacher reflection. This rubric is de-signed to address three related questions aboutpreservice teacher reflection that takes place withintoday’s standards-based educational systems.First, what are the qualities that distinguish moreand less meaningful reflection? Second, how arethese qualities related to a focus on studentlearning and student learning outcomes? Finally,how can we describe these qualities in such a wayas to make them visible and valued outcomes intheir own right?In the following sections we will describe our

process for developing the rubric and present therubric along with samples of preservice teacherreflective writing exemplifying different levels ofthe rubric. We started the development process byexamining existing frameworks for describing thequalities of reflection. We found these frameworksuseful for developing guiding principles for eval-uating the quality of reflection. None of theexisting frameworks, however, were sufficient foranswering all three of our questions. In general,existing reflection frameworks are not sufficientfor evaluating reflection, especially for evaluatingreflection on student learning.

2. Reflection frameworks

The essential quality of reflection is thinkingabout practice in order to improve (Hatton &Smith, 1995). This is clear in Dewey’s (1933, p. 17)justification of the importance of reflection foreducation, ‘‘By putting the consequences ofdifferent ways and lines of action before the mind,it enables us to know what we are about when weact.’’ Sch .on expanded on Dewey’s work bydifferentiating reflection-in-action from technicalrationality or the application of scientific princi-ples to practice. For Sch .on (1983), reflection isbound up with practice; teachers and otherprofessionals understand and act on their situa-tions in ways that cannot be reduced to rules ordescribed fully by educational theory. This earlywork has led to a widely accepted understandingthat reflection is an important process for teachers(Darling-Hammond & Snyder, 2000; Valli, 1992;Zeichner, 1996).Recent frameworks for reflection build on the

theories of Dewey and Sch .on, and includecommon process elements that owe a debt to earlyadvocates for reflection. Dewey’s focus on reflec-tion situated in practice is a typical characteristicof reflection in teaching and teacher education(Collier, 1999; Korthagen & Kessels, 1999; Lough-ran, 2002; Reiman, 1999). The tendency to reflectabout problems in that practice, and to ‘‘frameand reframe’’ those problems over time wasoriginally discussed by Sch .on (1983), but has beenechoed in other frameworks (e.g., Clarke, 1995;Korthagen, 1999; Loughran, 2002; Reiman, 1999;Stanley, 1998). The notion that problems cannotbe easily solved lends a cyclical nature to manymodels of reflection (Clarke, 1995; Korthagen,1999; Reiman, 1999; Stanley, 1998). Seeking otherviewpoints, or multiple perspectives, to gain in-sight on problems is another common element ofreflective practice (Collier, 1999; Hatton & Smith,1995; Loughran, 2002; Rearick & Feldman, 1998).The common elements of many frameworks of

reflection that we have identified (reflection issituated in practice, is cyclic in nature, and makesuse of multiple perspectives) have also beenapplied in much of the research on how to improvethe quality of teacher reflection. Several methods,

ARTICLE IN PRESS


such as action research (Dinkelman, 1997; Yost,Sentner, & Forlenza-Bailey, 2000; Zygouris-Coe,Pace, Malecki, & Weade, 2001), and cases(Gillespie, 1996; Harrington, Quinn-Leering, &Hodson, 1996) pointedly make use of the idea thatreflection should be situated in practice. Methodssuch as logbooks and journals (Francis, 1995;Korthagen, 1999; Reiman, 1999) emphasize theongoing cyclic nature of meaningful reflection.Other methods such as peer interviews andconferences (Collier, 1999) emphasize considera-tion of multiple perspectives. Many of thesestrategies, in fact, make use of all three of thesequalities.In addition to qualities of reflection that are

valued both from a theoretical and an appliedperspective, there has been much work aimed atdescribing developmental or hierarchical qualitiesof reflection (Collier, 1999; Dinkelman, 2000;Galvez-Martin, Bowman, & Morrison, 1998;Hatton & Smith, 1995; Kitchener & King, 1981;Rearick & Feldman, 1998; Ross, 1989). In thesestudies, lower levels of reflection are identified bythe absence of evidence of one or more qualitiesascribed to higher levels (situated, cyclic, usingmultiple perspectives, and consideration of mo-rally important questions). Low levels of reflectionhave been described as Technical (Collier, 1999),Routine (Yost et al., 2000), or simply descriptivewriting (Hatton & Smith, 1995). Higher reflectionis classified based on evidence of these qualities,and is often called critical reflection (Collier, 1999;Dinkelman, 2000; Hatton & Smith, 1995; Yostet al., 2000).There are missing elements from the existing

frameworks, however, which make them difficultto use for evaluating the quality of preserviceteacher reflection. A common shortcoming ofthese frameworks as tools for evaluation is a lackof attention to how teachers situate their thinkingwithin the context of their practice (Kitchener &King, 1981; Ross, 1989). Other frameworks areincident-based, which has the advantage of beingsituated in practice, but misses the ongoing,cyclical aspect of framing and reframing problems(Galvez-Martin et al., 1998). Still other frame-works include many of the general principlesabout the process of reflection, but lack a method

to either practically use them in evaluating thereflection of teachers (Stanley, 1998; Clarke, 1995)or to evaluate both the qualities and dimensions ofreflection (Hatton & Smith, 1995; Korthagen,1999; Korthagen & Kessels, 1999).Although these frameworks have limitations,

they do make some aspects of reflection visible,and are potentially helpful in working withpreservice teachers. The most significant problemto date, however, is that existing frameworks donot address the challenge of outcomes-basededucation or deal seriously with student learning.They are designed to describe a process, but notdesigned to identify or recognize the qualities ofreflection that are related to improvement ofpractice. The missing link for reflection onteaching is a tool for the comprehensive assess-ment of reflection that gives shape to the generalprinciples of reflection, helps teachers visualizehow reflection can improve their practice, andexplicitly links reflection to student learning. In thenext section we describe how we used samples ofpreservice teacher reflection on student learningalong with general principles about the process ofreflection to develop such a system.

3. Methodology

After reviewing the literature on qualities ofreflection and recognizing that there was nocomprehensive framework for the evaluation ofreflection it was important for us to focus on ourdata. We utilized a grounded theory approach todevelop our rubric. Grounded theory is a con-trolled, systematically inductive approach to de-velopment of theory from data (Strauss & Corbin,1990).

3.1. Data sources

We used two rich sources of preservice teacherreflection on student learning as our data, sevenexemplars from the Renaissance Teacher WorkSample (TWS) and six exemplars from a qualita-tive approach (Collaborative Inquiry: Reflection,Questions about Student Learning—CIRQL).TWS and CIRQL products are essentially

ARTICLE IN PRESS


enhanced and improved unit plans. Each approachrequires student teachers to plan and teach a unitof instruction along with a well-balanced assess-ment plan. Student teachers use their assessmentsas a basis for reflection on their teaching. Theresulting student teacher products contain ap-proximately 15–20 pages of reflective text alongwith lesson plans and samples of student work.These two sources were selected because theyrepresent contrasting approaches to reflection onstudent learning. The TWS places greater empha-sis on summative assessment and calculation ofgain scores using pre- and post-assessment.CIRQL emphasizes formative assessment an on-going reflection. The exemplars we choose were theculminating work of 13 different student teachersfrom 11 different teacher education institutions.The 13 units we used as our data sources werefreely available online1 for purposes of trainingstudent teachers and their mentors in each of therespective methodologies.The TWS and CIRQL are methodologies

developed primarily at public universities withgrant funding. The TWS is supported by theRenaissance Partnership, a national consortium ofteacher education programs, funded with a federalTitle II grant. CIRQL was developed with thesupport of a National Science Foundation Colla-borative Level II grant. The TWS, used for teacherlicensing purposes in Oregon since 1986, has beenpromoted by both the National Council forAccreditation in Teacher Education (NCATE)and the American Association of Colleges forTeacher Education (AACTE) (Girod, 2002; TheNewsletter of the National Council for theAccreditation of Teacher Education, 2000; Scha-lock, Schalock, & Myton, 1998).As a methodology, the TWS asks student

teachers to clearly align learning goals, instruction,and assessment with standards; analyze gains instudent learning using pre- and post-assessmentmeasures; and reflect on their teaching given

1Examples of TWS student teacher units may be viewed

online at The Renaissance Partnership for Improving Teacher

Quality (2002) website. Examples of CIRQL student teacher

units are available online through the Collaborative Inquiry–

Reflection, Questions about Learning (2003) website.

student learning gains. By requiring samples ofboth the teacher’s work and the students’ work,the TWS provides a structured way for noviceteachers to meaningfully connect the two (Scha-lock & Myton, 2002). Pre- and post-assessment ofstudents is used as a basis for measuring effective-ness: ‘‘yteacher work sampling highlights learn-ing gains made by pupils in relation to outcomesdesired from a unit of instruction. In this context,TWS measures devised by the student are used toassess learning before and after instruction’’(Girod, 2002, p. 11). The focus on standards isapparent both in instruction, where a shift fromcoverage to outcomes is counseled, as well as inassessment, where clear alignment between mea-sures and outcomes is advocated.Like TWS, CIRQL products involve a unit plan,

along with robust analysis of student learning.Rather than using pre–post-test scores as the basisfor analysis of student learning as in the TWS,student teachers working under the CIRQL modelanalyze the learning process of a few studentsusing a case study approach. Student teacherscompleting a CIRQL unit are asked to useformative assessments on a daily basis for ongoingreflection. This varying selection of data sourcesallowed us to engage in theoretical sampling(Preissle, Tesche, & LeCompte, 1993), as wedeveloped the theory from our data.

3.2. Analysis of reflective writing

Our process of using the data to build the theorydemonstrated four key procedural elements ofgrounded theory coding: fracturing data (identify-ing reflective ‘‘chunks’’ within lengthy reflectiveunits) to gain higher levels of abstraction, usinggenerative questions, moving toward the discoveryof core categories, and ultimately integrating theentire analysis and yielding conceptual density(Strauss, 1987).Our first step in analyzing the data was to

identify reflective ‘‘chunks’’ from the TWS ex-emplars. This gave us the first procedural elementof grounded theory coding, fracturing the data.Chunks were identified as writing on a particulartheme or topic. The end of a chunk was identifiedby a change in the focus of reflection without a

ARTICLE IN PRESS


clear transition or connection to previous reflec-tion. Chunks ranged in length from several shortsentences to longer pieces of several hundredwords. We considered any text that focused on aspecific teaching action to be reflective. The factthat the action was being described implieddeliberate thinking about the action and desiredimprovement. This liberal definition of reflectionallowed us to pull in as many chunks as possiblefrom the exemplars. The two authors did the initialidentification independently, and then agreed uponthe reflective chunks to code, thus providingtriangulation.Our review of the literature led us to develop

generative questions, the second procedural ele-ment of grounded theory, that allowed us tointerrogate our data in detail. We used threeframes for our investigation of samples of pre-service teacher reflection: cyclic nature, breadth ofperspective, and inquisitive stance. The cyclicqualities of reflective writing referred to whetherpreservice teachers engaged in a process of framingand reframing problems, or whether they consid-ered issues over a period of time or across a varietyof situations. Breadth of perspective referred towhether preservice teachers considered other view-points, whose viewpoints they considered, andwhether they considered moral questions. Inquisi-tive stance referred to whether reflective writingwas centered on questions, and indeed whetheruncertainty was expressed at all. We also con-sidered the situated nature of reflection, although

Table 1

Precipitants by reflective level

Precipitant Routine Techni

Student interest (high) 2 1

Student interest (low) 2

TWS 2 2

Assessment/learning goals 4 5

Content consideration 3

External constraints 3 2

Instructional strategy 8 7

Prior-knowledge/experience 11 12

Relations/environment

Self-lauding 3 2

Struggling students/failure 1 12

Student learning/excitement 3 3

not as one of our guiding frames. Situatedreflection was operationalized as the events,problems, questions, and context that precipitatedreflective writing. Specifically, the qualities ofreflection related to student work and learningwere considered as part of the situated nature ofreflection. We analyzed these chunks askingquestions such as ‘‘how well do our initial guidingframes describe this data?’’ and ‘‘What types ofprecipitants tended to lead to greater use of otherperspectives or ongoing questions?’’We coded reflective chunks from the TWS

samples along the frames we had initially identifiedas part of reflection (cyclic nature, and breadth ofperspective, and inquisitive stance), and noted theevents or themes that precipitated and situated thereflective chunks. We conducted w2 analysis on therelationship between precipitant type and fourlevels of a continuum we devised sorting reflectivechunks along a continuum from Routine toCritical (Table 1). There was clear significancebetween precipitant type and level of Reflection(w2 ¼ 121:25; df=33, p ¼ 0:000), Breadth of Per-spective (w2 ¼ 123:174; df=33, p ¼ 0:000), andCyclic Level (w2 ¼ 80:562; df=33, p ¼ 0:000).The relationship between precipitant and Inquisi-tive Stance was less clear (w2 ¼ 39:368; df=33,p ¼ 0:206).The most effective precipitant in terms of getting

student teachers to reflect critically was whenthey focused on Struggling Students or Student

Failure. Our original frames did not capture the

cal Dialogic Critical Total

3

2

4

9

3

5

1 16

23

1 1

5

4 17

6

ARTICLE IN PRESS


importance of precipitants or the importanceof focusing on struggling students. As weadded CIRQL reflections to our data set, wesaw a broader range of reflections that werenot well described by our original frames.Unlike the TWS, students involved in the CIRQLprocess were encouraged to reflect in depthon individual students and also on a wide rangeof self-initiated questions. This led us to distin-guish between reflections focusing on the processof learning as opposed to merely productsof learning. We also saw a clear distinctionbetween reflection on self and reflection onteaching tasks undertaken without considerationof other perspectives.This process allowed us to develop other key

descriptive distinctions that appear in our finalrubric. For example, it became clear that self-lauding or blaming problems on constraints out-side of the perceived control of the preserviceteacher were associated with many of the leastreflective chunks. Consideration of teaching stra-tegies and other teaching behaviors were oftenassociated with intention to change, but rarely ledto ongoing thinking or involved breadth ofperspective.The third element of grounded theory coding is

the discovery of core categories. In our work thismeant revising our initial frames until theyprovided a comprehensive method for analyzingall of our reflective chunks. Going back and forthfrom data to our initial guiding frames led us todevelop the three dimensions of our final rubric(Focus, Inquiry, and Change).Using this process our developing rubric was

revised multiple times until we reached the fourthelement of grounded theory, conceptual density. Inother words, we continued asking questions aboutour reflective chunks, until our rubric provided agood description of all of our reflective chunks andwas consistent with the guiding frames we hadinitially identified through the literature. For themost part this process resulted in more specificdescriptions for each level and dimension of ourrubric, but in other cases our descriptions becamemore general. For example, instead of strictlyconsidering the number of cycles of questions,insights, and resulting teacher actions our final

rubric describes the general qualities of ongoingreflection.Using a controlled and systematic approach to

coding helped us meet the challenge of creating arubric that can be used to define qualities thatdistinguish more from less meaningful reflection. Itis a challenge to ask preservice teachers to usestudent learning and student work as a basis forreflecting on their practice. Our response to thischallenge was clarified through the process ofclosely examining the available data. Under out-comes-based systems, the most reportable out-comes are also the most visible and valued. Ourrubric achieves this goal of making the qualities ofmeaningful reflection more visible and valuedwhile honoring the richness and depth of mean-ingful reflection on practice.

4. Reflection rubric

The final rubric is depicted in Table 2, and theremainder of this section describes the dimensionsand levels.

4.1. Dimensions of reflection

Our cyclical examination of theory and data ledus to refine our generative frames (cyclic nature,breadth of perspective, and inquisitive stance) intothree dimensions (Focus, Inquiry, and Change).The Focus dimension encompassed elements ofour original breadth of perspective frame withwhat we learned about the precipitants of reflec-tion and reflections on student learning. The datarevealed that broadening perspective was oftenassociated with attempts to understand studentsand the process of their learning, especially whenstudents were struggling or failing. Conversely, anarrow perspective was frequently revealed as afocus on problems that were perceived as causedby others and outside of the control of the teacher.The revised dimension of Focus asks, ‘‘What is thefocus of concerns about practice?’’ and rangesfrom focus on self, to focus on students, to focuson the impact of broader concerns on studentlearning. This dimension most clearly describes the

ARTICLE IN PRESS

Table 2

Reflection rubric

Levels

Routine Technical Dialogic Transformative

Self-disengaged from

change

Instrumental response

to specific situations

without changing

perspective

Inquiry part of a process

involving cycles of

situated questions and

action, consideration for

others’ perspectives, new

insights

Fundamental questions

and change

Focus (What is the focus

of concerns about

practice?)

Focus is on self-centered

concerns (how does this

affect me?) or on issues

that do not involve a

personal stake. Primary

concerns may include

control of students, time

and workload, gaining

recognition for personal

success (including

grades), avoiding blame

for failure.

Focus is on specific

teaching tasks such as

planning and

management, but does

not consider

connections between

teaching issues. Uses

assessment and

observations to mark

success or failure

without evaluating

specific qualities of

student learning for

formative purposes.

Focus is on students.

Uses assessment and

interactions with

students to interpret

how or in what ways

students are learning in

order to help them.

Especially concerned

with struggling students.

Focus is on personal

involvement with

fundamental

pedagogical, ethical,

moral, cultural, or

historical concerns and

how these impact

students and others.

Inquiry (What is the

process of inquiry?)

Questions about needed

personal change are not

asked or implied; often

not acknowledging

problems or blaming

problems on others or

limited time and

resources. Critical

questions and analysis

are limited to critique of

others. Analysis tends to

be definitive and

generalized.

Questions are asked by

oneself about specific

situations or are implied

by frustration,

unexpected results,

exciting results, or

analysis that indicates

the issue is complex.

Stops asking questions

after initial problem is

addressed.

Situated questions lead

to new questions.

Questions are asked

with others, with open

consideration of new

ideas. Seeks the

perspectives of students,

peers, and others.

Long-term ongoing

inquiry including

engagement with model

mentors, critical friends,

critical texts, students,

careful examination of

critical incidents, and

student learning. Asks

hard questions that

challenge personally

held assumptions.

Change (How does

inquiry change practice

and perspective?)

Analysis of practice

without personal

response—as if analysis

is done for its own sake

or as if there is a

distance between self

and the situation.

Personally responds to a

situation, but does not

use the situation to

change perspective.

Synthesizes situated

inquiry to develop new

insights about teaching

or learners or about

personal teaching

strengths and

weaknesses leading to

improvement of

practice.

A transformative

reframing of perspective

leading to fundamental

change of practice.


situated qualities of reflection that tend to lead toimprovement of practice.We further refined the rubric by combining the

frames for cyclic nature (framing and reframing of

questions) and inquisitive stance into the singledimension of Inquiry. This dimension addressesthe question ‘‘What is the process of inquiry?’’ andexamines how questions are asked rather than just

ARTICLE IN PRESS


their presence. In our original analysis of the data,there was an almost complete and unrealisticabsence of explicit questions in student writing.We came to see this as a style of writing ratherthan a true absence of curiosity. This led us toconsider implied questions, as well as explicitquestions, as relevant to the reflective process.Analysis of student writing led us to distinguishbetween questions that were unconnected topractice, and situated questions.The final dimension, Change, asks the question

‘‘How does inquiry change practice and perspec-tive?’’ Again, our data revealed several seeminglyimportant reflective statements that did not easilyfit into our original frames. These were oftenstatements like, ‘‘The biggest thing I learned fromthis experience wasy’’ The problem these state-ments posed was that they were identified by thewriter as extremely important, yet they oftenlacked any clear development or basis in reflectivewritings. Our original dimensions lacked a de-scriptive way to account for these insights. Thechange dimension evolved to include the conceptof new insights arising from a synthesis of situatedexperiences.

4.2. Qualitative levels

The labels for our first three levels of our rubric(Routine, Technical, Dialogic) are terms that havebeen traditionally used in the literature on reflec-tion (Hatton & Smith, 1995; Sch .on, 1983). Inkeeping with the literature, we had initially labeledthe fourth level as critical. In our final rubric, thehighest level has been renamed and is now calledTransformative.2 This change resulted from apossible confusion we see arising between Routineand Transformative reflection. Our Transforma-tive level includes traditional ‘‘critical’’ issues suchas historical and moral concerns. However, we didnot want this confused with ‘‘critical’’ in the senseof blaming problems on conditions beyond one’s

2The use of the term Transformative is not meant to imply

the Freireian notion of emancipatory reform through educa-

tion. Although some teachers who reflect at the Transformative

level may indeed incorporate aspects of emancipation or

liberation in their practice, those ideals are not necessary for

reflection at the Transformative level on our rubric.

control. Our finding that this type of perceptionwas closely related to low-level reflection led us todistinguish between moral questions about one’sown practice and merely being critical about themoral practices of others (which demonstratesRoutine reflection in our rubric).During the early stages of coding we viewed

these levels as descriptive of markers on acontinuum with important qualities being moreor less present at each level. As we continuedevaluating reflective writing we began to see theselevels as also having clear qualitative distinctionsthat crossed all dimensions. This is most clearlyseen in the Focus dimension. The distinctionsbetween levels of Focus are not merely in terms ofthe amount of focus on the perspectives of others;each level also has a different focus. Qualitativedistinctions exist between levels of the other twodimensions as well. The Inquiry dimension, forexample, distinguishes between questions that areasked by oneself and questions that are asked withothers.Analysis of reflective chunks using the final

rubric revealed that the vast majority of studentreflections were scored with a spread of no morethan one level across all dimensions. For example,it was rare for a reflective chunk to fit the Routinequalities of Inquiry yet yield Change at theTransformative level. In the following sectionswe provide samples of preservice teacher reflectivewriting and explain how our rubric was used toevaluate them. We begin first with the more typicalcases, where the level of reflection was relativelyconsistent across each dimension. Following adiscussion of the four levels of reflection, we willdiscuss two specific and illuminating patterns ofwide variance across dimensions.

4.3. Routine

Routine reflections tended to contain verydefinitive statements that revealed either a lack ofcuriosity or a lack of attention to complexity.These writings usually did not focus on problems,but when they did, the tendency was to blameproblems on others or on a lack of time andresources. The lack of questioning and especiallythe lack of a sense of responsibility for change are

ARTICLE IN PRESS


probably most characteristic of these low-levelreflections. These reflective writings also tended tobe fairly short as illustrated by the followingexample of Routine reflection:

The other barrier I found was the ability ofmany of my students. As an entire class, theydid not have much experience working hands-on. I would have liked to teach many moreconcepts hands-on, but due to the lack ofexperience in the class it was not feasible.Classroom management was a problem the firstfew times we tried a hands-on activity. If thishad been my classroom, students would havebeen familiar with my mode of teaching andclassroom management would not have been anissue. When I taught my fall week I did not runinto any classroom management problemsbecause they knew my expectations.

This exemplar was rated as Routine in alldimensions. The focus is on typical self-concernssuch as the ability to control or manage students.Inquiry was also rated as Routine, there does notseem to be any doubt or questioning of the sourceof problems. Placing blame on others is not likelyto lead the student teacher to change practice orperspective.

4.4. Technical

Technical reflection can best be thought of asinstrumental, in that the reflection is used as ameans to solve specific problems, but does notquestion the nature of the problem itself. Mosttypically, these reflections focus on teaching tasks,as seen in this exemplar:

I could use more professional developmentinygetting students more involved in ‘‘lecture’’material and making a connection from classnotes and lecture material to the overall under-standing of the lesson. Sometimes there is notthe opportunity to do a hands-on activityrelated to a particular topic. The material canbe very dry, but definitely necessary to theunderstanding of the topic. I try to play reviewgames, and get the students involved in thelecture by asking questions that make them

more active participants. But, I feel I need tofind some more strategies on how to makelecture material more interesting and engagingfor the students.

This exemplar typifies a Focus on narrowteaching tasks; in this case how to make lecturesmore engaging. There is a clear desire to learnmore related to this topic, but Inquiry aboutlectures does not seem to lead to further or deeperquestioning of practice, nor does this teacher usethe perspectives of students or peers in reflectingon this issue. These are the hallmark character-istics of Technical Inquiry. Finally, there is anindication that this student teacher is committed toimprovement of practice in this area, though thereare no new insights gained from this reflection.This student teacher wants to engage students, butdoes not question lecturing as an approach.Narrow change or change without new insight ischaracteristic of Technical qualities of the Changedimension.

4.5. Dialogic

Dialogic reflection is best thought of as ongoingprocess. The term itself connotes discussion andconsideration of the views of others. The theme ofprocess at this level can also be seen in a Focus onthe process of learning (as contrasted with theoutcomes of learning) and often in the process ofInquiry (i.e., a sustained process of asking ques-tions, trying new approaches, and asking newquestions). One of the most common forms thatthis reflection takes is grappling with the learningprocess for a struggling student, as seen in thisexemplar:

Student one, who is an English as a secondlanguage student, did very poorly on the pre-assessment. My first reaction was to have hisESL teacher give him the assessment. When Ifound out that this was not feasible, I decided totry it myself. My first step was to borrow one ofthe student’s English-Spanish dictionaries. Iwas surprised to find out that a lot of words Ineeded were not in the dictionary. After findingas many words as I could, I made notes on ablank assessment and set a time to meet with

ARTICLE IN PRESS


student one. I went through the assessmentagain with this student, only to find that myefforts did not help. My analysis of thisexercise, however, allowed me to understand alittle better why he did poorly on the pre-assessment. I found that it was not just hisEnglish deficiency that hindered him on theassessment. Student one did not have the priorknowledge needed to answer the questions onthe assessment.

I also discovered other helpful informationfrom this exercise. Student one is able to answerquestions that require one-word answers, butcould not answer questions that required him towrite sentences. If I had not discovered this, Iwould have just assumed he didn’t know thematerial. Because of this discovery, I was ableto make modifications on the rest of hisassessments.

Clearly this reflection chunk has a Focus on astruggling student. The key to a Dialogic rating ofInquiry for this sample is the ongoing nature ofquestions with original questions leading to newquestions. Unlike many reflections at the Dialogiclevel, however, this reflection did not includeconsideration of the student’s perspective on theproblem. Our rubric does not require considera-tion of others’ perspectives to achieve a Dialogiclevel; instead, another’s perspective is usually thedriving force behind consideration of new ideasand new questions. In this case, the student teacheris essentially involved in a dialogue with thesituation itself. The situated nature of Inquiryoften motivates ongoing questions. Finally, thisreflection reveals new insights and Change stem-ming from the process.

4.6. Transformative

While Dialogic reflection considers and synthe-sizes new ideas, Transformative reflection ques-tions fundamental assumptions and purpose moredeeply. Although in our experience it is notunusual for preservice teachers to express Trans-formative ideas in response to theoretical readings,it is rare for preservice teachers to reach this levelwhen reflecting on their own teaching. The

primary focus of teachers at this level on teachingtasks and self-concerns tends to crowd outdeeper questioning. It is also important topoint out that our rubric describes this processas taking place over a long period of time andso we would not expect to see true examples ofthis type of change within a single semesteror year. In our sample, only one studentteacher approached this level. The sample shownhere is abstracted from the original writing. Itseems that this student reached this higher levelonly after sustained inquiry and self-questioningduring her experience. In this case, the studentteacher was part of a cohort that met regularlywith mentors to discuss ongoing questions andshare student work.

During the unit I found that TD workedindependently on his essay and appearedconfident in his work. DK in comparison whorarely asks for assistance with her writingwanted to conference all the time. Her continualneed for teacher approval concerned me andforced me to question why this was occurring.After questioning her behavior I began toquestion the assignment. Yes, DK is a verygood writer, but why was she struggling? y

Wow, in the middle of my unit I then began toquestion the success of the unit. Am I reallymeeting the needs of all of my students or is thistoo easy? y

Finally after weeks of teaching, reflecting andquestioning the unit it was over and I spenthours grading countless persuasive essays. Afterlooking over the drafts and then the final essaysI found a correlation between what wasoccurring in the classroom and what thestudents were writing; they were making theconnectionsy

First, I would never just do a persuasive writingunit again (even though it was in conjunctionwith westward expansion). I would love tomake a connection between a relevant issue instudent’s lives and how they can utilize persua-sive writing to assist them with it. I feel that it isso important to not label units, such aspersuasive writing because, it deters students

ARTICLE IN PRESS


from getting involved. In the future I think thatI will continue to ponder many of the samequestions that arose throughout the teaching ofmy persuasive writing unit (here I go labeling itagain) and even more.

The Focus on students in this sample leads to adeeper questioning of what is truly valuable andmeaningful for them. This sample exemplifieswhole-hearted Inquiry revealing a sincere habitof continually asking questions and looking foranswers over time. Though not completely reach-ing the full Transformative level in the Changedimension, this reflection does lead to a change ofperspective on the unit that was taught, potentiallyhaving a profound effect on practice.

4.7. Patterns that cross levels of the rubric

As mentioned earlier, there were two patterns inthe data where student reflection showed a rangeof levels on the Reflection Rubric. The first patterninvolved deep reflection on personal qualities thataffect teaching. The following exemplar was ratedas having a Routine Focus on self, but reached theDialogic level for Inquiry:

One student suggested that I teach moreaggressively and to develop more confidencein myself as well as (I love this) to not ‘‘take anycrap from us students’’. I understand what thatstudent was saying. I was definitely notconfident in teaching this unit, partly becauseprobability was a topic in mathematics that Iwasn’t comfortable with and partly because thiswas the first unit I taught with this class. As thesemester went on, I did develop more con-fidence in teaching them as well as became morecomfortable with displaying my personality,which another student had suggested. I nevergot to that point with my calculus students.They did ask me lots of questions, but I alsowas not completely comfortable with the topic Iwas teaching (volumes of revolution), whichmade it difficult to become comfortable andconfident in the classroom within in the twoweeks I had with them. I may have been able todevelop a better relationship with them if I hadmore time to do so.

This example brings out an important point inusing the reflection rubric. Although the Dialogicand Transformative levels of the rubric representdeeper reflection, they are not always the mostappropriate or needed forms of reflection. Thedevelopmental path for many preservice teacherssuggests that concern for self and gaining compe-tency in teaching tasks is and probably should bethe most immediate focus. The question is whetherthese beginning stages of reflection will contain theseeds for deeper reflection later or whetherreflection is undertaken as a process that aimsfor improvement and is open to the ideas of others.In this case, the student teacher’s focus on self wasclearly important: the writing reveals a struggle forpersonal empowerment as a teacher. Unlike otherreflective writings with a Routine self-focus,however, this student teacher sought the perspec-tives of others in the process of reflecting on self.There is also a difference in personal responsibilityas evidenced by the Focus on self-improvementrather than on blaming others for problems.A second pattern involved summative reflective

statements describing teaching success as seen inthe following exemplar:

My greatest insight in teaching this unit was thefact that children want to learn. When conceptsare presented to children in a way that ismeaningful to them, they will naturally learn.My students wanted to know why, when,where, and how. They wanted to show theycould apply their knowledge in a new situation.They wanted to analyze their learning bybreaking it down and then they wanted tocreate thereby using synthesis. My students hadno problems evaluating their work. They wereable to justify their need and their ability toconquer the subject matter. The students hadwithin them the desire to learn. I just had toallow that desire to express itself. I realized thatI did not have the ability to ‘‘force’’ a student tolearn. Instead, I recognized that I could lead thestudent and provide an environment thatallowed the student to learn.

This type of writing is perhaps the most difficultto evaluate. On the one hand, the level ofgenerality and the definitive stance on complex

ARTICLE IN PRESS


issues reveals a lack of recognition of how muchthere is to learn as a teacher. On the other hand,the student explicitly relates this as important. TheFocus is clearly on students, but one mightquestion whether this is more of a self-congratu-latory reflection using students to illustrate perso-nal success. This exemplar came from the TWSdata; students completing the TWS know that theyare being evaluated as candidates based on theirreported success. This highlights the differencebetween evaluating and making visible the quali-ties of reflection (as we have attempted to do withour rubric), and merely using reflection as a meansfor reporting other professional outcomes. Thefirst approach emphasizes the value of reflection asa means toward growth; the second approachtends to stifle sincere curiosity and change. In theeffort to add value to the reflective process byconnecting it to student learning, it is critical thatquality reflection be considered a valuable out-come on its own merit, and not merely a measureof other outcomes.

5. Conclusions

The standards movement poses several chal-lenges to the value of meaningful reflection. Onthe most basic level, reflection is in dangerof becoming nothing more than a tool towardgreater student achievement. Taken to an extreme,this narrow view of the purpose of reflectionredefines reflection as a means of documentingstudent outcomes. As teacher educators, wemust be able to make a clear case for reflec-tion as an outcome above and beyond itsshort-term instrumental value. Further, weshould be able to define reflection in a way thatmakes the qualities we value visible. Developmentof a rubric that includes dimensions for processand emphasizes broadening perspective and fun-damental questions is one means of doing so.Incorporating student learning within this rubric isone way of recognizing the importance of studentlearning outcomes without making reflectioninferior to them.The patterns we saw in the work of preservice

teachers echoed Fuller’s (1969) theory about the

stages that often describe preservice teachergrowth. The reflections of beginning teachersreinforce the fact that reaching levels of Transfor-mative reflection is unusual and difficult. This doesnot give us or them permission to aim lower, butgives us reasons to identify lower levels ofreflection and provide the scaffolding needed toreach higher. We believe our matrix can help serveas some of that scaffolding. It is not supportive toask preservice teachers to ignore their self-focusedconcerns, but it is more helpful (and morechallenging) to engage them in a Dialogic processof carefully considering their self-related concerns.Although we hope to move our students closer toTransformative reflection, it is not our vision thatmost of our preservice teachers will be able toquickly reach the highest levels of our rubric, northat all of them will do so sequentially. It is ourvision that all preservice teachers should consis-tently reach the Dialogic level of Reflection by theend of their preparation.The process of developing the Reflection Rubric

was eye opening for us. We realized that we haveoften asked our students to reflect on fieldexperiences without ever discussing the qualitiesof good reflection, often with disappointingresults. Students do not automatically know whatwe mean by reflection; often they assume reflectionis an introspective after-the-fact description ofteaching. Reflection, meant to make teaching andlearning understandable and open, has itself beenan invisible process to many of our preserviceteachers. The dimensions of Focus, Inquiry, andChange can be used as formative guides to helppreservice teachers evaluate, understand, andimprove their own reflection.Though we feel the Reflection Rubric should be

used as a formative tool for individual preserviceteachers, it may also be used as a summativeevaluation instrument for innovations such as theTeacher Work Sample, which seek to encouragereflection on student learning. As teacher educa-tion programs attempt to articulate and measureoutcomes for their programs, our reflection rubricprovides a means for evaluating reflection as a coreprogram goal. Our reflection rubric would alsowork well as a research tool for evaluating theeffectiveness of a wide variety of strategies

ARTICLE IN PRESS


designed to promote teacher reflection such ascases and journals as well as newer innovationssuch as the use of electronic portfolios and digitalvideo.Development of the rubric using data from

two models for reflection on student outcomesled us to see how an emphasis on student learningcould either enhance or detract from the reflectiveprocess. When reflection is focused on formativeassessment and the process of student learning,Technical questions tend to lead to ongoingreflection and broadening of perspective. Alterna-tively, when reflection is based on summativeassessment of student learning there is littleopportunity for new questions to arise. Ifpreservice teachers reflect only on whether ornot their students scored well on tests, studentlearning is evaluated in a very narrow way,rather than a broader reflection about theteaching and learning process. In order forreflection to be evaluated, we must overtly connectthe qualities of reflection to the process of teachingand learning. It is only through this integralconnection that we can prevent reflection frombecoming a rote process, or seeing it eliminatedaltogether.Given both our passion as a profession and the

political climate, we need to continue to activelysearch for ways to demonstrate that our studentteachers understand this connection betweenteaching and learning. The new emphasis onstudent outcomes creates an opportunity for usto go to new levels of reflection and understandingof student learning. We can, should, must developmethods of stepping back to take a broad view ofthis essential connection without giving up theprocess of reflection.

References

Clarke, A. (1995). Professional development in practicum

settings: Reflective practice under scrutiny. Teaching and

Teacher Education, 11(3), 243–261.

Cochran-Smith, M. (2000). Teacher education at the turn of the

century. Journal of Teacher Education, 51, 163–165.

Collaborative Inquiry-Reflection, Questions about Learning.

(2003). CIRQL unit exemplars. From the CIRQL web site:

http://harmony2.millersville.edu/cirql.

Collier, S. T. (1999). Characteristics of effective teachers.

Journal of Teacher Education, 50(3), 173–181.

Darling-Hammond, L., & Snyder, J. (2000). Authentic assess-

ment of teaching in context. Teaching and Teacher Educa-

tion, 16(5–6), 523–545.

Dewey, J. (1933). How we think: A restatement of the relation of

reflective thinking to the educative process. Boston: Heath.

Dinkelman, T. (1997). The promise of action research for

critically reflective teacher education. Teacher Educator,

32(4), 250–274.

Dinkelman, T. (2000). An inquiry into the development of

critical reflection in secondary student teachers. Teaching

and Teacher Education, 16, 195–222.

Francis, D. (1995). The reflective journal: A window to

preservice teachers’ practical knowledge. Teaching and

Teacher Education, 11(3), 229–241.

Fuller, E. F. (1969). Concerns of teaching: A developmental

conceptualization. American Educational Research Journal,

6, 207–226.

Galvez-Martin, M. E., Bowman, C. L., & Morrison, M. A.

(1998). An exploratory study of the level of reflection

attained by preservice teachers. Mid-Western Educational

Researcher, 11(2), 9–18.

Gillespie, M. L. (1996). Stories, cases, and practical wisdom.

Innovative Higher Education, 21(1), 49–66.

Girod, G. R. (2002). An overview of teacher work sample

methodology. In G. Girod (Ed.), Connecting teaching and

learning: A handbook for teacher educators on teacher work

sample methodology. Washington, DC: AACTE.

Harrington, H. L., Quinn-Leering, K., & Hodson, L. (1996).

Written case analyses and critical reflection. Teaching and

Teacher Education, 12, 25–27.

Hatton, N., & Smith, D. (1995). Reflection in teacher

education: Towards definition and implementation. Teach-

ing and Teacher Education, 11, 33–49.

Ketter, J., & Pool, J. (2001). Exploring the impact of a high-

stakes direct writing assessment in two high school class-

rooms. Research in the Teaching of English, 35, 344–393.

Kitchener, K. S., & King, P. M. (1981). Reflective judgment:

Concepts of justification and their relationship to age and

education. Journal of Applied Developmental Psychology, 2,

89–116.

Korthagen, F. A. J. (1999). Linking reflection and technical

competence: The logbook as an instrument in teacher

education. European Journal of Teacher Education, 22(2/3),

191–207.

Korthagen, F. A. J., & Kessels, J. P. A. M. (1999). Linking

theory and practice: Changing the pedagogy of teacher

education. Educational Researcher, 28(4), 4–17.

Loughran, J. J. (2002). Effective reflective practice: In search of

meaning in learning about teaching. Journal of Teacher

Education, 53(1), 33–43.

Meier, D. (2000). Will standards save public education?. Boston:

Beacon Press.

Mitchell, R. (1996). Front-end alignment: Using standards to

steer educational change. Washington, DC: The Education

Trust.

http://harmony2.millersville.edu/cirql

ARTICLE IN PRESS


National Commission on Teaching & America’s Future. (1996).

What matters most: Teaching for America’s future. New

York: Author.

National Council for the Accreditation of Teacher Education.

(1999). NCATE 2000: Continuing accreditation and beyond.

Some questions and answers. Washington, DC: Author.

National Council for the Accreditation of Teacher Education.

(2000). Performance assessment in action. Quality Teaching:

NCATE Newsletter, Spring 2000, pp. 4–8.

No Child Left Behind Act (Accessed June 9, 2002). Retrieved

June 5, 2002, from http://www.nochildleftbehind.gov/next/

overview/index.html.

Preissle, J., Tesch, R., & LeCompte, M. D. (1997). Ethnography

and qualitative design in educational research. San Diego:

Academic Press.

Rearick, M. L., & Feldman, A. (1998). Orientations, purposes

and reflection: A framework for understanding action

research. Teaching and Teacher Education, 15, 333–349.

Reiman, A. J. (1999). The evolution of the social roletaking and

guided reflection framework in teacher education: Recent

theory and quantitative synthesis of research. Teaching and

Teacher Education, 15, 597–612.

Ross, D. D. (1989). First steps in developing a reflective

approach. Journal of Teacher Education, 40(2), 22–30.

Schalock, D., Schalock, M., & Myton, D. (1998). Effective—

along with quality—should be the focus. Phi Delta Kappan,

79, 468–470.

Schalock, H. D., & Myton, D. (2002). Connecting teaching and

learning: An introduction to teacher work sample metho-

dology. In G. Girod (Ed.), Connecting teaching and learning:

A handbook for teacher educators on teacher work sample

methodology. Washington, DC: AACTE.

Sch .on, D. A. (1983). Reflective practitioner: How professionals

think in action. New York: Basic Books.

Stanley, C. (1998). A framework for teacher reflectivity.

TESOL Quarterly, 32(3), 584–591.

Strauss, A. L. (1987). Qualitative analysis for social scientists.

New York: Cambridge University Press.

Strauss, A. L., & Corbin, J. (1990). Basics of qualitative

research: Grounded theory procedures and techniques. New-

bury Park, CA: Sage.

The Renaissance Partnership for Improving Teacher Quality.

(2002). Teacher work sample: Performance prompt, teaching

process standards, & scoring rubrics. From The Renaissance

Partnership for Improving Teacher Quality web site: http://

fp.uni.edu/itq.

Valli, L. (1992). Reflective teacher education: Cases and

critiques. Albany: SUNY Press.

Yost, D. S., Sentner, S. M., & Forlenza-Bailey, A. (2000). An

examination of the construct of critical reflection: Implica-

tions for teacher education programming in the 21st

century. Journal of Teacher Education, 51(1), 39–49.

Zeichner, K. M. (1996). Teachers as reflective practitioners and

the democratization of school reform. In K. M. Zeichner, S.

Melnick, & M. L. Gomez (Eds.), Currents of reform in

preservice teacher education (pp. 199–214). New York:

Teachers College Press.

Zygouris-Coe, V., Pace, B., Malecki, C., & Weade, R. (2001).

Action research: A situated perspective. Qualitative Studies

in Education, 14(3), 399–412.

http://www.nochildleftbehind.gov/next/overview/index.html

http://www.nochildleftbehind.gov/next/overview/index.html

http://fp.uni.edu/itq

http://fp.uni.edu/itq

reflection as a visible outcome for preservice teachers

Documents