generating, deepening, and documenting learning: …...critical reflection oriented toward...

25

Generating, Deepening, and Documenting Learning: the Power of

Critical Reflection in Applied Learning

SARAh L. AShNorthCarolinaStateUniversity

PAttI h. CLAytonPHCVentures/IndianaUniversity-PurdueUniversityIndianapolis

Applied learning pedagogies—including service-learning, internships/practica, study abroad, and undergraduate research—have in common both the potential for significant student learning and the challenges of facilitating and assessing that learning, often in non-traditional ways that involve experiential strategies outside the classroom as well as individualized outcomes. Critical reflection oriented toward well-articulated learning outcomes is key to generating, deepening, and documenting student learning in applied learning. This article will consider the meaning of critical reflection and principles of good practice for designing it effectively and will present a research-grounded, flexible model for integrating critical reflection and assessment.

Applied learning pedagogies share a design fundamental: the nur-turingoflearningandgrowththroughareflective,experientialprocessthattakesstudentsoutoftraditionalclassroomsettings.Theapproachisgroundedintheconvictionthatlearningismaximizedwhenitisactive,engaged, and collaborative. Each applied learning pedagogy providesstudentswithopportunities toconnect theoryandpractice, to learn inunfamiliar contexts, to interact with others unlike themselves, and topracticeusingknowledgeandskills. Despite the oft-cited maxim that “experience is the best teacher,”we know that experience alone can, in fact, be a problematic teacher

JournalofAppliedLearninginHigherEducationVol.1,Fall200925-48©2009MissouriWesternStateUniversity

26 JournalofAppliedLearninginHigherEducation/Fall2009 Ash,Clayton/CRITICALREFLECTIONINAPPLIEDLEARNING 27

(Dewey,1910;Conrad&Hedin,1990;Hondagneu-Sotelo&Raskoff,1994;Stanton,1990;Strand,1999).Experientiallearningcanalltooeas-ilyallowstudentstoreinforcestereotypesaboutdifference,todevelopsimplisticsolutionstocomplexproblems,andtogeneralizeinaccuratelybasedon limiteddata.Theservice-learningstudent, forexample,maythinkthatallfoodassistanceprogramsfunctionexactlyliketheoneatwhich he is working, causing him to make sweeping generalizationsabouttheeffectivenessofsuchprogramsdespitewidespreadvariationsinsize,structure,andsourcesoffoodandfunding. Inaddition, studentsmaynotderive themost importantorsignifi-cant learning from their experiences.Theundergraduate researcher inthephysiologylabmaybefrustratedbythetediousnessoftheresearchandnotappreciatethatscientificinquiryisintentionallyaslowprocessoftrialanderror.Shemaynotfullyunderstandwhytheresearchques-tionssheisinvestigatingareimportantorhowthedatasheiscollectingfitintopreviousfindings. Studentsmayleaveappliedlearningexperienceswithlittlecapacityto turn learning into improved action. The study abroad student maybelievehehasdevelopedagreatersensitivitytoculturesdifferentfromhisownbutsixmonthslaterfindhimselfjumpingtoconclusionsaboutothersbasedontheirbackgroundorethnicity.Theinternwhofindshercollaborativeprojectfrustratingmayenduprepeatingpatternsofpoorteamworkinhernextgroupproject. Finally,students inapplied learningpedagogiesmayhaveavaguesenseoftheimpacttheirexperienceshavehadonthembutnotbefullyawareofthenatureoftheirownlearning,itssources,oritssignificance.Theymayonlybeabletodescribeoutcomesvaguely,withphrasessuchas“I learneda lot fromworkingwithcommunitymembers”or“Igotsomuchoutoflivingabroad.”Theservice-learningstudentmayfailtounderstandthedifferentwaysinwhichtheclassroomandthecommu-nitypresentherwithlearningchallenges.Thestudyabroadstudentmaybeunabletoidentifyspecificchangesinherattitudestowardothersor

AUTHORNOTE:Sarah L. Ash, Ph.D., Department of Food, Bioprocessing and Nutrition Sciences, North Carolina State University; Patti H. Clayton, Ph.D., PHC Ventures, Cary, North Carolina/Center for Service and Learning, Indiana Univer-sity-Purdue University Indianapolis. The authors would like to express apprecia-tion to the many student and faculty colleagues who have contributed over the past decade to the work overviewed in this article and to Kelly Henry, Audrey Jaeger, Jessica Jameson, George Hess, and Roy Schwartzman for their review of earlier versions of this manuscript. Correspondence concerning this article should be addressed to: Patti H. Clayton, 300 Swiss Lake Drive, Cary, NC 27513; e-mail: [email protected]

toarticulatewhatledtothechanges.Studentsmay,inotherwords,misstheopportunitytolearnabouttheirownlearningprocesses—todevelopthemeta-cognitiveskillsrequiredforlifelong,self-directedlearningthatappliedlearningissowellsuitedtocultivate. The students in these examples would all benefit from a processof strong reflection, to help them avoid what T.S. Eliot (1943) oncedescribedashavingtheexperiencebutmissingthemeaning.Learning—and understanding learning processes—does not happen maximallythroughexperiencealonebut ratherasa resultof thinkingabout—re-flectingon—it.AsnotedbyStanton(1990),whenreflectiononexperi-enceisweak,students’“learning”maybe“haphazard,accidental,andsuperficial” (p. 185). When it is well designed, reflection promotessignificantlearning,includingproblem-solvingskills,higherorderrea-soning, integrative thinking,goalclarification,openness tonew ideas,abilitytoadoptnewperspectives,andsystemicthinking(Eyler&Giles,1999;Conrad&Hedin,1987). However,reflectionanditscentralroleinappliedlearningareoftenmisunderstoodorseenasunnecessary.Theworditselffrequentlycon-notesstream-of-consciousnesswriting,keepingadiary,orproducingasummaryofactivities. It caneasilybeassociatedwith“touchy-feely”introspection,toosubjectivetoevaluateinameaningfulwayandlack-ingintherigorrequiredforsubstantiveacademicwork.Dewey(1910),oneof theearlychampionsofexperiential learning,providesastrongfoundation for re-conceptualizing reflection,defining it as the“active,persistentandcarefulconsiderationofanybelieforsupposedformofknowledge in the light of the grounds that support it and the furtherconclusions to which it tends” (p. 6). Schön (1983) emphasizes thelinkbetweenreflectionandaction;hedefinesreflectionas“acontinualinterweavingofthinkinganddoing”andsuggeststhatwhathecallsthereflectivepractitionerisonewho“reflectsontheunderstandingswhichhave been implicit in [one’s] action, which [one] surfaces, criticizes,restructures, and embodies in further action” (p. 281). The reflectionrequiredifappliedlearningpedagogiesaretobemaximizedaslearningopportunities is best understood in these terms, as a process ofmeta-cognitionthatfunctionstoimprovethequalityofthoughtandofactionandtherelationshipbetweenthem. Whenunderstoodin this lightanddesignedaccordingly, reflectionbecomes “critical reflection.” It generates learning (articulating ques-tions, confronting bias, examining causality, contrasting theory withpractice, pointing to systemic issues), deepens learning (challengingsimplistic conclusions, invitingalternativeperspectives, asking“why”iteratively),anddocumentslearning(producingtangibleexpressionsofnewunderstandingsforevaluation)(Ash&Clayton,2009aand2009b;Whitney&Clayton, inpress). Asweunderstandit,criticalreflection


DETERMInInGDESIREDLEARnInGouTCoMES Just as with any other intentional design process, designing criti-cal reflection requires beginning with the end in mind (Covey, 1989;Wiggins&McTighe,1998).Specifically,itbeginswiththeidentifica-tionofdesiredlearningoutcomes.Itthenproceedswiththeexpressionoflearninggoalsintermsofassessablelearningobjectivesandcontin-uestothedesignandimplementationofteachingandlearningstrategies(suchasreflection)alignedwiththoseobjectives,allthewhiledevelop-ingassessmentstrategiesthatarewell-matchedtotheobjectivesandtotheteachingandlearningstrategiesandthatcanbeusedtoinformfuturerevisionsofeitherorboth. Instructors,aswellastheprogramsthatsupportthem,havearangeof desired learning outcomes that underlie their use of any particularappliedlearningpedagogy(orcombinationofthem).Figure1providesa conceptual framework for articulatinga categorizationof theseout-comesandtheroleofcriticalreflectioninadvancingthem,usingservice-learningasanexample.Most instructorsuse service-learning tohelptheir students engage more effectively with the content of the courseor the perspective of the discipline while also learning about citizen-shipandaboutthemselvesasindividuals.Inotherwords,theyuseser-vice-learningtohelpstudentslearnatleastinthegeneralcategoriesofacademic enhancement,civic learning,andpersonal growth.Thesecat-egories can apply toother applied learningpedagogies aswell, alongwithadditionalonessuchasintercultural learning(particularlyrelevant

isanevidence-basedexaminationofthesourcesofandgapsinknowl-edgeandpractice,withtheintenttoimproveboth.Designingreflectioneffectively so as to make applied learning educationally meaningfulfirstrequiresthatwemakeclearitsmeaningasanintegrative,analyti-cal, capacity-building process rather than as a superficial exercise innavel-gazing (Ash & Clayton, 2009b; Whitney & Clayton, in press;Zlotkowski&Clayton,2005). Acriticalreflectionprocessthatgenerates,deepens,anddocumentslearningdoesnotoccurautomatically—rather,itmustbecarefullyandintentionallydesigned.Welch(1999)pointsoutthatitisnotenoughtotellstudents“itisnowtimetoreflect”(p.1).Eyler,Giles,andSchmiede(1996)notethatreflection“neednotbeadifficultprocess,but itdoesneedtobeapurposefulandstrategicprocess”(p.16).Especiallygivenhow unfamiliar most students are with learning through reflection onexperience(Clayton&Ash,2004),theyneedastructureandguidancetohelpthemderivemeaningfullearningwhentheyareoutsidethetradi-tionalclassroomsetting,otherwisereflectiontendstobelittlemorethandescriptiveaccountsofexperiencesorventingofpersonalfeelings. Thisarticleexploresprinciplesofgoodpracticeacrossthreestepsinthedesignofcriticalreflectioninappliedlearning: 1) determiningthedesiredoutcomes:learninggoalsandassociated objecives, 2) designingreflectionsoastoachievethoseoutcomes,and 3) integratingformativeandsummativeassessmentintothereflectionprocess.

Itthenpresentsamodelforcriticalreflection—theDEALmodel—thathas been explicitly designed to embody these principles and refinedthroughseveralyearsofresearch. Thediscussionhereisgroundedintheconvictionthatfacilitatorsofstudentlearninginappliedlearningpedagogiesareinstructionaldesign-ers;theymakechoicesthroughoutthedesignprocessthatareinfluencedbytheirgoalsandconstraintsandbytheirstudents’abilitiesaswellastheirown.Designingreflectionproceedsbestwhenframedinscholarlyterms: as a process of experimentation, of continual assessment andrefinement,oflearningwithandalongsidethestudents.Inotherwords,the designer of applied learning opportunities is best understood as areflective practitioner herself—one who engages in the same criticalreflection that sheexpects fromher students—thereby improvingherthinkingandactionrelativeto theworkofgenerating,deepening,anddocumentingstudentlearninginappliedlearning.

SERVICE-

LEARNING

Components

academic material

relevant service

critical reflection

personal growth

civic learning

academic enhancement

Figure 1: Conceptual Framework for the Role of Reflection in Achieving Categories of

Learning

Goals (service-learning example)

Learning Goal Categories

critical thinking within all categories

SERVICE-

LEARNING

Figure1:ConceptualFrameworkfortheRoleofReflectioninAchievingCategoriesofLearningGoals(service-learningexample)


tobecultivatedthroughthepedagogy.Towardthatend,itmaybehelp-fultobeginbylistingmorespecificgoals(suchastheonesgivenaboveforcollaboration)andthendetermininghowbesttoorganizethemintomoregeneralcategories.Suchanactivitymaybeparticularlyimportantwhenagroupofinstructorsundertakesinstructionaldesigntogetheraspartofaprogramorcurriculum,soastomakesurethateveryoneisinagreementwithandworkingtowardsthesamedesiredoutcomes. Figure 2 provides an example of the use of Venn diagrams to ex-press the learninggoalsassociatedwithvariouscategoriesof learningthatmightbedevelopedforinternships.AstheuseoftheVenndiagramssuggests, learning outcomes are often conceptualized as the intersec-tionoftwoormorecategories.ANonprofitStudiescurriculumatNorthCarolinaStateUniversitythatisdesignedwiththreadedservice-learning,forexample,articulateslearningoutcomesattheintersectionofacadem-icenhancementandciviclearningintermsoflearninggoalsincluding:aligningmission,methods,andresources;balancingindividualinterestsandthecommongood;movingbeyondcharitytosystemicchange;capi-talizingonopportunitiesassociatedwithdiversity;andearningthepub-lictrust(Jameson,Clayton,&Bringle,2008).

instudyabroad),professional development(especiallyforinternships),and research skill development (in undergraduate research). Critical thinkingmightbeseenasitsowncategoryofoutcomesorasadimen-sionofothercategories;additionalmeta-leveloutcomesrelatedtolearn-ingprocessesmightincludeemotional intelligenceortheabilitytomakeconnectionsbetweenideas. Given thepublicpurposesofhighereducation (Boyer,1996;Salt-marsh, Hartley, & Clayton, 2009; O’Meara & Rice, 2005; Saltmarsh,2005),designersofanyappliedlearningpedagogymightwellconsidercivic learningasarelevantcategoryoflearning.Battistoni(2002)offersthirteenconceptualframeworksforunderstanding“civic”thatarelinkedtovariousdisciplinesandtherebysuggestsawidevarietyofwaysitcanbedefined,suchasintermsofparticipatorydemocracy,socialjustice,oranethicofcare.Specificlearninggoalsinthiscategorymightrelatetosuchissuesaschangeagency,power,privilege,leadership,economicandpoliticalsystems,governmentalprocesses,communityorganizing,andpublicproblem-solving.Inlightofthemulti-facetednatureofthiscategory,appliedlearningopportunitiesofalltypescanbedesignedtoincludeit.Forexample,studentsinvolvedinundergraduateresearchcanconsider the social drivers for and implicationsof both their researchquestionsandtheirprocessofinquiry;thosestudyingabroadcanfocusattentionontheinterconnectionsbetweenlocalandglobalissuesandonthewayscultureshapesnotionsofcitizenship;internscanexploretherolesofcorporationsascitizensandtherangeofopportunitiestointe-gratetheirprofessionalandciviclives. Similarly, applied learning pedagogies often involve interactionswith others—classmates, mentors, community members, lab partners,officemates—and therefore lend themselves readily to learning in thegeneralcategoryofcollaboration.Associatedlearninggoalsmightin-cludedevelopingstudents’abilitiestocommunicatewithdiverseothers,makedecisionsasagroup,assessgroupmembers’strengthsandweak-nessesandallocateresponsibilityaccordingly,handleinterpersonalcon-flicteffectively,holdthemselvesandothersaccountabletogroupnorms,developsharedvisions,andmonitorprogress towardcollectiveobjec-tivesandreachconsensusonappropriatechangesintheirapproach. Asthepreviousexamplesuggests,learninggoalswithinanyonecat-egoryoflearningcanoftencrossintoanothercategory—collaborationcouldalsobeunderstoodasanelementofdiversitylearning,profession-aldevelopment,personalgrowth,orciviclearning.Itisthereforeuptoinstructors,programadministrators,and/orstudentstodecidehowbesttoexpressthecategoriesoflearningandtheassociatedlearninggoalsfortheirparticularsituation.Becausethesecategoriesarelikelygoingtobe-comeheadingsin,forexample,assessmentreports,particularattentionshouldbepaidtowhatbestrepresentsthekeyarenasoflearningthatare

Components of Internships

Learning Goals of Internships

(critical thinking within all categories)

academic discipline

critical reflection

work experience

Figure 2: Conceptual Framework for the Role of Reflection in Achieving Learning Goals

(internship example)

Professional development:

• teamwork

• business etiquette

• dynamics and skills of

business networking

Personal growth: • communicating

across difference

• assessing strengths

and weaknesses

• exploring relationship

between personal

values and business

values

Civic learning: • corporations as local

and global citizens

• power dynamics within

organizations

• organizational

leadership

•

Figure2:ConceptualFrameworkfortheRoleofReflectioninAchievingLearningGoals(internshipexample)


thenext, fromsimpler tomorecomplexdimensionsof reasoning.Forexample, applying an academic concept effectively requires having agoodunderstandingofit,whichitselfinvolveshavingbasicknowledgeoftheunderlyingfactsortheories. Table2providesanexample,drawnfromservice-learning,oftheuseofBloom’sTaxonomytomovefromgeneralcategoriesof learning tospecificlearninggoalsandthentoassessablelearningobjectives.

Whetherstartingwith thegeneralcategoriesandworkingdown tomorespecificlearninggoalswithinthemorstartingwithlearninggoalsandthendeterminingthemostusefulwaytocategorizethem,developingthisbroadstructuretoexpressandorganizedesiredlearningoutcomesiskeytoundertakinganintentionalinstructionaldesignprocess,tocom-municatingtherationalesforappliedlearningtostudentsandcolleagues,andtostructuringassessmentstrategiesandsharingresultantdata.Thisstructure for thinking about learning outcomes provides an importantfoundationfordevelopingstrongapproachestocriticalreflection.

FRoMLEARnInGGoALSToLEARnInGoBJECTIVES

Oncethegeneralcategoriesoflearningandtheirassociatedlearninggoalshavebeendetermined,theinstructionaldesigner’snexttaskistoexpressthelearninggoalsasassessablelearningobjectives.Goalssuchas“studentswilllearnaboutprojectmanagement”(internship),“studentswillunderstandthechallengesfacingschoolsintheirattemptstoimple-mentstateandfederaleducationpolicies”(service-learning),“studentswillappreciatethesimilaritiesaswellasthedifferencesbetweentheirhome and host cultures” (study abroad), or “students will understandthedifferencesbetweenquantitativeandqualitativeresearchmethods”(undergraduate research) are difficult to translate into effectivepedagogicalpractice. Bloom’s Taxonomy of Educational Objectives (1956) provides afoundationforturninglearninggoalsintoassessablelearningobjectives,whichthendrivetherestofthedesignprocess.Thetaxonomyincludeslearning in threedomains: cognitive, affective, andpsychomotor; thisdiscussion refers to the Taxonomy of Educational Objectives in theCognitiveDomain.Althoughmodifiedandre-orderedbysomescholarsinrecentyears,Bloometal.originallyidentifiedsixlevels,eachwithas-sociated—andassessable—learningbehaviors,assummarizedinTable1.Acentralandwidelyshared,althoughnotuniversal,tenetofourread-ingofthetaxonomyisitshierarchicalnature—eachlevelbuildstoward

Table 1: Bloom’s Taxonomy and Associated Learning Behaviors

Bloom’s Classification Examples of Learning-Related

Behaviors

Knowledge

Identify, define, order

Comprehension Explain describe, restate

Application Apply, solve, choose

Analysis Analyze, compare, contrast

Synthesis Synthesize, develop, propose

Evaluation Evaluate, assess, judge, critique

Table1:Bloom’sTaxonomyandAssociatedLearningBehaviors

Using Bloom’s Taxonomy in this way, to achieve a high level ofclarityregardingdesiredlearningoutcomesandtoexpresstheminas-sessable language, enables instructors to design reflection that targetslearning objectives in developmentally-appropriate ways, buildingtoward thehighest level of learningdeemed appropriate in anygiveninstance.The learningobjectives thusbecomeboth the roadmap thatguides thedesignofreflectionactivitiesand thebasis fordeterminingwhether the intendeddestinationhasbeenreachedandadequatelyex-pressedintheproductsofreflection.

Table2:usingBloom’sTaxonomytoMovefromGeneralCategoriesofLearningtoSpecificLearningGoalstoAssessableLearningobjectives(service-learningexample)

Table 2: Using Bloom’s Taxonomy to Move from General Categories of Learning to Specific

Learning Goals to Assessable Learning Objectives (service-learning example)

Learning Objective

Level

Category: Personal Growth

Learning Goal:

Students will consider ways to refine their

skills

Category: Civic Learning

Learning Goal:

Students will become more effective change

agents

Category: Academic Enhancement

Learning Goal:

Students will understand the Stages of Change

model

LO 1: Identify

Identify a particular skill of

yours that you need to develop

further.

Identify the collective

objectives at stake and the

approach you or others took

toward meeting them.

Identify the Stages of Change

model.

LO 2: Explain

Explain the skill so that

someone who does not know

you can understand it.

Explain the objectives and the

approach you and / or others

took toward meeting them so

that someone not involved can

understand.

Explain the Stages of Change

model so that someone not in the

course can understand it.

LO 3: Apply

Apply your understanding of

this skill in the context of your

service-learning experience

and (as applicable) in other

areas of your life.

Apply your understanding of

the approach in the context of

the objectives at stake.

Apply your understanding of the

Stages of Change model in the

context of the experience.

LO 4: Analyze

Analyze the sources of this

skill in your life.

Analyze the approach in light of

alternatives.

Analyze the similarities and

differences between the Stages of

Change model as presented in the

text and as it emerged in the

community.

LO 5: Synthesize

Develop the steps necessary to

improve upon this skill in the

short term, in your service-

learning activities and (as

applicable) in other areas of

your life.

Develop the steps necessary to

make any needed improvements

in your / their approaches

(and/or in the objectives) in the

short term.

Develop an enhanced

understanding of the Stages of

Change model in light of the

experience.

LO 6: Evaluate

Evaluate your strategies for

refining your skills over the

long term.

Evaluate your / their

approaches in terms of the

prospects for long-term,

sustainable, and/or systemic

change.

Evaluate the completeness of

your understanding of the Stages

of Change model and of its use in

the community.


DESIGnInGREFLECTIonToAChIEVEDESIREDLEARnInG

Effectivelydesigningcriticalreflectioninvolvesmakingaseriesofchoices thatare informedby thedesired learningoutcomesaswellasbytheopportunitiesandconstraintsthatcomewiththespecificcontextinwhichappliedlearningisbeingimplementedandbytheabilitiesofthe participants. These choices produce an overall reflection strategyorover-archingstructurethatmaycombinevariousreflectionactivitiesor mechanisms—such as journal entries, online chat sessions, posterpresentations, worksheets, or discussion sessions. Questions suchas those in Table 3 can help guide the design of reflection strategiesandmechanisms. Theresultofsuchintentionaldesignworkisacustomizedplanthatintegratescriticalreflectionintothecoreofappliedlearningexperiences.Thisplanmaybemaximizedbydesigningthereflectionstrategysuchthatindividualreflectionmechanismsbuildononeanothercumulatively,sothatstudentslearnhowtolearnthroughreflectionaswellasimprovethequalityoftheirlearningandtheirpracticeovertime.Table4sum-marizesabodyofprinciplesofgoodpracticethathasemergedtosupporttheinstructionaldesignerinmakingthechoicesthatproducehighqual-ityreflectionstrategiesandmechanisms. Eachofthesesetsofcharacteristicsofhighqualitycriticalreflection

includesexplicitlinkagetodesiredlearningoutcomes,andBloom’sTax-onomyprovidesastructuretofacilitatethedesignofreflectionaccord-ingly.TheexamplereflectionactivityprovidedinFigure3demonstratesthedesignof reflectionprompts—for the learninggoalofunderstand-ingstrengthsandweaknesses,inthecategoryofpersonalgrowth—thatguidestudents step-by-step toever-higher levelsof reasoning throughprompts thatareexplicitly structured inaccordancewith the levelsofthetaxonomy. Afocusoncriticalthinkingisakeycharacteristicofcriticalreflec-tion.ThereflectionguidedbythepromptsinFigure3canprogresstoever-higherlevelsofreasoningbutdosopoorly,inanillogical,unclearway that is uninformedby considerationofmultipleperspectives andthatfailstoengagewiththetruecomplexityoftheissues.Criticalthink-ing,asoutlinedbyPaulandElder(2002),isbasedonuniversalintellec-tualstandardsthatincludeaccuracy,clarity,relevance,depth,breadth,logic, significance, and fairness. Many of the potential shortcomingsof reflection described in the introduction—reinforcing stereotypes,generalizing inappropriately on the basis of limited data, missing themostsignificantlearninginanexperience—areindicativeofandresultfrom poorly developed critical thinking abilities. Providing guidancein thisarea is, therefore,anecessarycorollary to theuseofhierarchi-callearningobjectivesinthedesignofcriticalreflection(Ash,Clayton,&Atkinson, 2005).Table 5 provides anoverviewof the standardsof

Table 4: Characteristics of High Quality Reflection

High Quality Reflection …

is continuous (ongoing)

is connected (with assignments and activities related to and building on one

another and including explicit integration with learning goals and academic

material)

is challenging (including in terms of the expectation that students take

responsibility for their own learning)

Eyler et al. (1996)

is contextualized (to the community setting and broader public issues and

to the students’ own particular roles)

links experience to learning

is guided

occurs regularly

involves feedback to the learner to enhance the learning

Bringle & Hatcher

(1999)

helps clarify values

is oriented toward specific learning objectives

is integrative

is assessed in terms of critical thinking

includes goal setting

Zlotkowski &

Clayton (2005)

generates change in the learner’s life

Table4:CharacteristicsofhighQualityReflection

Table 3: Questions to Guide the Design of Reflection Strategies and Mechanisms

Reflection Strategies

When and how often will reflection occur?

Before, during, and after the experience?

Will students reflect iteratively such that reflection builds on itself over time?

Where will reflection occur?

In or outside the classroom?

Who will facilitate and/or particpate in reflection?

Instructors, members of the community or workplace, peers?

How will feedback be provided and/or reflection products graded?

What is the relationship between amount of feedback and level of expected outcomes?

What is the relationship between the reflection products and the overall grade?echanisms

Reflection Mechanisms

Toward what specific learning goals and objectives will the particular activity be guided?

What medium will be used for the activity: written assignments, worksheets, spectrum activities,

photographs, videos, games, drawings, online forums, in-class discussion, out-of-class reflection

sessions, concept maps, etc.?

What prompts will be used to guide the activity?

What products will demonstrate the learning the activity generates: essays, PowerPoint or poster

presentations, oral exams, etc.?

Note that in a critical reflection process, the products used to demonstrate learning are in

many cases the same as the medium used to generate it

What criteria will be used to assess the learning so demonstrated ?

Table3:QuestionstoGuidetheDesignofReflectionStrategiesandMechanisms

ReflectionStrategies


critical thinking (with the addition of integration and writing quality),alongwithpromptingquestionsthatcanbeusedbystudentsthemselvestoimprovethequalityoftheirreasoningandbypeersand/orinstructorsasfeedbackonreflectionproducts. Using these tools together—designing reflection mechanismsthrough the use of hierarchical learning objectives and improving thequalityofthinkingateachofthelevelsofreasoningthroughtheuseofcriticalthinkingstandards—willhelpto generateanddeepenlearninginanappliedlearningenvironment.Theproductsofsuchintentionallyde-signedreflection,inturn,documentlearningforpurposesofgradingorresearchaswellasforstudentuseinguidingfuturethinkingandaction.

InTEGRATInGFoRMATIVEAnDSuMMATIVEASSESSMEnTInToThEREFLECTIonPRoCESS

Designing an intentional approach to critical reflection in appliedlearningalsoinvolvesthedevelopmentofanassessmentstrategy.Justasreflectionismuchmoreeffectivelyimplementednotonlyattheendofanappliedlearningcourseorprojectbutthroughout,sotooisassess-mentmorevaluablewhenitisdesignedfromthebeginningandisitselfevaluatedandmodifiedasneededthroughout.

Figure3:SampleBloom-basedReflectionMechanism(undergraduateresearchexample)

Figure 3: Sample Bloom-based Reflection Mechanism (undergraduate research example)

According to Parker Palmer (2000), “limitations and liabilities are the flip side of our

gifts … a particular weakness is the inevitable trade-off for a particular strength.”

There is nothing “wrong” with us that we need to “fix,” he suggests. Rather, we are

who we are; sometimes our personal characteristics serve us well (and we think of

them as strengths), and sometimes they serve us ill (and we think of them as

weaknesses) The attempt to “fix” our liabilities will inevitably alter their “flip side”

gifts as well. (pp. 52-53).

Individually and in writing …

Identify and explain a personal characteristic that you tend to think of as a weakness

in your role as a researcher

Apply Palmer’s distinction to this characteristic: What gift or strength could be the

“flip side” of this “weakness”?

Discuss with a partner …

Compare and contrast a research-related situation in which the “weakness” emerged

and one in which its “flip side” strength emerged. Why do you think each

emerged as it did and what were the consequences?

If Palmer is correct regarding the relationship between our gifts or strengths and our

limitations or liabilities, what do you think are the implications for your

approach to personal and professional development as a researcher?

Individually and in writing …

Critique Palmer’s distinction: Do you agree with him? Why or why not? What, if

anything, would you change in his thinking?

Table

5:

Cri

tica

l T

hin

king S

tandard

s

Cri

tica

l

Th

ink

ing

Sta

nd

ard

Des

crip

tion

Ass

oci

ate

d Q

ues

tion

s to

Ch

eck

you

r T

hin

kin

g

Inte

gra

tio

n

Ser

vic

e ex

per

ien

ce c

lear

ly r

elat

ed t

o t

he

lear

nin

g.

Hav

e I

clea

rly

sh

ow

n t

he

con

nec

tio

n b

etw

een

my

ex

per

ien

ce

and

my

lea

rnin

g?

Cla

rity

E

xp

and

s o

n i

dea

s, e

xp

ress

id

eas

in a

no

ther

way

,

pro

vid

es e

xam

ple

s o

r il

lust

rati

on

s w

her

e ap

pro

pri

ate.

Did

I g

ive

an e

xam

ple

?

Is i

t cl

ear

wh

at I

mea

n b

y t

his

?

Co

uld

I e

lab

ora

te f

urt

her

?

Acc

ura

cy

All

sta

tem

ents

are

fac

tual

ly c

orr

ect

and

/or

sup

po

rted

wit

h e

vid

ence

.

Ho

w d

o I

kn

ow

th

is?

Is t

his

tru

e?

Ho

w c

ou

ld I

ch

eck

on

th

is o

r v

erif

y i

t?

Pre

cisi

on

S

tate

men

ts c

on

tain

sp

ecif

ic i

nfo

rmat

ion

.

Can

I b

e m

ore

sp

ecif

ic?

Hav

e I

pro

vid

ed s

uff

icie

nt

det

ail?

Rel

evan

ce

All

sta

tem

ents

are

rel

evan

t to

th

e q

ues

tio

n a

t h

and

; al

l

stat

emen

ts c

on

nec

t to

th

e ce

ntr

al p

oin

t.

Ho

w d

oes

th

is r

elat

e to

th

e is

sue

bei

ng

dis

cuss

ed?

Ho

w d

oes

th

is h

elp

us/

me

dea

l w

ith

th

e is

sue

bei

ng

dis

cuss

ed?

Dep

th

Ex

pla

ins

the

reas

on

s b

ehin

d c

on

clu

sio

ns

and

anti

cip

ates

an

d a

nsw

ers

the

qu

esti

on

s th

at t

he

reas

on

ing

rai

ses

and

/or

ack

no

wle

dg

es t

he

com

ple

xit

y

of

the

issu

e.

Wh

y i

s th

is s

o?

Wh

at a

re s

om

e o

f th

e co

mp

lex

itie

s h

ere?

Wh

at w

ou

ld i

t ta

ke

for

this

to

hap

pen

?

Wo

uld

th

is b

e ea

sy t

o d

o?

Bre

adth

C

on

sid

ers

alte

rnat

ive

po

ints

of

vie

w o

r h

ow

so

meo

ne

else

mig

ht

hav

e in

terp

rete

d t

he

situ

atio

n.

Wo

uld

th

is l

oo

k t

he

sam

e fr

om

th

e p

ersp

ecti

ve

of…

.?

Is t

her

e an

oth

er w

ay t

o i

nte

rpre

t w

hat

th

is m

ean

s?

Lo

gic

T

he

lin

e o

f re

aso

nin

g m

akes

sen

se a

nd

fo

llo

ws

fro

m

the

fact

s an

d/o

r w

hat

has

bee

n s

aid

.

Do

es w

hat

I s

aid

at

the

beg

inn

ing

fit

wit

h w

hat

I c

on

clu

ded

at

the

end

?

Do

my

co

ncl

usi

on

s m

atch

th

e ev

iden

ce t

hat

I h

ave

pre

sen

ted

?

Sig

nif

ican

ce

Th

e co

ncl

usi

on

s o

r g

oal

s re

pre

sen

t a

(th

e) m

ajo

r is

sue

rais

ed b

y t

he

refl

ecti

on

on

ex

per

ien

ce.

Is t

his

th

e m

ost

im

po

rtan

t is

sue

to f

ocu

s o

n?

Is t

his

mo

st s

ign

ific

ant

pro

ble

m t

o c

on

sid

er?

Fai

rnes

s

Oth

er p

oin

ts o

f v

iew

are

rep

rese

nte

d w

ith

in

teg

rity

(wit

ho

ut

bia

s o

r d

isto

rtio

n).

Hav

e I

rep

rese

nte

d t

his

vie

wp

oin

t in

su

ch a

way

th

at t

he

per

son

wh

o h

old

s it

wo

uld

ag

ree

wit

h m

y c

har

acte

riza

tio

n?

Mo

dif

ied

so

urc

e: P

aul,

R &

Eld

er,

L.

20

01

. T

he

Min

iatu

re G

uid

e to

Cri

tica

l T

hin

kin

g.

Th

e F

ou

nd

atio

n f

or

Cri

tica

l T

hin

kin

g.

San

ta R

osa

, C

A.

ww

w.c

riti

calt

hin

kin

g.o

rg

Mod

ified

sou

rce:

Pau

l,R

.P.&

Eld

er,L

.200

1.T

heM

inia

ture

Gui

deto

Cri

tica

lThi

nkin

g.T

heF

ound

atio

nfo

rC

riti

calT

hink

ing.

San

taR

osa,

CA

.ww

w.c

riti

calt

hink

ing.

org

Table5:CriticalThinkingStandards


provedifficultforstudentstograsp;suchinformationcaninformdiscus-sionofhowtheseconceptsorskillsaretaughtinthecoursesorprogramsassociatedwithorprerequisitetotheappliedlearningactivity. Havingsorted through thevariouspurposesofassessment, thede-signer of applied learning pedagogies faces additional choice pointsrelated tohowassessment is implemented, includingthenatureof theproductsorevidencethatwillbeexamined.Willassessmentinvolveex-traactivitiesthatarenotrelatedtothelearningprocess(forexample,preandpostquestionnaires)or assignments andproducts that are alreadypartofthecourseorproject(forexample,reflectionproductsoressays)?Inaquestionnaire, studentsmightbeasked towhatdegree they thinktheyhavemet thelearningobjectivesof theirappliedlearningexperi-ence;inacourse-embeddedassignment,studentswouldbeaskedtore-spondtoapromptorprompts,andtheresultantproductwouldbeevalu-ated against the objectives. Practitioner-scholars such as Eyler (2000)suggestthattheformeroftenconfusesstudentsatisfactionwithstudentlearningandthereforecallforthedevelopmentofapproachesthatsup-portstudentsindoingthelatter.Inaddition,acourse-embeddedprocessisgenerallylesstime-consuming,forbothstudentsandinstructors,thanthe interview, focus group, or portfolio methods often used (Eyler &Giles,1999;Gelmon,Holland,Driscoll,Spring,&Kerrigan,2001)andrequiresmoreintentionalintegrationofassessmentwiththeteachingandlearningprocess. Anotherissueisthedeterminationofcriteriathatwillbeusedtoeval-uate theproducts thatdemonstrate learning. Inotherwords,whatwillbetheindicatorsofthedegreetowhichthestudenthasmetthelearningobjectivesorofthequalityoflearningoutcomes?Thecreationofaru-bricthatexpressesvaryinglevelsofqualityormastery,fromnovicetoexpertorfromunder-developedtoexcellent,canbeextremelyhelpfulinguidingthisprocess.Forexample,iftheobjectiveisforinternstobeabletodeterminetheappropriateapproachtoaparticularworkplacesituation(e.g.,ateammembernotpullingherweight),arubricinwhichresponsesarecategorizedbydegreeofsophisticationand/orefficacycouldbeusedforassessment.Ifanobjectiveismeta-cognitiveand/orisuniquetothestudents as individuals and their particular experiences (for example,thatstudentsareabletoevaluateapersonalstrengthorweaknessinlightoftheirprofessionalgoals),thenarubricbasedonBloom’sTaxonomythat expresses levels of reasoning may be in order. The Standards ofCriticalThinkingdescribedearliercanalsobeturnedintoarubric(seeTable6,forexample)thatcanbeusedtoassessqualityofreasoning. Anintegratedapproachtoassessmentandreflectionincludesusingthesamesetofobjectivesandstandardsandtoolstogeneratelearning(throughreflectionprompts),todeepenlearning(throughformativeas-sessmentor feedback), and todocument learning (throughsummative

Assessment can be designed for summative purposes and used attheendofaprocesstomeasureanddocumentoutcomes,anditcanbedesignedfor formativepurposesandusedduringaprocessasawaytocontinuouslyimproveboththeprocessandtheoutcomes.Asummativeassessmentprocessthatisgroundedinwell-articulatedlearningobjec-tivescanbeusedbothtogradestudentproductsandtoreportoutcomesatprogramorcurriculumlevels.Summativeassessmentintheformofgradinggenerally involves judging thedegree towhich studentshavemet the learning objectives. Such assessment can be standards basedand therefore measure the ultimate attainment of an objective at theendoftheexperience,oritcanbebasedonimprovementandthereforemeasurechangeovertime.Arelateddesignchoicethatoftenemergesat theprogramor curriculum level iswhether theultimate attainmentorthechangeovertimeistobeassessedwithinasinglecourseorap-pliedlearningproject,acrossasequenceofcoursesorprojects,orboth.Instructorsand/oradministratorsneedtodecideontheformsummativeassessmentreportsshouldtake, in lightof theuses towhichtheywillbeputandtheaudiencesforwhomtheyareintended.Forexample,willthe assessment be expressed quantitatively, such as the percentage ofstudentswhosereflectiononexperiencedemonstratesfulfillmentofthedesiredoutcomes,orwillthereportprovidequalitativeinformationwithexamplesofstudentlearningoutcomes,orboth? Facultyandstudentsusingappliedlearningpedagogieswillfindval-ueinformativelyassessingbothlearningandtheteachingandlearningprocessandprogramsthatgenerateit.Formativeassessmentisincreas-inglyrecognizedaskeytoeffectivelydesigningteachingandlearning.AsnotedbytheNationalResearchCouncil(2001),“Studentswilllearnmoreifinstruction[inthiscase,reflectioninappliedlearning]andas-sessment are integrally related. [P]roviding students with informationaboutparticularqualitiesoftheirworkandwhattheycandotoimproveit is crucial for maximizing learning” (p. 258). Feedback combinedwith opportunities to apply it (e.g. through revision of their work) isanapproachtoformativeassessmentthathelpsstudentslearnnotonlycontent,butmeta-cognitiveskillsaswell—inthiscase,learninghowtolearnthroughtheoftenunfamiliarprocessofcriticalreflection. Formativeassessmentcanalsobeusedtocheckthereflectionprocessagainstthelearningoutcomesitgeneratessoastorefineboththelearn-inggoals andobjectives and the reflection strategies andmechanismsdesignedtomeetthem.Instructorsmightreviewstudentproductscriti-callynotonlyinordertoprovidehelpfulfeedbacktoimprovestudents’thinkingbutalsotogaugetheeffectivenessoftheirowndesign(e.g.,theclarityofthereflectionprompts)andtoprovidethemselveswithfeed-back to improve it.Such formativeassessmentalsoprovidesvaluablefeedback to instructors regarding, for example, concepts or skills that


ThEDEALMoDELFoRCRITICALREFLECTIon Anexampleofanapproachtocriticalreflectionexplicitlydesignedin accordancewith theprinciplesofgoodpracticediscussedabove istheDEALModelforCriticalReflection(Ash&Clayton,2004;Ash&Clayton, 2009a, 2009b)—the product of a multi-year scholarship ofteachingandlearningprojectinvolvingstudentsandfacultyfromava-rietyofdisciplines.Originallydevelopedinthecontextofservice-learn-ing,DEALhasbeenusedacrossarangeoftraditionalandexperientialpedagogies;inK-12,undergraduate,andgraduatecoursesandcurricula;andinco-curricularaswellasprofessionaltrainingsettings.TheDEALmodelconsistsofthreesequentialsteps(seeFigure4): 1. Descriptionofexperiencesinanobjectiveanddetailedmanner; 2. Examinationofthoseexperiencesinlightofspecificlearning goalsorobjectives;and 3. ArticulationofLearning,includinggoalsforfutureactionthat canthenbetakenforwardintothenextexperienceforim- provedpracticeandfurtherrefinementoflearning.

assessment or grading and reporting outcomes). Reflection promptsbasedonBloom’sTaxonomycanbothguidestudentstodesiredlevelsofreasoninganddeterminethelevelofreasoningtheyhaveattained.Criti-calthinkingstandardscanbeusedasbothaformativeguidetoimprovestudentreasoningandasummativetooltoevaluateitsqualityintheend.Makingvisiblesuch integrationofreflectionandassessment iskey inhelpingstudentsbecomeincreasinglyawareofandresponsiblefortheirownlearningprocesses. Thecreationofanassessmentstrategyisasimportantasthearticula-tionof the learninggoalsandassociatedobjectives,andall shouldbedevelopedinparallelduringthedesignofthereflectionactivities.Tryingtoassessalearninggoalthathasnotbeenarticulatedasanassessableobjective(e.g.,“studentswillunderstand…,”“studentswillappreciate…,”“studentswilllearnabout…”)isusuallyanexerciseinfrustration.Areflectionmechanismthatisnotmappedtolearningobjectivesisoftenamissedopportunityformaximizedlearningaswellasahindrancetousing reflectionproducts toassess learning.Andanobjective thatex-pressesdesiredlearningthatcannotbeachievedthroughthepedagogyinquestion,muchlessassessed,should,likealloftheabove,sendthedesignerbacktothedrawingboard.

Table 6: Critical Thinking Rubric [excerpts]

completely lacking (1) under-developed (2) good (3) excellent (4)

Accuracy

Consistently makes

inaccurate statements

and/or fails to provide

supporting evidence for

claims

Makes several

inaccurate statements

and/or supports few

statements with evidence

Usually but not always

makes statements that

are accurate and well-

supported with evidence

Consistently makes

statements that are

accurate and well-

supported with evidence

Clarity

Consistently fails to

provide examples, to

illustrate points, to define

terms, and/or to express

ideas in other ways

Only occasionally

provides examples,

illustrates points, defines

terms, and/or expresses

ideas in other ways

Usually but not always

provides examples,

illustrates points, defines

terms, and/or expresses

ideas in other ways

Consistently provides

examples, illustrates

points, defines terms,

and/or expresses ideas in

other ways

Depth

Fails to address salient

questions that arise from

statements being made;

consistently over-

simplifies when making

connections; fails to

consider any of the

complexities of the issue

Addresses few of the

salient questions that

arise from statements

being made; often over-


connections; considers

little of the complexity

of the issue

Addresses some but not

all of the salient questions

that arise from statements

being made; rarely over-


connections; considers

some but not all of the

full complexity of the

issue

Thoroughly addresses

salient questions that arise

from statements being

made; avoids over-

simplifying when making

connections; considers the

full complexity of the

issue

Breadth

Ignores or superficially

considers alternative

points of view and/or

interpretations

Gives minimal

consideration to

alternative points of

view and/or

interpretations and

makes very limited use

of them in shaping the

learning being

articulated

Gives some consideration

to alternative points of

view and/or

interpretations and makes

some use of them in

shaping the learning

being articulated

Gives meaningful

consideration to

alternative points of view

and/or interpretations and

makes very good use of

them in shaping the

learning being articulated

Fairness

Consistently represents

others’ perspectives in a

biased or distorted way

Occasionally represents

others’ perspectives in a

biased or distorted way

Often but not always

represents others’

perspectives with

integrity

Consistently represents

others’ perspectives with

integrity (without bias or

distortion)

[Modified source: Paul, R & Elder, L. 2001. The Miniature Guide to Critical Thinking. The Foundation for Critical Thinking. Santa Rosa, CA. www.criticalthinking.org]

Table6:CriticalThinkingRubric[excerpts]

[Modifiedsource:Paul,R.P.&Elder,L.2001.The Miniature Guide to Critical Thinking.TheFoundationforCriti-calThinking.SantaRosa,CA.www.criticalthinking.org]

Figure 4: Schematic Overview of the DEAL Model for Critical Reflection

Category #1 Goals &

Objectives

Category #3 Goals &

Objectives

Engage in experience

Engage in experience and

test learning and/or

implement goals

Examine

per learning goals/objectives in each category of learning

Describe experience objectively

Articulate Learning including setting goals

in each category

Category #2 Goals &

Objectives

Figure4:SchematicoverviewoftheDEALModelforCriticalReflection


ingandtheirfutureactions.Itconsistsoffourprompts:(a)What did I learn?;(b)How did I learn it?;(c)Whydoesitmatter?;and(d)What will I do in light of it? TheDEALmodelthusdoesnotbeginbutratherends with the question “What did you learn,” in accordance with theunderstanding of reflection as the component of applied learning thatgenerateslearning. ThegeneralstructureprovidedbytheDEALmodelcanbeusedtoguidecriticalreflectiononline,inanoraldiscussion,inawrittenjournalentryoressay,orinanycombinationofmechanisms.Forexample,De-scriptionmightbedoneonlinebyeachstudentindividually,Examina-tionorallybyagroupofstudents,andArticulationofLearningasawrit-tenessay.TheDEALmodelcanbeusedtostructure“light”reflection,asina30minutein-classactivityoranonlinechatthatproducessimple(e.g.,foursentence)ArticulatedLearnings.

ABLooM-BASEDuSEoFDEAL DEAL can also guide more in-depth critical reflection that targetshigher order reasoning and critical thinking through prompts that aretieddirectlytohierarchicallearningobjectives.Suchanapproachmightbeusednotmerelytostimulatequestionsandsurfaceissuesforfurtherdiscussion,as in thegoal-basedexampleinTable7above,butalsotosupportstudentsexplicitlyindevelopingreasoningabilitiesandtoassessthequalityoftheirreasoning. InaparticularlycomprehensiveversionoftheDEALmodel(Ash&Clayton,2009a,2009b),designedtofacilitatestudentreasoningallthewayuptothelevelofevaluationinBloom’sTaxonomy,theExamine

Eachstepofthismodelrequiresspecificprompts,whichprovidetheguidancenecessaryforstudentstoengageintheoftentimescounter-nor-mativeactivityofdevelopingtheirownlearningratherthanreproducingwhattheirinstructorshavetaughtthem(Clayton&Ash,2004;Howard,1998).ThediscussionthatfollowssummarizeseachstepintheDEALmodelandprovidessampleprompts.

DESCRIBE Objective, detailed description of an experience provides a strongfoundationformeaning-makinginthecriticalreflectionprocess;itisawaytomaketheexperiencepresentandtoensurethatstudentshaveac-cesstoallrelevantaspectsofitastheyengageinreflection.Thisstepisnotassimpleasitmightappear,asstudentsoftenprefertojumpstraightintointerpretation.Itisalsoeasytooverlookorunder-valuethedetailsthatareoftenmostsignificant,soenhancedskillsofmindfulnessandat-tentivenessareoftenrequiredfor—anddevelopedby—thisstep.Reflec-tionpromptsassociatedwiththeDescribestepaskstudentstoaddresssuch issuesaswhenandwhere theexperience inquestion tookplace,whowasandwasnotpresent,whattheyandothersdidanddidnotdo,whattheysawandheard,andsoon.

ExAMInE The DEAL model is explicitly designed to move students beyondsummarizing their experiences,whichall toooften resultswhena re-flectionactivityisassigned,intomeaning-making.InthesecondstepofDEAL,promptsthathelpstudentsExaminetheirexperiencesarelinkedtothedesiredlearningoutcomes—whetherexpressedaslearninggoalsor, in a more assessable fashion, as learning objectives—within eachcategoryoflearning.Table7providesexamplesofpromptsdrawnfromlearninggoals in thegeneralcategoryofcivic learning; someinstruc-torsmayprefertodevelopExaminepromptsfromlearninggoalssuchastheseratherthanfromassessableobjectiveswhen,forexample,theintent is tostimulatequestionsorsurface issuesfor furtherdiscussionratherthantoevaluatestudents’reasoning.ARTICuLATELEARnInG

The third step of the DEAL model supports students inArticulating the Learning that the two previous steps have begun togenerate,whileprovidingfurtherguidanceincontinuingtoexpandanddeepenthatlearning.Ithelpsthemcapturetheirlearninginsuchawayastobeabletoactonitandtherebyimprovethequalityoftheirlearn-

Table 7: DEAL Model Sample “Examine” Prompts Based on Learning Goals in the

General

Category of Civic Learning

Learning Goals Sample Examine Prompts

What was I / someone else trying to accomplish?

In taking the actions I / they did, was the focus on symptoms of problems

or causes of problems?

Was the focus (symptom or cause) appropriate to the situation?

Students will explore

the dynamics of change

agency

How might I / they focus more on underlying causes in the future?

In what ways did differentials in power and privilege emerge in this

experience?

What are the sources of power and privilege in this situation?

Students will learn

about power and

privilege

Who benefits and who is harmed?

What is in the interest of the common good in this situation? What is in

the interest of (whose) individual interests or rights?

In what ways is the individual good (mine / others) linked to and/or

contrary to the common good?

Students will appreciate

the tension between

individual interests and

the common good

What trade-offs between them are involved? Who made the trade-offs?

Were the trade-offs made appropriate or inappropriate and why?

Table7:DEALModelSample“Examine”PromptsBasedonLearningGoalsintheGeneralCategoryofCivicLearning


InPartIIoftheArticulateLearningstep,studentsareaskedtoevalu-atetheirwrittenproductsusingachecklist,whichincludesthestandardsof critical thinking, and to rewrite their “I learned that” statement asneededtoensurethatitexpressesthehighestleveloflearningtheyhaveachieved.

and theArticulateLearning steps eachhave twoparts.After studentsDescribeanexperience,theysurfaceoneormorekeyideasforfurtherthoughtinExaminePartIandthentakeoneofthoseideasfromidenti-ficationandexplanationthroughapplicationandtoanalysisinExaminePartII.InPartIoftheArticulateLearningsteptheysynthesizeanewunderstandingofthekeyideaandevaluatechangesintheirthinking,andinPartIItheyevaluatethewrittenexpressionofthatthinkingandreviseitasneeded. For example, in the category of personal growth, Examine Part ImightincludesomeorallofthepromptsinTable8,whichareorientedtowardthelearninggoalsthatcomprisethiscategoryandwhichencour-agestudentstofocusontheirownparticularpersonalcharacteristics. ThenPartIIoftheExaminestepmightusepromptssuchasthoseinTable9—specificallymappedtoBloom-basedlearningobjectivesuptothe levelofanalysis—tosupportstudents indeveloping their thinkingaboutthatcharacteristicfurther. TheArticulateLearning step then supports students in re-thinkingand extending the thinking from the Examine step, to create a moremeaningfulandfullythoughtoutreflectiveessay,movingthemthroughSynthesis and Evaluation with additional sub-prompts and supportingthemindocumentingallsixlevelsofreasoninginBloom’sTaxonomy.Continuingwiththeexampleinthecategoryofpersonalgrowth,PartIofthisstepincludestheexpandedpromptsrepresentedinTable10.

Table 8: Bloom-based Version of DEAL: Sample “Examine” Part I Prompts (Personal

Growth

Category)

Examine Part I (Personal Growth): Sample Prompts to Surface a Personal

Characteristic

What assumptions or expectations did I bring to the situation? How did they affect what I

did or didn’t think, feel, decide, or do? To what extent did they prove true? If they did not

prove true, why was there a discrepancy?

How did this experience make me feel (positively and/or negatively)? How did I handle

my emotional reactions? Should I have felt differently? Why or why not?

How did I interpret the thoughts, feelings, decisions, and/or behaviors of others What

evidence do I have that my interpretations were or were not accurate?

In what ways did I succeed or do well in this situation (e.g., interacting with others,

accomplishing tasks, handling difficulties) and what personal characteristics helped me to

be successful (e.g., skills, abilities, perspectives, attitudes, tendencies, knowledge)? In

what ways did I experience difficulties (e.g., interacting with others, accomplishing tasks)

and what personal characteristics contributed to the difficulties (e.g., skills, abilities,

perspectives, attitudes, tendencies, knowledge)?

How did this situation challenge or reinforce my values, beliefs, convictions (e.g., my

sense of right and wrong, my priorities, my judgments)? My sense of personal identity

(e.g., how I think of myself in terms of gender, sexual orientation, socioeconomic status,

age, education level, ethnicity, nationality, mental/physical health)?

Table8:Bloom-basedVersionofDEAL:Sample“Examine”PartIPrompts(PersonalGrowthCategory)

Table 9: Bloom-based Version of DEAL: Sample “Examine” Part II Prompts (Personal

Growth

Category)

Examine Part II (Personal Growth): Prompts to Develop Understanding of a Personal

Characteristic Using Bloom’s Taxonomy

Identify What personal characteristic are you coming to understand better as a result of

reflection on your applied learning experiences?

Explain Explain the characteristic so that someone who does not know you would

understand it.

Apply How does / might this characteristic positively and/or negatively affect your

interactions with others, your decisions, and/or your actions in your applied

activities and (as applicable) in other areas of your life?

Analyze What are the possible sources of / reasons for this characteristic? How does

your understanding of these sources / reasons help you to better understand

what will be involved in using, improving, or changing this characteristic in the

future?

Table9:Bloom-basedVersionofDEAL:Sample“Examine”PartIIPrompts(PersonalGrowthCategory)

Regardlessofhowitisimplemented—writtenororal,individualorcollaborative,lightlyorin-depth—theDEALmodeloffersstudentstheopportunity to use writing or speaking as vehicles for learning ratherthan as expressionsof learningafter it has alreadyoccurred (Clayton&Ash,2004).Generatingtheirownlearninginthiswayisyetanothercounter-normativeaspectofcriticalreflectiononexperienceand,assug-gestedinthesetofcharacteristicsofhighqualityreflectioninTable4,students will benefit from feedback on their thinking, with associatedopportunitiestorevisitandrevise(e.g.,throughapplicationoftheStan-dardsofCriticalThinkingpresentedinTables5and6)tomaximizethequalityoftheirlearning. Inaddition, thedevelopmentof a critical reflectionmodel suchasDEALfacilitatesscholarlyworkrelativetoteachingandlearninginanappliedlearningpedagogy,helpinginstructorsimprovetheformertoen-hancethelatter.Forexample,DEALanditsassociatedrubrics(includ-ingthecriticalthinkingrubricinTable6)wereusedtoexaminechangesinstudents’criticalthinkingandhigherorderreasoningabilitiesacrossdraftsofasinglereflectionproductandoverthecourseofasemester,aswellasacrossthecategoriesofacademicenhancement,civiclearning,andpersonalgrowthinseveralservice-learningenhancedclasses(Ashetal.,2005).Buildingonthiswork,Jamesonetal.(2008)modifiedtheDEALreflectionpromptsandrubricsforapplicationacrossthecoursesequence of a Nonprofit Studies minor, investigating changes in stu-dents’criticalthinkingandreasoningabilitiesacrossthelearninggoals


REFEREnCES

Ash,S.L.,&Clayton,P.H.(2004).Thearticulatedlearning:Anapproachtoreflectionandassessment.Innovative Higher Education, 29,137-154.

Ash,S.L.,Clayton,P.H.,&Atkinson,M.P.(2005).Integratingreflectionandassessmentto improve and capture student learning. Michigan Journal of Community Service

Learning, 11(2),49-59.Ash,S.L.,&Clayton,P.H.(2009a).Learning through critical reflection: A tutorial

for students in service-learning.Raleigh,NC.(Availablefromhttp://www.curricular-engagement.com/Publications.html)

Ash,S.L.,&Clayton,P.H.(2009b). Learning through critical reflection: A tutorial for students in service-learning (Instructor version).Raleigh,NC.(Availablefromhttp://www.curricularengagement.com/Publications.html)

Battistoni,R.(2002).Civic engagement across the curriculum.Providence,RI:CampusCompact.

Bloom,B.S. (Ed.). (1956).Taxonomy of educational objectives, handbook I: Cognitive domain.NewYork,NY:DavidMcKayCompany.

Boyer, E. (1996). The scholarship of engagement. The Journal of Public Service and Outreach, 1(1),11-20.

Bringle,R.G.,&Hatcher,J.A.(1999).Reflectioninservice-learning:Makingmeaningofexperience.Educational Horizons, 7(4),179-185.

Clayton,P.H.,&Ash,S.L. (2004).Shifts inperspective:Capitalizingon thecounter-normativenatureofservice-learning.Michigan Journal of Community Service Learn-ing, 11,59-70.

Clayton,P.,Ash,S.,&Jameson,J. (2009).Assessingcritical thinkingandhigherorderreasoninginservice-learningenhancedcoursesandcoursesequences.InT.Banta,B.Jones,&K.Black(Eds.),Planning, implementing, and sustaining assessment: Prin-ciples and profiles of good practice(pp.58-62).SanFrancisco,CA:Jossey-Bass.

Conrad,D.,&Hedin,D.(1990).Learningfromservice:Experienceisthebestteacher--orisit?InJaneKendallandAssociates(Eds.), Combining service and learning. I(pp.87-98).Raleigh,NC:NationalSocietyforInternshipsandExperientialEducation.

Covey,S. (1989).7 habits of highly effective people.NewYork:FreePress /Simon&Schuster.

Dewey,J.(1910).How we think.Boston,MA:D.C.HeathandCompany.Eliot,T.S.(1943).The four quartets.NewYork,NY:Harcourt,Brace,andCompany.Eyler,J.(2000,Fall).Whatdowemostneedtoknowabouttheimpactofservice-learning

onstudentlearning?Michigan Journal of Community Service Learning, SpecialIssue,11-17.

Eyler,J.,&Giles,D.E.(1999).Where’s the learning in service-learning?SanFrancisco,CA:Jossey-Bass.

Eyler,J.,Giles,D.E.,&Schmiede,A.(1996).A practitioner’s guide to reflection in ser-vice-learning.Nashville,TN:VanderbiltUniversity.

Gelmon,S.B.,Holland,B.A.,Driscoll,A.,Spring,A.,&Kerrigan,S.(2001).Assessing service-learning and civic engagement.Providence,RI:CampusCompact.

Hondagneu-Sotelo,P.,&Raskoff,S.(1994).Communityservice-learning:Promisesand problems.Teaching Sociology, 22,248-254.Howard, J. (1998). Academic service learning: A counter normative pedagogy. In R.

Rhoades&J.Howard(Eds,),Academic service learning: A pedagogy of action and reflection(pp.21-29).SanFrancisco,CA:Jossey-Bass.

Jameson,J.,Clayton,P.,&Bringle,R.(2008).Investigatingstudentlearningwithinandacrosslinkedservice-learningcourses.InM.Bowden,S.Billig,&B.Holland(Eds.),Scholarship for sustaining service-learning and civic engagement (pp. 3-27). Char-lotte,NC:InformationAgePublishing.

offiveleadershipchallengesfacingthenonprofitsector.McGuireetal.(2009)examinedcriticalthinkingdemonstratedinArticulatedLearningsproducedbystudentsinmultipledisciplinesusingavarietyofassign-mentandfeedback-revisionformats. TheDEALmodeland its associated rubrics thereforedemonstratethe intentionaldesignofcritical reflection: identifyingdesiredstudentlearningoutcomes,articulatingthemasspecificgoalsandasassessablelearningobjectives,andthencraftinganintegratedreflectionandassess-mentapproacharoundthem. Inadditiontoprovidingtoolsneededtogenerate,deepen,anddocumentstudentlearning,DEALfacilitatesin-vestigationofthelearningprocesses(Clayton,Ash,&Jameson,2009).

ConCLuSIon

Itisourhopethatourworkcanserveasamodelforfaculty,staff,andstudentsastheyseektodesignreflectionassociatedwithappliedlearn-ingopportunities,courses,andprograms.Ourindividualandcollectivelearningaspractitioner-scholarsacrossthefieldofappliedlearningcanbeenhancedthroughascholarlyapproachtotheinstructionaldesignpro-cess.Inturn,itcancontributetoadvancingtheacademy’sunderstandingofbothhowourstudentsthinkandhowwecansupporttheminlearningtothinkmoredeeplyandwithgreatercapacityforself-directedlearning.

Table 10: Bloom-based Version of DEAL: “Articulate Learning” Part I Prompts (Personal

Growth Category)

1. What did I learn?

• Identify and explain (so that someone who doesn’t know you can understand it) a

personal characteristic that you are beginning to understand better

• Express the learning in general terms, not just in the context of the experience, so

that it can be applied more broadly to other areas of your life (personally or

professionally) and help you in your ongoing personal growth process

• Introduce a judgment regarding whether the characteristic serves you well (and

thus needs to be capitalized on) or poorly (and thus needs to be changed) – or both

2. How did I learn it?

• Clearly connect the learning to your specific applied learning activities so that

someone who was not involved would understand, including discussion of the

positive and negative impacts of the personal characteristic

3. Why does it matter?

• Consider how the learning has value over the short and long term, both in terms of

your applied learning activities and in terms of your life more generally

4. What will I do in light of it?

• Set specific goals and assessable goals (that you could come back to and check on

to see if they are being met) relative to this learning over the short and long term

• Consider the benefits and challenges associated with fulfilling these goals,

especially in light of the sources of or reasons for the characteristic

Table10:Bloom-basedVersionofDEAL:“ArticulateLearning”PartIPrompts(PersonalGrowthCategory)

48 JournalofAppliedLearninginHigherEducation/Fall2009

49

McGuire,L.,Strong,D.,Lay,K.,Ardemagni,E.,Wittberg,P.,&Clayton,P. (2009).Acasestudyoffacultylearningaroundreflection:Acollaborativefacultydevelopmentproject.InB.Moeley,S.Billig,&B.Holland(Eds.), Creating our identities in service-learning and community engagement (pp. 53-72). Charlotte, NC: Information AgePublishing.

NationalResearchCouncil.(2001).Knowing what students know: The science and design of educational assessment.Washington,DC:NationalAcademyPress.

O’Meara,K.,&Rice,E.(Eds.).(2005).Faculty priorities reconsidered: Rewarding mul-tiple forms of scholarship.SanFrancisco,CA:Jossey-Bass.

Palmer,P.(2000).Let your life speak: Listening for the voice of vocation.SanFrancisco,CA:Jossey-Bass.

Paul,R.P.,&Elder,L.(2001).The miniature guide to critical thinking.SantaRosa,CA:FoundationforCriticalThinking(www.criticalthinking.org).

Paul,R.P.,&Elder,L.(2002).Critical thinking: Tools for taking charge of your profes-sional and personal Life.UpperSaddleRiver,NJ:PearsonEducationInc.

Saltmarsh,J.(2005).Thecivicpromiseofservice-learning.Liberal Education, 91(2),50-55.

Saltmarsh, J., Hartley, M., & Clayton, P. (2009). Democratic engagement white paper.Boston,MA:NewEnglandResourceCenterforHigherEducation.

Schön,D.(1983).The reflective practitioner: How professionals think in action.NewYork,NY:BasicBooks.

Stanton,T.K.(1990).Liberalarts,experientiallearningandpublicservice:Necessaryin-gredientsforsociallyresponsibleundergraduateeducation.InJaneKendallandAsso-ciates(Eds.),Combining service and learning. I (pp.175-189).Raleigh,NC:NationalSocietyforInternshipsandExperientialEducation.

Strand,KJ.(1999).Sociologyandservice-learning:Acriticallook.InJ.Ostrow,G.Hesser&S.Enos(Eds.),Cultivating the sociological imagination(pp.29-37).NewYork,NY:AmericanAssociationforHigherEducation.

Welch,M.(1999).TheABCsofreflection:Atemplateforstudentsandinstructorstoim-plementwrittenreflectioninservice-learning. NSEE Quarterly, 25,22-25.

Whitney,B.,&Clayton,P.(inpress).Researchontheroleofreflectionininternationalservice-learning. In R. Bringle, J. Hatcher, & S. Jones (Eds.), International service learning: Conceptual frameworks and research.Sterling,VA:StylusPublishing.

Wiggins,G.,&McTighe,J.(1998).Understanding by design.Alexandria,VA:Associa-tionforSupervision&CurriculumDevelopment.

Zlotkowski,E.,&Clayton,P.(2005,April).Reclaiming reflection.PaperpresentedatthemeetingoftheGulfSouthSummitonService-LearningandCivicEngagement,CocoaBeach,FL.

examining the Development of the Victorian Certificate of Applied

Learning and Its Implications for Schools and teacher

education in Australia

DAmIAn BLAKeDeakinUniversity,Australia

DAVID GALLAGheRVictorianCurriculumandAssessmentAuthority(VCAA),Australia

The Victorian Certificate of Applied Learning (VCAL) is a very successful senior second-ary school qualification introduced in the Australian state of Victoria in 2002. Applied learning in the VCAL engages senior students in a combination of work-based learning, service-learning, and project-based learning and aims to provide them with the skills, knowledge, and attitudes to make informed choices regarding pathways to work and further education. The program has enjoyed rapid growth and its system-wide adoption by Victorian secondary schools, Technical and Further Education (TAFE) institutions, Registered Training Organizations (RTOs), and Adult and Community Education (ACE) providers has broadened significantly the range of senior schooling pathway options for young people. This paper will examine reasons for developing an applied learning senior secondary certificate and its rapid growth in Victoria since 2002. The authors draw on a number of case studies to profile the unique nature of applied learning in the VCAL, includ-ing its dimensions of service learning, work-based learning, and project-based learning. These case studies are also used to discuss a number of implications that have emerged from the use of applied learning in the VCAL, including approaches to teaching and as-sessment that will support applied learning and the development of new partnerships be-tween VCAL providers and community partners. Finally, the paper considers significant implications the VCAL has created for teacher education in Victoria by discussing the new Graduate Diploma of Education (Applied Learning) developed by Deakin University.

JournalofAppliedLearninginHigherEducationVol.1,Fall200949-71©2009MissouriWesternStateUniversity

generating, deepening, and documenting learning: …...critical reflection oriented toward...

Documents