developing reflective judgment through msw education

23
This article was downloaded by: [Washington University in St Louis] On: 05 October 2014, At: 03:31 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Journal of Teaching in Social Work Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/wtsw20 Developing Reflective Judgment Through MSW Education Cathryn C. Potter PhD a & Jean F. East PhD a a University of Denver, Graduate School of Social Work , Denver, CO, 80208, USA Published online: 13 Oct 2008. To cite this article: Cathryn C. Potter PhD & Jean F. East PhD (2000) Developing Reflective Judgment Through MSW Education, Journal of Teaching in Social Work, 20:1-2, 217-237, DOI: 10.1300/J067v20n01_14 To link to this article: http://dx.doi.org/10.1300/J067v20n01_14 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.

Upload: jean-f

Post on 23-Feb-2017

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Developing Reflective Judgment Through MSW Education

This article was downloaded by: [Washington University in St Louis]On: 05 October 2014, At: 03:31Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Journal of Teaching in SocialWorkPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/wtsw20

Developing ReflectiveJudgment Through MSWEducationCathryn C. Potter PhD a & Jean F. East PhD aa University of Denver, Graduate School of SocialWork , Denver, CO, 80208, USAPublished online: 13 Oct 2008.

To cite this article: Cathryn C. Potter PhD & Jean F. East PhD (2000) DevelopingReflective Judgment Through MSW Education, Journal of Teaching in Social Work,20:1-2, 217-237, DOI: 10.1300/J067v20n01_14

To link to this article: http://dx.doi.org/10.1300/J067v20n01_14

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

Page 2: Developing Reflective Judgment Through MSW Education

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 3: Developing Reflective Judgment Through MSW Education

Developing Reflective JudgmentThrough MSW Education

Cathryn C. PotterJean F. East

ABSTRACT. Discussion of the nature of critical thinking and the waysin which it can be enhanced has increased in recent years among socialwork educators, and yet little is known about the impact of MSWeducation on the critical thinking of students. This paper examines thenature of critical thinking, presents an adult cognitive developmentalmodel, the Reflective Judgment Model (King & Kitchener, 1994),which holds promise for enlarging social work’s understanding of criti-cal thinking and examines approaches to teaching which enhance criti-cal thinking. The Reflective Judgment Model focuses on the develop-ment of assumptions about knowledge and the process of knowing asdevelopment occurs through adulthood. Specific implications for creat-ing classroom and field learning environments are detailed, includingcharacteristics of effective teachers and of effective assignments. [Articlecopies available for a fee from The Haworth Document Delivery Service:1-800-342-9678. E-mail address: <[email protected]> Website:<http://www.haworthpressinc.com>]

KEYWORDS. Critical thinking, MSW education, reflective judgment,clinical judgment, problem solving, professional decision-making

Social Work education’s mandate to focus on the critical thinking ofMSW students is clearly articulated by the Council on Social WorkEducation (1997). Students are expected to ‘‘apply critical thinking

Cathryn C. Potter, PhD, is Assistant Professor, University of Denver, GraduateSchool of Social Work, Denver, CO 80208 (E-mail: [email protected]). Jean F. East,PhD, is Assistant Professor, University of Denver, Graduate School of Social Work.

Journal of Teaching in Social Work, Vol. 20(1/2) 2000E 2000 by The Haworth Press, Inc. All rights reserved. 217

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 4: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK218

skills within educational contexts, including synthesizing and applyingappropriate theories and knowledge to practice interventions’’ (p. 4).Some have argued that critical thinking is a necessary condition for anintegrated practice in social work (Gambrill, 1990). Discussion of thenature of critical thinking and the ways in which it can be enhancedhas increased in recent years (Gambrill, 1990; Gibbs, 1991; MacMor-ris, 1997; Mumm & Kersting, 1997; Selig, 1991), and yet research onthe impact of MSW education on students’ critical thinking is not yetavailable (Gambrill, 1994). This paper examines the nature of criticalthinking, presents an adult cognitive developmental model, ReflectiveJudgment (King & Kitchener, 1994), which holds promise for enlarg-ing social work’s understanding of critical thinking, and examinesapproaches to teaching which enhance critical thinking.

WHAT IS CRITICAL THINKING?

Critical thinking refers to thinking which is purposeful and goaldirected (Dewey, 1933; Halpem, 1989; Knowles, 1990; Paul, 1990;1992). Critical thinking involves rendering a judgment based on rele-vant information, while following accepted rules of thought. To thinkcritically is to wrestle with a complex issue by successfully definingthe issue at hand, exploring alternate views of both the question andthe potential ‘‘answers,’’ identifying relevant information, selectingand applying self generated criteria for judging, and acknowledgingthe potential need to re-examine the issue. Critical thinking ‘‘requirespaying attention to the process of reasoning, not just the product’’(Gibbs & Gambrill, 1996, p. 3).Two traditions in conceptualizing critical thinking are described

(MacMorris, 1997; Wolcott & Lynch, 1997). The first approach em-phasizes the importance of identifying logical fallacies in reasoningand decision making (Gibbs & Gambrill, 1996; Golub, 1997), whilethe second focuses on the ability to construct and test frameworksbased on experience, and to emerge the logical processes which under-lie those frameworks (Schon, 1983, 1987; Van Maanen, 1991). Thefirst approach leads educators to focus on teaching critical thinkingskills; whereas the second approach leads educators to focus on creat-ing learning environments where thinking abilities can emerge.Still others have argued that critical thinking in professional con-

texts requires acknowledging and building upon both approaches

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 5: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 219

(Lynch & Wolcott, 1997). That is, students need both frameworks forthinking and experiences in shared thinking with others. Assumptionsabout knowledge which provide a foundation for thinking must alsobe addressed. Some students frame all questions in concrete terms, andconclude that certain approaches to intervention and policy are abso-lutely correct, a dangerous assumption in social work practice. Otherstudents resist the move to conclusion, or to evaluation of evidence, onthe basis that knowledge is so uncertain that we can only make ourown idiosyncratic choices. These assumptions influence students’ un-derstanding of rules for thinking, whether these rules are framed interms of scientific methods, problem-solving models or clinical deci-sion-making frameworks. A model which incorporates epistemic as-sumptions has the potential to expand our understanding of criticalthinking.

THE REFLECTIVE JUDGMENT MODEL

The Reflective Judgment (RJ) Model (Kitchener & King, 1990;King & Kitchener, 1994), is a neo-Piagetian model which broadensthe developmental focus on cognition beyond logic to examination ofthe development of ontological and epistemic assumptions aboutknowledge. The model presents a framework for the development ofepistemic assumptions and the resulting approaches to thinking aboutill-structured problems, that is, problems for which true answers arenot known (Churchman, 1971). The term reflective judgment comesfrom Dewey (1933) who used it in discussing the nature of reasoningthat is required to pose a tentative solution to an unstructured problem.The RJ model builds upon Perry’s (1970) model of ethical and intel-lectual development, but differs in that the RJ model has a strong focuson the ways in which people render and justify their judgments aboutill-structured problems.The model postulates that there are several levels or stages in the

way people understand the process of knowing. Progress throughthese stages typically occurs between adolescence and middle adult-hood. Each of the levels includes a ‘‘distinct set of assumptions aboutknowledge and how knowledge is acquired’’ (King & Kitchener,1994, p. 13). These assumptions include perspectives on views ofknowledge, what can be known, how certain we can be, and the

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 6: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK220

justification of beliefs. Figure 1 summarizes the seven stages of theReflective Judgment model.Reasoning in the early stages (Stages 1, 2 and 3) is characterized by

the assumption that knowledge is certain; that is, there is a correctanswer to be known for any problem. Knowledge is gained eitherthrough personal observations (Stage 1) or from authorities (Stages 2 &3). Stage 3 is characterized by the acknowledgement that some knowl-edge is uncertain though only temporarily. Interpretation of evidence

FIGURE 1. Reflective Judgment Stages

RJ STAGE WHAT CAN WE HOW CERTAIN THROUGH WHAT HOW MAY BELIEFSKNOW? CAN WE KNOW? PROCESS CAN BE JUSTIFIED?

WE KNOW?

1 Reality Absolutely certain By direct Beliefs are a directobservation reflection of reality. No

need to justify them.

2 True reality and false Absolutely certain By direct Direct observation or viaclaims or certain, but not observation and via authorities

immediately what authorities sayavailable is true

3 True reality, false Absolutely certain Via authorities in Via authorities in someclaims, and uncertainty about some things; some areas; through areas; via what feels right

temporarily our own biases in the moment whereuncertain about when knowledge is knowledge is uncertainothers uncertain

4 While there is a reality, No certainty because Via our own and Via idiosyncraticit can never be known. of situational other’s biases, data evaluations of evidenceKnowledge is variables and logic and un-evaluated beliefsindividuallyidiosyncratic

5 Personal interpretations No certainty except Via evidence and By rules of inquiry for aof individual realities via personal rules of inquiry particular context

perspectives within appropriate for thea specific context context

6 Reality assumed. Some personal Via personal Via generalized rules ofEvaluated personal certainty about assessment of inquiry, personalinterpretations beliefs based on arguments and data, evaluations that apply

evaluations of via evaluated across contexts, evaluatedevidence on different opinions of experts views of expertssides of the question

7 Reality is never ‘‘given.’’ Certainty that some Via process of As more or lessFacts and assumptions knowledge claims critical inquiry or reasonable conjecturesmay be constructed into are better or more synthesis about reality or the worldevaluated knowledge complete than based on an integrationclaims about reality others although they and evaluation of data,

are open to evidence and opinionreevaluation

Adult Cognitive Development: Methods and Models, Mines, R.A., and Kitchener, K.S., eds., Adult Cognitive Development (Praeger Publishers, animprint of Greenwood Publishing Group, Inc., Westport, CT, 1985). Reprinted with permission.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 7: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 221

in these situations is seen as merely ‘‘guessing.’’ Stage 3 thinkerscannot distinguish bias from interpretation, and often express a pro-found mistrust of experts who are seen as making arbitrary guessesabout solutions. King and Kitchener (1994) refer to these stages aspre-reflective. Examples given here and in succeeding paragraphscome from Reflective Judgment interviews with first-year MSW stu-dents (Potter, East & Wolcott, 1997).

Experts disagree all the time. Again, it’s going to go back to whoare these experts working for . . . It’s a big macho thing. Likeexperts titled ‘‘experts’’ or ‘‘scholars.’’ That weighs very heavy,so they are protecting themselves. (Stage 3)They [experts] can read the information different; they can

twist it. Experts are just people. I don’t know if they know thebest. (Stage 3)

Stages 4 and 5 are known as quasi-reflective. In these stages indi-viduals recognize that problems are multi-faceted and that knowledgeabout these problems is uncertain. They understand that ill-structuredproblems exist. Stage 4 thinkers believe that knowledge is both uncer-tain and idiosyncratic to the individual since situational variablesmake judging between knowledge-claims impossible. For example,Stage 4 thinkers do not typically distinguish between the opinions ofexperts and their own opinions. They understand knowledge as anabstraction and can make some distinction between beliefs and evi-dence, but their use of evidence is sporadic and idiosyncratic. Theyfrequently articulate the view that everyone has a right to his or herown opinion.

When you don’t have any certainty about whether it is right orwrong, then they are just opinions. Everybody has to choosewhat they think is best for them, but there is no right or wrongopinion. (Stage 4)Everyone has unique experiences, and they are at a different

stage in life in their education, in a multitude of variables bom-barding them every day. This is why each person’s judgmentwould be correct for their experience. (Stage 4)

Stage 5 thinkers have the ability to relate two abstractions, resulting inthe ability to compare and contrast. These skills result in a more

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 8: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK222

complex view of problems and a focus on knowing within a specificcontext. However Stage 5 thinkers still have difficulty applying frame-works for judging across contexts, and their use of evidence ofteninvolves stacking evidence into separate piles, thus giving a balancedpicture without feeling comfortable justifying a particular conclusion.

Somebody who works in the health field may say it has more todo with genetics, because that would be more of the cases thatthey see. Somebody who works in the human services field maytend to feel it is more environmental. It’s just what you focus onor what type of group you look at. (Stage 5)There is a lot of conflicting evidence and opinions. You could

easily argue either position fairly effectively. So, I wouldn’t beable to say that either one was wrong. (Stage 5)

Stages 6 and 7 are the true reflective stages. Individuals here ‘‘arguethat knowledge is not a ‘‘given’’ but must be actively constructed, andthat claims of knowledge must be understood in relation to the contextin which they were generated’’ (King & Kitchener, 1994, p. 66). AtStage 6 individuals understand that knowing is a process, that prob-lems can be approached from multiple perspectives using a variety ofevidence, and therefore one must examine and evaluate the context ofevidence before reaching a conclusion. By comparing evidence andopinion across contexts, an initial judgment can be made based on aview that is supported by greater evidence. Stage 6 thinkers are likelyto rely on interpretations of evidence made by those they judge to beexperts, but are not fully comfortable offering an integrated, personalsystem for evaluating evidence.

If you can check the studies out and see how they did it, and reallysee whether it was sound, then I would agree . . . I would say toothers ‘‘listen to the news, read it in the papers, check on thestudies and look at the facts. Then, what do you think? (Stage 6)

At Stage 7 individuals understand that knowledge is uncertain andsubject to interpretation but argue that evidence and opinion can besynthesized so that judgments can be made about a better solution toan ill-structured problem. At the same time, a Stage 7 thinker is will-ing to critique her own reasoning and recognize the ever-evolvingnature of knowledge. ‘‘Conclusions are defended as representing the

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 9: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 223

most complete, plausible, or compelling understanding of an issue onthe basis of available evidence.’’ (King & Kitchener, 1994, p. 72)

It’s always better to keep an open view to see what is going tocome about. Once we get stuck in one tradition, we’re not opento hearing the new advances and research (Stage 7). As researchprogresses and as we become more familiar with it, I think wewill be able to say, ‘‘yes, genetics have more of an effect thanenvironment.’’ (Stage 7)

Stage 7 thinkers can often sound remarkably certain in their conclu-sions, but are distinguished from earlier absolutist reasoning stages(Stages 1 and 2) by the acknowledgement of uncertainty and thedemonstration of an integrated, personal process for evaluatingknowledge.In summary, the Reflective Judgment Model presents an adult cog-

nitive developmental framework for understanding the progressionfrom observation and authority-based ways of knowing, through theacknowledgement of uncertainty to an integrated, personal process forknowing.

ASSESSING REFLECTIVE JUDGMENT

Reflective judgment is assessed using the Reflective Judgment In-terview (RJI), a semistructured interview format in which subjects areasked to think about a series of 4 dilemmas and respond to a set ofquestions designed to focus their thought on their reasoning. The fourstandard RJI dilemmas cover a range of issues, including the questionof how the Egyptian pyramids were built, the objectivity of newsreporting, how human beings were created and the safety of chemicaladditives in food (King & Kitchener, 1994). Researchers have begunto fashion RJI dilemmas which are profession-specific. Some of thesedilemmas include the causes of alcoholism (psychology) and themethods of valuing assets (accountancy).Subjects are asked to respond to a short statement of the dilemma,

and are then asked a series of questions which target critical aspects ofthe RJ model. These questions include:

S How did you come to hold that point of view?S On what do you base that point of view?

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 10: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK224

S Can you ever know for sure that your position on this issue iscorrect? How or Why not?

S When two people differ about matters such as this, is it the casethat one opinion is right and one is wrong?If yes, what do you mean by ‘‘right’’?If no, can you say that one opinion is in some way better thanthe other? What do you mean by ‘‘better’’?

S How is it possible that people have such different points of viewabout this subject?

S How is it possible that experts in the field disagree about this sub-ject? (King & Kitchener, 1994)

The interviewer constructs follow-up questions within this frameworkusing the participant’s own language to explore the meaning of state-ments. For example, it is common to inquire about the meaning ofinterpretation. Some interviewees use interpretation to mean an idio-syncratic imposition of personal convictions on facts, while othersmay articulate a systematic and criterion-referenced approach to draw-ing conclusions.Extensive psychometric work has been done with the RJI. For a

complete discussion see Wood (1997), which reports the results ofsecondary data analysis from 74% of all subjects from the last 18 yearsof Reflective Judgment research (n = 1509). The internal consistencyof the RJI is high (alpha = .94.) Rater agreement is strong, rangingfrom 80 to 100 percent.

RESEARCH ON REFLECTIVE JUDGMENT

The past twenty years of reflective judgment research is well sum-marized by King and Kitchener (1994). Here we will focus on fourkey issues:

S What is the nature of reflective judgment over time?S What is the nature of reflective thinking across educational levels?S Are there gender differences in the development of reflectivethinking?

S What is known about the efficacy of educational models in pro-moting reflective thinking?

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 11: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 225

The first question is, does the ability to make reflective judgmentsdevelop over time, particularly between adolescence and middle adult-hood? Although there have been numerous studies conducted, perhapsthe most critical information is found in the results of a ten yearlongitudinal study of the development of reflective thinking whichtraced the development of 80 individuals from 1977 to 1987. RJIscores increase with age. The RJI scores for those subjects who werehigh school students in 1977 increased the most dramatically from 2.9to 5.5, a difference of two and half stages (Kitchener & King, 1990).Six other longitudinal studies have been conducted (Brabeck & Wood,1990; Pokosnik & Winston, 1989; Sakalys, 1984; Schmidt, 1985; VanTine, 1990; Welfel & Davison, 1986). With two exceptions, the meanscore increased significantly for all groups tested at one- to four-yearintervals. (Exceptions were adult freshmen (Schmidt, 1985) and ad-vanced doctoral students (Kitchener, King, Wood & Davison, 1989)).What is the nature of reflective judgment across educational levels?

A meta-analysis of all available cross sectional studies using the RJIprovides information about these differences. RJI scores increase witheducational level. In addition, the variability among scores increasesover time (King & Kitchener, 1994). Of some interest here is thefinding that masters level students score between stages 4 and 5 onaverage; even advanced doctoral students are not, on average, yetconsistently reasoning at stages 6 and 7.There has been much discussion about the validity of developmen-

tal models tested predominantly on male samples (Belenky, Clinchy,Goldberger & Tarule, 1986; Commons et al., 1990). Are there genderdifferences in reflective thinking as assessed by the RJI? Of the eigh-teen studies involving both male and female participants, seven stud-ies have reported gender differences, and six of these reported thatmen scored higher (King & Kitchener, 1994). Unfortunately, widedifferences in sampling and research design make drawing conclu-sions difficult. For example, not all studies examined interaction ef-fects which might provide insight into the nature of the relationshipbetween gender and other variables. Both Schmidt (1985) and the tenyear longitudinal study (King & Kitchener, 1990) found a significantinteraction between gender and educational year, suggesting that menand women may differ in the timing of development of reflectivejudgment during their educational experience. Given the gender bal-ance of social work programs and the wide age ranges present among

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 12: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK226

MSW students, exploration of the patterns of thinking among bothmale and female social work students is needed.What are effective classroom strategies for promoting reflective

judgment? While there has been discussion of the implications of thereflective judgment model for teaching at the undergraduate and grad-uate levels (King & Kitchener, 1994; Lynch & Wolcott, 1997; Wolcott& Lynch, 1998), there is little rigorous research on the effectiveness ofspecific curriculum and classroom approaches. This represents a ma-jor area of needed research.

ENHANCING THE DEVELOPMENT OF REFLECTIVEJUDGMENT TAKING A DEVELOPMENTAL APPROACH

Before examining some of the specific implications of this modelfor teaching in social work, it is important to revisit the implications ofa developmental model of critical thinking. Any developmental modelincludes theory about (1) what is developing, in this case reflectivethinking, and (2) the mechanisms which drive development forward.In this case, the reflective judgment model is grounded in Fischer’sSkill Theory (Fischer, 1980; Kitchener & Fischer, 1990), which em-phasizes the role of environment, specifically joint thinking with oth-ers, as a primary mechanism for the development of increasing com-plex sets of skills. Fischer notes that an individual’s approach to taskscan vary within a range of stages depending on the amount of contex-tual support and the nature of the task (Fischer, 1980). Thus, it isimportant to distinguish between a person’s functional level of reason-ing, used under typical circumstances, and their optimal level of rea-soning, the most advanced level they can use under supportive circum-stances.Development can be stimulated through learning environments

which are supportive to the development of thinking. As Dewey(1944) said, ‘‘We never educate directly, but indirectly by means ofthe environment. Whether we permit chance environments to do thework, or whether we design environments for the purpose makes agreat difference’’ (p. 19). Fischer (1980) and Rogoff (1990) haveemphasized the role of ‘‘shared thought’’ in promoting the develop-ment of reasoning. That is, persons engaging in shared thinking taskswith others who are operating at slightly higher stages of thinking are‘‘drawn forward’’ in development of their own reasoning. It is impor-

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 13: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 227

tant to note that the discrepancy between stages cannot be too large orthe connections are not made. King and Kitchener (1994) have notedthat students have difficulty comprehending epistemic assumptionsmore than one or two stages higher than the stage they typically use.Models of critical thinking which focus on the logical, systematic

aspects of reasoning suggest approaches to teaching critical thinkingskills (Gibbs & Gambrill, 1996). This implies that students can betaught a set of skills within a relative short time frame. Taking adevelopmental approach focuses the instructor on the construction oflearning tasks and experiences which will push the student’s thinkinglevel forward. A developmental focus requires a significantly longerterm view of critical thinking, a view which may be less than comfort-able for the instructor who knows the level of thinking required insocial work practice.

APPLYING THE REFLECTIVE JUDGMENT MODELTO SOCIAL WORK EDUCATION

Here we will highlight three critical areas of the curriculumconstruction and educational processes in which Reflective Judgmentcan help guide our thinking: (1) focusing on ill-structured problems,(2) creating classroom and field environments and (3) constructingassignments. These do not represent the entire range of issues in-volved in the integrating and understanding of reflective judgmentinto our education approaches; however, we believe that they repre-sent essential starting points.

TEACHING IN THE CONTEXTOF ILL-STRUCTURED PROBLEMS

How can we ensure that MSW students use complex thought whenfaced with ill-structured problems? Social work education uses a num-ber of frameworks which are entirely supportive of developing anunderstanding of the nature of ill-structured problems, including theprofession’s focus on analyzing social problems, a person-in-environ-ment framework and emphasis on problem solving models. We mustmake the nature of the problems explicit and address the essential

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 14: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK228

messiness that uncertainty interjects into professional decision-mak-ing. For those students who come to us in pre-reflective thinkingstages, this realization is a major one. In response to the question:‘‘How would you summarize the major concepts of the first yearcurriculum?’’, one beginning second year MSW student mournfullyresponded, ‘‘It depends.’’ The learning environment must live square-ly in ‘‘it depends,’’ but also provide students with experiences indeciding upon what it depends, rendering a judgment about a preferredsolution and choosing a course of action.How can instructors maintain a focus on solutions to ill-structured

problems? Much of the solution lies in the characteristics of the teach-er. Here we offer a few salient suggestions.

1. Effective teachers operate at high reflective judgment levels.This may seem obvious, but it is worth stating.

2. Effective teachers frame course content as ill-structured prob-lems for which there may be preferred solutions. Course contentcannot be presented as immutable fact, no matter how much stu-dents demand to be handed the ‘‘right’’ answers, or how clearlythe instructor sees the evidence for a solution.

2. Effective teachers are comfortable with ambiguity. Strugglingwith ill-structured problems is anxiety-producing for students.The instructor must be comfortable holding the focus.

3. Effective teachers don’t simplify complex concepts. In an at-tempt to summarize basic concepts, instructors may be temptedto allow students to assume complex problems are well-struc-tured. It takes longer to stay ill-structured, but it is essential.

4. Effective teachers are extraordinarily patient and accepting ofmultiple levels of reasoning. Pre-reflective thinkers can be tre-mendously difficult to reason with since they reframe every is-sue in concrete terms. Quasi-reflective thinkers resist making ajudgment, or even evaluating evidence, and this can be irritating.It is easier to join with students with higher reasoning abilities.All students, however, are on a journey.

5. Effective teachers join with students and enjoy the developmen-tal journey. Reflective judgement varies with task and with thelevel of support available. Effective teachers resist rigid labelsfor students’ thinking; instead, they enjoy the variety and workto support progress.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 15: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 229

CLASSROOM AND FIELD ENVIRONMENTS

What are critical elements of classroom and field environmentswhich are supportive of the development of Reflective Judgment? TheRJ model presented in Figure 1 consists of two dimensions. The hori-zontal dimension describes four aspects of epistemic assumptions:what we can know, how certain we can be, what the process of know-ing is and how beliefs are justified. The vertical dimension describesprogression in thinking. Classroom and field environments must re-quire students to articulate their assumptions about each dimension ofknowledge while creating opportunities for increasingly complex rea-soning. Three characteristics of learning environments can supportthis effort.First, the importance of shared thought to the development of Re-

flective Judgment has obvious implications for both classroom andfield environments. Students need opportunities to witness the reason-ing of those who may be operating at more advanced thinking levels.More importantly, they need the opportunity to engage in the thinkingprocess as judgments are made and justified. Group decision-makingis an important teaching tool, but the construction of the group isimportant. Groups must represent a range of thinking, and shouldchange in their composition, so that all thinkers have an opportunity towork with others whose thinking may be slightly more advanced thantheir own. Individual tutorials with students also offer an opportunityto join their thinking and to introduce more advanced approaches toknowing.In the field environment, partnership with the field instructor and

with other professionals in thinking about ill-structured problems andarriving at decisions is essential. It is here that students have therichest opportunity to engage in the full process of professional judg-ment, including the articulation of personal frameworks for knowingand the process of constantly re-evaluating evidence.Second, learning environments must encourage and support articu-

lation of multiple perspectives, including multiple solutions to com-plex problems. Specifically, they must support articulation of differingassumptions about the four dimensions of epistemic assumptions. Thepresence of multiple perspectives alone does not support the develop-ment of reflective judgement. Bringing forth the assumptions aboutknowledge that undergird those perspectives gives students the oppor-tunity to grapple with the nature of complex reasoning.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 16: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK230

Third, the learning experience must mirror the iterative thinkingprocess that is characteristic of higher level reflective judgment. Com-plex thinkers can articulate a framework for using available informa-tion to arrive at a preferred solution. They can use this frameworkacross contexts. They can also articulate a process for re-evaluatinginformation and re-assessing their own judgment. The classroom ex-perience must provide experiences in which this ability is required andsupported.The RJ literature indicates that most masters level students operate

in Stages 4 and 5; thus social work educators are often promoting thetransition between quasi-reflective and reflective thinking. It is at thistransition between Stage 5 and Stage 6 that students understand thatframeworks for knowing can be assessed and applied across differingcontexts, such as classroom and field settings. Our experience with anMSW, Integrative Field Seminar leads us to believe that this can be animportant setting for promoting the development of Reflective Judg-ment.This seminar has several features which make it conducive to pro-

moting thinking in a developmental context:

S It is a year long experience in which the same group of thinkerswork together,

S It consists of a smaller group of students, typically, 10-12, with aconsistent instructor,

S Its focus is on the integration of classroom and field learning,S Its form is process oriented.

The focus on the integration of classroom and field learning places theinstructor firmly in the role of developing critical thinking. The classis often focused on professional decisions, almost all ill-structured innature, and on the exploration, evaluation and construction of frame-works for practice decisions. Students must use classroom learning toaddress difficult issues in field and use field learning to examine andevaluate what they are hearing and reading in the classroom instruc-tion. An instructor who is focused on promoting the development ofReflective Judgment finds fertile ground for teaching. Creating thefeatures of the Field Seminar in all classroom settings could assist inthe development of Reflective Judgment.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 17: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 231

CONSTRUCTING ASSIGNMENTS

How then do classroom and field instructors construct assignmentsthat promote higher level thinking, but can be completed by a rangeof thinkers? And, to what extent are higher levels of thinking valuedin grading of assignments? An important initial step in answeringthese questions is to explore the nature of critical thinking with ourstudents.In our experience, it is common for grading criteria to include a

reference to demonstration of critical thinking, critical analysis, etc.,without a clear articulation of what this means for students. Manystudents have approached us asking for guidance in what constitutes‘‘critical thinking.’’ Paul (1990; 1992) articulates several importantsuggestions, including that we ‘‘let students know what they are infor . . . and . . . spell out explicitly the intellectual standards of thesystem of grading’’ (1990, p. 20). Instructors need to introduce stu-dents to the nature of critical thinking and the methods by which it isbelieved to be acquired. It may go without saying that this must beframed as an ill-structured problem; and the discussion of it mustillustrate both conclusions and justifications.Our colleague Susan Wolcott, an Accounting Professor teaching

in both undergraduate andMBA programs, has given an example ofan up-front invitation to students to focus on their own thinking(Lynch & Wolcott, 1997). Following is a framework for such anintroduction:

S An introduction to the nature of ill-structured problems.S A clearly stated expectation that the course will emphasizethinking: ‘‘You are responsible for monitoring and working to-ward improvement in your own approach to unstructured prob-lems’’ (p. 1).

S An acknowledgement that thinking is a developmental process,and that education plays a role in moving development forward.

S A framework of important questions which links the problemsolving process to reflective judgment concepts.

Figure 2 presents the structure for such a framework from the work ofLynch and Wolcott (1997). The instructor may provide students withboth the general framework and examples drawn from problemswhich the class addresses.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 18: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK232

FIGURE 2. Process for Addressing Unstructured Problems

Phase 5: Re-Addressing Unstructured ProblemsS What are the limitations of the solution and how

will you articulate these?S What are the implications of those limitations?S Under what conditions would you need to

re-consider your solution?S What strategies need to be implemented to

monitor the solution results?

Phase 4: Resolving Unstructured Problems:S What are your relative evaluations of related information and

opinions?S Have you adequately compensated for any initial biases you

might have?S Which solution is likely to be most viable?S What is your justification for this belief?S Is your belief based on objective, qualitative evaluations of

the relevant information?S How would you respond to arguments that support other

viable solutions?S Given your setting and audience, how will you articulate

and justify your solution?

Phase 3: Framing Unstructured Problems:

S What is your initial reaction to the problem? What are your initial biases?

S How might you compensate for any initial biases you might have?

S How is the problem related to other problems you have considered?

S What is the context in which the problem is found?

S What information and evidence can you find that is related to the problem?

S What are the various points of view about the problem?

S What are the arguments for and against each point of view?

S What assumptions are embedded in different perspectives on the problem?

S How might you organize the factors and information so that they are usefulas you think about problem complexities?

Phase 2: Identifying the Nature of Problems:

S Do experts disagree about the best solution to this problem?If so, why do they disagree?

S Why is there no single correct solution to this problem?

S What aspects of the problem are unstructured?

Phase 1: Gaining Prerequisite Skills:

S What highly structured problem solving skills are necessary for addressing the problem?For example, calculations, definitions, authoritative pronouncements, laws, rules, and libraryand electronic information search skills

by Cindy L. Lynch and Susan K. Wolcott (1997)

STARTHERE

This approach to naming critical thinking as a central course objec-tive provides the foundation for constructing assignments which willchallenge students’ thinking. This framework can provide guidancefor the construction of written classroom assignments, an approachwhich will explicitly value higher level thinking, while simultaneously

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 19: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 233

introducing students to the tasks involved in thinking beyond theirpersonal optimal level. This is congruent with Paul’s (1992) sugges-tion that we construct assignments that students must ‘‘think their waythrough’’ (p. 19).One approach we have used is to break assignments down into

components, where the earliest papers focus on identification andframing of problems, while later assignments focus on resolving andre-addressing problems. This gives students an opportunity to strugglewith the nature of the problem before proceeding to choosing andjustifying solutions. This approach will not, however, guarantee thatpre-reflective thinkers can adequately identify problems. They mayrespond to all problems as highly structured. Similarly, quasi-reflec-tive thinkers may not frame problems adequately since they may bevaluing all evidence, opinion and interpretation equally.

CONCLUDING THOUGHTS

What advice can we take from the reflective judgment literature andcongruent critical thinking literature? First and foremost we mustrecognize that students differ in their assumptions about knowledgeand in their resulting approaches to thinking. Instructors must assessthe range of thinking levels in their classrooms. Students’ cognitivedevelopment is maximized when the instructor meets them where theyare and constructs a learning environment that promotes developmentto the next level. We must familiarize students with ill-structuredsocial work problems and ground much of our teaching in these diffi-cult issues. Students need the opportunity to explore differing pointsof view on topics, and be encouraged to make judgments for whichthey must articulate a justification. As much as possible, studentsshould be given the opportunity to engage in shared thought with theinstructor and with fellow students. A delicate balance must be foundbetween providing a supportive environment and consistently chal-lenging students to think more reflectively, and both classroom teach-ing and assignments must be designed with this in mind. Studentsmust see evidence of critical examination of issues in the life of thesocial work department. That is, they must engage with us, both in andout of the classroom, in tasks which require high levels of reflectivethinking, so that they experience critical thinking at the heart of thesocial work profession.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 20: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK234

THORNY ISSUES

It is worth noting that teaching for the promotion of ReflectiveJudgment is an ill-structured challenge. One of the more irritating as-pects of ill-structured problems is the way in which approaches tosolution give rise to other ill-structured problems! Here we raise two ofthese issues: questions of student outcome and questions of studentsupport.Social work educators have a choice to make in their use of devel-

opmental principles. Is the goal to help students maximize their func-tioning in a current level of development, or is the goal to help themachieve a more complex level of thinking? Clearly, we have opted forthe latter approach; and we believe it to be the most responsible choicegiven the decision-making demands of social work practice. However,human development is not a quick process. Moreover, social workstudents are very diverse in terms of age, education, life experienceand, very likely, thinking level. This is, of course, a great advantagefor our teaching and for our ability to expose students to the range ofthinking, but it presents both the teaching challenges we have dis-cussed above, and questions regarding student outcome. Is there alevel of reflective judgment which is required for effective social workpractice? Are we content to support the process of development duringstudents’ time with us in the hope that the required thinking level isreached during the early years of MSW practice? What are the mini-mum expected critical thinking outcomes for social work education?The question of student support is equally important. Students re-

quire both challenge and support as their thinking develops.

Students seem to perceive the move from absolutist thinking . . . toa view where everyone’s point of view has importance and legiti-macy as very expansive and liberating. On the other hand inter-ventions designed to promote development from the mid-range toupper levels of development is seen as narrowing or restrictingoptions as students face the need to evaluate and eventually rejectsome alternatives. (Schmidt & Davison, 1983, p. 568)

How well we balance our commitment to promoting critical thinkingby providing (1) cognitively challenging learning environments and(2) the level of support needed for students to function optimally maybe the key to successfully moving students to more complex modes ofthinking.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 21: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 235

OPPORTUNITIES

The Reflective Judgment model offers social work education anexciting opportunity to deepen our thought about critical thinking. TheReflective Judgment Model’s emphasis on assumptions about know-ing helps us to identify another kind of diversity in our educationalenvironments, and a developmental approach helps us to understandways to use that diversity to help all students advance their thinking.The RJ Model’s approach helps us to identify and examine our as-sumptions about the cognitive requirements for effective practice andto construct learning environments to support development of criticalthinking. Social work, as a profession, is well placed to apply theseconcepts to the educational experience. Thorny issues aside, applyingprinciples from the Reflective Judgment Model presents an opportuni-ty to re-examine our professional assumptions about knowing, ourdesired goals for professional thinking and decision-making, and ourapproaches to teaching and learning.

REFERENCES

Belenky, M., Clinchy, B., Goldberger, Y. & Tarule, J. (1986). Women’s ways ofknowing. New York: Basic Books.

Brabeck, M.M. (1984). Longitudinal studies of intellectual development duringadulthood: Theoretical and research models. Journal of Research and Develop-ment in Educations, 17(3), 12-27.

Brabeck, M., & Wood, P. (1990). Cross-sectional and longitudinal evidence fordifferences between well-structured and ill-structured problem solving abilities.In M.L. Commons, C. Armon, L. Kohlberg, F.A. Richards, T.A. Grotzer and J.D.Sinnot (Eds.)Adult development 2: Models and methods in the study of adolescentand adult thought. New York: Praeger, 1990.

Churchman, C.W. (1971). The design of inquiring systems: Basic concepts of systemsand organization. New York: Basic Books.

Commons, Michael L. and others (Eds.) (1990). Adult development, Vol. 2: Modelsand methods in the study of adolescent and adult thought. New York: PraegerPublishers.

Council on Social Work Education (1997).Handbook of accreditation standards andprocedures. Alexandria, VA: Author.

Dewey, J. (1933). How we thinking: A restatement of the relation of reflective think-ing to the educative process. Lexington, MA: Heath.

Dewey, J. (1944). Democracy and education: An introduction to the philosophy ofeducation. New York: Free Press.

Fischer, K.W. (1980). A theory of cognitive development: The control and construc-tion of hierarchies of skills. Psychological Review, 87(6), 477-531.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 22: Developing Reflective Judgment Through MSW Education

JOURNAL OF TEACHING IN SOCIAL WORK236

Gambrill, E.D. (1990). Critical thinking in clinical practice. San Francisco: Jossey-Bass.

Gambrill, E.D. (1994). Social work research: Priorities and obstacles. Research onSocial Work Practice, 4(3), 359-388.

Gibbs, L.E. (1991). Scientific reasoning for social workers: Bridging the gap be-tween research and practice. New York: Macmillan.

Gibbs, L.E. & Gambrill, E.D. (1996). Critical thinking for social workers: A work-book. Newbury Park, CA: Pine Forge Press.

Golub, A.L. (1997). Decision analysis: An integrated approach. New York: JohnWiley and Sons.

Halpern, D.E. (1989). Thought and knowledge: An introduction to critical thinking(2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

King, P.M. &Kitchener, K.S. (1994).Developing reflective judgment. San Francisco,CA: Jossey-Bass.

Kitchener, K. S. & King, P. M. (1990). The Reflective Judgment model: Ten years ofresearch. In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, R. A.Grotzer, & J. D. Sinnott (Eds.), Adult development: Vol 2. Models and methods inthe study of adolescent and adult thought (pp. 63-78). New York: Praeger.

Kitchener, K. S., King, P.M., Wood P.K., and Davison, M.L. (1989). Consistency andsequentiality in the development of reflective judgment: A six year longitudinalstudy. Journal of Applied Developmental Psychology, 10, 73-95.

Kitchener, K.S., Lynch, C.L., Fischer, K.W. & Wood, P.K. (1993). Developmentalrange of reflective judgment: The effect of contextual support and practice ondevelopmental stage. Developmental Psychology, 29(5), 893-906.

Kitchener, K. S. & Fischer, K. W. (1990). A skill approach to the development ofreflective thinking. In D. Kuhn (Ed.) Developmental Perspectives on Teachingand Learning Thinking Skills. Contributions to Human Development, 21, 48-62.

Knowles, M.C. (1990). The adult learner: A neglected species (4th Ed.). Houston:Gulf Publishing Co.

Lynch, C.L., & Wolcott, S.K. (1997). Optimizing professional problem solving.Manuscript submitted for publication.

MacMorris, S.H. (1997). Linking critical thinking with empirical and reflectivepractice. Presentation at the Council of Social Work Education Annual ProgramMeeting, Chicago, IL.

Mumm, A.M. & Kersting, R.C. (1997). Teaching critical thinking in social workpractice courses. Journal of Social Work Education, 33(1), 75-84.

Perry, W.A. (1970). Forms of intellectual and ethical development the college years:A scheme. Troy, MO: Holt, Rinehard & Winston, 1970.

Paul, R.W. (1990).Critical thinking: What every person needs to survive in a rapidlychanging world. Rohne, Park, CA: Center for Critical Thinking and Moral Cri-tique.

Paul, R. (1992). Critical thinking: What, why and how. New Directions for Commu-nity Colleges, 77, 3-23.

Polkosnik,M.C. &Winston, R.B. (1989). Relationships between student’s intellectu-al and psychological development: An exploratory investigation. Journal of Col-lege Student Development, 30, 10-19.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014

Page 23: Developing Reflective Judgment Through MSW Education

Cathryn C. Potter and Jean F. East 237

Potter, C.C., East, J. &Wolcott, S.K. (1997). Evaluating critical thinking outcomes ofMSW and MBA programs. Working manuscript.

Rogoff, Barbara. (1990). Apprenticeship in Thinking: Cognitive Development inSocial Context. New York, NY: Oxford University Press.

Sakalys, J.A. (1984). Effects of a research methods course on nursing students’research attitudes and cognitive development.Nursing Research, 33(5), 290-295.

Schon, D.A. (1983). The reflective practitioner: How professionals think in action.New York: Basic Books.

Schon, D.A. (1987). Educating the reflective practitioner. San Francisco, CA:Jossey-Bass Publishers.

Schmidt, J.A. & Davison, M.L. (1983). Helping students think. The Personnel andGuidance Journal. 563-569.

Schmidt, J.A. (1985). Older and wiser: A longitudinal study of the impact of collegeon intellectual development. Journal of College Student Personnel, 26, 388-394.

Selig, J. (1991). Social work and the critical thinking movement. Journal of Teachingin Social Work, 5(1), 21-34.

Van Maanen, M. (1991). The tact of teaching: The meaning of pedagogical thought-fulness. Albany, NY: State University of New York Press.

Van Tine, N.B.(1990). The development of reflective judgment in adolescents. Dis-sertation Abstracts International, 51, 26-59.

Welfel, E.R. & Davison, M.L. (1986). The development of reflective judgment in thecollege years: A four year longitudinal study. Journal of College Student Person-nel, 27, 209-216.

Wolcott, S.K. (1997). Superior managerial analysis and problem solving: A process.Classroom Handout.

Wolcott, S.K. & Lynch, C.L. (In press). Critical thinking in the accounting class-room: A reflective judgment developmental process perspective. Accountingeducation: A journal of theory. practice and research.

Wolcott, S.K. & Lynch, C.L. (1998). Optimizing professional problem solving. ToImprove the Academy (in press).

Wood, P.K. (1997). A secondary analysis of claims regarding the Reflective Judg-ment Interview: Internal consistency, sequentiality and intra-individual differ-ences in ill-structured problem solving. In J. Smart (Ed.),Higher education hand-book of theory and research. Bronx, NY: Agathon.

Wood, P.K., Kitchener, K. S. & King, P.M. (1997). Instructions for giving the reason-ing about critical issues questionaire. Columbia, MO: University of Missouri.

Dow

nloa

ded

by [

Was

hing

ton

Uni

vers

ity in

St L

ouis

] at

03:

31 0

5 O

ctob

er 2

014