http://jmd.sagepub.com/Journal of Marketing Education

http://jmd.sagepub.com/content/27/3/250The online version of this article can be found at:

 DOI: 10.1177/0273475305279657

2005 27: 250Journal of Marketing EducationJames W. Peltier, Amanda Hay and William Drago

The Reflective Learning Continuum: Reflecting on Reflection  

Published by:

http://www.sagepublications.com

can be found at:Journal of Marketing EducationAdditional services and information for    

  http://jmd.sagepub.com/cgi/alertsEmail Alerts:

 

http://jmd.sagepub.com/subscriptionsSubscriptions:  

http://www.sagepub.com/journalsReprints.navReprints:  

http://www.sagepub.com/journalsPermissions.navPermissions:  

http://jmd.sagepub.com/content/27/3/250.refs.htmlCitations:  

What is This? 

- Nov 11, 2005Version of Record >>

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

ARTICLEDECEMBER 2005JOURNAL OF MARKETING EDUCATION

The Reflective Learning Continuum:Reflecting on Reflection

James W. Peltier, Amanda Hay, and William Drago

The importance of reflection to marketing educators is in-creasingly recognized. However, there is a lack of empiricalresearch that considers reflection within the context of boththe marketing and general business education literature. Thisarticle describes the use of an instrument that can be used tomeasure four identified levels of a reflection hierarchy: habit-ual action, understanding, reflection, and intensive reflectionand two conditions for reflection: instructor-to-student inter-action and student-to-student interaction. The authors alsodemonstrate the importance of reflective learning in predict-ing graduates’ perception of program quality. Although thefocus was on assessment of MBA-level curricula, the findingshave great importance to marketing education andeducators.

Keywords: reflection; critical reflection; learning styles;habitual learning; deep learning

Porter and McKibbin’s (1988) now infamous report titled“Management Education and Development: Drift or Thrustin the 21st Century” was an ominous warning to administra-tors and educators alike. Although many deficiencies werehighlighted in that report, of particular relevance to marketingeducators was the perceived inability of students to identifybusiness problems across a wide range of integrated decisionenvironments, to understand and appreciate the entire gamutof decision-oriented processes and experiences, and to com-prehend and prepare for a lifetime of learning. While nearlytwo decades have passed since the time frame of that studywas initiated, these same sentiments are being echoed abouttoday’s MBA programs (Richards-Wilson 2002; Stevens2000). Pfeffer and Fong (2002) contended that there is anoveremphasis on analysis at the expense of decision integra-tion and developing wisdom. Similarly, Mintzberg and Gos-ling (2002) criticized MBA programs for placing too muchemphasis on the function of business rather than on the prac-tice of managing business across a wide spectrum of personalexperiences and decision dilemmas. The net result is that stu-dents entering the business world often lack the reflective

thinking skills necessary for discovering insights throughexperience, necessary requisites to becoming lifelonglearners (Bourner 2003; Braun 2004).

In response to these concerns, pedagogy researchers areadvocating curricular and educational practices that moti-vate, develop, and nurture reflective thinking in students(Hay, Peltier, and Drago 2004; Kember and Leung 2005;Pfeffer and Fong 2002; Van Velzen 2004). Although rela-tively scant discussions are found in the marketing literature(cf. Catterall, Maclaran, and Stevens 2002; Graeff 1997),reflection as a learning tool and action process has long beenseen as a crucial prerequisite to effective decision making,especially in decision environments that contain ambiguousproblems, unique elements, and no defined solutions (Peeet al., 2000; Schön 1983, 1987). Boud, Keogh, and Walker(1996) portrayed reflection as cognitive and/or affectiveresponses resulting from personal experiences that revise orcreate new understandings. Maclellan (2004) noted that thisintention to revise extant understanding or to construct newmeaning as experience unfolds incorporates a variety of men-tal and emotional processes, including dealing with fuzzyideas to reconcile ambiguity and inconsistency; to recognizethat one’s current knowledge set may be confused, incom-plete, or misconceived, culminating in the purposefulrealignment of meaning to that which is disparate. Mintzbergand Gosling (2004) argued persuasively that reflecting doesnot mean musing; reflection “means wondering, probing,analyzing, synthesizing—and struggling” (p. 21).

250

James W. Peltier, PhD, is the Arno Kleimenhagen Professor of Marketing inthe College of Business and Economics at the University of Wisconsin–Whitewater. Amanda Hay, PhD, is a senior lecturer in the Department ofHuman Resource Management, Nottingham Business School, NottinghamTrent University, United Kingdom. William Drago, PhD, is a professor ofmanagement in the College of Business and Economics at the University ofWisconsin–Whitewater. Please address correspondence to James W. Peltier,University of Wisconsin–Whitewater, College of Business & Economics,Whitewater, WI 53190; phone: (262) 472-5474; fax: (262) 472-4863; e-mail:peltierj@uww.edu.

Journal of Marketing Education, Vol. 27 No. 3, December 2005 250-263DOI: 10.1177/0273475305279657© 2005 Sage Publications

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

Despite the lack of theoretical development and empiricalinvestigations in the marketing and business education litera-ture, a greater understanding and use of reflection has impor-tant ramifications for both the academic and business com-munity. Arguably, one of the most important objectives ofgraduate business education is to develop leadership andleaders (Friga, Bettis, and Sullivan 2003). Graduates whoclearly understand that learning is a process of self-discovery,who challenge their own personal assumptions and beliefs,who question the actions that they and others take, and have acomprehensive view of effective managerial practices arewell on their way to becoming effective leaders (Kayes2002). As Kouzes and Posner (2002) so aptly elaborated,effective leaders are constantly learning and see all experi-ences as learning experiences. In turn, leaders steeped in thetradition of reflection can move constituents to heighteneddegrees of consciousness and self-actualization (Bennis andNanus 1997).

Adding to the general lack of reflection research in themarketing education literature is the paucity of instrumentsdesigned to operationalize and test this construct (Kember,Biggs, and Leung 2004; Kember et al. 1999; Kember et al.2000; Leung and Kember 2003). More important, the vastmajority of research on reflection within and external to busi-ness education has been at the conceptual rather than empiri-cal level. Part of this dearth is due to the fact that there is noone consistent theory on reflection (van Woerkom 2004).Moreover, nearly all of the empirical studies of reflectionhave either been qualitative in nature (cf. Cope 2003; Demp-sey, Halton, and Murphy 2001; Loo 2002; Maclellan 2004) orused instruments with relatively low reliability scores and/orthat lacked sufficient validation (cf. Kember, Biggs, andLeung 2004; Kember et al. 2000; Leung and Kember 2003).Of significance, the Association to Advance CollegiateSchools of Business (AACSB), the primary accrediting orga-nization for business programs, is requiring increased out-come measurements and assurance of learning reporting(AACSB International 2004). To meet this assurance oflearning standards, particularly as they relate to the role ofreflective learning in the management curriculum, research isneeded that develops the nature and scope of the reflectionconstruct and that offers reliable and valid assessment tools.

Although the reflection construct has received relativelylittle attention in the marketing education and general busi-ness literature (Catterall et al. 2002), research on measuringstudent perceptions of their educational experience is perva-sive. Although too extensive to review here, a wealth of recentmarketing education literature relevant to our area of investi-gation suggests that marketing educators and learning envi-ronments are perceived to be most effective when students areproactive and engaged; when instructors and studentsactively engage and coproduce what is learned; when coursecontent is competent, challenging, experiential, and has real-world relevancy; and student-to-student interactions enhance

the learning experience (for recent reviews, see Faranda andClarke 2004; Paswan and Young 2002; Peltier, Drago, andSchibrowsky 2003; Smart, Kelley, and Conant 2003; Smithand Van Doren 2004; Wilhelm 2004; Young 2005; Young,Klemz, and Murphy 2003).

The intent of this article is to address the limited attentiongiven to reflection within the marketing education literatureby bringing together research from multiple disciplines forthe purpose of describing and developing the theoreticalunderpinnings of the reflective learning process, and moreimportant, to use an instrument that measures reflectivelearning. As part of this endeavor, we seek to identify thedimensions of reflective learning that affect how graduate-degree recipients value the educational experiences that theyreceived and how this education has benefited them uponentering the workforce. In this way, our research is consistentwith calls by marketing educators to assess the value of thelearning experience and how well these experiences carryover to students’ lifelong learning process once in the work-force (Duke 2002). Here, we provide evidence for the reli-ability and validity of an instrument for measuring thedimensionality of reflective learning and its impact on out-come assessment. It is important to note that because theextant literature in reflective thinking in marketing educationis underdeveloped, we are directing our theoretical focus atthe business curriculum and not specifically for marketingfaculty and courses.

REVIEW OF REFLECTION LITERATUREAND MODEL DEVELOPMENT

Within the marketing education literature, conceptual andempirical research on reflection is in its infancy. However, thereflection construct is receiving considerable attention in awide range of disciplines including social work (Dempsey,Halton, and Murphy 2001; Fisher and Somerton 2000), infor-mation systems (Mukherjee 2004), earth sciences (Harrison,Short, and Roberts 2003), leadership development (Denstenand Gray 2001), and particularly in the areas of humanresource development (Bourner 2003; Corley and Eades2004; Maclellan 2004; Van Woerkom 2004; Van Woerkom,Nijhof, and Nieuwenhuis 2002) and health sciences (Biggs,Kember, and Leung 2001; Kember et al. 2000; Leung andKember 2003; Liimatainen, Poskiparta, Karhila, and Sjogren2001). We examine this diverse literature base and blend itwith emerging business education literature for the purposeof defining the reflection construct, identifying its dimen-sions and processes, and as input for further consideration ofits measurement.

Reflection Defined

Dewey (1933), one of the earliest pioneers in the field,defined reflection as “active, persistent and careful consider-ation of any belief or supposed form of knowledge in the light

JOURNAL OF MARKETING EDUCATION 251

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

of the grounds that support it and the further conclusion towhich it tends” (p. 9). Dewey’s view of reflective learningencompassed two interrelated ideas: “(1) a state of doubt,hesitation, perplexity, mental difficulty, in which thinkingoriginates, and (2) an act of searching, hunting, inquiring, tofind material that will resolve the doubt, settle and dispose ofthe perplexity” (p. 12). Building on Dewey, Kolb’s (1984)experiential loop concept expressed reflection as one of fourkey processes through which individuals make sense of theirexperiences and in turn yields a basis for future action and ini-tiates new or adapted ideas in the process. Similarly, Schön(1983) linked reflection and experience through his conceptof reflection-in-action as “on the spot surfacing, criticizing,restructuring and testing of intuitive understandings of expe-rienced phenomena” (pp. 241-242). Focusing on reflection asa mechanism for change, Boyd and Fales (1983) definedreflective learning as “the process of internally examiningand exploring an issue of concern triggered by an experience,which creates and clarifies meaning in terms of self andwhich results in a changed conceptual perspective” (p. 100).More recently, reflection and reflective learning have beendefined in terms of adaptive learning (van Woerkom 2004),self-construction (Dempsey, Halton, and Murphy 2001), self-awareness (Loo 2002), coordinated understanding(Maclellan 2004), and critical self-reflection (Cope 2003).From these definitions, it is clear that an important outcomeof this exploration and internal examination process is chang-ing one’s perspective as new information and experiences areencountered (Hay, Peltier, and Drago 2004). As we note later,there is also a difference between general reflection andcritical reflection.

The Reflection Process andthe Nonreflection ↔ Reflection Hierarchy

Boud, Keogh, and Walker (1985) noted that reflection isan integrated learning process with multiple, although non-linear, dimensions. They underscored the importance ofexperience and viewed the reflection process as having threeinterrelated stages: return to experience, analysis, and reeval-uation in terms of the emotional experiences and resultingreintegration of the outcomes that add to the learner’s iden-tity. Correspondingly, and based in part on the work of Dewey(1933), Mezirow (1981), Kolb (1984), Kember et al. (1999),Kember et al. (2000), Hay, Peltier, and Drago (2004), and oth-ers, the reflection process can be conceptualized in terms ofawareness—critical analysis—and change. Awareness,which is initiated when an individual becomes conscious of aprevious experience, is often stimulated by positive or nega-tive feelings about a learning situation. If salient, the eventtriggers the reflection process. In the second stage, the indi-vidual performs a critical analysis of the experience, whichinvolves identifying existing and pertinent knowledge, chal-lenging internal and external assumptions, and seeking possi-ble resolution alternatives. In the final stage, and if cognitive

and/or emotional change is necessary, the individual reachesa new perspective, sometimes called perspective transforma-tion (Mezirow 1991). As a result of this reflection-basedchange, the individual alters firmly held beliefs, and ulti-mately, behavior.

The majority of research on reflection has focused on itsprocess orientation. More recently, and based on the initialwork of Mezirow (1977, 1981, 1991), empirical research isemerging that investigates reflection in terms of hierarchicallevels of learning (Cope 2003; Hay, Peltier, and Drago 2004;Kember et al. 2000; Kember et al., 2004; Leung and Kember2003; Liimatainen et al. 2001; Maclellan 2004; vanWoerkom, Nijhof, and Nieuwenhuis 2002). Learning hierar-chies are distinguishable in terms of the amount of effortneeded for more basic, routine learning activities, comparedto deeper cognitive processing necessary for generating newunderstandings and modifying internal theories for action(Argyris and Schön 1996). Mezirow (1991) posited twobroad hierarchies of learning actions: reflective (i.e., higherlevel learning) and nonreflective (i.e., lower level learning).Reflective learning is therefore seen as being associated withwhat has generally been described as higher level learning.Higher level learning is seen to encompass various forms ofmore intensive learning within the literature such as doubleloop learning, transformational learning, and deep learning(Argyris 2002; Cope 2003). Moreover, Leung and Kember(2003) provided evidence for a relationship between a deepapproach to learning and reflection and encourage the twointerpretive frameworks to be used in concert.

Research by van Woerkom, Nijhof, and Nieuwenhuis(2002) supports the notion that higher level learning (reflec-tion) is in part a function of self-efficacy or the confidencemanagers have in their ability and courage to withstand socialpressure, to be critical, to take a vulnerable position, to ask forfeedback, and to evaluate their own performance. Like-wise, building on Dewey (1933), Densten and Gray (2001)contended that the depth of learning is a function of open-mindedness (i.e., listen to multiple perspectives), responsibil-ity (i.e., desire to seek the truth), and wholeheartedness (i.e.,willingness to make change by critically evaluating self andothers).

The type of learning process different students are likely toemploy is contingent on personal motivation, the amount ofeffort they give to solving a problem, and their individual per-formance goals (Pee et al. 2000; van Woerkom, Nijhof, andNieuwenhuis 2002). Biggs (1987) argued that students differin terms of when and under what conditions they would applya deep versus a surface approach to learning. Specifically, astudent engaging in a surface approach to learning sees thetask as a demand to be met or as an imposition to reaching agoal, views the various aspects of the task to be discrete andindependent of each other, prefers to solve the task quickly, isnot concerned with attaching personal or other well-ingrainedmeanings to the task, relies on memorization, and is most

252 DECEMBER 2005

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

concerned with basic-level aspects. In contrast, studentsengaging in deep learning are interested in the task itself andenjoy completing the task; search for the inherent meaning inthe task; make the task personal, attaching meaning from botha personal and a real-world perspective; view the task holisti-cally and try to integrate the individual components of thedecision scenario; and try to apply theories to the task and inturn create testable hypotheses.

Related to business education, Cole, Feild, and Harris(2004) investigated “psychological hardiness” and its rela-tion to students’motivation to learn (commitment), their abil-ity to learn (control), and their willingness to change (chal-lenge). Of particular relevance to reflective learning is thechallenge construct, described by Kobasa and Puccetti (1983)as a “belief that change rather than stability is normal in lifeand that the anticipation of changes are interesting incentivesto growth rather than threats to security” (p. 169). Cole, Feild,and Harris concluded that “hardy persons experience activi-ties as interesting and enjoyable, as being a matter of personalchoice, and as important stimuli for learning. In ambiguoussituations, hardiness equips individuals to draw upon a per-sonal sense of commitment and control to find meaning intheir tasks as well as exercise decision-making and goal-setting techniques” (p. 66).

Adapting Mezirow’s (1991) multitiered levels of reflec-tive and nonreflection learning, Kember et al. (2000) positedfour different “hierarchical” levels of reflection: habitualaction/learning, understanding, reflection, and critical reflec-tion. Although they did not link reflection to learning out-comes, they did develop and test an instrument to determinewhether students differed in their level of reflection. Theirprimary research hypothesis was supported in that studentsdid differ across the four hierarchical learning dimensions.Table 1 presents our visualization of where each of Kemberet al.’s four dimensions fall along the nonreflection/surfacelearning ↔ reflection/deep learning hierarchy and how wedefine each dimension. We have thus attempted to integrate

the various levels of reflection with surface and deepapproaches to learning, following Leung and Kember (2003)who have encouraged the use of the two interpretive frame-works in concert. More important, although our dimensionsin Table 1 occupy different scale positions on the lower-higher level learning hierarchy, as a continuum, each dimen-sion has a certain amount of gray area as it transitions to thenext higher learning dimension.1

Habitual action/Habitual learning

Habitual action refers to “that which has been learnt beforeand through frequent use becomes an activity that is per-formed automatically or with little conscious thought”(Kember et al., 2000, p. 383). Habitual actions are those thathave been learned through repetition, in which a particularproblem has been encountered a number of times and hasbeen resolved in a consistent and routine fashion. Habitualaction, which is consistent with a surface approach to learn-ing, occupies the lowest level of the reflection hierarchy andinvolves minimal thought and engagement (Biggs, Kember,and Leung 2001; Leung and Kember 2003). According toEntwhistle and Ramsden (1983), surface learning is an attitu-dinal state of “unreflectiveness” through which specific tasksare processed as unrelated activities and where memorizationis the most common method for learning. Schön (1983)referred to these types of decision environment and problemresolution scenarios as “knowing-in-action”; that which hasbecome almost unconscious routine. Consistent withMezirow’s (1991) view that habitual actions involvenonreflective learning, Cope (2003) suggested that the incre-mental accumulation of routinized, habitual events stagnatesthe learning process and that which can be learned.

From a management education perspective, as a lowerlevel learning perspective, we contend that habitual action/learning is characteristic of students who memorize ratherthan reflect, prefer tests over strategic learning exercises,focus only on what is required rather than exploring addi-

JOURNAL OF MARKETING EDUCATION 253

TABLE 1THE NONREFLECTION ↔ REFLECTION CONTINUUM

Nonreflection/Surface Learning Reflection/Deep Learning

Habitual Action Understanding Reflection Intensive Reflection

Minimal thought andengagement, correlated with asurface approach tolearning—specific tasks aretreated as unrelated activities,memorization is emphasized,and which embodies anattitudinal state ofunreflectiveness.

Focuses on comprehensionwithout relation to one’spersonal experience or otherlearning situations. Booklearning as beingunderstanding-oriented inthat the learner need onlycomprehend the readmaterials. Most of what islearned stays within theboundaries of preexistingperspectives.

Learning is related to personalexperience and otherknowledge. Reflection alsoinvolves challengingassumptions, seekingalternatives, identifying areasfor improvement. Showsactive and consciousengagement, characteristicscommonly associated with adeep approach to learning.

Intensive reflection is at thehighest level of the reflectivelearning hierarchy, andlearners become aware ofwhy they think, perceive, oract as they do. Learner mightalter or even completelychange firmly held beliefsand ways of thinking.Intensive reflection is thusseen as involving a changein personal beliefs.

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

tional learning materials, and who focus more on the gradethan what is learned.

Hypothesis 1: Habitual action/learning is negatively related toalumni’s perceived quality of the learning experience.

Understanding

Understanding is situated at the second level in the reflec-tion hierarchy and focuses for the most part on comprehen-sion without relation to one’s personal experience or otherlearning situations (Kember et al. 2000). Kember et al.’soperationalization of “understanding” draws on Bloom’sknowledge taxonomy (1979), in which understanding isequated to comprehension in that students learn without relat-ing to other situations and/or to personal experiences. Leungand Kember (2003) contended that much of the “book learn-ing” activities that students use when studying is characteris-tic of the understanding construct. As with habitual action,the understanding stage of the reflection hierarchy is alsoconsistent with nonreflective thinking in that comprehensionis valued over mental integration and personal assimilationbut does, however, represent a deeper form of learning.Mezirow (1991) termed this type of learning “thoughtfulaction” and noted that although understanding requires moreactive engagement in learning than habitual action, most ofwhat is learned stays within the boundaries of preexistingperspectives. In this way, a student using a thoughtful actionlearning rule would access existing knowledge without ap-praising that knowledge and would process the informationwithin preexisting schemas.

Within a marketing education context, habitual action/learning may be seen as more process oriented versus under-standing, which is more content driven. Understanding maybe seen to relate to the extent to which content required con-scious thought of the student. However, as understanding isnot seen to involve personal assimilation, thus representing alower level of learning, we hypothesize that a greater focus onunderstanding relative to reflective learning will negativelyaffect perceptions of the learning experience.

Hypothesis 2: Understanding is negatively related to alumni’sperceived quality of the learning experience.

Reflection

Habitual action and understanding are viewed as beingpositioned nearer to the surface learning portion of thenonreflection ↔ reflection hierarchy. The third and finalhierarchical levels of the reflection process, reflection andcritical/intensive reflection, are both associated more closelywith deeper and more thoughtful learner and learning traits(Dewey 1933; Mezirow 1991). The primary differencebetween reflection and critical/intensive reflection is that thelast stage involves a change in personal beliefs and assump-tions. In defining reflection, Kember et al. (2000) drew on

Mezirow (1991), who expounded that “reflection involvesthe critique of assumptions about the content or process ofproblem solving. . . . The critique of premises or presupposi-tions pertains to problem posing as distinct from problemsolving. Problem posing involves making a taken-for-grantedsituation problematic, raising questions regarding its validity”(p. 105). Mezirow further outlined two types of reflection dis-tinct from critical reflection: content reflection, which refersto “reflections on what we perceive, think, feel or act upon”(p. 107), and process reflection, which refers to “examinationsof how one performs the functions of perceiving, thinking,feeling, or acting and an assessment of the efficacy of perform-ing them” (pp. 107-108). Kember et al. (2000) subsequentlycollapsed these two types of reflection into s single dimen-sion. Central to reflection is thus a questioning of experiencesand in turn a search for alternatives and identification of areasfor improvement (Pee et al. 2000). A focus on alternativeways of seeing and doing is seen as especially important inthe case of the management student since exploring differentperspectives will assist managers in handling complexity,uncertainty, and uniqueness that are seen to characterizetoday’s managerial world (Dehler, Welsh, and Lewis 2001).A more active form of learning is thus suggested, and indeed,Atkins and Murphy (1993) argued that reflection involvesconsideration of the wider implications of learning. Reflec-tion is seen to be closely associated with a deep approach tolearning where formal learning is integrated with personalknowledge and the formation of relationships between partsof knowledge and search for meaning (Entwhistle andRamsden 1983). Leung and Kember (2003) further suggestedthat deep learning is a prerequisite to reflection.

Marketing courses that encourage reflective learning maybe seen to assist students in appraising their experiences andin turn identifying new and alternative solutions to problems,improving their past actions, and thinking about the widerimplications of their experiences. For example, case studiesor experiential exercises would be better received than lec-tures and exams. As reflective learning is seen to represent aform of higher level learning, we hypothesise that alumni willevaluate their learning experience more highly where reflec-tive learning is invoked.

Hypothesis 3: Reflection is positively related to alumni’s per-ceived quality of the learning experience.

Critical/Intensive reflection

Critical reflection is at the pinnacle of the nonreflection ↔reflection hierarchy. Critical reflection represents a moreintense form of reflection and involves a transformation ofmeaning frameworks. Drawing on Mezirow (1991), Kemberet al. (2000) suggested that this form of reflection “involvesus becoming aware of why we perceive, think, feel or act aswe do” (p. 110). Kember et al. further proposed that integralto such reflection is a perspective transformation where an

254 DECEMBER 2005

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

individual critically reviews his or her internal prepositions,cumulating in changing one’s assumptions and beliefs of self.Along these same lines, Boyd and Fales (1983) pointed outthat higher level reflection, and subsequent change orienta-tion, “is the process of internally examining and exploring anissue of concern, triggered by an experience, which createsand clarifies meaning in terms of self, and which results in achanged conceptual perspective” (p. 100). Kember et al.(1999) contend that critical reflection and resulting internaltransformations are difficult tasks in that conventional wis-dom and ingrained assumptions become so deeply embeddedthat we may not even be aware that they are assumptions orthat they actually exist.

Within the literature, there appears to be some debate sur-rounding the description of critical reflection in relation tobusiness education. There is disagreement between scholarsas to whether critical reflection is really critical in the senseintended by critical theorists. Reynolds (1998) argued thatcritical reflection “involves analysis of power and control andan examination of the ‘taken for granteds’ within which thetask or problem is situated” (p.189) and is therefore associ-ated with critical theory, which seeks to challenge power rela-tions in the pursuit of emancipatory and political ideals(Willmott 1997). Reynolds argued that Mezirow’s concept ofcritical reflection is not critical in this sense as it presents anexplanation of personal, psychological change rather thansocial change. Therefore, the conceptualization of criticalreflection used here cannot be seen to be consistent with criti-cal theory because our focus is on the individual. However,we would argue that the self-analysis and self-questioningassociated with reflection and the perspective self-transfor-mation in critical reflection as used here may offer the poten-tial for social change and the development of “fundamentallybetter organizations, better societies and a better world byfreeing managers from socially embedded assumptions”(Kayes 2002, p. 138). Indeed, Cope (2003) highlighted thatcritical self-reflection can stimulate both challenges to per-sonal assumptions and a deeper understanding of one’s busi-ness. We therefore propose that qualitative differences in lev-els of reflection exist and following Cope (2003), we suggestthat critical reflection within our hierarchy may be more help-fully termed intensive reflection to describe a deeply chal-lenging form of reflection.

Ultimately, marketing educators stimulating intensivereflection will have accomplished the highest of all learninggoals by motivating students to rethink assumptions aboutthemselves and about business, and in turn to change howthey think of the world and their actions in response to thisepiphany.

Hypothesis 4: Intensive reflection is positively related toalumni’s perceived quality of the learning experience.

Conditions for Reflection

Reflection and intensive reflection are high-order learningand change processes that are often difficult to reach(Mezirow 1998). Moreover, both types of reflection occur invarying degrees across individuals and in different types oflearning environments, and as such, a better understanding ofthe conditions for sparking reflective processes is a key toacademic success, and ultimately, as a means of enhancingthe educational experience (Hay, Peltier, and Drago 2004).Although underdeveloped, a review of the emerging litera-ture suggests that two key conditions within the academiclearning environment may foster reflective thinking:instructor-to-student interactions and student-to-studentinteractions.

Instructor-to-student interactions

For higher-order reflections to occur, the learning environ-ment must allow for individuals to express doubt, to exploreuncertainties, and to become aware of internal and externalcontradictions (Boud, Keogh, and Walker 1985). The role ofthe educational instructor is thus paramount to elicitingreflective learning. More important, the instructor must cre-ate a learning environment in which open discussion is facili-tated, allowing individuals to express doubt in what is beinglearned, and that affords the opportunity to raise points of dis-agreement (Thorpe 2001). Because reflective thinking is notnecessarily an innate trait, the instructor must design thecourse in such a way as to build students’ metacognitiveawareness of their personal thought processes, that hones stu-dents’ basic skills, and provides dialogues with students thatare supportive and trusting (Braun 2004; Brown and Posner2001). In many ways, the instructor is a coach who leads stu-dents down the path to reflection-in-action (Schön 1987).This importance is magnified in that many students are“reflection novices,” and as such, need considerable guidanceto move from lower to higher forms of reflective thinking(Bailey et al. 1997). Conceptually, for reflective learning tooccur, students should view the lecturer as a “mentor” whopromotes learning in a nonthreatening learning environment,by encouraging questions and by highlighting connectionsamong and between concepts (Bailey et al. 1997; Fisher andSomerton 2000; Marton and Saljo 1984). Clearly, the instruc-tor is one of the most critical components of “trans-formational” learning and internal change (Mintzberg andGosling 2004; Hay, Peltier, and Drago 2004; Taylor 2000).As Hypothesis 5 states below, positive instructor-to-studentinteractions enhance the learning experience, whereasnegative interactions impair the learning experience.

Hypothesis 5: Instructor-to-student interactions are directly re-lated to alumni’s perceived quality of the learning experience.

JOURNAL OF MARKETING EDUCATION 255

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

Student-to-student interactions

Without question, instructor-to-student interactions arecritical for introducing, developing, and nurturing higher-order reflective learning. However, student-to-student inter-actions must also play an important role in any learning pro-cess (Peltier, Drago, and Schibrowsky 2003; Hay,Hodgkinson, et al. 2004). Dempsey, Halton, and Murphy(2001) contended that the sharing of thoughts and feelingswith others is a fundamental requisite to reflection. In theirview, transformational change is in part a function of the dia-logue that takes place between students that gives support toand motivates the learner to move beyond intellectual grasp-ing of ideas to find personal meaning, and thus, to progressfrom understanding to reflection. Similarly, Gray (2001)argued that student-to-student interactions facilitate theapplication of learning to existing knowledge by introducingreal-world examples of the learning topic. Of interest, theseinteractions are no longer exclusively within the classroom;they can also occur in the electronic learning space (Hay,Hodgkinson, et al. 2004; Peltier, Drago, and Schibrowsky2003). In combination, student-to-student interactions willassist the transition to a higher-order learning process andreflection-oriented transformations through vision sharing(van Woerkom 2004), coproduction of outcomes (Biggs,Kember, and Leung 2001), analyzing and comparing one’sresponses to others (Thorpe 2001), and enhancing team lead-ership skills (Brown and Posner 2001). Ultimately, to achievereflection within both an academic and an organization learn-ing environment, learning participants need to share reflec-tions and experiences, support and encourage, disseminateideas and feelings outward, and assimilate the views of others(Hodgkinson and Brown 2003). As Hypothesis 6 identifiesbelow, positive student-to-student interactions enhance thelearning experience, whereas negative interactions impair thelearning experience.

Hypothesis 6: Student-to-student interactions are directly relatedto alumni’s perceived quality of the learning experience.

Model Development

Our model, which is shown in Figure 1, incorporates threecomponents: (1) the four stages along the nonreflection ↔reflection hierarchy (habitual action, understanding, reflec-tion, and intensive reflection), (2) two conditions within aneducational setting for creating higher-order reflection(instructor-to-student and student-to-student interactions),and (3) outcome measurements pertaining to the value of thelearning experience. From our review of the scant literature inthis area discussed above, we hypothesize that except forhabitual action and understanding, all of the dimensionswill be positively related to perceived quality of the learningexperience.

METHOD

Data Collection and Sample

The study was conducted at a midwestern university with alarge College of Business and Economics. In total, 323 indi-viduals who had received their MBA in the preceding 3-yearperiod were mailed the questionnaire. MBA students wereused because the college was undertaking an assessment of itsgraduate business program. A postcard promoting the surveywas mailed prior to distribution of the actual questionnaire.All graduates received a $2 participation incentive, and ano-nymity was assured. A total of 220 questionnaires werereturned after two mailings, for a response rate of 68.1%.

Questionnaire Development

A detailed and extensive questionnaire was developedto assess perceptions regarding all four stages in thenonreflection ↔ reflection hierarchy (habitual action, under-standing, reflection, and intensive reflection) and the twoeducational-setting variables (instructor-to-student andstudent-to-student interactions). Given the suggested impor-tance of reflective learning, it was surprising that few mea-sures actually exist. Indeed, we only identified four instru-ments designed to assess reflection (Boenink et al. 2004;Kember et al. 2000; Sobral 2000; van Woerkom, Nijhoff, andNieuwenhuis 2002). Furthermore, none of these were devel-oped within a management education context. Moreover,although there are a wide range of topics that could be viewedas important to reflective learning, the literature is scant interms of integrating these topical areas.

256 DECEMBER 2005

LEARNING OUTCOMES

Student-to-Student Interactions

(+)

Instructor-to-Student Interactions

(+) R

E F L E C T I O N

H I E R A

C

HY

Intensive Reflection

(+)

LEARNING CONDITIONS

Reflection

(+)

Habitual Action

(-)

Understanding

(-)

R

FIGURE 1: Model of Reflective Learning

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

When developing our questionnaire, we did an extensivereview of the reflection literature identified above to identifythe possible domain of useful questions. We then integratedand refined statements from this literature to create our instru-ment. In developing the questionnaire, the work of Kemberet al. (2000) was especially helpful. As described above,Kember et al.’s scale comprises four levels: habitual action,understanding, reflection, and critical reflection, which arebased on Mezirow’s model of reflection. However, there wasan issue with respect to the reliability of the individual scaleswith three of the four scales having alpha coefficients belowNunnally and Bernstein’s (1994) accepted level of .70. There-fore, an attempt was made to improve the clarity/understanding of certain items and in some cases to relateitems to a management education context. Additional itemswere also added to examine aspects of reflection and inten-sive reflection such as integration of learning with existingknowledge (Fisher and Somerton 2000) and consideration ofthe wider implications of learning (Atkins and Murphy1993). Given the suggested relationship between approachesto learning and levels of reflective learning, items assessingsurface and deep approaches to learning were included fromBiggs, Kember, and Leung (2001). The literature also sug-gested the importance of conditions for reflection; accord-ingly, items were developed to assess both the role of the fel-low students (Braun 2004; Brown and Posner 2001;Dempsey, Halton, and Murphy 2001; Gray 2001;Hodgkinson and Brown 2003; Peltier, Drago, andSchibrowsky 2003) and the role of the professor (Bailey et al.1997; Fisher and Somerton 2000; Liimatainen et al. 2001;Thorpe 2001) in fostering reflection. Upon completion of thisreview, we developed 56 questions across the six dimensionsoutlined in our model. The questionnaire also contained sixglobal dependent measures designed to describe the value ofstudents’ overall learning experience. These are as follows: Ilearned a lot in the MBA program, I enjoyed the MBA pro-gram, I would recommend the MBA program to others, theMBA program has benefited my career, the MBA programhas benefited my life generally, and overall, I am satisfiedwith the MBA program. A factor analysis revealed that all sixquestions loaded on one global learning assessment measure-ment. We thus collapsed all six items into one summeddimension. All items were measured via a 5-point Likert-typescale that ranged from strongly agree to strongly disagree.

FINDINGS

Assessing the Underlying Dimensionalityand Measure Reliability

As noted, one of our major objectives was to use a reflec-tion scale that tapped into the nonreflection ↔ reflection hier-archy (habitual action, understanding, reflection, and inten-sive reflection) and that addressed conditions for reflection(instructor-to-student and student-to-student interactions). In

doing so, we first factor analyzed the 56 questions concerningvarious facets of the MBA program. First, a principle compo-nents analysis employing varimax rotation was conducted toidentify the major commonalities in the data. Next, the indi-vidual questions associated with each measure were evalu-ated using an item-to-total correlation analysis. In that previ-ous reflection instruments were constrained by havingrelatively low reliability scores, no priority was given tomaintaining any item from previous reflection question-naires. Given the goal was to create highly reliable dimen-sions, items with low item-to-total correlations and thatwould raise the coefficient alpha score if removed were elimi-nated. The coefficient alphas for final measures were thencalculated and ranged from .74 to .87, indicating that the mea-sures were reliable. The specific items making up each of themeasures and the coefficient alphas are shown in Table 2. Aspredicted, all six dimensions were present in the data andsupport the framework of the model.

Model Validity and ExplainingOutcome Assessment

Once dimensionality and measure reliability were con-firmed, we statistically assessed the impact of each of thedimensions on MBA graduates’ perceptions of the quality ofthe program and how it has benefited them upon entering theworkforce. As we show in Tables 3 and 4, all six of our hypo-theses were supported. Using regression, we first tested therelationship that each of the four nonreflection ↔ reflectiondimensions had on perceived learning outcomes. We wereparticularly interested in the direction of the relationship andthe relative importance each had in explaining the global out-come variable. Specifically, the factor scores for habitualaction, understanding, reflection, and intensive reflectionwere used as the independent variables; the summed globalmeasure was the dependent variable. The findings can befound in Table 3.

A number of meaningful conclusions can be drawn fromthe regression findings in Table 3. First, as expected, all fournonreflection ↔ reflection dimensions significantly affectedthe outcome variable (F = 36.5, model significant at p < .001,R2 = .43). Second, although not well documented in the litera-ture, our expectation was that habitual action and understand-ing would be negatively correlated with outcomes and thatreflection and intensive reflection would have a positive impacton outcome assessment. An examination of the standardcoefficients in Table 3 provides empirical support for ourdirectional hypotheses. Last, and important, the standardizedbeta coefficients show support for our premise that each stageof the nonreflection ↔ reflection hierarchy becomes increas-ingly important in explaining perceived learning outcomes.Specifically, and in order, intensive reflection (standardizedβ = 3.65, t = 7.0, p < .001) had the highest standardized betas,followed by reflection (standardized β = 3.59, t = 6.9, p <.001), understanding (standardized β = –.351, t = –6.7, p <

JOURNAL OF MARKETING EDUCATION 257

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

258

TAB

LE

2FA

CTO

R M

OD

EL

OF

DIM

EN

SIO

NS

OF

RE

FL

EC

TIV

E L

EA

RN

ING

AN

D IN

TE

RA

CT

ION

Inte

nsiv

eS

tude

ntIn

stru

ctor

Hab

itual

Ref

lect

ion

Ref

lect

ion

Inte

ract

ion

Inte

ract

ion

Lear

ning

Und

erst

andi

ng

Wha

t I le

arne

d m

ade

me

reth

ink

my

assu

mpt

ions

abo

ut b

usin

ess.

.687

I lea

rned

man

y ne

w th

ings

abo

ut m

ysel

f..6

65A

s a

resu

lt of

this

pro

gram

, I h

ave

chan

ged

the

way

I no

rmal

ly d

o th

ings

..6

58A

s a

resu

lt of

this

pro

gram

, I h

ave

chan

ged

the

way

I lo

ok a

t mys

elf.

.656

Wha

t I le

arne

d fo

rced

me

to r

ethi

nk h

ow I

view

the

wor

ld.

.647

I dis

cove

red

faul

ts in

wha

t I p

revi

ousl

y be

lieve

d to

be

right

..6

41C

ours

e co

nten

t cha

nged

man

y of

my

firm

ly h

eld

idea

s..6

39I l

earn

ed m

ore

abou

t my

own

lear

ning

pro

cess

..5

32I o

ften

reap

prai

sed

my

expe

rienc

es s

o I c

ould

lear

n fr

om th

em.

.682

I ofte

n re

flect

ed o

n m

y ac

tions

to s

ee w

heth

er I

coul

d im

prov

e th

em.

.650

I ofte

n tr

ied

to th

ink

abou

t how

I co

uld

do s

omet

hing

bet

ter

next

tim

e..6

38I l

iked

to th

ink

abou

t my

actio

ns to

find

alte

rnat

ive

way

s of

doi

ng th

em.

.613

I exp

lore

d m

y pa

st e

xper

ienc

es a

s a

way

of u

nder

stan

ding

new

idea

s..5

96I s

pent

con

side

rabl

e tim

e ex

plor

ing

alte

rnat

ive

solu

tions

to p

robl

ems.

.585

I ofte

n th

ough

t of w

ider

impl

icat

ions

of w

hat I

was

lear

ning

..5

57I o

ften

rela

ted

cour

se m

ater

ial t

o m

y ow

n ex

perie

nces

..5

12M

y fe

llow

stu

dent

s ch

alle

nged

me

to th

ink.

.700

I enj

oyed

lear

ning

new

way

s of

thin

king

from

oth

er s

tude

nts.

.671

I sou

ght f

eedb

ack

from

oth

ers

abou

t the

dec

isio

ns th

at I

mad

e..6

64O

ther

stu

dent

s he

lped

me

lear

n ab

out s

olvi

ng r

eal-w

orld

pro

blem

s..6

60M

y fe

llow

stu

dent

s va

lued

my

opin

ions

..6

38T

here

was

an

open

exc

hang

e of

new

idea

s be

twee

n st

uden

ts.

.606

Wer

e w

illin

g to

talk

abo

ut th

ings

that

I di

sagr

eed

with

..7

42E

ncou

rage

d st

uden

t que

stio

ns a

nd c

omm

ents

..7

20V

alue

d m

y op

inio

ns.

.696

Allo

wed

me

to e

xpre

ss d

oubt

in w

hat I

was

lear

ning

..6

95C

reat

ed a

n at

mos

pher

e th

at a

llow

ed m

e to

be

open

abo

ut m

y vi

ews.

.609

Ofte

n as

ked

ques

tions

to h

elp

me

thin

k m

ore

deep

ly.

.531

Mem

oriz

ing

thin

gs w

as o

ften

mor

e im

port

ant t

han

unde

rsta

ndin

g th

em.

.697

I usu

ally

onl

y fo

cuse

d on

wha

t I th

ough

t the

inst

ruct

or w

ould

test

me

on.

.692

I was

bet

ter

at ta

king

test

s th

an d

oing

mor

e st

rate

gic

assi

gnm

ents

..6

65If

it w

as n

ot g

oing

to b

e ev

alua

ted,

we

shou

ld n

ot h

ave

had

to s

tudy

it.

.617

My

goal

was

to g

et a

goo

d gr

ade

whi

le d

oing

as

little

wor

k as

pos

sibl

e..4

71C

ours

e co

nten

t was

rep

etiti

ous

so I

ofte

n di

d no

t nee

d to

thin

k..7

20M

uch

of w

hat I

lear

ned

I alre

ady

knew

..7

06M

uch

of w

hat I

lear

ned

requ

ired

little

or

no th

inki

ng.

.697

The

pro

gram

req

uire

d un

ders

tand

ing

conc

epts

taug

ht b

y th

e in

stru

ctor

..5

01I h

ad to

con

tinua

lly th

ink

abou

t mat

eria

l bei

ng ta

ught

..4

51

Var

ianc

e ex

plai

ned

= 5

6.3%

11.8

10.8

8.8

8.5

7.4

7.0

Coe

ffici

ent a

lpha

.87

.85

.80

.80

.75

.75

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

.001), and habitual action (standardized β = –211, t = –4.1, p <

.01). We believe this to be the first empirical test of the reflec-tive learning hierarchy in predicting program outcomes.

Table 4 contains the findings when learning conditionsrelated to perceptions of student-to-student and instructor-to-student interactions are included in the model (F = 47.2,model significant at p < .001, R2 = .58). As can be seen,instructor-to-student and student-to-student interactionswere found to be significant and positive predictors of perfor-mance outcomes. Of significance, the relative importance ofintensive reflection (standardized β = .364, t = 8.1, p < .001),reflection (standardized β = .358, t = 7.9, p < .001), andunderstanding (standardized β = –.351, t = –7.8, p < .001 waspreserved. Student-to-student interactions (standardized β =.283, t = 6.3, p < .001) and instructor-to-student interactions(standardized β = .259, t = 5.7, p < .001) had a greater impactthan did habitual action (standardized β = –.212, t = –4.7, p <.001).

DISCUSSION

The article has reported the results from a study designedto measure reflective learning in graduate business education.We address both the limited attention given to reflectivelearning within the marketing and business education literatureand the scarce attention given to measurement of reflectivelearning in general. Our work integrates theoretical under-standings of reflective learning from diverse subject areas tooffer an instrument that assesses both levels of reflective

learning and conditions for reflection. We also demonstrate theimportance of reflective learning in predicting graduates’per-ceptions of program quality and career enhancement.

Our review of the reflective learning literature identifiedsix dimensions associated with reflection. The resulting fac-tor analysis confirmed that the six dimensions were separateand important factors associated with reflective learning asperceived by graduate alumni. Four of these dimensionsrelated to identified levels of reflective learning: habituallearning, understanding, reflection and intensive reflection.This provides further empirical support for Kember et al.’s(2000) and Mezirow’s (1991) conceptualization of levels ofreflective learning. More important, there is clear evidencehere of two distinct forms of reflection: reflection and a moreintensive reflection that involves changes to underlyingassumptions about the self and business. This is significant asthere is some debate within the literature as to the nature ofthe highest level of reflection within Mezirow’s andKember’s frameworks (Reynolds 1998). It seems thatalthough the most intensive reflection identified here may notgo as far as the emancipatory ideals of critical theory, there isevidence of a type of reflection that involves a profound ques-tioning and challenging of deeply held beliefs and assump-tions that may in turn provide a catalyst for wider socialreflection (Reynolds and Vince 2004).

The findings here seem to support the dualistic dichoto-mies found within the wider learning literature that suggesttwo general forms of learning: lower and higher level learn-ing (Cope 2003). Both habitual action and understandingmay be seen to represent lower levels of learning and corre-late negatively with program outcomes. Reflection and inten-sive reflection are seen to represent higher levels of learningand correlate positively with program outcomes.

Two of the dimensions identified by the factor analysisrelated to conditions for reflection, namely, instructor-to-student interactions and student-to-student interactions. Thisaccentuates the importance of interaction generally in foster-ing enhanced learning experiences and provides evidencethat supports the espoused role of social exchanges to studentlearning (e.g., Dempsey, Halton, and Murphy 2001; Hay,Hodgkinson, et al. 2004; Hodgkinson and Brown 2003;Peltier, Drago, and Schibrowsky 2003; Thorpe 2001). Draw-ing on the items contained in the instructor-to-student inter-action dimension, the instructor in encouraging an improvedlearning experience may usefully encourage student ques-tions and comments, allow doubt and disagreement withinlearning, and create an atmosphere that allows the student tobe open about their views. It is important to create a learningenvironment that promotes divergence and complexity ofthought as is consistent with the arguments of Dehler, Welsh,and Lewis (2001), who proposed the value of promotion of“complicated understanding” in business education wherebythe student is encouraged to increase the variety of ways inwhich events can be understood. Furthermore, it seems that

JOURNAL OF MARKETING EDUCATION 259

TABLE 3REGRESSION MODEL FOR NONREFLECTION ↔

REFLECTION DIMENSIONS

Nonreflection ReflectionDimensions Standardized β t p

Intensive reflection .365 7.0 .001Reflection .359 6.9 .001Understanding –.351 –6.7 .001Habitual action –.211 –4.1 .001

NOTE: R 2 = .43, F = 39.45, model significant at p < .001.

TABLE 4FULL REFLECTION MODEL

Reflection Dimension Standardized β t p

Intensive reflection .364 8.1 .001Reflection .358 7.9 .001Understanding –.351 –7.8 .001Student interaction .283 6.3 .001Instructor interaction .259 5.7 .001Habitual action –.212 –4.7 .001

NOTE: R 2 = .58, F = 47.2, model significant at p < .001.

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

fellow students play an important role in stimulating diver-gent thinking by bringing to the classroom their range ofideas and ways of solving problems. Taken together, this maysuggest that a change in the relationship between classroomparticipants is necessary for enhanced learning where bothstudents and instructors play important roles, with neithertaking center stage. Accordingly, the notion of a decenteredclassroom where both student and instructor jointly constructknowledge has merit here (Dehler, Welsh, and Lewis 2001;Watson 2001).

More important, the study also offers a reliable measurefor reflective learning and thus makes a contribution to theexisting field as the paucity of instruments that measurereflective learning has been observed. Of note, we have high-lighted that in the case of marketing education, there havebeen calls from the AACSB (2004) for increased outcomemeasurements and assurance of learning reporting, and thusthe measures can be seen to, in part, respond to this demand.Given the importance of reflection to managerial practice(Schön 1983, 1987), it is hoped that programs that demon-strate the development of reflective learning in students canprovide some form of assurance of their contribution tolearning.

In the context of the wider reflective learning field, wehave added to the growing consideration of ways to oper-ationalize the construct (Kember et al. 2000; van Woerkom,Nijhof, and Nieuwenhuis 2002). Our scale draws on the workof Kember et al. (2000), providing further evidence of theusefulness of the four levels of reflective learning identifiedby these authors. We have expanded this scale to includeadditional items designed to tap into these four levels anditems that assess approaches to learning following Leung andKember (2003). The result of this development has been fourscales that demonstrate increased reliability. In addition, thescale developed also includes items that assess conditions forreflective learning that have not previously been operational-ized. Of importance, the results also show that the “contin-uum” nature of the reflection hierarchy makes it difficult tomake mutually exclusive and/or completely delineated hier-archal dimensions along the continuum. This is evident by thefact that even though the factor loadings showed that habit-ual and understanding dimensions were separate dimensions,like any continuum, there are some gray areas at theirintersection.

Our regression analysis that assessed each of the sixdimensions on MBA graduates’perceptions of program qual-ity and career enhancement reveals a number of importantfindings. All six dimensions were found to significantlyaffect the outcome variable. This provides evidence of thevalidity of the measures and indicates that each factor affectsthe perception of program quality and career enhancement indifferent ways. Of particular interest is the negative relation-ship found between habitual learning and understanding andperceived program outcomes. This explicitly suggests that

lower levels of learning are associated with diminished stu-dent ratings and thus highlights a concern for programs thatare seen to rely on routine and nonchallenging learning expe-riences. The analysis also established the relative importanceof each factor in explaining perceived program outcomes.Consistent with our nonreflection-reflection hierarchy, eachlevel becomes of increasing importance in explaining per-ceived program outcomes. Accordingly, intensive reflectionis most important in accounting for variance in program out-comes, followed by reflection, understanding, and habituallearning. This is important for a number of reasons. First, webelieve this to be the first empirical test of the reflective learn-ing hierarchy in predicting program outcomes. Second, thefinding that intensive reflection and reflection are the twomost important factors in explaining program outcomesempirically demonstrates that reflective learning is funda-mental to an enhanced student learning experience. More-over, as our outcome measure includes items assessingoverall benefit to the individual’s career, there is animplication that reflective learning is also positively related tothe workplace.

The regression analyses also revealed that our identifiedconditions for reflection, instructor-to-student interactionand student-to-student interaction, were also significant inexplaining program outcomes; however, these were less im-portant than understanding, reflection, and intensive reflec-tion. It is encouraging that the students’learning experience isof greater significance in explaining program outcomes sinceit is sometimes bemoaned that factors such as the instructorplay an overriding part in course evaluation. In practice, mar-keting faculty could redesign their curriculum and class activ-ities to match students’ level of critical and reflective thinkingso as to enhance their evaluative abilities (Chan et al. 2002).

The contribution of the role of the instructor and fellowstudents is still important, and suggestions have been madeabove as to how classroom interactions can strengthen learn-ing experiences.

LIMITATIONS AND FUTURE RESEARCH

The work presented here is subject to a number of limita-tions. First, the study is based on one MBA program at oneuniversity, and thus questions regarding generalizability arenaturally raised. Second, as we have already noted, our studyrepresents an individual perspective of reflective learning,and thus future work may seek to incorporate a social per-spective consistent with critical theory (Reynolds 1998).Third, our work only considers two important conditionsfor reflective learning, student-to-student interactions andinstructor-to-student interactions; there are obviously furtherconditions worthy of consideration such as, for instance,assessment and course design (Thorpe 2001).

Our study also raises a number of suggestions for areas offuture research. Further work may seek to examine the inter-

260 DECEMBER 2005

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

relationships between important dimensions of reflectivelearning; for example, does reflection influence intensivereflection? This can be investigated by the use of structuralequation modeling. Our work is based on graduates’ percep-tions of an overall program; there may be merit in conductingsimilar research at the course level to identify differencesrelating to specific courses, for example, a comparison ofcourses such as quantitative analysis and organizational the-ory. Furthermore, it may also be beneficial to conduct a longi-tudinal study that is able to follow the development of the stu-dent’s ability to reflect throughout the program. It may also beinteresting to assess international differences in reflectivelearning. For example, a comparison between the UnitedStates and the United Kingdom may be of interest sincescholars have recognized the increased consideration of criti-cal management education in the United Kingdom relative tothe United States (Grey 2004), and we have noted the poten-tial importance of reflection to this. Last, future researchshould also address whether reflective thinking differs bybusiness discipline and for graduate versus undergraduatestudents.

CONCLUSION

In conclusion, our work is seen to progress the consider-ation of reflective learning within the context of business edu-cation and provides a reliable and valid instrument for man-agement educators to assess reflective learning in theirstudents. Further empirical investigation of the reflectionconstruct will significantly enhance conceptual develop-ments within the field. This is seen to be of vital importance tomarketing education given the increasing value placed onhelping current and future managers to become morecritically reflective.

NOTE

1. As will be seen through the orthogonal factor analysis, the learningdimensions are distinct and uncorrelated. However, adjacent learning dimen-sions had factor loadings in the .2 to .25 range; significant but not meeting thestandards for inclusion in a dimension.

REFERENCES

American Assembly of Collegiate Schools of Business International. 2004.Assurance of learning requirements . http://www.aacsb.edu/resource_centers/assessment/standards.asp.

Argyris, C. 2002. Double-loop learning, teaching, andresearch. Academy ofManagement Learning & Education 1 (2): 206-18.

Argyris, C., and D. A. Schön. 1996. Organizational learning II: Theory,method and practice. Reading, MA: Addison-Wesley.

Atkins, S., and K. Murphy. 1993. Reflection: A review of the literature. Jour-nal of Advanced Nursing 18:1188-92.

Bailey, J. R., P. Saparito, K. Kressel, E. Christensen, and R. Hooijberg. 1997.A model for reflective pedagogy. Journal of Management Education 21(2): 155-67.

Bennis, W., and B. Nanus. 1997. Leaders: The strategies for taking charge.New York: Harper & Row.

Biggs, J. 1987. Student approaches to learning and studying. Melbourne:Australian Council for Educational Research.

Biggs, J., D. Kember, and D. Y. P. Leung. 2001. The revised two-factor studyprocess questionnaire: R-SPQ-2F. British Journal of Educational Psy-chology 71:1333-49.

Bloom, B. S. 1979. Taxonomy of educational objectives. Book 1: Cognitivedomain. London: Longman.

Boenink, A. D., A. K. Oderwald, P. De Jonge, W. Van Tilburgh, and J. A.Smal. 2004. Assessing student reflection in medical practice. The devel-opment of an observer rated instrument: Reliability, validity and initialexperiences. Medical Education 38:368-77.

Boud, D., R. Keogh, and D. Walker. 1985. Reflection: Turning experienceinto learning. London: Kegan Page.

———. 1996. Promoting reflection in learning: A model. In Knowing, learn-ing and instruction, edited by L. Resnick. London: Routledge.

Bourner, T. 2003. Assessing reflective learning. Education & Training 45 (4/5): 267-73.

Boyd, E. M., and A. W. Fales. 1983. Reflective learning: Key to learning fromexperience. Journal of Humanistic Psychology 23 (2): 99-117.

Braun, N. M. 2004. Critical thinking in the business curriculum. Journal ofEducation for Business 79 (4): 232-36.

Brown, L., and B. Z. Posner. 2001. Exploring the relationships betweenlearning and leadership. Leadership and Organisation DevelopmentJournal 22 (6): 274-80.

Catterall, M., P. Maclaran, and L. Stevens. 2002. Critical reflection in themarketing curriculum. Journal of Marketing Education 24 (December):184-92.

Chan, C. C., M. S. Tsui, M. Y. C. Chan, and J. H. Hong. 2002. Applying thestructure of the observed learning outcomes (SOLO) taxonomy on stu-dent’s learning outcomes: An empirical study. Assessment & Evaluationin Higher Education 27 (6): 511-27.

Cole, M. S., H. S. Feild, and S. G. Harris. 2004. Student learning motivationand psychological hardiness: Interactive effects on students’ reactions toa management class. Academy of Management Learning & Education 3(1): 64-85.

Cope, J. 2003. Entrepreneurial learning and critical reflection. ManagementLearning 34 (4): 429-50.

Corley, A., and E. Eades. 2004. Becoming critically reflective practitioners:Academics’ and students’ reflections on the issues involved. HumanResource Development International 7 (1): 137-44.

Dehler, G. E., A. Welsh, and M. W. Lewis. 2001. Critical pedagogy in the“new paradigm.” Management Learning 32 (4): 493-511.

Dempsey, M., C. Halton, and M. Murphy. 2001. Reflective learning in socialwork education: Scaffolding the process. Social Work Education 20 (6):631-41.

Densten, I. L., and J. Gray. 2001. Leadership development and reflection:What is the connection? International Journal of Educational Manage-ment 15 (3): 119-24.

Dewey, J. 1933. How we think: A restatement of the relation of reflectivethinking to the education process. Boston: D. C. Health.

Duke, C. R. 2002. Learning outcomes: Comparing student perceptions ofskill level and importance. Journal of Marketing Education 24 (3): 203-17.

Entwhistle, N., and P. Ramsden. 1983. Understanding student learning. Lon-don: Croon Helm.

Faranda, W. T., and I. Clarke III. 2004. Student observations of outstandingteaching: Implications for marketing educators. Journal of MarketingEducation 26 (3): 271-81.

Fisher, T., and J. Somerton. 2000. Reflection on action: The process of help-ing social work students develop their theory in practice. Social WorkEducation 19 (4): 387-401.

Friga, P. N., R. A. Bettis, and R. S. Sullivan. 2003. Changes in graduate man-agement education and new business school strategies for the 21st cen-tury. Academy of Management Learning and Education 2 (3): 233-48.

JOURNAL OF MARKETING EDUCATION 261

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

Graeff, H. 1997. Bringing reflective learning to the marketing researchcourse: A co-operative learning project using intergroup critique. Journalof Marketing Education 19 (spring): 53-64.

Gray, D. 2001. Work based learning, action learning and the virtual paradigm.Journal of Further and Higher Education 25 (3): 315-24.

Grey, C. 2004. Reinventing business schools: The contribution of criticalmanagement education. Academy of Management Learning and Educa-tion 3 (2): 178-86.

Harrison, M., C. Short, and C. Roberts. 2003. Reflecting on reflective learn-ing: The case of geography, earth and environmental sciences. Journal ofGeography in Higher Education 27 (2): 133-52.

Hay, A., J. W. Peltier, and W. A. Drago. 2004. Reflective learning and on-linemanagement education: A comparison of traditional and on-line MBAstudents. Strategic Change 13 (4): 169-82.

Hay, A., M. Hodgkinson, J. W. Peltier, and W. A. Drago. 2004. Interactionand virtual learning. Strategic Change 13 (4): 193-204.

Hodgkinson, M., and G. Brown. 2003. Enhancing the quality of education: Acase study and some emerging principles. Higher Education 45:337-52.

Kayes, D. C. 2002. Experimental learning and its critics: Preserving the roleof experience in management learning and education. Academy of Man-agement Learning and Education 1 (2): 137-49.

Kember, D., J. Biggs, and D. Leung. 2004. Examining the multidimension-ality of approaches to learning through the development of a revised ver-sion of the learning process questionnaire. British Journal of EducationalPsychology 74 (2): 261-78.

Kember, D., A. Jones, A. Loke, J. McKay, K. Sinclair, H. Tse, C. Webb, F.Wong, M. Wong, and E. Yeung. 1999. Determining the level of reflectivethinking from students’ written journals using a coding scheme based onthe work of Mezirow. International Journal of Lifelong Education 18 (1):18-30.

Kember, D., and D. Y. P. Leung. 2005. The influence of active learning expe-riences on the development of graduate capabilities. Studies in HigherEducation 30 (2): 155-60.

Kember, D., D. Y. P. Leung, A. Jones, A. Y. Loke, J. Mckay, K. Sinclair, H.Tse, et al. 2000. Development of a questionnaire to measure the level ofreflective thinking. Assessment and Evaluation in Higher Education, 25(4): 382-95.

Kobasa, S. C., and M. C. Puccetti. 1983. Personality and social resourcesin stress resistance. Journal of Personality and Social Psychology 45:839-50.

Kolb, D. A. 1984. Experiential learning as sources of learning and develop-ment. Englewood Cliffs, NJ: Prentice Hall.

Kouzes, J. M., and B. Z. Posner. 2002. The leadership challenge. Hoboken,NJ: Wiley.

Leung, D. Y. P., and D. Kember. 2003. The relationship between approachesto learning and reflection upon practice. Educational Psychology 23 (1):61-71.

Liimatainen, L., M. Poskiparta, P. Karhila, and A. Sjogren. 2001. The devel-opment of reflective learning in the contest of health counselling andhealth promotion during nurse education. Journal of Advanced Nursing34 (5): 648-58.

Loo, R. 2002. Journaling: A learning tool for project management trainingand team-building. Project Management Journal 33 (4): 61-66.

Maclellan, E. 2004. How effective is the academic essay? Studies in HigherEducation 29 (1): 75-89.

Marton, F., and R. Saljo. 1984. Approaches to learning. In The experience oflearning, edited by F. Marton, D. Hounsell, and N. Entwhistle. Edin-burgh: Scottish Academic Press.

Mezirow, J. 1977. Perspective transformation. Studies in Adult Education9:153-64.

———. 1981. A critical theory of adult learning and education. Adult Educa-tion 32:3-24.

———. 1991. Transformative dimensions of adult learning. San Francisco:Jossey-Bass.

———. 1998. On critical reflection. Adult Education Quarterly 48:185-98.Mintzberg, H., and J. R. Gosling. 2002. Reality programming for MBA’s.

Strategy and Business 26 (1): 28-32.———. 2004. The education of practicing managers. MIT Sloan Manage-

ment Review 45 (4): 19-22.Mukherjee, A. 2004. Promoting higher order thinking in MIS/CSI students

using class exercises. Journal of Information Systems Education 15 (2):171-79.

Nunnally, J., and I. Bernstein. 1994. Psychometric theory. 3rd ed. New York:McGraw-Hill.

Paswan, A. K., and J. A. Young. 2002. Student evaluation of instructor: Anomological investigation using structural equation modelling. Journalof Marketing Education 24 (3): 193-202.

Pee, B., T. Woodman, H. Fry, and E. S. Davenport. 2000. Practice-basedlearning: Views in the development of a reflective learning tool. MedicalEducation, 34: 754-761.

Peltier, J., W. Drago, and J. Schibrowsky. 2003. Virtual communities and theassessment of online marketing education. Journal of Marketing Educa-tion 25 (3): 260-76.

Pfeffer, J., and C. T. Fong. 2002. The end of business schools? Less successthan meets the eye. Academy of Management Learning and Education 1(1): 78-95.

Porter, L., and L. E. McKibbin. 1988. Management education and develop-ment: Drift or thrust in the 21st century. New York: McGraw-Hill.

Reynolds, M. 1998. Reflection and critical reflection in management learn-ing. Management Learning 29 (2): 182-200.

Reynolds, M., and R. Vince. 2004. Critical management education andaction-based learning: Synergies and contradictions. Academy of Man-agement Learning & Education 3 (4): 442-57.

Richards-Wilson, S. 2002. Changing the way MBA programs do business:Lead or languish. Journal of Education for Business 77 (5): 296-300.

Schön, D. A. 1983. The reflective practitioner. London: Temple Smith.———. 1987. Educating the reflective practitioner. San Francisco: Jossey-

Bass.Smart, D. T., C. A. Kelley, and J. S. Conant. 2003. Mastering the art of teach-

ing: Pursuing excellence in a new millennium. Journal of Marketing Edu-cation 25 (1): 71-78.

Smith, L. W., and D. C. Van Doren. 2004. The reality-based learning method:A simple method for keeping teaching activities relevant and effective.Journal of Marketing Education 26 (1): 66-74.

Sobral, D. T. 2000. An appraisal of medical students’ reflection-in-learning.Medical Education 34 (3): 182-87.

Stevens, G. E. 2000. The art of running a business school in the new millen-nium: A dean’s perspective. Society for Advancement of ManagementAdvanced Management Journal 65 (3): 21-24.

Taylor, E. 2000. Analyzing research on transformational learning. In Learn-ing as transformation: Critical perspectives on a theory, by J. Mezirowand Associates. San Francisco: Jossey-Bass.

Thorpe, M. 2001. Reflective learning and distance learning-made to mix bydesign and assessment. Information Services and Use 20:145-58.

Van Velzen, J. H. 2004. Assessing students’ self-reflective thinking in theclassroom: The self-reflective thinking questionnaire. PsychologicalReports 95 (December): 1175-86.

Van Woerkom, M. 2004. The concept of critical reflection and its implica-tions for human resource development. Advances in Developing HumanResources 6 (2): 178-92.

Van Woerkom, M., W. J. Nijhof, and L. F. M. Nieuwenhuis. 2002. Criticalreflective working behavior: A survey. Journal of European IndustrialTraining 28 (8): 375-83.

262 DECEMBER 2005

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from

Watson, T. J. 2001. Beyond managism: Negotiated narratives and criticalmanagement education in practice. British Journal of Management12:385-96.

Wilhelm, W. B. 2004. The relative influence of published teaching evalua-tions and other instructor attributes on course choice. Journal of Market-ing Education 26 (1): 17-30.

Willmott, H. 1997. Critical management learning. In Management learning:Integrating perspectives in theory and practice, edited by J. Burgoyneand M. Reynolds. London: Sage.

Young, M. R. 2005. The motivational effects of the classroom environment infacilitating self-regulated learning. Journal of Marketing Education 27(1): 25-40.

Young, M. R., B. R. Klemz, and J. W. Murphy. 2003. Enhancing learning out-comes: The effects of instructional technology, learning styles, instruc-tional methods, and student behavior. Journal of Marketing Education 25(2): 130-42.

JOURNAL OF MARKETING EDUCATION 263

at Umea University Library on August 24, 2014jmd.sagepub.comDownloaded from