concept mapping to reveal prior knowledge and conceptual change in a mock summit course

ABSTRACT

The com plex na ture of cli mate change sci ence poses spe -cial chal lenges for ed u ca tors. Learners come to the class -room with prior knowl edge on the topic, which serves asa foun da tion for fur ther knowl edge build ing, but canalso pose bar ri ers to con cep tual change. Learners haveex ist ing men tal mod els that may limit their per cep tionand pro cess ing of con flict ing in for ma tion and pre ventadop tion of sci en tific con cep tions. In struc tional strat e-gies that at tempt con cep tual change by sim ply pro vok-ing cog ni tive con flict have had lim ited suc cess due to theim por tance of epistemological be liefs and mo ti va tion tothe con cep tual change pro cess. The Mock En vi ron mentSum mit course uses role-playing, ar gu men ta tion anddis cus sion to heighten epistemological aware ness andmo ti va tion and thereby fa cil i tate con cep tual change. The pre/post-course con cept map eval u a tion of stu dents'knowl edge about the sci ence of global cli mate change re-ported here shows ev i dence of sig nif i cant learn ing andcon cep tual change. Our study also pro vides use ful in for-ma tion about gaps in knowl edge and the types of mis-con cep tions stu dents are likely to have about this topic.In sight gained from this as sess ment study can be used totai lor the cur ric u lum and en hance stu dent prog ress to-wards more sci en tific con cep tions of the prob lem.

CLIMATE CHANGE SCIENCE - THENATURE OF THE DISCIPLINE

Interdisciplinary Nature - Climate change scienceembodies an effort to understand an inherentlyinterdisciplinary problem that has interwoven humanand natural causes. Since global climate change hasconnections to society that are mediated by a complexrange of political, social, technological and economicfactors, the study of the problem in the context of each ofthese fields is equally relevant and important. Onlythrough consideration of the forces and processesoperating in not only the natural sphere, but also in eachof these social spheres will society be able to formulate areasonable path for future human-environmentinteractions.

Uncertainty and Ill-defined Problem Solving - Climatechange science necessitates the ability to deal withuncertainty on several levels-not only uncertainty aboutthe workings of the complex physical climate system, butalso uncertainty with respect to social and culturalprocesses that mediate human response to changeswithin the system. The interdisciplinary and complexnature of climate change science results in an abundanceof ill-defined problems, and finding solutions to suchproblems requires skills that go beyond the relativelyconstrained problems generally presented in science

textbooks (Schraw, Dunkle and Bendixen, 1995).Ill-defined or ill-structured problems are those that 1)begin with a lack of all information necessary to developa solution or even to precisely define the problem, 2)have no single right approach for solution, 3) haveproblem definitions that change as new information isgathered, and 4) have no identifiable 'correct' solution(Gallagher, et al., 1995).

Research suggests that problem-based learningapproaches that use ill-defined problems facilitatelearning and conceptual change and the ability totransfer that learning to other domains (Kitchener andKing, 1981; Kuhn, 1991). In a problem-based learningenvironment, instructors function as "metacognitivecoaches" (Barrows, 1988) rather than simply informationpresenters or discussion leaders. In their well-knownreport, Science for All Americans, Rutherford andAhlgren (1990) discuss the potentials of ill-definedproblem solving for enhancing not only subject matterlearning, but also the metacognitive skills that areintegral to scientific literacy: "Students should be givenproblems-at levels appropriate to their maturity-thatrequire them to decide what evidence is relevant and tooffer their own interpretations of what the evidencemeans. This puts a premium, just as science does, oncareful observations and thoughtful analysis. Studentsneed guidance, encouragement, and practice incollecting, sorting and analyzing evidence, and inbuilding arguments based on it." Instructionalapproaches that utilize ill-structured problem solvingnot only result in increased learning (Gallagher et al.,1995) and information retention (Boud and Feletti, 1991),but also encourage epistemological understanding of thediscipline (Wilkinson and Maxwell, 1991) and enhancemotivation (Carter, 1988; Tobias, 1990). In this paper wediscuss an instructional approach that makes use ofcollaborative problem-based learning to accrue thesebenefits and encourage conceptual change.

LEARNING AND CONCEPTUAL CHANGE

Factors that Determine Conceptual Change - Thebasis for many current conceptual change theories is anidea that learning is a process of integration that involvesboth individual and social processes and consists ofrevising or fitting new information into existing mentalmodels (Driver et al., 1994; Mayer, 2002). Not all learningis considered conceptual change; this term is generallyreserved for learning that results from deep processingof knowledge (Strike and Posner, 1992). Since conceptualchange involves modifications in core knowledge andbeliefs, it is generally not easily achieved (Hynd, 1998)and occurs as a continuous gradual process (Smith,diSessa and Roschelle, 1993). In spite of the fact thattheorists often differ with regards to the exact nature ofthe underlying cognitive processes, there is generalagreement about a group of cognitive and social factorsthat seem to be vital to this type of learning. Prior

Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge 355

Concept Mapping to Reveal Prior Knowledge and ConceptualChange in a Mock Summit Course on Global Climate Change

Stacy Rebich Department of Geography, Institute for Computational Earth Systems Science,University of California, Santa Barbara, CA 93106 [email protected]

Catherine Gautier Department of Geography, Institute for Computational Earth Systems Science,University of California, Santa Barbara, CA 93106 [email protected]

knowledge is generally considered of high importance asit is deemed to serve both as the foundation forintegration of new concepts and as a potential obstacle toconceptual change (Mason, 2002; Vosniadou, 2002; Chiand Roscoe, 2002). Metacogntive awareness isknowledge and self-awareness of one's own learning,and is considered a necessary but not sufficient conditionfor conceptual change (Carey, 1985; Limon, 2002).Epistemology, or knowledge and beliefs about thenature of the discipline and the nature of knowledge andlearning, is linked to both the learner's ability torecognize incongruous beliefs and to motivation tochange them (Leach and Lewis, 2002; Mason, 2002).Motivation as a factor was initially ignored in many 'coldcognition' models (Pintrich et al., 1993) but now plays arole in many models. Motivation is intrinsically linked togoals, and there is evidence that learners with masterygoals who focus on learning and understanding morereadily achieve conceptual change than those withperformance goals who focus on demonstrating ability(Linnenbrink and Pintrich, 1999). Awareness ofmeta-concepts or second-order concepts refers to havingappropriate conceptions of higher-order concepts suchas evidence, cause, explanation, time, space, changesource, fact, and description (Limon, 2002) that influencereasoning across the subject matter. If conceptual changecan only occur in the space where these factors intersect,it is easy to appreciate that achieving conceptual changeshould present quite a challenge. In the next section, wefocus on three of these factors prior knowledge,epistemology and motivation which are of particularinterest in our discussion of the Mock EnvironmentSummit course.

Focus on Prior Knowledge - Researchers in cognitivescience have consistently found that the knowledgelearners possess is a very strong determinant in whatinformation they attend to, how that information isperceived, what learners judge to be important orrelevant, and what they are able to understand andremember (Alexander, 1996). With this in mind, alearner's knowledge base can be thought of as a scaffoldfor all of his or her future learning. Results of theoverwhelming number of studies on prior knowledgeand conceptual change illustrate the inadequacy andinaccuracy of a conception of the learner as an 'emptyvessel' or 'blank slate' that needs to be 'filled' withknowledge. Awareness of the critical role of priorknowledge in the acquisition of new knowledgenaturally leads to attempts to elicit and evaluate alearner's relevant knowledge prior to instruction. Manystudies of science learning have utilized an approach thatinvolves identification of prior knowledge andapplication of specific instructional strategies intendedto build upon and/or modify or replace that knowledge.The study reported here is based on a similar framework;our primary purpose here, however, is limited to thediscussion of a strategy for eliciting and evaluating priorknowledge and suggested ways in which this evaluationcould be used for instructional enhancement.

MISCONCEPTIONS

While prior knowledge can be seen as the foundation forintegration of new concepts, it is also commonly viewedas an obstacle to conceptual change (Chi and Roscoe,2002; Mason, 2002; Vosniadou, 2002). In research onscience education, much of the attention given to prior

knowledge has been focused on identifying andeliminating misconceptions. (Some find the term"misconceptions" pejorative or reserve it for a specifictype of non-scientific conception; Guzzetti et al. (1993)discuss a number of alternate, and possibly moreappropriate, terms that have been proposed. However,for the sake of simplicity, "misconceptions" will sufficefor the purposes of our discussion.) Misconceptions arecommon features of learners' prior knowledgethroughout the sciences and have proven resistant toinstruction (Champagne, Gunstone and Klopfer, 1983;Limon, 2001). A significant portion of the misconceptionsresearch has been devoted to identifying causes for causethe persistence of misconceptions, and a variety offactors seems to be at play. The difficulty associated withovercoming an inadequate conception can be related towhether it is derived from interactions with the physicalenvironment, the social environment or a formalinstructional environment (Guzzetti et al., 1993). Priorknowledge may be characterized by varying levelsaffective entrenchment related to social values, ideologyand identity (Limon, 2002; Pintrich et al., 1993), andpresumably higher levels of affective entrenchmentwould correspond with greater difficulty in achievingconceptual change. Revision of misconceptions may alsoprove costly at the level of cognitive processing ifrevision of a particular mental model will requirerevision of a number of related models.

Focus on Epistemological Beliefs - Beliefs about thenature of knowledge, knowing and learning and aboutthe nature of science have a strong impact on a learner'scapacity for conceptual change (Gregoire, 2003; Leachand Lewis, 2002; Limon, 2001; Mason, 2002).Development of epistemological beliefs is generallyviewed as a progression through stages (King andKitchener, 1994). Kuhn (1999) classifies these stages asabsolutist and overly-objective (characterized by beliefsthat knowledge is absolute, certain, non-problematic,right or wrong, and does not require justification since itoriginates from observations of reality or the authorities),multiplist and overly-subjective (characterized by beliefsthat knowledge is ambiguous, idiosyncratic, and thinkseach individual has his or her own views and owntruths), evaluativist and having an appropriate balanceof objectivity and subjectivity (characterized by beliefsthat there are shared norms of inquiry and knowing, andsome positions are reasonably more justified andsustainable than others) (see also Schommer, 1993).Learners' epistemological beliefs are not likely to beequally developed or applicable across a variety ofsubject contexts (White, 2002), which implies a need for aspecific focus on beliefs about the nature of science.Specifically, science learning environments should focuson the development of understanding that "the objects ofscience are not the phenomena of nature but constructsthat are advanced by the scientific community tointerpret nature" (Driver et al., 1994).

Research has shown that learners who have moresophisticated epistemological beliefs are more likely toaccept evidence that conflicts with their prior knowledgeand achieve conceptual change (Mason, 2002; 2003).Schraw, Dunkle and Bendixen (1995) found that lessadvanced epistemological beliefs were associated withlower performance in ill-defined problem solving, butfound no correlation between epistemological beliefsand success at solving well-defined problems.Interpretation of and learning about controversial issues

356 Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365

seems to be particularly affected by levels ofepistemological belief development (Kardash andScholes, 1996; Sinatra et al., 2003) and only individualswith advanced epistemological understanding seem tobe able to contemplate, reason and judge evidence,arguments and alternative perspectives (Mason andBoscolo, 2004). In a reciprocal fashion, includingcontroversial issues in curricula promotesepistemological development (Schommer-Aikins andHutter, 2002).

Focus on Motivation - Pintrich, Marx and Boyle (1993)speculate on the reasons why "cold and isolatedcognition models" are not able to explain students'inability to activate prior knowledge, and assert that alearner's goals, intentions, purposes, expectations, andneeds must be taken into account. They propose thatmotivational beliefs (including goals, values,self-efficacy, and control beliefs) play a major role inconceptual change, and caution against assumption thatstudents will find it easy to 'act like scientists' whenmotivational issues for students and scientists are likelydifferent and students’ motivations are constrained bythe larger context of the educational system. Inexamining the goal dimension of motivation, learners areoften distinguished based on the possession of masterygoals (focus on learning and understanding) orperformance goals (focus on good grades, approval ofpeers, or viewing the classroom as a competition)(Linnenbrink and Pintrich, 2002). Pintrich (1999)proposes that adoption of a mastery goal orientation willfacilitate conceptual change, and cites evidence thatsupports this proposition. He goes on to suggest that thestructure of the learning environment plays a major rolein determining whether students will adopt performanceor mastery goals. Constructivist approaches to teachingand learning and problem-based learning environmentsin which learners are given challenging, meaningful andauthentic tasks, especially those that involve ill-definedproblems, promote mastery goals and influence learners'motivation and cognition (Carter, 1988; Gallagher et al.,1995; Pintrich, Marx and Boyle, 1993). This is especiallytrue when students are encouraged to engage indiscussions and develop scientific arguments as theycollaborate in their problem solving efforts (Nussbaumand Sinatra, 2003).

The Cognitive Conflict Approach to ConceptualChange - The most popular instructional approachintended to facilitate conceptual change focuses onidentifying and removing misconceptions and involvesthe presentation of anomalous data or information thatconflicts with inadequate prior knowledge. Thecognitive conflict approach generally involvesevaluating a learner's current knowledge, presentingcontradictory information, and then re-evaluating toidentify changes in the learner's conceptions. While thisapproach appears logical enough and there is evidencethat the stimulation of cognitive conflict is a necessarycondition for conceptual change, the results of numerousstudies suggest that this approach in and of itself is notsufficient. While a handful of studies report success withthis approach (e.g., Jensen and Finely, 1995), many morehave reported their unsuccessful attempts to supportconceptual change through the presentation ofconflicting information or anomalous data to createcognitive conflict (e.g., Champagne, Gunstone andKlopfer, 1985; Dreyfus, Jungwirth and Eliovitch, 1990;

Guzzetti et al., 1993). The observation that cognitiveconflict in the absence of knowledge-building activitydid not result in conceptual change (Chan, Burtis andBereiter, 1997) suggests that multiple methods should beused to encourage meaningful learning. Based on ananalysis of the difficulties of achieving conceptualchange through the cognitive conflict strategy, Limon(2001) proposes that presentation of anomalous datacould be augmented with cooperative and sharedlearning to promote collective discussion of ideas.

The Mock En vi ron ment Sum mit course dis cussedhere uti lizes the com ple men tary strat egy of co op er a tivelearn ing through dis cus sion and ar gu men ta tion as theba sis for en cour ag ing con cep tual change. Ar gu men ta-tion in volves "con struct ing a ra tio nale for a par tic u larout come, re fut ing op pos ing ar gu ments, and weigh ingcom pet ing con sid er ations" (Nussbaum and Si na tra,2003). As learn ing tools, dis cus sion and ar gu men ta tionhave proven to en hance mo ti va tion, en cour age de vel op-ment of more so phis ti cated epistemological be liefs,heighten aware ness of the so cial na ture of sci en tificknowl edge con struc tion, and fa cil i tate con cep tualchange (Ma son, 1998; Ma son and Santi, 1998; Mortimerand Machado, 2000; Soja and Huerta 2001).

MOCK ENVIRONMENT SUMMIT

Course Overview - The "Mock Environment Summit"(Geography 135) upper-division undergraduate coursehas been offered at The University of California SantaBarbara for the past several years as a means toencourage students to gain a deeper understanding ofthe scientific evidence of global climate change, considerproblem-solving approaches that arise from within avariety of disciplines, and utilize a variety of skills andknowledge of different topics to negotiate an'international agreement' that represents a collaborativeeffort to deal with the complex problem of climatechange. (see more detailed description in Gautier andRebich, 2005) The design of the Mock EnvironmentSummit curriculum is firmly rooted in constructivistpedagogy, which departs from the traditional paradigmof the student as an "empty vessel" and embraces thenotion that each individual comes into a class with aunique existing knowledge framework. The "MockEnvironment Summit" course has been designed tocreate a learning environment where individuals usespecially designed tools to build upon their ownknowledge in an effort to experience enrichedunderstanding and conceptual change. Integration ofnew knowledge with prior knowledge is accomplishedthrough research, role playing, and the presentation ofarguments to support the position appropriate to achosen role. As students apply newly-gained knowledgeto problem-solving tasks, meaningful learning isenhanced and the likelihood of the transfer of thisknowledge to other relevant problem-solving situationsincreases (Ausubel, 1963).

The "Mock Environment Summit" is aninterdisciplinary, student-directed, inquiry-based EarthScience course during which students attend severallectures on the science of global change and then engagein a series of role-playing activities for which they act aspolicymaker-representatives of different countries. Thecourse focuses on the drafting and negotiation of aninternational agreement similar to the Kyoto protocol. Intheir roles as policymakers, students conduct (primarily)web research to investigate the present and potential


physical, social and economic impacts that global changemay have on their countries. On a regular basis, thestudents present their research findings to the class andbegin to build the foundation for the position they plan totake in the global change debate. Each countryrepresentative also joins with representatives from othercountries to research a topic related to global climatechange (e.g., fresh water availability, technologytransfer, carbon trading, and population control) and asthey present information about these topics to the class,they lobby to have these issues included as elements ofthe final agreement. In addition to feedback andcoaching throughout the in-class discussion andargumentation, the instructor and teaching assistantprovide individualized written feedback to the studentsto help them improve their research and related writingand presentation assignments. Grades for the course arebased on participation in class discussions,presentations, writing assignments and contributions tothe final negotiated agreement.

Course Evaluation and Learning Assessment - Sincewe are aware of the role of prior knowledge in theconceptual change process, we also appreciate the valueof assessing learners' existing conceptions of topicscentral to a course before instruction begins. Informationabout learners' initial beliefs and conceptions can then beused to guide the design and refinement of the learningenvironment. With this transition from more traditionalteaching methods, there comes a set of challenges forassessing learning. When applied to constructivistlearning environments, the assessment methods thathave been considered appropriate under traditionalpedagogy seem increasingly inappropriate. In responseto this need for new evaluation tools, new methods thatinvolve portfolio and project-based assessment havebeen proposed and utilized successfully (Mintzes,Wandersee and Novak, 2001). The current assessmentmethods utilized in Geography 135 are of this sort, andstudents are evaluated on the basis of performance onpresentation, writing and negotiation activities. No testsare given as part of the course, and the final means forevaluation is the quality of the final agreementnegotiated by the class.

While these evaluation methods have proven veryuseful for evaluating individual student performance, itis relatively more difficult to extract from them detailedinformation about how well the course is achieving itsstated goals and intended learning outcomes in terms ofcontent knowledge. In particular, we have beeninterested in obtaining a measure of how much scientificknowledge is being gained during the course as thestudents listen to and participate in discussions of thevarious positions presented and the evidence thatsupports these positions. While the course is focused onachieving an interdisciplinary perspective on the climatechange issue rather than on acquisition of specificclimate science content, we are interested in evaluatingstudents' level of conceptual change for the physicalscience content. Throughout the history of the course,carefully designed multiple-choice tests have beenadministered prior to and immediately following thecourse. The insight into learning gained from thesemeasures, however, has seemed to be both incompleteand lacking in the complexity necessary for a morein-depth analysis.

CONCEPT MAPPING STUDY

Overview and Rationale - We recognized conceptmapping as a potentially useful strategy for assessing theconceptual change experienced by students enrolled inthe Mock Environment Summit class. Concept mappingpresents itself as a particularly adapted assessment toolfor this class since it allows an exploration of studentknowledge at a sufficient level of complexity, does notpresuppose that all students have mastered exactly thesame material, and has also been shown to create a moreequitable assessment situation for those who havedifficulty coping with test anxiety (Okebukola andJegede, 1989). The usefulness of concept mapping forassessment is partially due to its level of complexity,which distinguishes it from more conventionalevaluation techniques such as multiple-choice tests.Markham, Mintzes and Jones (1994) suggest that thesetraditional unidimensional assessment measuresrepresent a failure to recognize that much disciplinaryknowledge is based on an understanding ofrelationships among concepts. Other researchers havefound concept map-based evaluations to yield equallycomprehensive and accurate overviews of knowledge ascompared to well-planned structured personalinterviews (Edwards and Fraser, 1983) and assessmentthrough writing (Osmundson, et al., 1999). However,concept mapping allows for more efficient datacollection than interviews do, and presents an advantageover writing-based assessments in that it is inherentlynon-linear and facilitates self-monitoring. Students facedwith an essay-writing assessment task will oftencomplete it in a linear fashion, starting at the beginningand writing straight through. On the other hand,students constructing concept maps can easily put downconcepts and/or visual symbols and add details andconnections between concepts in any order, which offersa chance for metacognitive reflection. Concept maps maybe useful in revealing thought processes that generallyremain private to the learner (Cohen, 1987), and it hasbeen suggested that they may be more sensitive todevelopmental changes than traditional testing in whichquestions often focus on isolated ideas (Kinchin, Hayand Adams, 2000).

Before going into more detail about the specifics ofthe proposed research, a brief overview of the nature ofconcept maps and how they can be used for assessmentin the classroom may be useful. The basic element of aconcept map is a proposition (see Figure 1). Aproposition is a pair of concepts whose relationship isspecified in the form of a link. Concepts are thingsusually referred to by nouns or noun phrases, while linksare usually verbs. Concepts from a map on global climate


Figure 1. The basic element of a concept map is aproposition, consisting of two concepts connected bya link that shows the relationship between them.

change might include things like "aerosol emissions","industrial activities", "longwave radiation trapping",and "greenhouse effect." These individual conceptscould then be linked to form propositions such as"aerosol emissions [are generated by] industrialactivities," or "longwave radiation trapping [is associatedwith] greenhouse effect." Basically, a concept map is a(sometimes hierarchically) structured network ofpropositions.

Cognitive theories that emphasize the structure ofknowledge underlie instructional approaches andassessments that involve concept mapping. Anderson(1984) asserts that structure is the essence of knowledge,and the process of constructing a concept map focusesthe learner's attention on the structure of knowledge andthe importance of knowledge integration. Concept mapscan be used to elucidate a learner's knowledgerepresentation and organization of ideas - characteristicsof understanding are also related to a learner's ability toengage in higher-order thinking (Gobbo and Chi, 1989;Jonassen et al., 1997; Kinchin, 2000).

The objectives of the study reported here were to 1)elicit information about what students had learned in theclass at a level of knowledge complexity that had notbeen observable through the multiple-choice tests usedfor previous course assessments, and 2) to gain anunderstanding of both the types of learning occurringand the specific content knowledge gained by thestudents in the Mock Environmental Summit class. Thisenhanced evaluation of learning outcomes providesvaluable information that can be used to tailor and refinethe curriculum; at the same time, it helps to verify theusefulness and suitability of concept map assessment forevaluating the learning taking place in open-ended,student-directed science courses.

Re search De sign and Data Col lec tion - The par tic i-pants in the study were 17 un der grad u ates (ages 19 to 25;7 fe male, 10 male) who were en rolled in an up per di vi-sion un der grad u ate ge og ra phy course ti tled "Mock En vi-ron ment Sum mit" (Ge og ra phy 135). Ge og ra phy 135 wasof fered as an in ten sive course (4 class hours per day) andmet for a three-week pe riod dur ing the 2003 Sum merquar ter. The ma jor ity of the stu dents in the class wereGe og ra phy and En vi ron men tal Studies ma jors, but sev -eral were from the Eng lish, Phi los o phy and GlobalStudies de part ments. In this pa per we ex am ine a set ofpre- and post-course con cept maps on the topic of globalcli mate change that were con structed by the par tic i pants.

Prior to the con struc tion of the pre-course con ceptmap, stu dents were given a one-hour train ing ses sion oncon cept map con struc tion. Dur ing the train ing ses sion,the re searcher pre sented a brief in tro duc tion to con ceptmap ping, and pro vided some back ground in for ma tionon how con cept map ping can be use ful for learn ing, de -scribed the pur pose of the con cept maps the stu dentswould be con struct ing. A hands-on dem on stra tion/tu to-rial gave stu dents the op por tu nity to prac tice us ing theCmapTools™ soft ware (http://cmap.ihmc.us/) theywould be us ing to con struct their con cept maps. Stu dentswere also given time to browse a col lec tion of con ceptmaps on top ics un re lated to cli mate change, and thesemaps, as well as the oth ers used dur ing the train ing ses -sion, were avail able for brows ing through out the se riesof con cept map ping tasks. Fol low ing the pre sen ta tionand dem on stra tion, stu dents were asked to brain stormin groups on the topic of 'col lab o ra tive work.' (This topicwas cho sen be cause it also of fered a fo rum for dis cus sion

of the ben e fits and chal lenges of the group work that theywould be do ing through out the course.) When they hadfin ished brain storm ing, each stu dent con structed a con -cept map that re flected his or her group's thoughts aboutcol lab o ra tive work. As stu dents con structed maps, there searcher mon i tored their prog ress and pro vided feed -back on the con struc tion pro cess. Aside from this ini tialtrain ing ses sion, the stu dents re ceived no ad di tionaltrain ing or prac tice in con cept map con struc tion dur ingthe course.

The day following the training session, each studentconstructed a concept map on the topic of global climatechange. The following focus questions were given todirect their map construction:

• What is global climate change?• What is the evidence?• What are the causes?• What are the mechanisms?• What are the consequences?

Students were given 45 minutes to complete theconcept-mapping activity.

The concept-mapping activity completed on thesecond day of the course was repeated at the end of thecourse. Students were given equal amounts of time toconstruct their pre-course and post-course conceptmaps.

ANALYSIS AND RESULTS

The pre- and post-course student maps were analyzedfor the presence or absence of key concepts and forproposition accuracy and usefulness. In addition to thecalculation and evaluation of summary statistics, ouranalysis used a visualization method for examining thestructural nature of the concept maps, their levels ofinterconnectedness, and the content areas where mostlearning occurred. Although we were not interested in'grading' the concept maps, we based our evaluation on acombination of scoring methods that have proveneffective in other studies (Kinchin, 2000; McClure, Sonakand Suen, 1999; Nicoll, Francisco and Nakhleh, 2001;Novak and Gowin, 1984). The specifics of our evaluationprocedure are described below.

Before beginning the concept map evaluation, wereviewed the student maps for syntax, and propositionsthat were not in a suitable form for our analyses weremodified to a syntactically appropriate form. Thesemodifications were generally limited to ensuring thatnouns or noun phrases were in nodes and verbs were onlinking lines. In addition, in cases when it appeared that astudent had chained together several concepts intendingto have them to all refer to a particular concept, were-linked these concepts in a manner that was suitable forour analysis. Once the concept maps were in properform, we extracted all propositions (a feature offered bythe CmapTools™ software) and displayed them in athree-column format (concept-link-concept) for analysis.

We be gan our anal y sis by clas si fy ing con cepts fromthe stu dent maps into cat e go ries based on a con cept mapcre ated by the course in struc tor (sec ond au thor). This 'ex-pert' map may be viewed at http://www.icess.ucsb.edu/esrg/135_instrucor_map.html. Stu dent con ceptswere clas si fied by the first au thor as ei ther ex act uses ofthe con cepts used in the ex pert map, or as 'near' con ceptsthat were sim i lar to or ex am ples of the ex pert con cepts.Some stu dent con cepts did not fit well within the ex pert


con cept cat e go ries, and for these we cre ated ad di tionalcat e go ries. We eval u ated changes in the av er age num berof con cepts, use ful links and the ra tio of links to con ceptson stu dent's maps us ing paired Stu dent's t-tests. On av -er age, stu dents in cluded 45% (sig nif i cant at p=.01 levelfor all) more con cepts on their post-course maps than ontheir pre-course maps. While an in crease in the num berof con cepts alone is not con clu sive ev i dence of con cep-tual change, these re sults in di cate that at the very leastthe stu dents' knowl edge of global cli mate change wassig nif i cantly en riched.

Propositions were classified into four categories:useful propositions, examples (useful propositions thatsimply correspond to examples of members within acategory, e.g. "greenhouse gases include carbondioxide", "greenhouse gases include methane"), 'weak'propositions that indicate the student is likely to have anincomplete understanding of the relationship betweenthe concepts in question, and misconceptions. Theobserved increase in number of concepts reported abovewas accompanied by a 77% average increase in thenumber of useful links and a 22% average increase in theratio of links to concepts (see Figure 2). This indicatesthat the new knowledge gained by the students is onaverage characterized by a greater degree ofinterconnection, which is associated with the facilitationof knowledge retrieval and enhanced problem-solvingability. By the end of the course, students' incidence ofmisconceptions and weak conceptions had fallen from17% of the total propositions on their maps to 9%(difference significant at p=.01 level) (see Figure 3). This

significant decline in the appearance of misconceptionssuggests that some conceptual change had taken place.At the very least, a significant number of misconceptionshad been weakened to the point that students no longerstrongly associated them with global climate change andso did not include them on their concept maps.

DISCUSSION

Identification of Misconceptions - Identification ofcommon misconceptions that students were likely topossess when entering the Mock Environment Summitclass was one of the primary purposes of our conceptmap assessment. Our efforts to modify the coursecurriculum to facilitate meaningful learning will bebased in part on knowledge of these misconceptions, andmodifications to our instructional approach will dependon our interpretation of their likely causes.

Students' inappropriate mental models of shortwave and longwave radiative processes was evidenced by avariety of inadequate propositions. On some of thestudent maps, we found evidence of a flawed mentalmodel that attributes increased global temperatures toincreased solar input through the ozone hole. Somestudents with this mental model also understood thegreenhouse effect as the trapping of this extra (reflected)solar energy by greenhouse gases or clouds. Otherstudents thought it was the greenhouse gases themselvesbeing trapped. This misunderstanding of the greenhouseeffect may result in part from the direct analogy to a


Figure 2. This figure shows each student’s gains in number of concepts (light gray), number of propositions(dark gray bars), ratio of concepts to propositions (white bars) between the pre-course and post-courseconcept maps. Gains in concepts and links represent more elaborated knowledge; gains in the ratio ofpropositions to concepts represent increased interconnection of knowledge.

greenhouse maintaining heat by trapping warm airinside. In many cases, it seemed that longwave radiativeprocesses did not play any part in students' models of thegreenhouse effect, which indicates that they probably donot conceive of the earth (let alone greenhouse gases andaerosol particles) as radiating bodies.

There was also a great deal of confusion about whatgreenhouse gases are and the nature of their role inclimate and climate change. The term 'aerosol' was veryoften used to describe a type of greenhouse gas, whichwe considered to be evidence that many students weremaking colloquial use of the word aerosol (to mean CFC)and hadn't learned the scientific meaning of the term.Appearance of this misconception on several post-courseconcept maps indicates that some students were notlikely to have understood the discussions of aerosols thatoccurred throughout the course, and this continuingmisconception was somewhat surprising in light of thefact that the instructor and teaching assistant madeefforts on several occasions to point out the distinctionbetween the scientific and colloquial uses of the termaerosol.

Many students also seemed to use greenhouse gas,greenhouse gas emission and pollution indiscriminately. The lack of distinction between greenhouse gas andemission seemed to indicate that some students thoughtof greenhouse gases as "bad", while in reality they areessential to maintaining a habitable temperature onEarth. Students who thought of greenhouse gases assynonymous with pollution were likely to attribute alltypes of pollution-related health damages to greenhousegas emissions, associate ozone depletion withgreenhouse gas emissions in general, and think that alltypes of pollution enhance the greenhouse effect. Thesestudents demonstrated a lack of appreciation for the role

of aerosol pollution (both natural and anthropogenic) indetermining albedo and influencing the processes ofcloud seeding and precipitation.

In fact, several of the students seemed to considerlow albedo as something 'bad', most likely because oftheir knowledge of the urban heat island effect. Thislimited understanding of albedo mechanisms and theireffects on climate led them to associate all 'undesirable'land use changes with decreases in albedo, even thoughprocesses such as deforestation usually result inincreased albedo.

Gaps in Knowledge - In addition to information aboutstudents' misconceptions before and after the course, ourconcept map study allowed us to identify gaps instudents' knowledge of global climate change. Figure 4shows a small set of topics that were not present on thestudents' post-course concept maps, and these are areasthat we would like to address in future offerings of thecourse. Although most students were aware that globalclimate change is associated with sea level rise, nearly allof them attributed the rise to melting snow, glaciers andice caps and neglected to mention the effect of thermalexpansion. There was a general lack of appreciation forfeedbacks that occur within the climate system, andnearly no one mentioned the connection between thegreenhouse effect and the hydrological cycle via watervapor. In fact, water vapor was seldom mentioned as agreenhouse gas, and there was evidence that manystudents considered water vapor and clouds to be thesame thing, even though they are part of very differentclimate processes. Students also made very fewreferences to climate change in the long-term historicalcontext (for example, the effects of solar variability onclimate), and provided very little information about the


Figure 3. This figure illustrates the observed changes in the average quality of student’s propositions betweenthe pre-course and post-course concept maps. Frequencies of weak and misconceptions decreasedsignificantly, from 17% to 9%.


Figure 4. This figure shows the frequencies of concepts appearing on the students’ post-course maps that didnot appear on their pre-course maps. The concepts are arranged to reflect the locations of the concepts onthe instructor’s map, and they fit roughly into categories of: evidence, causes, mechanisms, predictedconsequences, mitigation & adaptation. Dark circles represent instances in which students used the conceptterm nearly exactly, while lighter circles represent examples of the concept (e.g. nitrogen oxide for GHGemission) or similar concepts. Concept names shown in bold type are those that appear on the instructor’smap. Concepts in regular type are those that appeared on student maps but not on the instructor map.Italicized concepts were on the instructor’s map but didn’t appear on any student maps.

mechanisms of global climate change. The role ofcomputational models as a source for data that underlieclimate change research was not mentioned by any of thestudents.

Primary Areas of Learning and Conceptual Change -Our concept mapping assessment strategy provided uswith evidence that students experienced significantlearning in the Mock Environment Summit class throughknowledge enrichment (increases in the number ofconcepts and useful links used) and conceptual change(decrease in weak and misconceptions). A closerexamination of the data gives some insight into thecontent areas where most of the learning was takingplace. Figure 4 shows the frequencies of conceptsappearing on the students' post-course maps that did notappear on their pre-course maps. To identify theseconcepts, each student's map pair was analyzed for newconcept appearances on the post-course map, and thenthese new concept appearances were aggregated acrossall students. This illustration shows large increases instudents' knowledge regarding both the causes of globalclimate change and mitigation and adaptationapproaches to the problem. There was also noticeablechange in the students understanding of the predictedconsequences of climate change, especially in the areas ofhuman health and agriculture. Our concept mapassessment also showed that students had achieved agreater level of interconnectedness and cohesion in theirknowledge about global climate change (Figure 2),which is related to greater ability and creativity inproblem solving processes.

Conceptual Change - On the level of conceptual changeinvolving prior misconceptions, we were also able toobserve significant progress. Figure 5 shows observeddecreases in the overall number of weak conceptions andmisconceptions, and also illustrates which concepts wereinvolved in these conceptions. Although the totalnumber of propositions on the post-course mapsincreased, the total numbers of weak and misconceptionsdecreased. In addition to illustrating this generallypositive trend, Figure 5 also shows how certain concepts(aerosol emissions, shortwave and longwave radiativeprocesses, changes in temperature, greenhouse effect,greenhouse gas emissions, trapping and pollution) areassociated with misconceptions that seem to have beenresistant to instruction. Future modifications to the MockEnvironment Summit curriculum will seek to addressthese misconceptions in multiple ways, takingadvantage of the unique opportunities for meaningfullearning provided by the role-playing, argumentationand discussion activities that already form the basis forthe learning environment.

This study did not did not focus on conceptualchange in areas outside of the physical science realm,which we believe make up a large portion of the learningoccurring throughout the class. In the future, we mayconduct further studies that seek to examine thedevelopments in knowledge regarding the economic,political and social aspects of global climate change. Weare also particularly interested in how epistemologicalbeliefs about the nature of science and the role of sciencein society affect learning about the topic. The MockEnvironment Summit course may prove a fruitfulground for exploring these questions.


Figure 5. These two diagrams show the incidence of weak conceptions and misconceptions on the pre-course(left) and post-course (right) concept maps. Circle sizes are proportional to the number of weak conceptionsor misconceptions that involve each concept; dark dotted lines represent misconceptions and light solidlines represent weak conceptions.

SUMMARY AND CONCLUSIONS

Our evaluation of student learning in the MockEnvironment Summit course through the use of pre- andpost-course concept mapping provided valuable insightinto the science learning taking place in the class.Concept mapping proved to be a valuable assessmenttool that allowed us to observe significant increases inthe breadth and interconnectedness of studentknowledge of climate change. This multi-dimensionaland open-ended approach to assessment also offeredinformation about student knowledge structures thatwas detailed enough to be suitable for identification of aset of commonly held misconceptions. This evaluation ofthe effectiveness of the Mock Summit course infacilitating conceptual change has provided a startingpoint for further development and improvement of thecourse, and the insight we gained may also be used toinform development of instructional materials aboutclimate change for a variety of audiences in both formaland informal educational contexts.

ACKNOWLEDGMENTS

This work was partially supported through theCurricular Assessment program of the University ofCalifornia Santa Barbara Office of InstructionalDevelopment. Special thanks to Joao Hespanha forassistance with data analysis and visualization, and toJulie Dillemuth, Dan Montello and anonymousreviewers for useful comments on a draft of thismanuscript.

REFERENCES

Alexander, P.A., 1996, The past, the present and future ofknowledge research: A reexamination of the role ofknowledge in learning and instruction, EducationalPsychologist, v. 31, p. 89-92.

Anderson, R.C., 1984, Some reflections on the acquisitionof knowledge, Educational Researcher, v. 13, p. 5-10.

Ausubel, D.P., 1963, The psychology of meaningfulverbal learning: an introduction to school learning,New York, Grune and Stratton.

Barrows, H.S., 1988, The tutorial process: IL, SouthernIllinois University School of Medicine.

Boud, D. and Feletti, G. (editors), 1991, The challenge ofproblem-based learning, New York, St. Martin'sPress.

Carey, S., 1985, Conceptual change in childhood,Cambridge, MA, MIT Press.

Carter, M., 1988, Problem solving reconsidered: apluralistic theory of problems, College Engligh, v. 50p. 551-565.

Champagne, A., Gunstone, R., and Klopfer, L., 1985,Effecting changes in cognitive structures amongphysics students, in West, L. and Pines, L.(editors), Cognitive structure and conceptualchange, Orlando, FL, Academic Press.

Chan, C., Burtis, J., and Bereiter, C., 1997, Knowledgebuilding as a mediator of conflict in conceptualchange, Cognition and Instruction, v. 15, p. 1-40.

Chi, M.T.H. and Roscoe, R.D, 2002, The processes andchallenges of conceptual change, in M. Limón and L.Mason (editors), Conceptual change reconsidered,issues in theory and practice (pp. 3-28), Dordrecht,NL, Kluwer Academic Publishers.

Cohen, D., 1987, The use of concept maps to representunique thought processes: toward more meaningfullearning, Journal of Curriculum and Supervision, v.2, p. 285-289.

Dreyfus, A., Jungwirth, E. , and Eliovitch, R., 1990,Applying the "cognitive conflict" strategy forconceptual change--some inplications, difficultiesand problems, Science Education, v. 74, p. 105-225.

Driver, R., Asoko, H., Leach, J., Mortimer, E., and Scott,P., 1994, Constructing scientific knowledge in theclassroom, Educational Researcher, v. 23, p. 5-12.

Edwards, J. and Fraser, K., 1983, Concept maps asreflectors of conceptual understanding, Research inScience Education, v. 13, p. 19-26.

Gallagher, S.A., Sher, B.T., Stepien, W.J., and Workman,D., 1995, Implementing problem-based learning inscience classrooms, School Science and Mathematics,v. 95, p. 136-140.

Gautier, C. and Rebich, S., 2005, The use of a MockEnvironment Summit to support learning aboutglobal climate change, Journal of GeoscienceEducation, v. 53, p. 5-16.

Gobbo, C. and Chi, M., 1989, How knowledge isstructured and used by expert and novice children,Cognitive Development, v. 1, p. 221-237.

Gregoire, M., 2003, Is it a challenge or a threat? Adual-process model of teachers' cognition andappraisal processes during conceptual change,Educational Psychology Review, v. 15, p. 147-179.

Guzzetti, B.J., Snyder, T.E., Glass, G.V., and Gamas, W.S.,1993, Promoting conceptual change in science: acomparative meta-analysis of instructionalinterventions from reading education and scienceeducation, Reading Research Quarterly, v. 28, p.116-159.

Hynd, C., 1998, Conceptual change in a high schoolphysics class: In B. Guzzetti and C. Hynd (editors),Perspectives on conceptual change: multiple ways to understand knowing and learning in a complexworld (p. 27-36), Mahwah, NJ, Lawrence EarlbaumAssociates.

Jensen, M.S. and Finely, F.N., 1995, Teaching evolutionusing historical arguments in a conceptual changestrategy, Science Education, v. 79, p. 147-166.

Jonassen, D.H., Reeves, T.; Hong, N., Harvey, D., andPeters, K., 1997, Concept mapping as cognitivelearning and assessment tools, Journal of InteractiveLearning Research, v. 8, p. 289-308.

Kardash, C.M. and Scholes, R.J., 1996, Effects ofpreexisting beliefs, epistemological beliefs, and needfor cognition on interpretation of controversialissues, Journal of Educational Psychology, v. 88, p.260-271.

Kinchin, I.M., 2000, Using concept maps to revealunderstanding: a two tier analysis, School ScienceReview, v. 8, p. 41-46

Kinchin, I.M., Hay, D.B., and Adams, A., 2000, How aqualitative approach to concept map analysis can beused to aid learning by illustrating patterns ofconceptual development, Educational Research,v. 42, p. 43-57.

King, P.M. and Kitchener, K.S., 1994, Developingreflective judgment: understanding and promotingintellectual growth and ciritical thinking inadolescents and adults (1st ed.), San Francisco:Jossey-Bass.

Kitchener, K.S. and King, P.A., 1981, Reflectivejudgment: concepts of justification and theirrelationship to age and education, Journal ofApplied Developmental Psychology, v. 2, p. 89-116.


Kuhn, D., 1991, The skills of argument, New York,Cambridge University Press.

Kuhn, D., 1999, A developmental model of criticalthinking, Educational Researcher, v.28, p. 16-26.

Leach, J. and Lewis, J., 2002, The role of students'epistemological knowledge in the process ofconceptual change in science, in M. Limón and L.Mason (editors), Conceptual change reconsidered,issues in theory and practice (pp. 201-216),Dordrecht, NL: Kluwer Academic Publishers.

Limón, M., 2001, On cognitive conflict as an instructionalstrategy for conceptual change: a critical appraisal,Learning and Instruction, v. 11, p, 357-380.

Limón, M., 2002, Conceptual change in history: In M.Limón and L. Mason (editors), Conceptual changereconsidered: issues in theory and practice (p.301-336), Dordrecht, NL, Kluwer AcademicPublishers.

Linnenbrink, E.A. and Pintrich, P.R., 2002, The role ofmotivational beliefs in conceptual change: In M.Limón and L. Mason (editors), Conceptual changereconsidered: issues in theory and practice (p.115-135), Dordrecht, NL, Kluwer AcademicPublishers.

Markham, K.M., Mintzes, J.J., and Jones, G.M., 1994, Theconcept map as a research and evaluation tool:further evidence of validity, Journal of Research inScience Teaching, v. 31, p. 91-101.

Mason, L., 1998, Sharing cognition to construct scientificknowledge in school context: the role of oral andwritten discourse, Instructional Science, v. 26, p.359-389.

Mason, L., 2002, Developing epistemological thinking tofoster conceputal changes in different domains: InM. Limón and L. Mason (editors), Conceptualchange reconsidered: issues in theory and practice(pp. 301-336), Dordrecht, NL, Kluwer AcademicPublishers.

Mason, L., 2003, Personal epistemologies and intentionalconceptual change: In G.M. Sinatra and P.R. Pintrich(editors), Intentional conceptual change (p. 199-236),Mahwah, NJ, Lawrence Erlbaum Associates.

Mason, L. and Boscolo, P., 2004, Role of epistemologicalunderstanding and interest in interpreting acontroversy and in topic-specific belief change,Contemporary Educational Psychology, v. 29, p.103-128.

Mason, L. and Santi, M., 1998, Discussing the greenhouseeffect: children's collaborative discourse reasoningand conceptual change, Environmental EducationResearch, v. 4, p. 67-85.

Mayer, R.E., 2002, Understanding conceptual change: acommentary: In M. Limón and L. Mason (editors),Conceptual change reconsidered: issues in theoryand practice (p. 101-114), Dordrecht, NL, KluwerAcademic Publishers

McClure, J.R., Sonak, B., and Suen, H.K., 1999, Conceptmap assessment of classroom learning: reliability,validity and logistical practicality, Journal ofResearch in Science Teaching, v. 36, p. 475-492.

Mortimer, E.F. and Machado, A.H., 2000, Anomalies andconflicts in classroom discourse, Science Education,v. 84, p. 429-444.

Nicoll, G., Francisco, J., and Nakhleh, M., 2001, Athree-tier system for assessing concept map links: amethodological study, International Journal ofScience Education, v. 23, p. 863-875.

Novak, J.D. and Gowin, D.B., 1984, Learning how tolearn: Cambridge, UK, Cambridge University Press.

Nussbaum, E.M. and Sinatra, G.M., 2003, Argument andconceptual engagement, Contemporary Educational Psychology, v. 28, p. 384-395.

Okebukola, P.A. and Jegede, O.J., 1989, Students' anxietytowards and perception of difficulty of somebiological concepts under the concept-mappingheuristic, Research in Science and TechnologicalEducation, v. 7, p. 85-92.

Osmundson, E., Chung, G.K.W.K., Herl, H.E., and Klein,D.C.D., 1999, Knowledge mapping in the classroom:a tool for examining the development of students'conceptual understandings, Report: CSE-TR-507,August 1999.

Pintrich, P.R., 1999, Motivational beliefs as resources forand constraints on conceptual change: In W.Schnotz, S. Vosniadou, and M. Carretero (editors),New perspectives on conceptual change (p,. 33-50),Amsterdam, Pergamon.

Pintrich, P.R., Marx, R.W., and Boyle, R.A., 1993, Beyondcold conceptual change: the role of motivationalbeliefs and classroom contextual factors in theprocess of conceptual change, Review of EducationalResearch, v. 63, p. 167-200.

Posner, G.J., Strike, K.A., Hewson, P.W., and Gertzon,W.A., 1982, Accommodation of a scientificconception: toward a theory of conceputal change,Science Education, v. 66m o, 211-227.

Rutherford, F.J. and Ahlgren, A., 1990, Science for allAmericans: New York, Oxford University Press.

Schommer, M., 1993, Comparisons of beliefs about thenature of knowledge and learning amongpostsecondary students, Research in HigherEducation, v. 34, p. 355-369.

Schommer-Aikins, M. and Hutter, R., 2002,Epistemological beliefs and thinking about everydaycontroversial issues, The Journal of Psychology, v.136, p. 5-20.

Schraw, G., Dunkle, M.E., and Bendixen, L.D., 1995,Cognitive processes in well-defined and ill-definedproblem solving, Applied Cognitive Psychology, v.9, p. 523-538.

Sinatra, G.M., Southerland, S.A., McConaughy, F., andDemastes, J., 2003, Intentions and beliefs in students'understanding and acceptance of biologicalevolution, Journal of Research in Science Teaching,v. 40, p. 510-528.

Smith, J., diSessa, A., and Roschelle, J., 1993,Misconceptions reconceived: a constructivistanalysis of knowledge in transition, Journal of theLearning Sciences, v. 3, p. 115-163.

Soja, C.M. and Huerta, D., 2001, Debating whetherdinosaurs should be "cloned" from ancient DNA topromote cooperative learning in an introductoryevolution course, Journal of Geoscience Education,v. 49, p. 150-157.

Strike, K.A. and Posner, G.J., 1992, A revisionist theory ofconceptual change: In Philosophy of science,cognitive psychology, and educational theory andpractice, R.A. Duschl and R.J. Hamilton (editors),Albany, State University of New York Press.

White, R.T., 2002, Content and conceptual change: acommentary, in M. Limón and L. Mason (editors),Conceptual change reconsidered: issues in theoryand practice (p. 301-336), Dordrecht, NL, KluwerAcademic Publishers.

Wilkinson, W.K. and Maxwell, S., 1991, The influence ofcollege students' epistemological style on selectedproblem-solving processes, Research in HigherEducation, v. 32, p. 333-350.


Membership Application or Renewal FormName: __________________________ Phone:

__________________________

Mailing Address: __________________________________

________________________________________________

Fax: ____________________

Email: ___________________

City: ______________________________ State: ________ Zip: _____________________

___College/University Professor @ ______________

___Precollege Teacher @ ________________________Other @ ________________________________

Checks, MasterCard, or VISA (US funds

only) are payable to:National Association ofGeoscience Teachers. Mail to: NAGT, POBox 5443, Bellingham, WA 98227-5443

Membership Rates (US funds)

Regular USA $35 _____Outside USA $47 _____

Student–USA $20 _____Student–outside USA $32 _____Retired NAGT member $30 _____

Library SubscriptionsRegular USA $55 _____

Outside USA $67 ____________New _______Renewal

q Checkq Credit card: MC/VISA (circle one) Number: ________________________Signature: __________________________ Exp. Date __________________________

The Journal and membership year runs from January to December. Subscriptions received afterJune 1 will begin receiving the Journal in January of the following year. Back issues are

available for $15 (foreign $18) each.

*To qualify for student rate, indicate and obtain verification from a NAGT member:

___Undergraduate ___Graduate ____________________________________Signature of NAGT member School

concept mapping to reveal prior knowledge and conceptual change in a mock summit course

Documents