Explorations of Metacognition Among Academically Talented Middle and High School Mathematics Students

by

Adena Elizabeth Young

A dissertation submitted in partial satisfaction of the requirements for the degree of

Doctor of Philosophy

in

Education

in the

Graduate Division

of the

University of California, Berkeley

Committee in charge: Frank C. Worrell, Ph.D., Chair

Alan H. Schoenfeld, Ph.D. Darlene D. Francis, Ph.D.

Spring 2010

Explorations of Metacognition Among Academically Talented Middle and High School Mathematics Students

© 2010

By Adena Elizabeth Young

1

Abstract

Explorations of Metacognition Among Academically Talented

Middle and High School Mathematics Students

by

Adena Elizabeth Young

Doctor of Philosophy in Education

University of California, Berkeley

Professor Frank C. Worrell, Chair

The purpose of this dissertation was to examine metacognition among academically talented middle and high school mathematics students from both educational psychology and mathematics education perspectives. A synthesis of the literatures and three studies employing quantitative, qualitative, and mixed methodologies were used to address three research questions: (a) What is metacognition, (b) What are the relationships between metacognition and academic achievement, and (c) How should educational psychologists measure metacognition? Literature review findings suggested four metacognition constructs: knowledge, regulation, beliefs, and awareness. Examples of students’ metacognition during mathematics problem solving with regard to each of these constructs were provided. Results of exploratory factor analysis indicated that scores on an existing metacognition questionnaire were structurally valid although they lacked concurrent and predictive validity. Metacognition as measured by the existing questionnaire was not significantly or meaningfully related to measures of academic achievement or problem solving metacognition. However, problem solving metacognition was related to both problem solving accuracy and students’ diagnostic test score and summer course grade. Findings from this study suggest that more research is needed in order to (a) create a coherent definition of metacognition that is both taxonomical and functional, (b) examine the complex relationships between metacognition and academic achievement, and (c) create a metacognition self-report questionnaire with sound psychometric properties.

i

Table of Contents List of Tables……………………………………………………………………………...….….. iii

List of Figures……………………………………………………………………………….…… iv

Acknowledgements………………………………………………………………….……………. v

1. Explorations of Metacognition Among Academically Talented Middle and High School

Mathematics Students……………………...………………………………………………… 1

Conceptualizations of Metacognition …………………………………..…………........... 2

Knowledge of cognition……………………………………………………........... 2 Regulation of cognition………………………………………………………........4 Beliefs about cognition……………………………………………………............ 8 Awareness of cognition………………………………………………………….. 14 Summary………………………………………………………………………… 15

Measuring Metacognition……………………………………………………………….. 15

Metacognition self-report questionnaires………………………………………...15 Problem solving interviews…………………………………………………........ 18 Limitations of current measurement techniques………………………….…....... 19

Metacognition and Academic Outcomes…………………………………………........... 19

Metacognition and academic achievement……………………………………… 19 Metacognition and problem solving outcomes………………………………….. 20 Summary………………………………………………………………………… 20

The Present Studies……………………………………………………………………… 20

2. Study 1……………………………………………………………………………...................21

Method……………………………………………………………………...…………… 21

Participants…………………………………………………………………......... 21 Measures………………………………………………………………………… 21 Procedure………………………………………………………………….…….. 22

Results…………………………………………………………………………………… 22

Preliminary analyses…………………………………………………………...... 22 Structural validity………………………………………………………………...22 Concurrent and predictive validity…………………………………………….....25

Discussion……………………………………………………………….…………......... 25

ii

3. Study 2………………………...………………………………………………………………27

Method………………………………………………………………..…………………. 27

Participants……………………………………………………………...……….. 27 Measures…………………….…………………………………………………... 27 Procedure………………………………………………………………...……… 28

Results………………………….…………………………………………………........... 29

Metacognition during problem solving………………………………………….. 29 Metacognition and solution accuracy…………………………………………… 42

Discussion……………………………………………………………………………….. 57

4. Study 3………………………………………………………………...………………………58

Method………………………………………………………………………..…………. 58

Participants and measures………………………….…………………………..... 58

Results……………………………………………………………………….……........... 58

Jr. MAI scores, problem solving metacognition, and solution accuracy………... 58 Problem solving metacognition and academic achievement…………………..... 60

Discussion…………………………………………………………………….…………. 60

5. General Discussion…………………………………………………………...…………….. 61

Defining Metacognition…………………………………………………………………. 61

Metacognition and Academic Achievement…………………………………………….. 61

Measuring Metacognition……………………………………………………………….. 61

Conclusion……………………………………………………………………………..... 62

References…………………………………………………………………………….………..... 63

Appendix A: Junior Metacognitive Awareness Inventory…………………………………....... 67

Appendix B: Problem Solving Interview Protocol……………………………………………... 68

iii

List of Tables Introduction Table 1. Theories of Metacognitive Knowledge…………………………….…………………. 4 Table 2. Theories and Frameworks of Metacognitive Regulation……………………………... 9 Table 3. Types of Metacognitive Beliefs……………………………………………............... 13 Table 4. Theories of Metacognitive Awareness………………………………………………. 16 Table 5. Metacognition constructs and subconstructs………………………………………... 17 Study 1 Table 6. Descriptive Statistics of Jr. MAI Subscale Scores and Academic Achievement

Variables……………………………………………………………………….......... 23 Table 7. Two-Factor Solution From Principal-Axis Extraction/Oblimin Rotation of Jr. MAI

Scores………………………………………………………………………………... 24 Table 8. Correlations of Jr. MAI Subscores with Achievement Variables (n = 183)………… 26 Study 3 Table 9. Descriptive Statistics of Metacognition and Academic Achievement Variables…… 59 Table 10. Mean Academic Achievement Scores by Problem Solving Metacognition………… 60

iv

List of Figures Introduction Figure 1. Metacognition constructs researched in the fields of educational psychology and

mathematics education………………………………………………………………. 3 Study 2 Figure 2. Train problem counting strategy.……………….………………………................. 28 Figure 3. Cailin’s initial representation………………………………………………………. 29 Figure 4. Cailin’s initial equation……………………………………….................................. 30 Figure 5. Cailin’s second picture representation………………………………………........... 31 Figure 6. Cailin’s final equation. ……………………………………….................................. 32 Figure 7. Brice’s picture representation. ………………………………………...................... 33 Figure 8. Brice’s DRT chart and equation. ……………………………………….................. 33 Figure 9. Amelia’s initial counting strategy. ………………………………………................ 36 Figure 10. Amelia’s initial counting strategy (cont.)………………………………………...... 36 Figure 11. Amelia’s revised counting strategy. ……………………………………….............. 37 Figure 12. Dylan’s counting strategy. ………………………………………............................ 38 Figure 13. Tori’s counting strategy. ………………………………………............................... 40 Figure 14. Diana’s algebraic strategy……………………………………….............................. 41 Figure 15. Problem solving trajectories influenced by metacognition………………………… 43 Figure 16. Trajectory 1……………………………………….................................................... 44 Figure 17. Trajectory 2……………………………………….................................................... 44 Figure 18. Trajectory 3……………………………………….................................................... 45 Figure 19. Jamal’s initial counting strategy………..………………………………………….. 46 Figure 20. Jamal’s second strategy………………………………………................................. 46 Figure 21. Jamal’s full counting strategy………………………………………........................ 47 Figure 22. Anthony’s initial algebraic strategy………………………………………............... 48 Figure 23. Anthony’s final algebraic strategy………………………………………................. 49 Figure 24. Trajectory 4……………………………………….................................................... 50 Figure 25. Trajectory 5……………………………………….................................................... 50 Figure 26. Trajectory 6……………………………………….................................................... 51 Figure 27. Trajectory 7……………………………………….................................................... 51 Figure 28. Trajectory 8……………………………………….................................................... 52 Figure 29. Jaylen’s complete algebraic strategy………………………………………............. 53 Figure 30. Fiona’s initial DRT chart………………………………………............................... 54 Figure 31. Fiona’s counting strategy………………………………………............................... 55 Figure 32. Fiona’s completed algebraic strategy………………………………………............ 55 Figure 33. Faith’s attempt to develop a plan………………………………………................... 57

v

Acknowledgements Many thanks to Crystal Simmons and Jeannette Coté for their assistance with data collection; to Jose Zaveleta for his audio-visual technology support; to Leo White for his assistance throughout the data collection and data analysis processes; to the Fall 2009/Spring 2010 School Psychology Research Groups and the Spring 2010 Functions Research Group for their feedback on data analysis and early written drafts; to Christine Coughlin for her enthusiasm and ever-helpful consultation; to Frank C. Worrell and Alan H. Schoenfeld for their wisdom, support, encouragement, and critical feedback; and to L. David Nefouse and Nina H. Gabelko for their encouragement and support.

1

Explorations of Metacognition Among Academically Talented Middle and High School Mathematics Students

Metacognition is a psychological construct that refers to people’s knowledge, regulation, beliefs, and awareness of their thinking, learning, and problem solving processes (Brown, 1987; Flavell, 1979). Over the past 30 years, metacognition has come to be recognized as a factor of primary importance in people’s learning and performance in a wide range of domains. For example the National Research Council identified metacognition as one of three components central to learning and teaching (Bransford, Brown, & Cocking, 2000).

Within the field of educational psychology, much work has been dedicated to creating taxonomies that define metacognition (e.g., Jacobs & Paris, 1987; Krathwohl, 2002), developing measurement tools based on these taxonomies to capture people’s levels of metacognition (e.g., Pintrich, Smith, Garcia, & McKeachie, 1993; Schraw & Dennison, 1994; Sperling, Howard, Miller, & Murphy, 2002), and using statistical analyses to study the relationships between metacognition and academic achievement (e.g., Pintrich, 2002; Zimmerman, 1990). These approaches are used to study metacognition across academic domains including mathematics. In contrast to the research on metacognition within the field of educational psychology, there is a separate line of research on metacognition within the field of mathematics education. Within the mathematics education research community, there have been a range of studies exploring the role of metacognition in performance, such as research on how metacognitive acts influence effective problem solving (e.g., Schoenfeld, 1985, 1987).

In this dissertation I address two overarching goals. The first goal is to examine the affordances and limitations of the educational psychology literature and research practices with regard to (a) conceptualizations of metacognition and its relationships to academic achievement and (b) metacognition measurement techniques. The second goal is to examine how theories and methodologies used within the field of mathematics education may enhance how educational psychologists conceptualize and measure metacognition, and how they study the relationships between metacognition and academic achievement.

To address these objectives, I begin by reviewing the existing literature within educational psychology and mathematics education regarding (a) conceptualizations of metacognition, (b) operationalizations of metacognition, and (c) metacognition and academic achievement. Specifically, I review the conceptualizations of four main metacognition constructs that emerge from educational psychology and mathematics education literatures (i.e., regulation of cognition, knowledge of cognition, beliefs about cognition, and awareness of cognition) and present a synthesis of these conceptualizations. Next, I review quantitative and qualitative research methods commonly used to examine metacognition within educational psychology and mathematics education, respectively. Finally, I briefly review existing empirical findings on the relationships between metacognition and learning outcomes such as academic achievement and mathematics problem solving outcomes.

Following this review of the literature, I present three studies. In Study 1, I examine the psychometric properties (i.e., item reliability, structural validity, concurrent and predictive validity with academic achievement) of scores on an existing metacognition self-report questionnaire. Findings from Study 1 are used to evaluate the affordances and limitations of the measurement techniques (i.e., using self-report questionnaires to study metacognition) commonly used in educational psychology to study the relationships between metacognition and academic achievement. In Study 2, I use methodologies commonly used in mathematics education (e.g., think-aloud interviews ) to examine students’ uses of metacognition during a

2

mathematics problem solving task. Findings from Study 2 are used to provide illustrative examples of the conceptualizations of metacognition synthesized in the review of the literature and to provide insight into the complexities of the relationships between metacognition and problem solving outcomes. In Study 3, I combine data from Studies 1 and 2 to examine the relationships between metacognition and academic achievement. Finally, I discuss how the findings from each study contribute to the two overarching goals of this dissertation. Conceptualizing Metacognition

Conceptualizations of metacognition have expanded over time within educational psychology and mathematics education. Despite the extensive theoretical and empirical literature on metacognition, there is no consensus definition of the construct. Metacognition was first defined by Flavell (1976) as “one’s knowledge concerning one’s own cognitive processes and products or anything related to them” (p. 232). Flavell (1976, 1979) described several aspects of metacognition, including metacognitive knowledge and metacognitive experiences, as well as the monitoring, regulation, and orchestration of cognitive processes. Soon after Flavell (1976) introduced the term, metacognition, Brown (1978) reviewed existing research on related phenomena, describing several aspects that later came to be recognized as aspects of metacognition, including planning, checking and monitoring, and knowing when and what you know.

Discussions of operational definitions of metacognition in the research literature followed next in the literature. In the educational psychology literature, Brown (1987) identified two distinct areas of metacognition research: knowledge of cognition and regulation of cognition. In the mathematics education literature, Schoenfeld (1987) distinguished between three areas of metacognition research: knowledge of one’s own thought processes, control or self-regulation, and beliefs and intuitions. Research in both fields of study have continued to expand based on these theoretical frameworks.

The early work of Flavell (1976, 1978) and Brown (1978) and the more recent work of Brown (1987) and Schoenfeld (1987) suggest four constructs of metacognition: knowledge of cognition, regulation of cognition, beliefs about cognition, and awareness of cognition (see Figure 1). The theoretical and empirical bases for each of these constructs in the educational psychology and mathematics education literature are reviewed in the following sections.

Knowledge of cognition. Two taxonomies of knowledge of cognition exist in the educational psychology literature on metacognition: Flavell’s (1979) categories of person, task, and strategy knowledge, and Jacobs and Paris’ (1987) categories of declarative, procedural, and conditional knowledge. Knowledge of cognition is not explicitly studied in mathematics education. Research about related phenomena in mathematics education is reviewed in the sections on beliefs about cognition and awareness of cognition.

Flavell’s theory of metacognitive knowledge. Flavell (1979) introduced metacognitive knowledge as “knowledge or beliefs about what factors or variables act and interact in what ways to affect the course and outcome of cognitive enterprises” (p. 907). Flavell presented these factors in three categories, arguing that a student’s learning and problem solving were influenced by metacognitive knowledge related to person, task, and strategy. Person knowledge refers to knowledge and beliefs about oneself and others as learners. This includes knowledge and beliefs about interindividual differences (e.g., Person A is better at math than Person B), beliefs about intraindividual differences (e.g., I learn math better from a teacher than a math book), and beliefs about universal cognitions (e.g., knowledge about how people learn math). Task knowledge refers to one’s metacognitive knowledge about the demands and goals of a cognitive task, as well

3

Flavell (1976, 1979)

Brown (1978)

Brown (1987)

Schoenfeld (1987)

Knowledge of

Cognition

Regulation of

Cognition

Beliefs about

Cognition

Awareness of

Cognition

as knowledge about what information is available during the task, how variations in the available information may affect the outcome of the task, and, therefore, how a cognitive enterprise should be managed to attain the goal given what information is available. Finally, strategy knowledge refers to the metacognitive knowledge of what strategies are most effective in a given learning or problem-solving situation to attain specific goals.

Figure 1. Metacognition constructs researched in the fields of educational psychology and mathematics education.

Jacobs and Paris’ theory of self-appraisal of cognition. Jacobs and Paris (1987) argued that one aspect of metacognition was self-appraisal, referring to one’s assessment of what one knows about a given domain or task (the other aspect was the self-management of thinking). According to Jacobs and Paris, self-appraisal of cognition, which was later termed by others (e.g., Schraw & Dennison, 1994; Sperling et al., 2002) as knowledge of cognition, consists of three categories of knowledge: declarative knowledge, procedural knowledge, and conditional knowledge. Declarative knowledge refers to knowledge of oneself as a learner and what factors influence one’s performance (e.g., knowing that one has difficulty subtracting negative numbers). Procedural knowledge refers to one’s knowledge about the execution of procedural skills (e.g., knowing how to solve a system of equations). Conditional knowledge refers to knowledge about when and why to apply certain cognitive actions or strategies (e.g., knowing when and why to use a pictorial representation).

Summary. Based on the work of Flavell (1979) and Jacobs and Paris (1987), there appear to be four types of metacognitive knowledge: person knowledge, task-specific knowledge, mathematical knowledge, and conditional knowledge (see Table 1). Metacognitive person knowledge refers to students’ knowledge of themselves as learners including knowledge of their strengths and weaknesses as mathematicians, as well as what factors (e.g., setting, mode of learning) influence their performance during a mathematical task. Task-specific metacognitive knowledge is knowledge that pertains to a specific task, and may vary between tasks. This

4

Table 1 Theories of Metacognitive Knowledge

Theoretical Frameworks

Types of Knowledge

Flavell (1979)

Jacobs and Paris (1987)

Knowledge about oneself and others as learners (Person Knowledge)

Person Knowledge Declarative Knowledge

Knowledge about the demands of a specific task (Task-Specific Knowledge)

Task Knowledge

Knowledge about procedural skills (Procedural Knowledge)

Procedural Knowledge

Knowledge about when and why certain strategies are effective to use (Conditional Knowledge)

Strategy Knowledge Conditional Knowledge

knowledge includes knowledge about the demands and goals of a task, as well as knowledge about what information and resources are available for the task (Flavell, 1979). Procedural knowledge refers to knowledge about the structure of mathematics knowledge and the execution of procedural skills (Schraw & Moshman, 1995). This knowledge, in contrast to task-specific knowledge, may be useful across mathematics tasks. Last, conditional knowledge pertains to knowing when and why to apply certain strategies on a task (Flavell, 1979; Schraw & Moshman, 1995).

Regulation of cognition. A second construct of metacognition, as described early on by Flavell (1976) and Brown (1978), is the regulation of cognition. Theories and research on students’ regulation of their learning processes are found in both educational psychology and mathematics education. The work in these two fields present similar models of the types of cognitive and metacognitive activities that students typically pursue during a learning or problem solving task, namely planning, monitoring, and evaluating. What differs most in these different bodies of literature is the specific behaviors each of these broad categories of activities encompasses. Additionally, it sometimes becomes difficult to differentiate metacognition from the cognitive and behavioral aspects of regulation. As metacognitive activities are often preceded, followed, or performed in conjunction with non-metacognitive but nonetheless important behaviors, my goal in this section is not to strictly review the metacognitive aspects of regulation, but to paint a larger picture of what the processes of metacognitive regulation entail with regards to mathematics learning and problem solving.

5

Regulation of cognition in educational psychology. Several theories of cognitive regulation are presented in this section. First, Brown’s (1987) theory of the components of regulation of cognition is discussed. Next, early work by Sternberg (1980) is presented to elaborate on some of the specific metacognitive behaviors that students may exhibit as they regulate their thinking. Finally Zimmerman’s (2002) and Pintrich’s (2004) theories of self-regulated learning are presented.

Brown’s theory of metacognitive regulation. Brown (1987) defined metacognitive regulation as “the activities used to regulate and oversee learning” (p. 68). Brown asserted that these activities consisted of three types: planning behaviors, monitoring behaviors, and checking outcomes. Brown provided a few examples of these behaviors (e.g., predicting outcomes as a planning behavior, revising learning strategies as a monitoring behavior), but did not provide a comprehensive definition to explain what these three types of activities fully entailed.

Sternberg’s componential subtheory of human intelligence. More detail about the specific learning behaviors that fall into the categories of planning and monitoring outlined by Brown (1987) is provided by Sternberg (1980), who outlined six metacomponents (i.e., higher-order control processes used for planning and decision making) that contribute to students’ problem solving abilities. According to Sternberg, planning behaviors include (a) understanding what the problem is that needs to be solved, (b) selecting lower-order components (e.g., skills involved in retrieving previously acquired information), (c) selecting representations or organizations for information (e.g., a spatial representation), and (d) selecting a strategy for combining the lower-order components selected. Monitoring behaviors include (a) deciding how much time to allot to each component of a task and how this allocation will affect the quality of the components, and (b) monitoring progress towards a solution (i.e., keeping track of what one has already done, what one is currently doing, and what one still has left to do). Sternberg did not identify any metacomponents related to checking outcomes. Instead, he conceptualized these metacomponents as being related only to students’ planning and monitoring behaviors. Theories of self-regulated learning. Similar to metacognitive regulation, self-regulated learning is defined as “the ways in which individuals regulate their own cognitive processes within an educational setting” (Puustinen & Pulkkinen, 2001, p. 269). Two models of self-regulated learning related to metacognition are presented by Zimmerman (2002) and Pintrich’s (2004).

Zimmerman (2002) defined self-regulated learning (SRL) as “the self-directive process by which learners transform their mental abilities into academic skills” (p. 65). In his model, SRL consists of a forethought phase, a performance phase, and a self-reflection phase, each consisting of two major classes. The forethought phase of Zimmerman’s model is comprised of task analysis processes (e.g., goal setting, strategic planning) and processes related to self-motivation beliefs (e.g., behaving a certain way as a result of self-efficacy beliefs). The performance phase consists of self-control processes (e.g., imagery, self-instruction) and self-observation (e.g., self-recording). Last, the self-reflection phase consists of self-judgment processes (e.g., self-evaluation) and self-reaction (e.g., comparing self to others). Zimmerman argued that self-regulatory processes are cyclical, and occur at different points within the learning process. According to Zimmerman, the forethought phase occurs prior to learning efforts, the performance phase occurs during behavioral implementation of learning, and the self-reflection phase occurs after each learning effort.

Pintrich’s (2004) model of SRL consists of four phases: (a) forethought, planning and activation, (b) monitoring, (c) control, and (d) reaction and reflection. Pintrich argued that these

6

phases, which are similar to the three phases of metacognitive regulation discussed by Brown (1987) and the three phases of self-regulated learning presented by Zimmerman (2002), represent a general time-ordered sequence (i.e., beginning with planning and ending with reflection); however, they may often occur simultaneously and dynamically. For example, students often exhibit control and monitoring behaviors concurrently. Unique to this model of SRL, Pintrich organized the behaviors that students regulate into four areas within each phase: cognition, motivation and affect, behavior, and context. For example, within the forethought, planning, and activation phase, students may regulate their learning behaviors by activating prior content knowledge (cognition), adopting a goal orientation (motivation and affect), planning how much effort they will exert (behavior), and analyzing the task (context).

Regulation of cognition in mathematics problem solving. The frameworks outlined by researchers in educational psychology are very similar to the processes of mathematics problem solving as described by Polya (1945), Schoenfeld (1981), and Garofalo and Lester (1985). The works of these mathematics researchers and educators contextualize regulation by studying how students regulate their thinking during problem solving situations.

Polya’s phases of mathematical problem solving. In his seminal book How to Solve It Polya (1945) outlined four phases to solving a mathematical problem, all of which are metacognitive in nature: understanding the problem, devising a plan, carrying out the plan, and looking back. In order to understand the problem, students must understand what the given condistions are, and what desired condition is. They may also question if it is possible to satisfy the desired condition. Consider the following mathematics problem: “Given a rectangle with a length of 3 cm and diagonal of 5 cm, find the width of the rectangle.” In the first phase of Polya’s problem solving framework, students need to understand that they are being asked to solve for the width of the rectangle (the unknown), that the rectangle has one set of sides measuring 3 cm and a diagonal of 5 cm (the data), and that the unknown is linked to the data because the unknown is the width of the triangle with a length of 3cm and a diagonal of 5 cm. Since the diagonal is larger than the length of the rectangle, it is possible to solve the problem. In the second phase of Polya’s (1945) framework of mathematical problem solving, devising a plan, students must determine which calculations, computations, and/or constructions they must perform in order to obtain the unknown. In the case of the rectangle with an unknown side, students may choose the Pythagorean theorem to solve for the unknown. In the third phase, carrying out the plan, students check each step of the plan to make sure that each step is correct. For example, in using the Pythagorean theorem to solve for the missing side of the rectangle, students must make sure that they set up the correct equation (i.e., 32 + x2 = 52), then check to make sure that they correctly go through the steps to solve for “x.” In the final phase, looking back, students must check their result and their argument. Students may choose to derive the result using a different method and then compare the results of the two plans, or may check to see if their result makes sense. In the case of the rectangle with the missing side, once the unknown is solved for, students may wish to plug their result back into the Pythagorean theorem and attempt to solve for one of the other sides, or may notice that their result is consistent with what they learned about special triangles, and recognize that “3-4-5” is one such triangle. Schoenfeld’s framework of episodes and executive decisions in mathematical problem solving. Schoenfeld (1981) presented a protocol for examining the metacognitive actions students use to solve mathematics problems. This protocol characterized seven types of episodes in which students engage as they solved a mathematics problem, beginning when students first read the problem statement and ending with the assessment of a solution. These phases are

7

reading, analysis, planning, implementation, exploration, verification, and transition. In the reading episode, students may note the conditions of the problem, state the goal of the problem, and assess their knowledge relative to the problem task. In other episodes, students attempt to fully understand the problem (analysis) and search for relevant information that may help them derive a plan (exploration). Students also create and implement a plan and assess the quality and implementation of the plan (planning/implementation), and may use metacognition between episodes to assess their current solution state and make decisions about pursuing new directions or approaches to solve the problem (transition). Once students finish the problem, they may review, test, and/or assess their solution (verification). Schoenfeld’s phases were used to analyze and describe people’s actual behavior in problem solving situations. These phases contrast Polya’s phases which were prescriptive (i.e., they prescribed what to do in order to solve problems) rather than descriptive.

Garofalo and Lester’s cognitive-metacognitive framework. Garofalo and Lester (1985) created a cognitive-metacognitive framework for studying students’ mathematical performance that combines the work of Polya (1945), Schoenfeld (1981), and Sternberg (1980). In this framework, students may engage in four types of activities consisting of orientation, organization, execution, and verification. In this framework, students use strategic behavior to assess and understand a problem (orientation), plan their learning behaviors and the actions they choose to take to solve the problem (organization), regulate their behavior to conform to their plans (execution), and evaluate the decisions they made as well as the outcomes of their executed plans (verification). These four categories are similar to those of Polya’s framework, but more broadly define the activities that occur in each phase.

Garofalo and Lester (1985) argued that the amount of metacognition used in each phase may differ depending on the type of mathematics problem. For example, a computational mathematics problem (e.g., 9876 - 5432) will require little orientation and organization for most students, as they may quickly recognize that the problem requires the application of a subtraction algorithm. For this problem, metacognitive decisions will mostly be applied during the execution and verification phases of problem solving. In contrast, given a word problem, students will most likely need to use more metacognitive strategies related to orientation and organization in addition to that related to execution and verification. Additionally, the selection and use of skills may differ depending on students’ mathematical knowledge and their knowledge and familiarity of different skills.

Summary. Researchers studying regulation processes have conceptualized metacognitive regulation in two ways. Some have created frameworks outlining distinct phases of activity (e.g., Brown, 1987; Garofalo & Lester, 1985; Pintrich, 2004; Polya, 1945; Zimmerman, 2002), and others have described individual behaviors without clustering them into categories (e.g., Schoenfeld, 1981; Sternberg, 1980). In general, the behaviors identified by these researchers fall into three categories: preparation activities, performance activities, and evaluation activities (see Table 2). Some researchers view preparation as a single group of activities such as planning (Brown, 1987) or forethought (Zimmerman, 2002), whereas others have identified groups of behaviors within the preparation phase, distinguishing orientation from organization (Garofalo & Lester, 1985), and understanding the problem from devising a plan (Polya, 1945).

Preparation activities, as described by researchers on metacognitive regulation fit into three subcategories: task analysis, planning, and knowledge and belief activation. Task analysis behaviors include analyzing the task at hand (Zimmerman, 2002), understanding the problem

8

(Polya, 1945; Sternberg, 1980), and analyzing the information given, the conditions of the problem, and one’s familiarity with the task (Garofalo & Lester, 1985). These metacognitive activities may take the form of reading and exploration (Schoenfeld, 1981). Planning activities include strategy selection (Brown, 1987; Zimmerman, 2002), time and effort planning (Pintrich, 2004), goal setting (Garofalo & Lester, 1985; Pintrich, 2004; Zimmerman, 2002), and making plans on how to achieve those goals (Garofalo & Lester, 1985). Last, knowledge and belief activation, which few researchers have identified in their conceptualizations of metacognitive regulation, refers to activation of metacognitive knowledge and content knowledge (Pintrich, 2004) as well as activation of metacognitive beliefs (Zimmerman, 2002).

In the performance phase, students engage in two types of metacognitive activities: monitoring and planning (Pintrich, 2004). Monitoring activities include self-recording and self-observation (Zimmerman, 2002), as well as monitoring one’s cognition, motivation, affect, time allocation, need for help, and changing task demands (Pintrich). Control activities (see Schoenfeld, 1985 for more detail) include selecting and adapting strategies (Pintrich), making decisions about speed-accuracy tradeoffs (Garofalo & Lester, 1985; Sternberg, 1980), self-instruction and attention focusing (Zimmerman), and controlling effort level (Pintrich). Monitoring and control often occur simultaneously, and control decisions may often be the result of monitoring. However, unlike some researchers who combine monitoring and control behaviors into a single category (e.g., Brown, 1987), in this framework, these two sets of behaviors are separated because it is important to identify them as distinct, although interrelated, behaviors.

Last, evaluation consists of three metacognitive activities: product evaluation, task evaluation, and self-evaluation. Product evaluation entails checking the outcomes of a mathematical task (Brown, 1987), making cognitive judgments about these outcomes (Pintrich, 2004), and verifying that the outcome is correct and meaningful (Schoenfeld, 1981). Task evaluation (Pintrich, 2004) includes evaluating the difficulty of the task, determining the causes of successes and errors within the task (Zimmerman, 2002), and evaluating orientation, organization, and execution of the task (Garofalo & Lester, 1985). Last, self evaluation includes comparing one’s performance against a standard (Zimmerman, 2002) and making self-efficacy judgments about one’s capabilities in mathematics (Bandura, 1997). Beliefs about cognition. Beliefs about cognition within educational psychology and mathematics education include epistemological beliefs, self-efficacy beliefs, and beliefs about mathematics learning and problem solving. Epistemological beliefs. Literature on epistemological beliefs includes research on students’ beliefs about knowledge and knowing, beliefs about learning, and beliefs about mathematics knowledge, knowing, and learning.

Beliefs about knowledge and knowing. Students’ beliefs about knowledge and knowing are often studied by those in the field of epistemology. Many epistemological theories have been proposed to explain how students view knowledge and knowing (Hofer & Pintrich, 1997). These theories pertain to students’ beliefs about the nature of knowledge, the nature of knowing, the nature of learning and instruction, and the nature of intelligence. With regard to the nature of knowledge, students may hold beliefs about the certainty of knowledge (e.g., knowledge is absolute, knowledge is tentative, knowledge is contextual) as well as the simplicity of knowledge (e.g., knowledge is simple or complex, knowledge is isolated or interrelated). With regard to the nature of knowing, students may hold beliefs about the source of knowledge (e.g., experts, self) or may believe that knowledge does not require a justification. With regard to the nature of

9

Table 2

Theories and Frameworks of Metacognitive Regulation

Regulation Activities

Researcher(s) Phases Preparation Performance Evaluation

Metacognitive Regulation

Sternberg (1980)

none

Understanding the problem Selecting components Selecting representations Selecting strategies

Speed-accuracy tradeoff Solution monitoring

Brown (1987)

Planning Monitoring Checking outcomes

Predicting outcomes Scheduling strategies Vicarious trial and error

Monitoring Testing Revising Re-scheduling learning

strategies

Checking outcomes

Self-Regulated Learning Zimmerman (2002)

Forethought Performance Self-reflection

Task analysis Goal setting Strategic planning

Imagery Self-instruction Attention focusing Task strategies Self-recording Self-experimentation

Self-evaluation Causal attribution

Pintrich (2004)

Forethought, planning, and activation

Monitoring Control Reaction and

reflection

Target goal setting Knowledge activation Time and effort planning Selecting cognitive strategies

Monitoring cognition Monitoring task and context

conditions Self-observation of behavior Adapting strategies Change/renegotiate task

Cognitive judgements

Evaluation of task

Mathematical Problem Solving Polya (1945)

Understanding the problem

Devising a plan Carrying out the

problem Looking back

Identify unknown Identify data Identify condition Connect data and unknown Create a plan

Carry out plan Check each step Prove correctness of steps

Check the result Check the argument

Schoenfeld (1981)

Reading Analysis Exploration Planning Implementation Verification Transition

Identify problem conditions Goal statement Knowledge assessment Understand the problem Search for relevant

information Create a plan

Monitor progress Assess plan quality and

implementation Assess current solution state Select new approach/direction

Review solution Test solution Evaluate solution

Garofalo & Lester (1985)

Orientation Organization Execution Verification

Comprehension strategies Analysis of information &

conditions Assessment of task familiarity Representation Difficulty assessment Goal identification Planning (global & local)

Performance of local actions Progress monitoring Trade-off decisions

Evaluation of orientation & organization

Evaluation of execution

10

learning and instruction, students may hold beliefs about the roles of the learner, peers, and instructor within an educational setting. Last, with regard to the nature of intelligence, students may believe that intelligence is an innate ability.

Beliefs about learning. In contrast to the large body of literature on epistemological theories, which predominantly focuses on beliefs about knowledge and knowing (e.g., Hofer & Pintrich, 1997; Schommer, 1994), a much smaller body of literature exists focusing specifically on beliefs about learning (Li, 2003, 2004, 2005). Li proposed that beliefs about learning include beliefs about the purposes of learning (e.g., what you gain from learning), beliefs about the learning process (e.g., what is required to learn something), beliefs about personal regard for learning (e.g., whether or not learning is important), beliefs related to affective experiences with learning (e.g., learning is fun), and beliefs related to social perceptions of learning (e.g., perceptions of those who learn well). Beliefs about mathematics through an epistemological lens. Some researchers have proposed examining students’ mathematical beliefs using an epistemological framework. Through the lens of this framework, mathematical beliefs include beliefs about the nature of mathematics knowledge (e.g., certainty of knowledge), justifications of mathematics knowledge, sources of mathematical knowledge, and acquisition of mathematics knowledge (Muis, 2004). In a review of studies on students’ epistemological beliefs about mathematics, Muis found that students generally believe that mathematics knowledge is unchanging, that mathematics knowledge is passively handed down to them from authority figures, and that various components of mathematics knowledge are unrelated. Although she attempted to use epistemology as a lens through which to analyze studies of students’ mathematical beliefs, Muis did not make connections between the beliefs she reviewed and the framework of epistemological beliefs.

Francisco (2005) examined themes of mathematical beliefs in five high school students. These themes included beliefs about the nature of knowledge and what it means to know, beliefs about the source of knowledge, and beliefs about the certainty of knowing, as well as students’ motivation to engage in learning and how students’ views vary with particular areas of mathematical activity. Students’ beliefs about the nature of knowledge and knowing (i.e., beliefs about what it means to know something) pertain to operational knowledge (e.g., knowing how to solve a problem), relational knowing (e.g., the ability to identify and articulate relationships), conceptual knowing (e.g., knowing the internal structure of a concept), personal knowing (e.g., knowing something in a personally meaningful way), and durability of knowing (e.g., knowledge should be lasting). In addition, these five students also believed that knowledge is evolving rather than static, that knowledge should be practical (e.g., knowing how to use knowledge), and that knowledge is acquired by discursive activity. With regard to beliefs about the certainty of knowledge, Francisco found that students believed that proving is an integral part of knowledge building. Last, students have different beliefs about the applicability of mathematics knowledge to other academic domains (e.g., English, history, science). Some believe that mathematics can be used in other areas, whereas others believe that mathematics knowledge is not useful in learning other subjects.

Self-efficacy beliefs. Self-efficacy was first introduced by Bandura (1977) and subsequently defined as “beliefs about one’s capabilities to organize and execute the courses of action required to produce given attainments” (Bandura, 1997, p. 3). Self-efficacy expectations are influenced by four sources of information: performance accomplishments, vicarious experience, verbal persuasion, and emotional arousal (Bandura, 1977), and according to

11

Bandura’s (1989) social cognitive theory, these judgments and expectations have a strong influence over people’s behaviors (Pajares, 1996). Self-efficacy beliefs may influence people’s choices and the courses of action they choose to pursue (Pajares, 1996) and can be both self-aiding and self-hindering (Bandura, 1989). Self-efficacy affects task effort, task perseverance, and resilience following adverse situations (Pajares, 1996). People’s self-efficacy also influences how high they sets their goals for a task and how committed they stays to achieving those goals (Bandura, 1989).

Researchers have studied self-efficacy beliefs that are specific to mathematics. Mathematics self-efficacy has been interpreted in many different ways in the literature. Taken literally, mathematics self-efficacy can be seen as people’s judgments of their capabilities in mathematics. However, in measuring mathematics self-efficacy, researchers have narrowed this definition to focus on more specific mathematics abilities. Randhawa, Beamer, and Lundberg (1993) distinguished between generalized mathematics self-efficacy, which deals with perceptions of competence in the subject of mathematics in general, and specific mathematics self-efficacy, which deals with perceptions of confidence in specific problems within a math subject such as percentages or simultaneous equations.

Even more specifically, Hackett and Betz (1989) defined mathematics self-efficacy as “a situational or problem-specific assessment of an individual’s confidence in her or his ability to successfully perform or accomplish a particular [mathematics] task or problem” (p. 262). Hence, there are varying levels at which mathematics self-efficacy can be examined. A researcher may choose to study (a) self-efficacy of mathematics as a domain, such as confidence in achieving certain outcomes in mathematics (Pietsch, Walker, & Chapman, 2003); (b) topic-specific mathematics self-efficacy such as confidence in achieving high marks on percentages tests (Pietsch et al., 2003); or most specifically (c), task-specific mathematics self-efficacy such as confidence in correctly completing specific math problems (e.g., Betz & Hackett, 1983; Chen, 2003; Hackett & Betz, 1989; Pajares & Graham, 1999; Pajares & Kranzler, 1995; Pajares & Miller, 1994; Randhawa et al., 1993).

Beliefs about mathematics in the context of learning and problem solving. Schoenfeld (1985) defined mathematical beliefs in the following way:

Belief systems are one’s mathematical world view, the perspective with which one approaches mathematics and mathematical tasks. One’s beliefs about mathematics can determine how one chooses to approach a problem, which techniques will be used or avoided, how long and how hard one will work on it, and so on. Beliefs establish the context within which resources, heuristics, and control operate. (p. 45)

According to Schoenfeld, beliefs are an important aspect of metacognition because these thoughts directly influence one’s performance on mathematics learning and problem solving tasks. Both Schoenfeld (1988, 1992) and Lampert (1990) have studied mathematical beliefs that they found to directly affect students’ mathematical behaviors. Schoenfeld found that such beliefs included the beliefs that mathematics taught in school has little to do with the real world, that only geniuses are capable of really understanding math, and that mathematics problems have only one correct answer. Lampert found that students often believe that doing mathematics entails memorizing and following rules, formulas, and facts. Additionally, she found that students believed it was not acceptable to have an opposing answer to a mathematics problem (i.e., an answer that differed from the teacher or the rest of the class). Neither Schoenfeld nor

12

Lampert used a framework for categorizing students’ mathematical beliefs. Instead, they examined the beliefs that they believed to most strongly shape students’ behavior. A categorization of mathematics-related beliefs. De Corte, Op 't Eynde, and Verschaffel (2002) developed a framework of mathematics-related beliefs that combined the work of other researchers in the field (e.g., Francisco, 2005; Lampert, 1990; Schoenfeld, 1988, 1989). De Corte et al. distinguished between three categories of beliefs: beliefs about mathematics education, beliefs about the self in relation to mathematics, and beliefs about the social context of mathematical learning and problem solving.

Beliefs about mathematics education include beliefs about mathematics (e.g., math has nothing to do with the real world), beliefs about mathematics learning and problem solving (e.g., learning math is about memorizing rules and formulas), and beliefs about mathematics teaching (e.g., a good teacher is good at explaining). Beliefs about the self in relation to mathematics include goal orientation (e.g., my goal is to understand the content), task value beliefs (e.g., it is important for me to learn this material), control beliefs (e.g., if I take notes, I will learn better), and self-efficacy beliefs (e.g., I am confident in my ability to understand this material). Finally, beliefs about the social context of mathematics learning include students’ beliefs about the roles of students and teachers in the classroom, beliefs about how a mathematics problem should be solved, and beliefs about what it means to be a good mathematics student.

Summary. The research on metacognitive beliefs is vast and spans many separate yet interrelated bodies of literature. Researchers have documented many individual beliefs held by students with regard to learning and mathematics in their empirical work, but much of that work lacks a guiding framework or organization of beliefs (e.g., De Corte et al., 2002; Francisco, 2005). From this review of the literature, there appears to be three types of metacognitive beliefs related to mathematics: beliefs about mathematics, beliefs about mathematics learning and problem solving, and beliefs about the self and others in relation to mathematics (see Table 3). Beliefs about mathematics refer to students’ beliefs pertaining to the subject of mathematics itself. Students hold beliefs about the relevance of mathematics. For example, students may believe that the mathematics learned in school has nothing to do with the real world, or that the processes of formal mathematics have little or nothing to do with discovery or invention (Schoenfeld, 1988, 1992). They may also have beliefs about the certainty of mathematics. For example, they may believe that mathematics problems have one and only one right answer (Schoenfeld, 1992), that the answer to an assigned problem is only correct when it is approved by a reliable authority (Lampert, 1990), or that math knowledge is unchanging (Muis, 2004). Students may also believe that components of math knowledge (e.g., algebra and geometry) are unrelated (Muis). Beliefs about mathematics learning and problem solving refer to beliefs about the learning and problem solving processes that students take on when doing mathematics. Some students hold very narrow beliefs about the processes required to do mathematics. For example, some students believe that doing math means to memorize mathematics and apply what they have learned mechanically and without understanding (Schoenfeld, 1992), or that doing math corresponds with rules, formulas, and facts (Lampert, 1990). Other beliefs of this nature include the beliefs that students who understand mathematics can solve an assigned problem quickly (Muis, 2004; Schoenfeld, 1988, 1992), and that there is only one correct way to solve any math problem, which is usually the rule the teacher has most recently demonstrated in class (Schoenfeld, 1992). Many students believe that one succeeds in school by performing the tasks

13

Table 3 Types of Metacognitive Beliefs

Types of Beliefs

Researcher(s)

Beliefs about Mathematics

Beliefs about Mathematics

Learning and Problem Solving

Beliefs about Oneself in Relation to Mathematics

Epistemology

Hofer and Pintrich (1997) Nature of knowledge Nature of knowing

Nature of learning and

instruction

Muis (2004) Nature of math knowledge

Justifications of math knowledge

Sources of math knowledge

Acquisition of Math

Knowledge

Francisco (2005) Nature of knowledge

Source of knowledge

Certainty of knowledge

Self-Efficacy

Bandura (1997) Self-efficacy beliefs

Randhawa et al. (1993) Mathematics self-efficacy

Topic-specific self efficacy

Hackett and Betz (1989) Task-specific self-efficacy

Pietsch et al. (2003) Mathematics self-efficacy

Topic-specific self-efficacy

Task-specific self-efficacy

Mathematics Education

Schoenfeld (1988, 1992) Beliefs about mathematics

Lampert (1990) Beliefs about mathematics

Beliefs about doing mathematics

14

as described by the teacher (Schoenfeld, 1988), and that students are not capable of constructing mathematical knowledge or solving problems on their own (Muis, 2004). Last, students hold beliefs about the environment in which learning and problem solving should occur. For example, some students believe that mathematics is a solitary activity that is done by individuals in isolation (Schoenfeld, 1992), implicitly and privately (Lampert, 1990).

Finally, students hold beliefs about themselves and others in relation to mathematics. These include beliefs about who is capable of doing math, for example, that only geniuses are capable of discovering, creating, or really understanding mathematics (Schoenfeld, 1988), that ordinary students cannot expect to understand mathematics (Schoenfeld, 1992), that those who are capable of doing math are born with a math gene and have innate ability (Muis, 2004). These beliefs also include self-efficacy beliefs, that is, beliefs about one’s own capabilities in mathematics (e.g., I am not good at proofs, I am capable of solving systems of equations, Hackett & Betz, 1989; Randhawa et al., 1993).

Awareness of cognition. The least studied metacognitive construct is what Brown (1978) described as knowing when and what you know and what Flavell (1979) defined as metacognitive experiences. In mathematics education, Schoenfeld (1987) referred to this construct as reflections on one’s own thinking, and Wilson and Clarke (2004) later described it as awareness of cognition. Although this phenomenon has been described in a handful of articles on metacognition, there is no agreement about how it is distinct from other metacognitive constructs, or even if it is a distinct construct. The limited literature on phenomena related to one’s awareness of their thinking is reviewed here.

Awareness of cognition in educational psychology. In their early work on metacognition, Brown (1978) and Flavell (1979) both described phenomena related to people’s awareness of their thinking. Unlike regulation and knowledge of cognition, the idea of awareness of cognition was not taken up by researchers in educational psychology in subsequent work on metacognition. However, this phenomenon is nonetheless relevant to an overall conceptualization of metacognition as it relates to mathematics learning and problem solving.

Knowing when and what you know. Brown (1978) argued that “a very basic form of self-awareness involved in all memory and problem-solving tasks is the realization that there is a problem of knowing what you know and what you do not know” (p. 82). Brown reviewed literature on several concepts related to knowing when and what one knows including (a) metacomprehension, or knowing that one has or has not understood something, (b) knowledge inference, or the ability for one to estimate the state of their own knowledge, (c) confidence in one’s response to a problem, and (d) awareness of task difficulty.

Metacognitive experiences. Flavell (1979) defined metacognitive experiences as a key aspect of metacognition. According to Flavell, metacognitive experiences are “any conscious cognitive or affective experiences that accompany and pertain to any intellectual enterprise” (p. 906). Such thoughts and feelings about one’s own thinking include feeling like you do not know or understand something, or feeling like you are confused. Metacognitive experiences can influence students’ regulation of cognition (e.g., a student who feels confused may go back to the planning phase and re-analyze the task) as well as their knowledge of cognition (e.g., a student who realizes he doesn’t know how to use a strategy to solve a problem may revise his strategy knowledge so that he does not select this strategy in the future).

Awareness of cognition in mathematics problem solving. In mathematics education, Schoenfeld (1987) named knowledge about one’s own thought processes as one of three categories of metacognition (control and beliefs being the other two). However, he argued that

15

research on students’ awareness and assessment of their thinking had fewer direct implications for mathematics educators than the research on control and beliefs. Subsequently, most metacognition research in mathematics education has focused on self-regulation and beliefs. One exception is Wilson and Clarke (2004) who examined students’ awareness of their thinking in relation to solving mathematics tasks.

Wilson and Clarke’s definition of metacognitive awareness. Similar to Flavell (1979), Wilson and Clarke (2004) argued that a key component of metacognition is metacognitive awareness. Wilson and Clarke argued that awareness is a distinct component of metacognition involving one’s awareness of their thinking. Specifically, metacognitive awareness refers to a students’ awareness of where they are in the learning or problem solving process, as well as knowledge the mental processes that are in progress as he or she is learning or solving a math problem.

Summary. Brown (1978), Flavell (1979), and Wilson and Clarke (2004) highlight a potentially valuable aspect of metacognition that has been overlooked in the majority of studies on metacognition (see Table 4). According to these authors, awareness of cognition includes both cognitive and affective components. It should be noted that metacognitive awareness is a term used in this paper to describe a unique aspect of metacognition. This perspective is in contrast with other researchers (e.g., Cheng, 1993; Schraw & Graham, 1997) who use the term, metacognitive awareness, as a synonym for the general term of metacognition (i.e., knowledge and regulation of one’s cognitive processes).

Summary. Metacognition is a psychological construct studied in the fields of educational psychology and mathematics education. Based on a review of the literatures in both fields, there appear to be four metacognition constructs related to mathematics learning and problem solving: metacognitive knowledge, metacognitive regulation, metacognitive beliefs, and metacognitive awareness. Each of these constructs has been defined in different ways by different researchers. A synthesis of the types of thinking entailed within each construct is presented in Table 5. In this dissertation, metacognition will be used as a broad term that encompasses these four constructs. Further research is needed to contextualize these types of thinking within actual learning and problem solving situations. Further research is also needed to examine the relationships between the four metacognition constructs. Measuring Metacognition It can be seen from the many conceptualizations of metacognition that there is not a single definition of metacognition. Similarly, there are multiple ways used to measure metacognition. When researchers design empirical studies of metacognition, they must not only decide which theory of metacognition from which to draw, but they must also decide how to operationalize that theory. Existing measurement techniques include self-report questionnaires, student interviews, teacher ratings, observations of students’ overt behaviors, and examinations of student work (Boekaerts & Corno, 2005; Desoete & Roeyers, 2006). Of these measurement techniques, two address students’ internal (i.e., non-observable) processes: self-report questionnaires and student interviews. These two techniques are described below.

Metacognition self-report questionnaires. Most scales measuring metacognition have been developed for adult populations (e.g., Pintrich et al., 1993; Schraw & Dennison, 1994) or measure only one aspect of metacognition (e.g., metacognitive regulation; Pintrich & De Groot, 1990). Currently, the Junior Metacognitive Awareness Inventory (Jr. MAI; Sperling et al., 2002) is the only scale developed to measure both metacognitive knowledge and metacognitive regulation among school-aged children and adolescents.

16

Table 4 Theories of Metacognitive Awareness

Theories of Metacognitive Awareness

Awareness of…

Flavell (1979)

Brown (1979)

Wilson and Clarke

(2004)

What one knows and doesn’t know

What one does and does not understand

The task difficulty for oneself

One’s affective state

Where one is in the learning or problem

solving process

One’s mental processes in progress

17

Table 5 Metacognition Constructs and Subconstructs Metacognitive Knowledge

Metacognitive Regulation

Person knowledge

Task-specific knowledge

Mathematics knowledge

Conditional knowledge

Preparation

Task analysis

Planning

Knowledge activation

Performance

Monitoring

Control

Evaluation

Product evaluation

Task evaluation

Self evaluation

Metacognitive Beliefs Metacognitive Awareness

Beliefs about mathematics

Beliefs about mathematics learning and

problem solving

Beliefs about oneself and others in relation to

mathematics

Awareness of what one knows

Awareness of what one understands

Awareness of personal task difficulty

Awareness of one’s affective state

Awareness of place in the learning process

Awareness of mental processes

18

Jr. MAI. Sperling et al. (2002) developed the 18-item Jr. MAI to measure both metacognitive knowledge and metacognitive regulation among adolescents. The metacognitive knowledge subscale of the Jr. MAI reflects the view of metacognitive knowledge as declarative knowledge, procedural knowledge, and conditional knowledge (Jacobs & Paris, 1987; Schraw & Dennison, 1994). The metacognitive regulation subscale reflects five regulatory behaviors: planning, information management, monitoring, debugging, and evaluation.

Structural validity of Jr. MAI scores. Sperling et al. (2002) used two studies to examine the psychometric properties of Jr. MAI scores in two samples of sixth through ninth graders. In both studies, exploratory factor analysis (principal component extraction with orthogonal varimax rotation) was used to determine the factor structure of the Jr. MAI. The authors used .35 as the minimum coefficient value to determine item salience.

In their first study, Sperling et al. (2002) presented a two-factor and a five-factor solution. In the two-factor solution, Factor 1 consisted of four knowledge items and all nine regulation items and Factor 2 consisted of three knowledge items and two regulation items (two of which were cross-loaded on Factor 1). In the five-factor solution, four of the five factors consisted of a combination of knowledge and regulation items, and the fifth factor was a non-viable duplet. Nine items had salient cross-loadings on multiple factors. In their second study, Sperling et al. also presented two-factor and five-factor solutions. In the two-factor solution, Factor 1 consisted of seven regulation items and two knowledge items. Factor 2 consisted of two regulation items and eight knowledge items (two of which were cross-loaded on Factor 1). In the five-factor solution, one factor consisted of six regulation items, three of the four factors consisted of a combination of knowledge and regulation items, and one factor was a non-viable duplet. Five items had salient cross-loadings on multiple factors. Sperling et al. concluded that the results of the second study supported the theoretical structure of the Jr. MAI. However, the results of their factor analysis did not yield clean metacognitive knowledge and metacognitive regulation factors.

Concurrent validity of the Jr. MAI scores with academic achievement. Sperling et al. (2002) used bivariate correlations to examine the relationships between Jr. MAI scores and academic achievement. They found that total Jr. MAI scores were not significantly correlated with problem solving (r = -.08) or reading comprehension (r = -.00). The authors did not examine the correlations between the individual Jr. MAI subscales (i.e., Factor 1 and Factor 2) and academic achievement. These results suggest that the Jr. MAI lacks concurrent validity with academic achievement.

Problem solving interviews. Problem solving interviews are another method for assessing internal cognitive processes such as metacognition. One type of problem solving interview is a concurrent report interview during which students think aloud while concurrently solving math problems. Another type of problem solving interview is a retrospective report interview during which students reflect on past learning or problem solving experiences. Examples of concurrent report (i.e., think-aloud) and retrospective report (i.e., stimulated-recall) interviews are described below.

Think-aloud interviews. A qualitative method for assessing students’ metacognition is using think-aloud interviews (e.g., Hammouri, 2003; Lawson & Rice, 1987; Montague & Applegate, 1993; Santos-Trigo, 1996). During think-aloud interviews, subjects are given problems to solve and asked to report their thinking aloud as they solve each problem. The purpose of the interview is not for the subjects to analyze or to explain their thinking, but merely to provide a monologue of their thoughts as they occur. Ericsson and Simon (1993) found that

19

verbalizing information is a way of gathering reliable data about subjects’ cognitive processes, and that these reports are generally valid as long as subjects are not being asked to verbalize information they would not otherwise attend to. Think-aloud interviews have been used by researchers to study problem solving behaviors among a variety of subject populations including elementary school children (Swanson, 1990), middle school students (Montague & Applegate, 1993), and college undergraduates (Hammouri, 2003).

Stimulated-recall interviews. Stimulated-recall interviews were first used by Bloom (1953, cited in O'Brien, 1993). During stimulated-recall interviews, students are shown video footage of themselves engaged in a learning or problem solving task and asked specific questions about what they were doing, thinking, and/or feeling at specific moments. Through these interviews, researchers can gather information about students’ cognitions and affects that may not have been apparent through observation alone. According to Edward-Leis (2006), reliability and validity of stimulated-recall data can be maximized by (a) adhering to strict protocols such as the use of non-directive questioning, (b) administering the interview soon after the recorded episode, and (c) allowing both the interviewer and the interviewee to pause the video to maximize the opportunities for the interviewee to identify and explain his or her internal thoughts. Edward-Leis argued that it is important for the interviewer to take a neutral stance during questioning. In doing so, the interviewer is less likely to lead the interviewee to make up responses that do not reflect what he or she was actually thinking or doing.

Limitations of current measurement techniques. Self-report questionnaires and problem solving interviews are two methodologically different ways to gather information about metacognition. Each has affordances and limitations. Metacognition self-report questionnaires provide information about students’ perceptions of their metacognitive behaviors. Due to the relative ease of administration, these questionnaires can be used to collect data from large samples, can be scored easily, and yield data that can be used for statistical analyses. However, they often lack acceptable psychometric properties (Desoete & Roeyers, 2006). Additionally, subjects’ beliefs about their behaviors as reported on questionnaires often do not match their actual behaviors, suggesting that questionnaire data may not always accurately reflect the subjects’ actual practices (Schoenfeld, 2002). Although metacognition questionnaires are most often tied to theoretical frameworks (e.g., Jacobs & Paris, 1987), they often lack concurrent validity with classroom behaviors (e.g., Sperling et al., 2002).

Problem solving interviews tend to elicit more information about students’ thinking than self-report questionnaires due to their qualitative nature. They are also directly connected with actual problem solving tasks and require less speculation on the part of the interviewee. However, administration and interpretation of problem solving interviews take substantially more time than self-report questionnaires, making it difficult to study large samples of students in a given study. More research is needed to determine the concurrent validity of metacognition self-report questionnaires with problem solving interviews. Metacognition and Academic Outcomes Metacognition and academic achievement. Substantial research has demonstrated the importance of metacognition on academic achievement in the field of educational psychology. For example, students with high metacognitive knowledge are (a) able to adjust their own cognition and thinking to be more adaptive when solving problems, (b) more capable of transferring their knowledge of strategies to new learning situations, and (c) learn and perform better in the classroom than those who have little or no knowledge of cognition (Pintrich, 2002). Similarly, students who regulate their own learning and problem solving processes demonstrate

20

superior academic functioning (e.g., placement into advanced level courses, high mathematics achievement test scores; Zimmerman, 1990), superior performance on classroom tasks and assignments (Pintrich & De Groot, 1990), and generally higher levels of academic achievement (Gaskill & Hoy, 2002). Most educational psychology studies of metacognition and academic achievement have used statistical analyses to establish the positive relationships. Metacognition and problem solving outcomes. Research in the field of mathematics education has shown that students’ regulation of and beliefs about cognition influence both their problem solving behaviors and problem solving outcomes (Schoenfeld, 1985, 1992). For example, Schoenfeld (1985) examined students’ problem solving on a challenging mathematics task and found that the absence of control contributed directly to students’ failure. Schoenfeld (1985) also found that students’ and experts’ belief systems influenced how they approached and attempted to solve math problems. Summary. Educational psychologists and mathematics educators have reported similar findings regarding the big picture of metacognition: Metacognition is related to academic achievement. Educational psychologists have used questionnaires to measure students’ general metacognitive knowledge and metacognitive regulation and shown statistically significant relationships to general outcomes such as classroom learning and achievement test performance. In contrast, mathematics educators have examined students’ metacognition during specific learning and problem solving situations and shown how this metacognition influences the outcomes of these situations. The Present Studies

The two overarching goals of this dissertation were (a) to examine the affordances and limitations of theory and research on metacognition within the field of educational psychology, and (b) to examine how theories and methodologies used within the field of mathematics education may enhance how educational psychologists conceptualize and measure metacognition and study the relationships between metacognition and academic achievement. In the preceding sections, these goals were addressed through a review of the existing educational psychology and mathematics education literatures. These goals were further addressed in the following three studies.

In Study 1, I examined the psychometric properties of students’ scores on Sperling et al.’s (2002) Jr. MAI. The specific research goals of Study 1 were to examine students’ Jr. MAI scores with regard to (a) reliability, (b) structural validity, and (c) concurrent and predictive validity with academic achievement measures. In Study 2, I examined students’ metacognition as they worked through a mathematics problem solving task. The specific research goals of Study 2 were (a) to describe students’ thinking during mathematics problem solving tasks and (b) to examine the ways in which metacognition influenced students’ problem solving solution accuracy. In Study 3, data from Studies 1 and 2 were combined to examine the relationships between metacognition and academic achievement. The specific research goals of Study 3 were (a) to examine the predictive validity of students’ Jr. MAI scores with their problem solving metacognition and problem solving accuracy and (b) to examine the relationship between students’ problem solving metacognition and academic achievement measures.

21

Study 1 In this study, the Jr. MAI (Sperling et al., 2002) was used to examine metacognition among a sample of middle and high school students attending a summer program for academically talented adolescents. The goals of this study were to examine students’ Jr. MAI scores with regard to (a) reliability, (b) structural validity, and (c) concurrent validity with grade point average (GPA), mathematics grade, and mathematics diagnostic test (MDT) score, and (d) predictive validity with summer course grade. Method Participants. Participants were 183 adolescents who completed a math course at a summer program for academically talented middle and high school students in 2009. Adolescents attending this summer program were admitted based on multiple indicators of academic talent (e.g., teacher recommendation, GPA; Klein, 1991). Participant demographic data were obtained from the program database. The sample was 54% female and ranged in age from 11 to 17 (M = 13.29, SD = 1.31). Participants represented a range of ethnic groups: Asian American (n = 132; 72%), European American (n = 14; 8%), African American (n = 4; 2%), Latino (n = 10; 6%), Multiethnic (n = 9; 5%), and other (n = 12, 7%), with two participants declining to state their ethnicities. Participants’ family incomes varied with higher incomes being overrepresented: Less than $30,000 (n = 14; 8%), $30,000 - $60,000 (n = 12; 7%), $60,000 - $100,000 (n = 37, 20%), more than $100,000 (n = 112; 61%). Eight participants (4%) declined to state their family income. The participants’ mean grade point average was 3.89 (SD = .19). Measures. In this study, metacognition was measured using the Jr. MAI (Sperling et al., 2002). Academic achievement measures included GPA, mathematics grade, MDT score, and summer course grade.

Jr. MAI. The 18-item Jr. MAI (Sperling et al., 2002) was used to measure participants’ metacognition (see Introduction and Appendix A for more details). The Jr. MAI was designed to measure aspects of metacognitive knowledge (i.e., declarative, procedural, and conditional knowledge) and metacognitive regulation (i.e., planning, information management, monitoring, debugging, and evaluation) as defined by Jacobs and Paris (1987). Following this theory of metacognition, the Jr. MAI consists of two subscales: metacognitive knowledge (Knowledge) and metacognitive regulation (Regulation). Each subscale consists of nine Likert-scale items with response options ranging from 1 (Never) to 5 (Always). For this study, the questionnaire instructions were modified to focus on metacognition during mathematics problem solving as opposed to studying in general. Mean subscale scores were calculated to determine students’ levels of metacognitive knowledge and regulation. In order to be consistent with Sperling et al.’s study, an overall Jr. MAI mean score (MCT) was also calculated.

Academic achievement. Achievement variables examined in this study were participants’ most recent mathematics grade and GPA, mathematics diagnostic test (MDT) score (Mathematics Diagnostic Testing Project, 2006), and final course grade in their summer program mathematics course.

Mathematics grade and GPA. Information regarding participants’ mathematics grades and GPA were gathered from school report cards from the school year preceding the summer program. Mathematics grades were coded on a scale of zero to four (e.g., A = 4.0, A- = 3.7, B+ = 3.3). To calculate GPA, participants’ most recent grades in academic subjects (e.g., English, History, Science, Mathematics) were scored on a scale of zero to four, and GPA consisted of the mean across the academic subjects. GPA and mathematics grade data were not available for four of the 183 participants, as these participants did not receive letter grades at their schools.

22

MDT. The MDT is a mathematics readiness test that assesses participants’ mastery of mathematics material that is required to succeed in the next level of mathematics (Mathematics Diagnostic Testing Project, 2006). The MDT offers readiness tests for Algebra I, Geometry, Algebra II, Pre-calculus, and Calculus. Each MDT contains 45 to 50 items, and is scored by the percentage of items correct.

Summer course grade. At the end of the summer program, all participants received a final mathematics course grade based on homework, participation, test scores, and projects. Grades ranged from A+ to B-, and students achieving below a B- received either a Pass or No Pass. Letter grades were coded on a scale of 2.7 to 4.3 (e.g., A = 4.0, A- = 3.7, B+ = 3.3), and Pass and No Pass grades were coded as 2 and 1, respectively. Procedure. Data were collected in three steps. Mathematics grades and GPA were gathered from the program database in the spring preceding the summer program. Two weeks prior to the start of the summer program, all participants attended a diagnostic testing session during which they completed the MDT followed by the Jr. MAI. At the end of the six-week summer program, final course grades were collected from the program database. Results Preliminary analyses. Means, standard deviations, and internal consistency reliability estimates of Knowledge, Regulation, and Total scores are presented in Table 6. Internal consistency reliability estimates were calculated using Cronbach’s alpha. All three estimates were greater than .70. Knowledge and Regulation subscales scores were significantly and moderately correlated (r = .56, p < .01). Means and standard deviations for participant academic achievement (i.e., GPA, math grade, MDT score, summer course grade) are also provided in Table 6.

Structural validity. Exploratory factor analysis (principal axis extraction) was used to determine the factor structure of the Jr. MAI. Factorability of Jr. MAI scores was based on the determinant of the correlation matrix (.007), Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO = .83), and Bartlett’s test of sphericity, χ2(153) = 869.64, p <.001, which all indicated that the correlation matrix of Jr. MAI scores was factorable. Communality estimates were in the low to moderate range. Given a variable to factor ratio greater than 20:3 (18:2) and a sample size of close to 200 participants (n =183), factor analysis should result in a convergent and admissible solution (MacCallum, Widaman, Zhang, & Hong, 1999). Multiple criteria were used to determine the number of factors to extract. The theoretical framework (Schraw & Dennison, 1994) suggested a two-factor structure, whereas parallel analysis (Hayton, Allen, & Scarpello, 2004; Watkins, 2000) suggested a three-factor structure, and the eigenvalue rule and scree test suggested a five-factor structure. Subsequently, two-, three-, four-, and five-factor solutions were examined. A floor of .40 was used to determine item salience (Floyd & Widaman, 1995). For each solution, both oblique and orthogonal rotations were examined. Two-factor solution. Structure coefficients from the two-factor oblique rotation are reported in Table 7. This solution accounted for 37.14% of the total variance in Jr. MAI scores. Fourteen of the 18 items achieved factor loadings greater than .40. Factor 1 (Regulation) consisted of seven of the nine metacognitive regulation items (excluding R6 and R17), and Factor 2 (Knowledge) consisted of seven of the nine metacognitive knowledge items (excluding K5 and K12). Reliability estimates for the Regulation factor (α = .80), Knowledge factor (α = .75), and all 14 items achieving a factor loading greater than .40 (α = .85) were all greater than

23

Table 6 Descriptive Statistics of Jr. MAI Scores and Academic Achievement Variables Subscale

n

M

SD

α

Range

Jr. MAI Scale

Knowledge 183 4.21 .66 .72 1.67 – 5.00

Regulation 183 3.54 .71 .81 1.89 – 4.78

Total 183 3.91 .46 .85 1.83 – 4.89

Academic Achievement

GPA 179 3.89 .19 2.57 – 4.00

Mathematics Grade 179 3.95 .20 3.30 – 4.30

MDT 183 87.42 10.94 44 – 100

Summer Course Grade 183 3.57 .61 1.00 – 4.30

24

Table 7 Two-Factor Solution From Principal-Axis Extraction/Oblimin Rotation of Jr. MAI Scores Item

Factor 1

Regulation

Factor 2

Knowledge

h2

R9 - I think about what I really need to learn before I begin a task .74 -.07 .43R7 - I ask myself if I learned as much as I could have once I finish a task .68 -.14 .32

R10 -

I ask myself questions about how well I am doing while I am

learning something new .64 .05 .43

R18 - I set specific goals before I begin a task .55 .06 .40R15 - I ask myself periodically if I am meeting my goals .54 .10 .47R11 - I focus on the meaning and significance of new information .49 .14 .43R8 - I ask myself if I have considered all options when solving a problem .43 .23 .37K1 - I am a good judge of how well I understand something -.16 .71 .35K2 - I can motivate myself to learn when I need to .04 .60 .39K4 - I know what the teacher expects me to learn -.07 .52 .23

K14 - I have control over how well I learn .16 .48 .38K16 - I find myself using helpful learning strategies automatically .17 .47 .36K3 - I try to use strategies that have worked in the past .06 .44 .27

K13 - I use my intellectual strengths to compensate for my weaknesses .24 .40 .39R17 - I ask myself if there was an easier way to do thing after I finish a task .33 .19 .29R6 - I draw pictures or diagrams to help me understand while learning .33 .03 .19

K12 - I learn more when I am interested in the topic .06 .19 .29K5 - I learn best when I know something about the topic .03 .13 .26

Eigenvalues (initial) 5.16 1.53

% of variance (initial) 28.64 8.50

α .80 .75

25

.70. Factors were moderately correlated (r = .57). Identical Regulation and Knowledge factors were found in the two-factor solution using orthogonal rotation. Three-, four-, and five-factor solutions. The three-factor, four-factor, and five-factor oblique rotations as well as the three-factor orthogonal rotation each resulted in a version of a Knowledge factor and a version of a Regulation factor with the remaining factors being non-viable duplets. The four-factor and five-factor orthogonal rotations resulted in a version of a Knowledge factor, a version of a Regulation factor, and a Mixed factor containing both knowledge and regulation items with the remaining factor(s) having either one or two salient items.

Concurrent and predictive validity. Bivariate correlations were used to examine the concurrent and predictive validity of (a) total Jr. MAI scores, (b) Jr. MAI subscale scores based on Sperling et al. (2002)’s Knowledge and Regulation factors, and (c) Jr. MAI subscale scores based on the Knowledge and Regulation factors identified in the current study with measures of academic achievement (i.e., GPA, mathematics grade, MDT score, summer course grade).

Correlation coefficients are presented in Table 8. Total Jr. MAI scores were not significantly correlated with any measure of academic achievement, r = │.00│ - │.12│. Scores based on Sperling et al.’s (2002) Knowledge subscale were not significantly correlated with any measure of academic achievement, r = │.01│ - │.10│, nor were scores for the original Regulation subscale, r = │.00│ - │.12│. Similarly, neither the Knowledge subscale scores, r = │.00│ - │.08│, nor the Regulation subscale scores, r = │.01│ - │.13│, identified in the current study had significant correlations with any measure of academic achievement. All academic achievement variables were significantly correlated with each other, r = │.20│ - │.45│, p < .01. Discussion The purpose of this study was to examine the psychometric properties of Jr. MAI scores among a sample of academically talented middle and high school students. Statistical analysis suggests that Jr. MAI scores are reliable in this academically talented sample. Furthermore, exploratory factor analysis yielded a two-factor structure consistent with Jacobs and Paris’s (1987) metacognition theory. These results contrast with previous exploratory factor analysis results by Sperling et al. (2002). Although Jr. MAI scores were found to be structurally valid, they lacked concurrent validity with measures of academic achievement (i.e., GPA, mathematics grade, MDT score) and predictive validity with final summer course grade in mathematics. The absence of a relationship between Jr. MAI scores and academic achievement is consistent with Sperling et al.’s findings.

There are three possible explanations for the lack of concurrent and predictive validity of Jr. MAI scores. The first is that metacognition (as measured by the Jr. MAI) is not significantly related to academic achievement. This explanation is contrary to metacognition theory (e.g., Brown, 1987) and past studies substantiating this relationship (e.g., Pintrich, 2002; Zimmerman, 1990). Second, the Jr. MAI scale may not assess the metacognitive behaviors that influence academic achievement. Third, participants’ self-reports may not accurately reflect the metacognitive behaviors that they employ in school (Schoenfeld, 2002).

26

Table 8 Correlations of Jr. MAI Scores with Achievement Variables (n = 183)

Achievement

GPAa

MGa

MDT

CG

Jr. MAI Total Score

-.00

.05

-.12

.01

Jr. MAI Subscale Scores – Sperling et al. (2002) Factors

Knowledge -.01 .04 -.10 .07

Regulation .00 .04 -.12 -.04

Jr. MAI Subscale Scores – Current Study Factors

Knowledge .00 .03 -.07 .08

Regulation .01 .04 -.13 -.04

Achievement

GPAa -- .40* .29* .24*

MGa -- .20* .17

MDT -- .45*

CG --

Note. MG = Mathematics grade; CG = Summer course grade. a n = 179. *p < .01

27

Study 2 In this study, problem solving interviews were used to examine metacognition among a

sample of the participants from Study 1. The goals of this study were (a) to describe students’ use of metacognition during each phase of problem solving and (b) to examine how students’ use of metacognition during each phase of problem solving affected their problem solving solution accuracy. Method

Participants. Participants included 30 adolescents from Study 1 (see Procedure for participant recruitment details). The participants were 63% female (n = 19) and ranged in age from 11 to 16 years old (M = 12.57, SD = 1.406). Participants represented several ethnicities including Asian American (n = 16, 53%), European American, (n = 6, 20%), African American (n = 1, 3%), East Indian (n = 3, 10%), Latino (n = 1, 3%), and Other (n = 3, 10%). Participants were enrolled in the following summer mathematics courses: Pre-Algebra (n = 6), Algebra I (n = 4), Geometry (n = 7), Algebra II (n = 6), and Pre-Calculus (n = 7).

Measures. Participants’ metacognition was examined through a problem solving interview. The interview process and the mathematics problem analyzed in this study are described below.

Problem solving interview. Individual problem solving interviews were used to assess participants’ metacognition. Interviews consisted of an introduction, a concurrent report section, and a stimulated-recall section (see Appendix B for Interview Script). During the introduction, participants were told that the purpose of the interview study was to better understand how students like themselves think when they solve math problems. During the concurrent report section, participants were presented with one practice problem and three mathematics problems, and asked to report their thinking aloud while solving each problem. During the stimulated recall section, participants were shown the video recordings of their concurrent report interviews and asked to elaborate upon what they were doing, thinking, and feeling as they solved each question.

The mathematics problems used in the problem solving interview were selected based on criteria presented by Hammouri (2003). Specifically, problems were chosen that (a) could be solved using multiple strategies, (b) required multiple cognitive/metacognitive strategies, and (c) could be solved with prior knowledge that all participants should have covered in school. Given these criteria, three problems were selected from recommendations made by Alan Schoenfeld and Betina Zolkower in personal communications (May, 2009). One of these three problems (the train problem) was selected for a fine-grained analysis.

The train problem. The train problem was selected for analysis because it elicited a wide range of metacognition among participants. The problem was presented as follows:

Train A leaves UC Berkeley station travelling at 50 miles an hour on Track X. Three hours later, Train B leaves the station travelling 60 miles an hour on Track Y, which is parallel to Track X. How long does it take Train B to catch up with Train A?

The train problem can be solved using many different strategies. The three most common strategies are described next.

Arithmetic catching up strategy. If the conditions of this problem are understood, it can be solved using arithmetic. Specifically, one could note that Train A travels for 150 miles (50 miles per hour times 3 hours) before Train B leaves the station. Once Train B leaves the station, it travels 10 miles an hour faster than Train A (60 miles per hour minus 50 miles per hour). Therefore, for every hour Train B travels, it catches up by 10 miles. Given that Train B has 150

28

miles to catch up with Train A, and that it takes one hour to catch up 10 miles, it will take 15 hours (150 divided by 10) to catch up with Train A.

Algebraic strategy. One can apply the formula for the distance travelled (distance = rate by time), knowing that when Train B catches up with Train A, they will have travelled the same distance. Therefore, the rate of Train A (50 mph) multiplied by the time that Train A travels will be equal to the rate of Train B (60 mph) multiplied by the time that Train B has travelled. Since the problem asks for the time that Train B travels, it is best to assign a variable (x) to describe the time that Train B travels. Since Train A left the station three hours before Train B, it will have travelled three more hours than Train B (x + 3) by the time the trains catch up. Therefore the equation 50(x + 3) = 60(x) can be used to find the time it takes for Train B to catch up with Train A, with the value of x that is determined by solving this equation being the final answer.

Counting strategy. Third, this problem can be solved by making a chart or other visual representation of the distances that each Train travels each hour (see Figure 2). Since Train A starts three hours ahead of Train B at 50 miles an hour, it will have travelled 150 miles before Train B begins. When Train B begins, it travels 60 miles in its first hour of traveling, meanwhile Train A travels another 50 miles, totaling 200 miles. For every hour after that, Train A will gain 50 miles and Train B will gain 60 miles. Using a chart, one can count the total number of hours it will take for Trains A and B to travel the same distance.

Miles Travelled by Train A 50 100 150 200 50 300 350 400 … 800 850 900

Miles Travelled by Train B -- -- -- 60 120 180 240 300 … 780 840 900

Hours Travelled by Train B -- -- -- 1 2 3 4 5 13 14 15

Figure 2. Train problem counting strategy.

Procedure. Participant recruitment took place on the first day of the summer program. Each mathematics course in the program (11 classrooms) was visited by the lead investigator and students were invited to participate in a mathematics problem solving interview study. Parental consent forms were distributed to all students, and the first 30 students who returned completed consent forms were selected to be the Study 2 participants.

Individual problem solving interviews were conducted during the six weeks of the summer program. Interviews lasted approximately one hour, and subjects received $10 upon completion of the interview. Two video cameras and a microphone were used to record interview data during each interview. Camera 1 was used to record the full duration of the interview capturing the participant, the interviewer, and the television used for the stimulated- recall. Camera 2 was used to record participants’ written work up close during the concurrent report section of the interview. The microphone was used for the full duration of each interview. During the interview, the interviewer recorded observational field notes and collected the participant’s written work at the end of the interview.

Several data sources emerged from the problem solving interviews. First, digital audio and video files were created from the video camera and microphone recordings. Second, transcripts were created from these recordings. Specifically, Camera 2 audio-video files were used to transcribe the concurrent report section of the interviews and microphone audio files

29

were used to transcribe the stimulated-recall section of the interviews. The microphone audio files were corrupted during data collection for three of the interviews. In these cases, Camera 1 audio-video files were used to transcribe the stimulated-recall section of the interviews. Finally, field notes and participants written work were treated as written artifacts. Results

Metacognition during problem solving. In this section, illustrative examples of the metacognition participants exhibits in each phase (i.e., preparation, performance, evaluation) of the problem solving process are presented. Illustrative examples of the absence of metacognition in each phase are also presented.

Metacognition in the preparation phase. Metacognition in the preparation phase involves assessing the train problem and identifying both what information was given and what the problem was asking for. It also involves analyzing the problem mathematically and then using the given information and the understanding developed through assessment and analysis to create an appropriate plan. Two examples are used to illustrate the presence and absence of metacognition. First, Cailin is used as an exemplar to illustrate how participants used sufficient metacognition. Next, Brice is used as an exemplar to illustrate the absence of sufficient metacognition. A summary contrasting the metacognition used by Cailin and Brice is presented following the two examples.

Cailin: An example of sufficient metacognition. Cailin exhibits a range of metacognitive thinking that contributes to her ability to generate a mathematically accurate algebraic strategy to solve the train problem. Cailin begins by noting the information given in the problem, drawing a picture representation of the problem, and deciding on the type of strategy – in this case, an algebraic strategy – to solve the problem. Line 1: So, here's the station, Track X, Track Y. 50 mph here and 3 hours. 3hours later. Ok.

3 hours later. 60 mph. [Draws Figure 3] Line 2: And the distance. Line 3: So the distances should be equal when they catch up to each other. Line 4: So distance equals rate times time [writes d = r*t].

Figure 3. Cailin’s initial representation. Cailin explains in her stimulated-recall interview that as she read the problem her first thought was that this was a “distance, rate, time problem.” She reasons that because of the nature of the information given in the problem (i.e., mph, trains, time, distance, rate), “I figured it was going to be something like this…I knew I had to use that [D = RT] equation” (D = RT equations are a standard form taught in high school mathematics curriculum).

30

After deciding to create an algebraic equation to solve the problem, Cailin continues to develop her understanding of the problem in ways that will allow her to create an equation (see Figure CR2). Line 5: Ok, and this [points to Track X]… Line 6: Oh, the time should be equal. Line 7: Oh, ok. So, let's see…50 miles per hour times [writes “50* =”]… Line 8: Oh, how long does it take, so you have to find the time [fills in t so the equation reads

50*t = ] Line 9: And then 60mph and that's 3 hours later. Line 10: So let's see. How can I show that in variables? Line 11: Um...hmm...60mph is the rate [writes 60 on the other side of the equal sign]. Line 12: And time, so, hmm, distance has to be equal. t [writes "(t"] hmm, minus. Line 13: Maybe I shouldn't use an equation.

Figure 4. Cailin’s initial equation Cailin tries to write an equation that represents her understanding of the problem. She also continues to develop her understanding of the problem in ways that will facilitate her ability to write an equation. She explains her thinking during Lines 5 – 12 in her stimulated-recall interview.

I was trying to figure out which was equal…I was trying to figure out, ok, time needs to be equal, does the distance need to be equal? When will they catch up…it was taking me a long time to figure out where I needed to place the variable and to show that one of the trains was three hours later. How do I make that equal when one is three hours ahead and how do I put that in correct units?

However, Cailin realizes that she is having difficulty and questions both her method and understanding of the problem.

I was like, I know the formula, I should be able to plug everything in and it should be ok. But then I was struggling a little bit to set up the equation. You know what should go where and what units, so I think I was just a little bit frustrated because I was like, hmmm, this seems like it’s not that difficult…I was getting frustrated with having to think of some way to put the 3 in there. And after the parentheses there I was trying to step back and think because it was getting too complicated…I was trying to rely on a formula and maybe a picture to help. But when I was, it was so difficult to visualize the three hours ahead and put it in the formula, it didn’t really fit.” So far, Cailin has read the problem, decided to use an algebraic equation to solve the

problem, developed an initial understanding of the problem, attempted to create an equation based on her understanding of the problem, and realized that she is having difficulty writing an equation based on her current understanding of the problem. Next, Cailin takes a step back and asks herself, “How does this make sense?” She attempts to better understand the problem by

31

drawing a second picture representation of the problem and making mathematical sense of the information she is given in the problem.

Line 14: How does this make sense? Line 15: Leaving from the same place. Why is this taking me so long? Leaving from the same

place. Line 16: 3 hours later…60mph. Line 17: Ok, so it's going 10 mph faster. Line 18: So, if it leaves 3 hours later, 3 hours later...ok, so they have to be...hmm. Line 19: So you don't know the time. Line 20: So this has to be…and 3 hours... Line 21:

So I'll show this behind. And then 3 hours later. It says how long does it take train B to catch up with train A. So they just have to be at the same place

Line 22: Ok. So, 50 should equal, hmm… Line 23 Why is this taking me so long? This doesn't seem this difficult.

Figure 5. Cailin’s second picture representation.

In her stimulated-recall interview Cailin explains that she drew a second representation in

order to solidify her understanding of the problem without worrying about the equation: “I tried to draw a second picture to maybe straighten out those thoughts and maybe not be so bogged down about the equation and just think about it…just thinking common sense.” This thinking allows her to develop a deeper conceptual understanding of the problem, but she struggles to connect her understanding with an equation. This indicates to her that her understanding is still incomplete.

I figure I knew everything, it was just sometimes when I get these problems, I get stumped just trying to get everything I know, all the data etc., together so it’s in a fluent equation or problem that I can solve. So I figured as soon as I just figured out what was going on that I could put it all together and it would make sense. Cailin’s first attempt to take a step back and figure out “how does this make sense” is

productive in the sense that it helps Cailin continue to develop her understanding of the problem. However, she knows that there is still more to understand. Moving forward, Cailin stops herself again and makes another change in her thinking. This time, she stops thinking about creating an equation altogether and focuses on understanding the problem mathematically.

32

Line 24: Let's see. How else can I show this? Hmm. I think I'm making this too complicated. Cailin explains her thinking in her stimulated-recall interview.

“I had to just take a step back and go 50 miles per hour means it’s going 50 miles in one hour, and three hours ahead, so 50 times 3…So then just stepping back and thinking about the broader picture and not worrying about this formula…then I was able to figure out how to put this in. But then I went back to the formula and fit it in. I think I just needed to step back and think about what was really happening because the formula didn’t really represent that 100%...I was transitioning to the need to think this out and not just worry about the equation.”

Once Cailin takes this step back and begins to think about the problem mathematically, she is quickly able to create an equation that reflects her mathematical understanding of the problem. Line 25: Hmm, ok, so if it's going 50 miles each hour and it leaves 3 hours…so that would be

at 150. Ok so this Track X will be 150 miles. Line 26: So how long will it take 60 miles to get there.

Line 27: 3 hours behind…So after 3 hours it's starting…It will continue…ok, so when Train Y is starting, that will already by 150 miles ahead and it's continuing at that rate.

Line 28: So plus 50 t. T can be the number of hours. And then this is starting when it's already at 150. So this should equal 60 times the time [Writes final equation].

Line 29: That makes more sense.

Figure 6. Cailin’s final equation.

Brice: An example of insufficient metacognition. Unlike Cailin, Brice has difficulty using metacognition to develop a mathematical understanding of the train problem. Instead, he spends his time in the preparation phase attempting to derive an equation from the given information and his vague memory of a formula using distance, rate, and time.

Line 1: ‘K. So this problem…I have to remember how to do this. Um, first I’m gonna set up the loca- the beginning point which will be UC Berkeley.

Starts to travel so Train X…Train Y…starts to travel at 50 miles per hour…Train Be leaves the station travelling…Wait. Three hours later, so the time between will be three hours starting at 60 miles per hour.

Line 2: How long will it take for them to catch up. Line 3: So, um, there’s three hours in between. Line 4: So I’m going to think about this for a second. Line 5: Um, 50 miles per hour…x is…Train X is equal to…the miles per hour…uh, how long

does it take for the train to catch up. Um, 3 hours later…so…

33

Figure 7. Brice’s picture representation.

In Lines 1 – 5, Brice begins to develop an understanding of the problem, as evidenced by

his diagram (see Figure 7). However, as he begins to develop a strategy to solve the problem, he recalls a memory of doing “this kind of math” – a train problem – in seventh grade. Based on his memory, he decided to use an equation using distance, rate, and time. At this point, he abandons his conceptual understanding of the problem and focuses on constructing an equation based on multiplying distance and rate to get time.

Line 6: Oh yeah, I remember the DRT problems. Line 7: So the distance which would be the distance is going to be…let me think about

that…that would be three hours, so, oh, 3 hours later. So I’m going to see if I can do this by…so maybe 50 times 3 which is 150…that would be…so Train X would already have been out for 150 miles. Ok, so basically, uh, Train X has already been out for 150 miles and that means…how long will Y take for it to reach…uh, wait. X…um…the distance is going to be, um…oh, oh, oh.

Line 8: So the distance is going to be X, and then Train Y will be Y plus 150. The rate will be 50…this is 60, and we’re looking for 50x…this will be 60(y + 150), um, so when will it equal.

Line 9: So, uh, Y…oh, no, this is actually [changes y’s to x’s]…because that’s x + 50, which is the same…um, 50x + 60(x + 150)…50x + 60x…150 times 60, um, let’s see, so pluss 9000. Oh no wait…no…the equal part of this…um…is going to…oh…those [50x and 60x+9000] are equal.

Figure 8. Brice’s DRT chart and equation.

34

Once Brice decides to use a DRT formula, he has difficulty stimulating his memory and

applying his knowledge of distance, rate, and time to this problem. He explains his thinking further in his stimulated-recall interview.

I filled in as distance because I knew it didn’t have anything to do with distance. I was trying to [figure out] time but I didn’t know anything about distance. The only thing the word problem gave was the rate and this little bit about time – the three hours – but it talked nothing about distance so I knew distance was a constant number…there’s this constant part that I’m talking about is distance and they stop – meet up at one point which is still in reference to distance and I’m looking for time…at first I was thinking about putting a variable as time, but that didn’t work out because I didn’t have – well, I don’t know exactly why I didn’t put something in time. I guess it didn’t feel right. I guess I always put it with distance. Summary. Cailin exemplified kinds of metacognitive thinking that facilitated her success

in the preparation phase of problem solving. With regard to metacognitive regulation, she was able to assess the problem right away. At first, her lack of analysis hindered her from developing an appropriate algebraic expression. However, once she took a step back and asked herself, “how does this make sense,” she was able to develop a conceptual and mathematical understanding of the problem that allowed her to develop an appropriate equation. In contrast, Brice did not exhibit sufficient metacognitive regulation during the preparation phase. Although he was able to identify the information given in the problem, he spent little to no time analyzing the problem, and therefore did not have a conceptual understanding of what the problem was asking. Brice focused on writing an equation based on the numbers given in the problem rather than on a mathematical understanding of the problem, resulting in an inappropriate equation.

Metacognitive knowledge, metacognitive beliefs, and metacognitive awareness influenced both Cailin’s and Brice’s thinking as they developed strategies to solve the train problem. Both participants drew on their metacognitive knowledge about the mathematical relationships between distance, rate, and time as a starting place for their strategies. Cailin’s knowledge about a DRT strategy was complete and correct and facilitated her development of an appropriate equation. Brice’s knowledge about the relationships between distance, rate, and time was incomplete, and his application of this knowledge was inaccurate. Brice knew that it was possible to apply a formula to find time when distances and rates are given, but he did not know the correct formula or how to apply it to the train problem. Cailin applied other metacognitive knowledge (person knowledge, to be exact) to generate her equation. She knew that she was most comfortable solving problems using equations and that diagrams helped her to understand math problems. These aspects of metacognitive knowledge helped her as she worked through the preparation phase by guiding how she chose to think about and solve the problem. In contrast, Brice did not appear to use any other metacognitive knowledge in trying to solve the problem.

There appeared to be several metacognitive beliefs that influenced Cailin’s thinking as she developed her equation. The three most influential beliefs were (a) that math problems, equations, and mathematics in general are supposed to make sense, (b) that she would be able to solve the problem once it made sense, and (c) that she was capable of making sense of the problem. These beliefs guided her thinking, which resulted in her taking a step back mid-way through the preparation phase in order to understand the broader picture. As a result, she focused

35

on developing her mathematical understanding of the problem and ultimately developed an appropriate equation.

Brice was operating under a very different set of metacognitive beliefs which were counterproductive. His very first statement—that he had to “remember how to solve this—was evidence that he believed that there is a right way to solve math problems. A belief that may be inferred from his thinking during the preparation phase is that mathematics is about creating and solving equations and that it need not (or does not) make sense. These beliefs affected his thinking as he created an equation that lacked mathematical meaning from a problem that he never completely understood.

Finally, Cailin and Brice exhibited very different levels of metacognitive awareness. Cailin not only identified when she was getting stuck and feeling frustrated, but also identified where and why she was stuck (e.g., not knowing how to represent the three hour difference between the trains). Cailin was also aware of when she was on the right track and was confident that her equation was correct. Brice, in contrast, had little metacognitive awareness as he created his equation. He did not appear to be aware of his lack of understanding of the problem or of his lack of understanding about how to correctly set up a DRT equation. He was also not aware that his equation was incorrect.

Metacognition in the performance phase. Metacognition in the performance phase involves monitoring both one’s progress and one’s execution of the strategy developed in the preparation phase of problem solving. In the case of a counting strategy, monitoring progress involves making sure that the trains were getting closer in distance to each other over time. Monitoring strategy execution involves making sure that distances were added correctly and numbers were lined up correctly. Two examples are used to illustrate the presence and absence of metacognition in the performance phase, both involving similar counting strategies. Amelia uses sufficient metacognition to monitor the execution of her counting strategy. In contrast, Dylan does not use sufficient metacognition and does not effectively monitor the execution of his counting strategy. A summary of the metacognition used by Amelia and Dylan as they execute their strategies is presented after these examples are described.

Amelia: An example of sufficient metacognition. Amelia is an example of a student who demonstrates metacognition, particularly in the performance phase, using a counting strategy.

Line 1: If it goes for 3 hours at 50 mph, it'll go 150 miles. Line 2: But if that goes...so if it keeps going, then for an hour, it'll reach 60 and it'll reach

200. That'll be 1 hour. Then 2 hours is 250, 120. 3 hours is 300, 180. 350, 240 is 4, 5, 6...400, 450, preset. ok, so that's 300, 360.

Line 3: 7 is 500, agg, it's going to take a while. Um, 420. Line 4: [Turns paper to landscape orientation] Then 8 is 550, 480. Then 9 is 600… 540, ah

ha! Line 5: 650, 600. Closer! Line 6: So what does that mean. ok. 700, 750. 640 [mistake -- added 40 instead of 60]. 680. Line 7: [Turns paper back to portrait orientation then back to landscape] Line 8: Not quite sure it will! Line 9: 800, 740.

36

Figure 9. Amelia’s initial counting strategy. Line 10: [flips to new paper] 14, 15… gonna set up my paper so I can keep going [labels

paper to 22 hours] Line 11: Ok, so 850, 900, 950...[lables 50's to 100]...start from there. Line 12: I left off at...ok that makes me 800, 860, um, 920, no!

Figure 10. Amelia’s initial counting strategy (cont.).

Amelia first thought that indicates monitoring in the performance phase is her statement in Line 3 that “it’s going to take a while.” She explains her thinking in her stimulated-recall: “Because at this point, they’re so far apart and they’ve been pretty far apart and they’re only catching up very very slowly. So it will take it quite a while to catch up.” Amelia confirms that she began noticing the difference in the distances around seven hours, and that she continued to monitor this difference on and off.

37

“I started to pay attention here [at 7 hours] and then 10, 12, 13, and so on. And then I realized that at about 16 it’s going back up because if the difference here is 50 but back here it’s 80 and then 60…and then went to 50 and then back here it’s gone back up…Here it was 50, here it was 40, but at this point it was starting to go back up.”

When asked why she stopped at Line X, she replies, “I think that’s the point where I’ve convinced myself that something is wrong.” Line 13: But then [looks back at first page of work]... OH!!! ok. I completely did this wrong. I

started going by 40's. Line 14: Let's...where'd I know I left off...ok, let's go back to 8, 9, 10, 11. ok so for 8 that'd be

550, 600, 650, 700. Line 15: Ok, so that'd be 480, plus 6, is 540. plus 6 is 600, 660. Line 16: 12, 13, 14...750, 800, 850. . Ah! 720, 780, 840. Line 17: 15 would be 900 and 900. Line 18: It'll take it 15 hours.

Figure 11. Amelia’s revised counting strategy. Once Amelia identifies that something is wrong, she goes back through her work, finds her initial mistake, and fixes the mistake by re-doing her work beginning at the eighth hour (see Figure EM3). She continues to monitor her addition accuracy and the decreasing distance between the two trains and arrives at the correct answer.

Dylan: An example of insufficient metacognition. Similar to Amelia, Dylan also uses a counting strategy to solve the train problem (see Figure 12). Although Dylan’s plan is mathematically correct, Dylan makes several calculation mistakes which lead him to an incorrect answer. He consistently adds 50 to the top number (representing the distance travelled by Train A) but does not consistently add 60 to the bottom number (representing the distance travelled by Train B). Instead of adding 60, he incorrectly adds 40 to 480 and adds 50 to 520, 630, and 740. As a result, he miscalculates the total distance Train B travelled in relation to Train A. Recall if this strategy is executed correctly, Trains A and B meet when they have each travelled 900 miles.

Line 1: So when Train B leaves Train A is already 150 miles away. Line 2: So this is 0, 150 at 1 hour. So then 2 hours, 2 hours, 3 hours. This is 4 hours.

38

Line 3: So at 4 hours this is at 200, this is at 60. At 5 hours 250, 120. 6 hours is 300 180. 7 hours is 350, 240 and then 8 hours is 400, 300.

Line 4: Yay. Line 5: 9 hours. 3, 4, 5, 6, 7, 8, 9 hours is 450, 360. 10 hours is 500, 420. 11 hours is 550,

480. 12 hours is 600, 520. Line 6: 7 hours – err – 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 hours is 650, 570. 14 hours is 700,

630. 15 hours is 750, 680. Line 7: Then 800, 740. 850, 790. 900, 850. 950, 910. 1000, 970. 1050, 1030. 1200 – or

1100, 1090. And then 1150 and 1150. Line 8: So that’s 3 hours, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23

hours. Line 9: So it’ll take 20 hours to catch up with Train A.

Figure 12. Dylan’s counting strategy.

Dylan shows some signs of monitoring (e.g., “yay”) at Line 4, counting hours to make sure he has written them out correctly at Line 5. But he does not monitor his addition as he adds 50 to the top number and 60 to the bottom number. When Dylan’s mistakes are pointed out to him, he is surprised. When asked if he was doing any thinking to monitor as he completed the addition, he replies, “I guess not.” However, he was monitoring his progress in some sense, saying that he did notice that “one is increasing faster than the other one.” Dylan best describes his monitoring in the following statement: “Yes, I was checking, but no, I wasn’t checking.” He explains that he thought he was monitoring his work sufficiently, but is convinced after seeing his mistakes that this was not the case. Summary. Both Amelia and Dylan exhibited some metacognitive regulation (i.e., monitoring) as they executed their counting strategies. Amelia’s regulation was effective and sufficient, and Dylan’s was not. Both Amelia and Dylan noticed how the distances between the trains changed over time and thought that this change was an indication that they were on the right track, so to speak. Amelia paid closer attention to these differences than Dylan. She

39

eventually noticed that there was something wrong with the way that the distances were changing over time (due to her addition mistakes) whereas Dylan did not. Once she established that something was wrong, she reviewed her work, identified her mistakes, and corrected the mistakes resulting in a correct solution. In contrast, Dylan’s lack of sufficient monitoring resulted in many addition mistakes that went unnoticed and resulted in an incorrect solution. The piece of metacognitive knowledge that appeared to directly influence Amelia and Dylan’s metacognitive regulation and overall thinking as they executed their counting strategies was their knowledge (for Amelia) or lack of knowledge (for Dylan) about how accurate they are at adding two and three digit numbers together. Amelia knew that she tended to make small arithmetic errors, which could have contributed to her active monitoring. In contrast, Dylan most likely did not know his own weakness in addition, which could have resulted in less close monitoring. It is not apparent how Amelia and Dylan’s metacognitive beliefs influenced their monitoring as they executed their counting strategies. Their metacognitive awareness, however, played a critical role in their monitoring. At first, both Amelia and Dylan thought that they were on the right track, which is evidence of metacognitive awareness. As they continued to execute their strategies, Amelia because aware that something was not correct, which led her to review her strategy. This feeling of uncertainty led to productive thinking that allowed her to correct her mistake. Dylan never became uncertain of his work, showing a lack of awareness that led him to complete his strategy without ever discovering his mistakes. Metacognition in the evaluation phase. The metacognition exhibited in the evaluation phase involves verification that (a) the answer was realistic, (b) the answer was derived correctly from the plan, and (c) that the answer made sense relative to the problem. Cailin, whose thinking was previously presented as an example of sufficient metacognition in the preparation phase, are used as an example of sufficient metacognition in the evaluation phase. Tori and Diana are used as examples of insufficient metacognition. A summary of the metacognition contributing to the presence and absence of verification follows. Cailin: Another example of sufficient metacognition. Recall Cailin, who used sufficient metacognition to develop an appropriate algebraic strategy in the preparation phase. During the evaluation phase she develops an appropriate equation and easily solves the equation to get her solution (Line 30). Before finishing the problem, Cailin evaluates her solution by questioning whether the solution makes sense in light of the problem. Line 30: Ok, so 10t would equal 150 and t would equal 15. Line 31: Hmmm, that seems a bit long. Line 32: Um, ok, [how does this] make sense? Line 33: So this is already going to be 150 miles ahead cause it’s leaving 3 hours ahead. So

it’ll have that 150 miles jump start plus 50. It’s rate time show ever long the time is taking should equal 60, which is the rate of Train Y, times the same amount of time, which is why they’ll be equal.

Line 34: Ok, so that makes sense. Line 35: Ok, so my final answer is 15 hours.

Cailin explains that after she got her answer, she wanted to check to see if her answer

made mathematical sense relative to the problem.: “I always finish my thought and then say,

40

‘Ok, does that answer really make sense? Does how I got it really make sense?’” Once Cailin relates her answer back to the original problem, she becomes confident in her answer. Tori and Diana: Examples of insufficient metacognition. Tori uses a counting strategy to solve the train problem and concludes incorrectly that it took 12 hours for Train B to catch up with Train A. Her mistakes, which go unmonitored in the performance phase, involve incorrectly set up her counting strategy and also making arithmetic errors (see Figure 13). When Tori finishes her counting strategy and both trains had reached a distance of 700 miles she says, “Ok, so it takes 12 hours for Train B to catch up with Train A” and hands in her paper without any further thinking about the problem. When asked why she didn’t check her solution at the end of the problem she replies, “Yeah, well, um, usually I just check the answers in the back of the book to see if I did it right. But if I got it wrong, I go back in and check.”

Figure 13. Tori’s counting strategy.

Diana uses the equation 50t = 60(t – 3) to solve the train problem (see Figure DM). In

this equation, the variable “t” represents the time that Train A travels and “(t – 3)” represents the time that Train B travels. Diana accurately simplifies the equation and solves for t and then says, “It’d be 18 hours until Train B catches up with Train A.” She finds the time that Train A travels, but not Train B. However, she does not take the time to consider her answer before handing in

41

her paper. When asked why she didn’t check his work, she states that she didn’t know how. “I was pretty sure it [my answer] fit…Sometimes I go back and check – plug it in – but I guess I didn’t this time since I knew I didn’t have distance so I wasn’t quite sure how I should check my work.”

Figure 14. Diana’s algebraic strategy.

Summary. The absence of verification in the evaluation phase, as illustrated by Tori and Diana, is a stark contrast to Cailin’s complete verification of her solution. Cailin returned to the original problem and verified that her answer made sense relative to the information given and asked for in the problem and her mathematical understanding of the problem. In contrast, Tori and Diana did not even attempt to verify their answers mathematically or conceptually. Metacognitive knowledge appeared to contribute, in part, to Diana’s lack of metacognition, as she claimed that she did not know how to verify her answer. For Cailin and Tori, metacognitive knowledge did not appear to have a direct relation to their choice to verify or not verify their solutions. Metacognitive beliefs, again, played a large roll in why Cailin chose to verify her solution and why Tori and Diana did not. Cailin’s beliefs about mathematics needing to make sense, as described earlier, led her to question whether or not her answer made sense once she solved her equation. It was not important for Tori or Diana to make sense of their answer. Their goal was to find an answer that fit their strategy, which is a very different way of viewing mathematics. Finally, Cailin showed metacognitive awareness by communicating uncertainty about her answer after she solved her equation. She knew that she was not completely confident in her answer. This changed after she verified her answer. Tori and Diana showed no awareness of their solution inaccuracy, which likely contributed to their lack of solution verification.

42

Metacognition and solution accuracy. Participants exhibited a range of metacognition as they solved the train problem. The presence and absence of metacognition influenced the plans they developed in the preparation phase, the solutions they came to as they carried out their plans in the performance phase, and the final solutions they reported in the evaluation phase. Additionally, metacognition led students to identify and correct mistakes made in the preparation (e.g., inappropriate plans, incomplete or inaccurate conceptual understanding of the problem) and performance (e.g., arithmetic mistakes) phases. A diagram illustrating the ways metacognition influenced participants’ trajectories through the problem solving process is presented in Figure 15. In this study, eight different problem solving trajectories were observed. Three of these trajectories resulted in a correct final solution and five resulted in either an incorrect final solution or termination of the problem.

Problem solving trajectories resulting in a correct solution. Eleven participants (37%) demonstrated productive thinking that resulted in correct solutions to the train problem. These participants followed three different problem solving trajectories (see Figures 16 – 18).

Trajectory 1. The most common trajectory among participants who reported a correct solution is presented in Figure 16. Participants following this trajectory (n = 7) exhibited sufficient understanding of the train problem in the preparation phase and created a plan that would work to solve the problem (for example, recall Cailin’s use of metacognition in the preparation phase). In the performance phase, they used sufficient metacognition (e.g., monitoring) to accurately complete their plan and obtain a correct solution (for example, recall Amelia’s use of metacognition in the performance phase). For these participants, their use of metacognition in the preparation and performance phases resulted in plans and solutions that were correct. Some of these participants used metacognition in the evaluation phase to verify their solutions and some did not. Since all of the participants who followed Trajectory 1 derived correct solutions in the performance phase, the absence of metacognition in the evaluation phase did not affect their solution accuracy.

Trajectory 2. Similar to participants who followed Trajectory 1, participants who followed Trajectory 2 (n = 2) exhibited sufficient metacognition in both the preparation and performance phases as they solved the train problem. However, Trajectory 2 was slightly more complex (see Figure 17). Following this trajectory, participants began the problem by exhibiting sufficient metacognition in the preparation phase and developing an appropriate plan to solve the problem. In the performance phase, they began to follow through with their plans, but as they monitored their performance and understanding of the problem, they made the decision to return to the preparation phase. Next, these participants focused their thinking on developing a deeper understanding of the problem which they took forward with them as they returned to the performance phase for a second time with new or enhanced plans to solve the problem. Finally, these participants solved the problem using their final plans. Similar to participants in Trajectory 1, the metacognition used in the preparation and performance phases resulted in accurate solutions, so they reported accurate final solutions regardless of the amount of metacognition they used in the evaluation phase. Jamal is one of the two participants who solved the train problem following Trajectory 2. He starts by using a variation of the counting strategy using two parallel number lines (see Figure 19).

43

EVA

LUA

TIO

N

Figure 15. Problem solving trajectories as influenced by metacognition (CONT = Continue with problem; MC = Sufficient metacognition; PERF = Return to performance phase; PLAN = Began with a good plan; PREP = Return to preparation phase; = Terminated problem; X = Inappropriate plan [in preparation] or incorrect solution [in performance/evaluation]; = Appropriate plan [in preparation] or correct solution [in performance/evaluation]).

X

MC?

CONT?

MC?

MC?

MC?

MC?

X

X

PREP?

CONT?

PREP?

Yes

N

PLAN?

PREP?

o

Yes

Yes

Yes Yes

Yes Yes

Yes

Yes Yes

Yes

No

PER

FOR

MA

NC

E

No No

No No

No

No No

PREP

AR

ATI

ON

No

No

44

MC?

MC?

MC?

Figure 16. Trajectory 1.

MC?

MC?

MC?

MC?

MC?

Figure 17. Trajectory 2.

45

Line 1: [Reads problem aloud] Line 2: Um, normally, since it says x and y, I would assume that it’s a graph. But it says

parallel so I’m going to put it as parallel lines. And I’m going to make segments – which are – I’m going to estimate how long they might be, so maybe I’ll put that as 10 miles and – are they coming from opposite directions? [waits for response]

Line 3: [AY: Read the problem again] Line 4: Ok. [Reads problem silently] Oh, ok. Oh, so they’re on parallel tracks leaving from

the same station. Line 5: Ok, so 10 miles for every segment. That’s 20. 30, 40, 50, 60, 70, 80, 90, 100, 110,

120. I might need more than that. And oh, so this is x and that’s y. 10, 20. 30, 40, 50, 60, 70, 80, 90, 100, 110, 120. I’m gonna recount that to make sure I’ve got it right. And, um, I’m gonna put points at each end.

Line 6: So for x it’s travelling at 50 miles per hour, so I’ll go over 5 – 1, 2, 3, 4, 5 – and I’ll put a dot there. And it’s 3 hours later, so I’m gonna do it 2 more times. And of course I’m gonna go off [the paper]. So 1, 2, 3, 4, 5. So I’ll continue the line. Can I go on the back [of the paper]

Line 7: [AY: I can give you more paper.] Line 8: 130, 140, 150, 160, 170, 180, 190, and 200 for now. And 1, 2, 3, 4, 5. Line 9: And finally, um, the other Train B starts. And it’s going 60 miles per hour so you

count 6. 1, 2, 3, 4, 5, 6. Put a dot there. And at the same time you do x again. 1, 2, 3, 4, 5. Then we’ll do y again. 1, 2, 3, 4, 5, 6. And we’ll do x again.

Line 10: 210, 220, 230, 240, 250, 260, 270, 280, 290, 300. Line 11: 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290,

and 300. Line 12: Then you go another 5. 2, 4, 5. And 6.

MC?

MC?

MC?

MC?

X

MC?MC?

Figure 18. Trajectory 3.

46

Line 13: Wait let me check. Oh, 2, 4, 6. Put a dot there. And then do it again. Line 14: Wait, no. [scans back down Track X with his pen] K, and then go up here. Line 15: And pretty much I’m gonna kinda do this quickly.

Figure 19. Jamal’s initial counting strategy. In Lines 1 – 15, Jamal’s strategy is mathematically correct and helps him to conceptualize how the distance between Trains A and B changed over time. However, at the time, Jamal feeels like his strategy is “confusing” and that it would be easier to attempt to solve the problem arithmetically, as he does in Lines 16 - 18. Line 16: And I’m gonna say that pretty much they’re the same except this is 150 miles ahead

cause 3 hours times 50 equals 150. Line 17: And then it goes – it gets – so you have to subtract 10 from it every time cause 60 is

10 more than 50. Line 18: So that means it would take 15 hours if you take off the zero. Line 19: Wait, no. I think I did that wrong. Um, I think so. 18. Um, yeah, I’m just going to

go ahead and do this. Cause now I’m confusing myself.

Figure 20. Jamal’s second strategy. Jamal elaborates on his thinking in his stimulated recall interview.

I was thinking that Y is ten miles faster than X…that was the speed difference…X was already 150 miles ahead when Y started, so 150…that is how many miles away it is...It’s basically 150 divided by 10 which is the number of miles divided by the number of miles per hour…10 goes into 150 15 times so that’s 15 hours.

Despite having a correct mathematical representation of the problem, Jamal is not confident with his answer. He becomes confused and feels like he “made a mistake or something.”

47

I was confused between the two [strategies] because this [first strategy] wasn’t fully completed and I went out of it and I went up here [to the second strategy] because I wanted to do it faster. And then I think I didn’t know the pattern yet. I probably had the pattern in my head and I lost my train of thought.”

Next, he returns to his original strategy, saying that although this first strategy had “seemed confusing” at the time, his second strategy “seemed more confusing.” Relative to the second strategy, his first strategy “seemed like a better way.” Line 20: Now from this mark I’m going to start doing 20 miles. Make it a little faster. Might

go up more eventually. And 320, 340, 360, 380, 400, 420, 440, 460, 480, and 500. And the same for the other line.

Line 21: And then 50, and let’s see. 2, 4, 6. And another 5. and then 1, 2, 3. So 1, 2, 3, 4, 5, 6. 1, 2, 3, 4, 5, 6, 7, 8.

Line 22: So you need another here ‘cause there’s a 3 hour difference. Line 23: And 60 and another 50. Line 24: Now they are, let’s see, 20, 40, 60, 80 miles away. Line 25: I’m gonna check how many hours that took. 1, 2, 3, 4, 5, 6, 7.

Figure 21. Jamal’s full counting strategy. In Lines 24 – 25, Jamal determines that Trains A and B have travelled for 7 hours and are now 80 miles apart. He then uses thinking similar to the thinking he uses in his second strategy to confirm that the answer is 15 hours, as he reports initially in Line 18.

I counted and found it [i.e., the remaining distance between Trains A and B] was 80 and to check that I counted up these – the dots [i.e., how many hours Trains B had already travelled]. 1 2, 3, 4, 5, 6, 7. That meant (7 + 8) / 10. Is, um, 15.

Line 26: So considering they were 150 miles behind, I’m going to say that my answer is correct.

Line 27: 15 hours. And I’m finished. Jamal’s success in this problem was due to the thinking that he developed using both his counting and arithmetic strategies. The counting strategy allows him to begin observing how the distance between the trains changed over time, and the arithmetic strategy allows him to begin conceptualizing the relationship between distance and time mathematically. Finally, returning to the counting strategy helps Jamal to fully understand the arithmetic strategy he has developed. Trajectory 3. Participants who followed Trajectory 3 (n = 2) differed from participants who followed Trajectories 1 and 2 in that they initially lacked sufficient metacognition in the preparation phase (see Figure 18). These participants developed an inappropriate plan due to an underdeveloped conceptual understanding of the problem. However, participants who followed

48

Trajectory 3 caught their mistaken plans in the performance phase and returned to the preparation phase to develop a deeper understanding of the problem and create a mathematically appropriate plan. Once their new plan was in place, these participants used sufficient metacognition as they worked through the performance phase for a second time, resulting in an accurate solution. Similar to participants who followed Trajectories 1 and 2, metacognition in the evaluation phase did not affect their solution accuracy. Anthony is one of the two participants who followed Trajectory 3. He initially develops an inappropriate plan (50t = 60[t + 3]) to solve the train problem. Line 3: So, Train A leaves UC Berkeley Station traveling 50 miles an hour on Track X. 3

hours later, train b leaves, so this is track x [draws track x], train b leaves the station travelling 60 mph on track y, so that's the same station, so train b, on track y [writes train b, draws track y]. And this train is travelling 50 mph, and this is travelling 60 mph [labels speeds next to tracks]. Which is parallel,

Line 4: Ok that's good.

Line 5: Um, how long does it take train b to catch up with train a. Line 6: So let's say t equals the amount of time [writes “t – time”], so it travels 50 miles in 1

hour and this travels 60 miles in 1 hour. 3 hours -- oh it's 3 hours later. Oh. So this time must be...let's say this is time in hours [writes in hours next to t -- time],

Line 7: So this is t here is t, and this is t plus 3 [writes t + 3]. so the time it takes [writes d=rt]

Line 8: So the distances have to be equal. Line 9: So let's say this is d1, d2. [writes d1 and d2 next to tracks x and y, writes d1=d2] d1

has to equal d2. this travels 50 miles in 1 hour, that's the speed. So 50t, and this is 60 (t+3) [labels tracks],

Line 10: So I have to look for when this equals... Line 11: [writes 50t = 60(t – 3), simplifies equation]

Line 12: 10t…well…

Figure 22. Anthony’s initial algebraic strategy.

49

Anthony describes his thinking as he realizes in the performance phase that he has made a mistake:

“I’m getting this negative time, which doesn’t make sense…It’s only when I got down here [Line 12] that I realized that something is wrong…I think I wanted to believe first that I made a mistake on how I did the problem instead of that I misinterpreted the problem, because that’s worse…That means I have to start all over. I just wanted to see where I made my mistake. And at the end, I looked at the problem again.”

Anthony’s metacognition in the performance phase leads him to reconsider the problem. Line 13: The distance this train [Train A] travels has to equal the distance this train [Train B]

travels for them to meet. Line 14: Travels 50 miles per hour…[looks over her page of work]…3 hours later. Line 15: [writes 50t + 150 = 60t] Line 16: [crosses out +3 from 60(t+3) and changes 50t to 50 (t+3) next to diagrams] Line 17: This is...[solves new equation, writes t=15]. 15 hours.

Figure 23. Anthony’s final algebraic strategy. Problem solving trajectories resulting in an incorrect or no solution. Nineteen participants (63%) exhibited thinking that resulted in incorrect or no solutions to the train problem. These participants followed five different problem solving trajectories (see Figures 25 – 29)

Trajectory 4. Two participants solved the train problem following Trajectory 4 (see Figure 24). Similar to participants who followed Trajectories 1 and 2, these participants exhibited sufficient metacognition in the preparation phase and developed accurate plans. Unlike participants in Trajectories 1, 2, and 3, these two participants made mistakes as they carried out their plans in the performance phase. These mistakes went unnoticed in the performance phase due to a lack of metacognition (i.e., monitoring). Finally, in the evaluation

50

MC?

MC?

MC?

MC?

X

Figure 24. Trajectory 4.

Figure 25. Trajectory 5.

MC?

X

MC?

MC?

X

MC?

MC?

X

X

51

Figure 27. Trajectory 7.

MC?

X

MC?

MC?

X

X

Figure 26. Trajectory 6.

MC?

X

MC?

MC?

X

X

MC?

MC?

52

MC?

MC?

Figure 28. Trajectory 8.

phase, these participants did not verify their solutions and reported incorrect final answers. For participants following Trajectory 4, the absence of metacognition in the performance and evaluations phases (despite sufficient metacognition in the preparation phase) resulted in incorrect solutions. Trajectory 5. Participants who worked the train problem following Trajectory 5 (n = 6) began in the preparation phase similarly to participants who followed Trajectory 3 (see Figure 25). These participants did not develop a sufficient conceptual understanding of the problem and therefore developed inappropriate plans to solve the problem. Also similar to participants who followed Trajectory 3, these participants realized in the performance phase that either their plans were not working or that they were confused and needed to re-evaluate the problem, leading them to return to the performance phase. Unlike participants who followed Trajectory 3, participants who followed Trajectory 5 were unable to use metacognition to develop a deeper understanding of the problem or an appropriate plan. Instead, they developed a second plan that was also incorrect.

Five of the six participants who followed Trajectory 5 carried out their second incorrect plans and reported incorrect answers without using sufficient the necessary metacognition to identify their mistakes in the evaluation phase. One participant realized as he was carrying out his second plan in the performance phase that the plan was not working, and returned to the preparation phase for a third time. Once in the preparation phase, this participant was still unable to use sufficient metacognition and developed a third inappropriate plan which he used, resulting in an incorrect answer. Similar to the other participants who followed this trajectory, he did not use sufficient metacognition in the evaluation phase to verify his answer, resulting in a final incorrect answer.

Jaylen is one of the six participants who followed Trajectory 5.

53

Line 1: [Reads problem aloud] Line 2: So, I’d make a chart and I’d put time and rate and distance. Line 3: So Train A and then Train B. And then it [Train A] goes 50 miles and then Train B

leaves 60 miles. Line 4: And since there’d be time, and then t plus 3 because it’s – it leaves 3 hours later. So

this would be 50t and then 60(t + 3) Line 5: So then it’d be 60t + 180 = 50t. Line 6: And then you’d subtract 50t and then subtract 50t so it’d be 10t + 180 = 0. And

subtract 180, so it’d be -180 = 10t. So you divide by 10, divide by 10, so it’d be t = -18.

Line 7: So then it’d be…so it’d be…so it’d be minus 3 instead of plus 3. [Changes t+3 in chart to t-3, changes equation from 60t + 180 = 50t to 60 – 180 = 50, changes 10t + 180 = 0 to 10t – 180 = 0].

Line 8: So you’d add 180 and then it’d be 18. Line 9: So it’d be 18 hours. Yeah.

Figure 29. Jaylen’s complete algebraic strategy. As Jaylen solves his first equation, he notices that something was wrong.

“[I realized] I didn’t really figure it out. It wouldn’t make sense…since it was -180 and it just would be a really weird answer…you want a positive number because you want to know how long it takes and that’s time. Then it wouldn’t make sense if it was -18 hours.”

Jaylen explains the thinking that led to changing the distance Train B travels from t + 3 to t – 3. Because I was going over my equation and I was thinking where I went wrong and when I checked I saw that I subtracted 180 and that got me a negative. And I was thinking it might be because it was t-3 and I got the minus from distributing 60 into t+3 so I thought of negative 3.

54

His thinking does not lead him to a more complete understanding of the problem or relate the math to the problem. Instead, he is focused on changing the math to give him a positive answer. Once he changes his equation and gets a positive answer, he is satisfied with the answer, and does not go back to verify that this answer maakes sense in terms of the problem.

Trajectory 6. Participants who followed Trajectory 6 (n = 2) used sufficient metacognition in the preparation phase to develop accurate plans (see Figure 26). In the performance phase, both participants decided that their counting strategies were not appropriate for solving the problem (although they were actually correct and working), and returned to the preparation phase. Once back in the preparation phase, the participants lacked the metacognition necessary to develop a second appropriate plan or a deeper understanding of their first plan. Instead, they developed one or more inappropriate plans which they carried out without sufficient metacognition in the performance or evaluation phases, resulting in an incorrect final solution. One participant moved back and forth through the preparation and performance phases multiple times, developing three inappropriate plans in addition to her appropriate plan before reporting her final incorrect solution. The other participant, Fiona, develops only one inappropriate plan following her correct plan, which she carries out resulting in an incorrect solution.

Line 1: [Reads problem aloud] Line 2: K, I’m gonna make a chart. So Train A, Train B. Then rate times time is distance. Line 3: Is that what they’re asking me? Line 4: 3 hours later. Yeah. I think so. Line 5: So 50 miles per hour. Um, this one’s going 60 miles per hour. 3 hours – so, 3 hours.

So if this train started at x, this one started at x+3. Line 6: But that would be how long it was travelling for, and that’s not what they told me.

Figure 30. Fiona’s initial DRT chart. Fiona reveals that her choice to use a DRT chart is based on her memory of past

experiences using DRT charts to solve similar problems at school. She explains that when she is given a rate, “I make a chart so I can see what’s going on and maybe use the rate times time equals distance thing.” However, she has difficulty filling out the chart.

I was thinking they didn’t give me any time for Train A, they just told me it was 50 miles per hour and that would probably be my variable. And it said 3 hours later. I was thinking, ‘ok, it’s x + 3’ and after that I was considering 50x and that would be 60x + 3. And I’m not sure that’s exactly what they want.

Fiona is unable to develop an algebraic strategy so instead, she develops a counting strategy. Line 7: So Train A is on Track X. Train B is on Track Y. Line 8: Ok, well if it’s going 50 miles per hour and I know that 3 hours later Train B left, then

Train A must be going for at least 3 hours. And it if was, then it covered how much

55

distance? 50 times 3 is 150 miles. And then Train B started and was going 60 miles per hour.

Line 9: If you add another hour, then this [Train A] would have gone 200 and this [Train B] would just be 60.

Line 10: If I give it another hour, that’d be 250 and 120. And another hour it’d be 300 and 180. And another one would be 240 and 250

Line 11: This is kindof just guess and check. It’s not really a good way to solve a problem.

Figure 31. Fiona’s counting strategy. Fiona describes her thinking about switching from an algebraic strategy to the counting

strategy, which she labels, “guess and check.” Here I was using guess and check because I couldn’t find a way to set up an equation. And if I could find the answer guessing and checking it, then that would be the only way [to solve the problem]. So I started adding 50 to the bottom and 60 to the top a couple of times and they were quite far apart each time so I figured that I was not getting very close any time soon. So guess and check would not be the best way to solve this.

Fiona’s counting strategy is mathematically correct, and she was on the right track. However, she does not realize this and decides to create an algebraic strategy instead. She explains that her counting strategy “would end up giving me the answer, but I thought it would be better to solve it using an equation because this might take a really long time.” Therefore, Fiona does not think her counting strategy is a realistic and viable way to solve the problem. Next, Fiona returns to her original idea of using a DRT chart to create an algebraic equation. Line 12: So that means, ok, ok. If this [the rates and times in the table] was correct then it’d be

50x and this would be 60(x + 3). Line 13: Which would be 60x + 180 and I guess that would be equal to 50x? Line 14: Would the distances be equal? Line 15: Ok, well, if they were, then 10x = -180. Line 16: But that doesn’t make sense. It can’t be negative. Line 17: I’m not really sure how to solve this.

Figure 32. Fiona’s completed algebraic strategy.

56

Fiona explains that she became stuck when she got a negative solution to her equation. She is unable to find the source of her mistake.

The train couldn’t be travelling for negative amount of time or for a negative distance, so I obviously had done something wrong…I thought maybe there was something wrong with my arithmetic but I checked and it was fine. So I wasn’t really sure [what I had done wrong]…I was pretty sure I was going to have to ask somebody else how to solve this and learn from them because these kind of problems require this kind of method.

After finding a negative solution, Fiona chooses to quit the problem instead of returning to the planning phase or revisiting her counting strategy which would have eventually led her to a correct solution (if she carried out the strategy accurately). She explains why she chose to quit instead of going back to her counting strategy.

I know from previous experiences that that’s not exactly how they want you to solve it. Because I’ve done that on a test before and gotten the right answer but still docked points because that’s not the way they taught me and the way they want me to solve it. So I guess in the real world you could do it that way, but I don’t know…

Trajectory 7. Participants who followed Trajectory 7 (n =7) lacked metacognition in all of the phases of problem solving (see Figure 27). Due to the absence of metacognition in the preparation phase, these participants developed inappropriate plans. Three of these plans were algebraic (i.e., 50x = 60(x - 3), 50x = 60(x + 3), 50x = 60x – 90), three of these plans involved counting strategies that were setup incorrectly (e.g., beginning sequence with Train B at 60 miles when Train A is at 150 miles), and one plan was a variation of the 150/10 strategy (i.e., 180/10 instead of 150/10). In the performance phase, these participants carried out their plans without monitoring their progress, resulting in inaccurate solutions. Finally, these participants lacked the metacognition that would have led them to identify that their solution was incorrect and to return to the preparation phase to re-analyze the problem.

Trajectory 8. Students who followed Trajectory 8 both lacked the most metacognition in the performance phase (see Figure 28). These participants were able to identify the information that was given in the problem, but were unable to conceptually understand the problem and could not generate any plan at all. Instead, these participants chose to terminate the problem in the preparation phase. Faith is one of the participants who followed Trajectory 8.

Line 1: [Reads Problem] Line 2: So Train A leaves UC Berkeley Station travelling at 50 miles…miles per hour. So

that’s Train A and this is Track, this is A and this is going to be Track X. Three hours later, Train B leaves the station travelling 60 miles per hour. So this is 50 miles and B is 60 miles. And this is on a different track, Y, which is parallel to track X, so these two are parallel.

Line 3: So how long does it take B to catch up with A. Line 4: [Writes 50] A can be…50A…60…[pause]…Oh wait. [Crosses out 50A = 60] Line 5: I actually don’t know how to solve these problems. Line 6: AY: Do you want to give it any more tries, or are you done? Line 7: Like, I don’t know how to solve it, like I had this problem at school and I just didn’t

know how to do it. Can I stop with this? I don’t think I can do this.

57

Figure 33. Faith’s attempt to develop a plan. Discussion In Study 2, think-aloud and stimulated-recall interviews were conducted to examine students’ metacognition as they solved mathematics problems. Data from problem solving interviews provided illustrative examples of metacognitive regulation, knowledge, beliefs, and awareness. In each phase, participants’ metacognitive knowledge, beliefs, and awareness contributed to their ability to regulate their thinking and problem solving. The illustrative examples presented in this study may contribute to conceptualizations of metacognition in the field of educational psychology by providing functional and contextual examples of metacognition. Data from problem solving interviews also showed that participants use or non-use of metacognition in the different phases of problem solving resulted in many different problem solving trajectories. Participants’ use of metacognition was consistent across phases in some trajectories, and inconsistent across phases in others. The variety of different trajectories follwed by participants indicate that the relationships between metacognition and problem solving accuracy are complex. Overall, participants’ use of metacognition in each phase had a substantial impact on their problem solving solution accuracy.

58

Study 3 In this study data from Studies 1 and 2 were used to analyze the relationships between

metacognition and academic achievement. The specific goals of this study were (a) to determine the predictive validity of students’ Jr. MAI scores with problem solving metacognition and problem solving accuracy, and (b) to examine the relationship between students’ problem solving metacognition and academic GPA, math grade, MDT score, and summer course grade. Method

Participants and measures. Participants were 30 middle and high school mathematics students attending a summer program for academically talented students who participated in both Studies 1 and 2. See Study 2 Method section for participant demographic information. Variables examined in this study were Jr. MAI scores, metacognition during problem solving, and measures of academic achievement.

Jr. MAI. Metacognition was measured quantitatively using the Jr. MAI (Sperling et al., 2002; see Study 1 for details). Scores for Knowledge and Regulation were derived from the 14 Jr. MAI items that were found to be salient in the exploratory factor analysis in Study 1.

Metacognition during problem solving. Participants’ level of problem solving metacognition was determined based on the metacognition they exhibited as they solved the train problem (see Study 2 for details). Participants who exhibited consistent metacognition across all phases of problem solving (i.e., Trajectories 1 and 2) as well as participants who exhibited strong metacognition in some but not all phases (i.e., Trajectories 3 and 4) were classified as having high problem solving metacognition. Participants whose absence of metacognition resulted in “wild goose chases” (Schoenfeld, 1985, p. 116; i.e., Trajectories 5 and 6) and participants who did not use metacognition in any phase of problem solving (i.e., Trajectories 7 and 8) were classified as having low problem solving metacognition.

Academic achievement. Academic achievement measures examined in this study were GPA, mathematics grade, MDT score, summer course grade (see Study 1 for details), and problem solving solution accuracy. Problem solving accuracy was measured by the total number of mathematics problems participants solved correctly during the problem solving interview. Scores ranged from zero to three problems correct. Results

Mean Jr. MAI subscale scores (i.e., Knowledge and Regulation) and mean academic achievement scores (i.e., GPA, math grade, summer course grade, and MDT score) are presented in Table 9. Frequencies of levels of problem solving metacognition and problem solving accuracy are also presented in Table 9.

Jr. MAI scores, problem solving metacognition, and solution accuracy. The predictive validity of Jr. MAI scores with problem solving metacognition was examined using independent t-tests. The predictive validity of Jr. MAI scores with problem solving accuracy was examined using bivariate correlation analysis.

There was no statistically significant or meaningful difference in Knowledge scores between participants with low problem solving metacognition (μ = 4.13, SD = .38) and participants with high problem solving metacognition (μ = 4.30, SD = .66; p > .05, Cohen’s d = .33). Similarly, there was no statistically significant or meaningful difference in Regulation scores between participants with low problem solving metacognition (μ = 3.72, SD = .69) and participants with high problem solving metacognition (μ = 3.55, SD = .72; p > .05, Cohen’s d =

59

Table 9

Descriptive Statistics of Metacognition and Academic Achievement Variables

n % M SD Range

Jr. MAI

Knowledge 30 -- 4.20 .52 2.86 - 5.00

Regulation 30 -- 3.65 .70 2.14 - 4.86

Problem Solving Metacognition

High 13 43%

Low 17 57%

Academic Achievement

GPA 27 3.90 .19 3.33 - 4.00

Math Grade 27 3.97 .17 3.30 - 4.30

Course Grade 30 3.45 .63 2.00 - 4.30

MDT 30 85.5 9.0 64.0 - 100

Problem Solving Solution Accuracy

0 Problems Correct 6 20%

1 Problem Correct 7 23%

2 Problems Correct 9 30%

3 Problems Correct 8 27%

60

.33). Neither Knowledge scores (r = .04) nor Regulation scores (r = -.07) were significantly correlated with problem solving accuracy.

Problem solving metacognition and academic achievement. Mean academic achievement scores for participants with low and high problem solving metacognition are presented in Table 10. There was no statistically significant or meaningful difference in GPA (p > .05, d = .26) or math grade (p > .05, d = .06) between participants with low and high problem solving metacognition. There was no statistically significant difference in summer course grade or MDT score between participants with low and high problem solving metacognition (p > .05). However, effect sizes indicated that the differences in these latter two variables were meaningfully different. That is, participants with higher problem solving metacognition received higher summer course grades (d = .55) and higher MDT scores (d = .54) than participants with low problem solving metacognition.

Table 10

Mean Academic Achievement Scores by Problem Solving Metacognition

GPA Math Grade CG MDT

n M SD M SD M SD M SD

PS-MC

Low 17 3.88a .22 3.98a .08 3.30 .68 83.4 9.3

High 13 3.93 .15 3.97 .25 3.64 .53 88.18 8.2

Note. CG = Summer course grade. PS-MC = Problem solving metacognition a n = 14

Discussion Results indicated that Jr. MAI scores were not predictive of participants’ use of metacognition during problem solving. They were also not predictive of participants’ problem solving solution accuracy. Participants’ problem solving metacognition did not have a meaningful relationship with their GPA or mathematics grades; however, problem solving metacognition did have a meaningful relationship, captured with medium effect sizes, on summer course grades and MDT scores.

61

General Discussion The two overarching goals of this dissertation were (a) to examine the affordances and

limitations of the ways that educational psychologists operationalize metacognition, measure metacognition, and study the relationships between metacognition and academic achievement, and (b) to consider how theories and methodologies from mathematics education may contribute to the theories and methodologies used by educational psychologists to study metacognition. A synthesis of the educational psychology and mathematics education literatures and three empirical studies were used to address these goals. Defining Metacognition Metacognition within the field of educational psychology has been considered a fuzzy construct. Most researchers in this field (e.g., Brown, 1987; Schraw & Dennison, 1994) define metacognition as metacognitive knowledge and metacognitive regulation, both of which are further defined by taxonomic categories of thinking. Literature from mathematics education and findings from Study 2 may contribute to educational psychologists’ definitions of metacognition in two ways. First, theories from mathematics education broaden the definition of metacognition by including metacognitive beliefs and metacognitive awareness (which were described early on by Flavell [1979] and Brown [1978] but never taken up by contemporary educational psychologists). The examples of students’ thinking during problem solving presented in Study 2 provide support for the inclusion of metacognitive beliefs and metacognitive awareness as important metacognition constructs. Beliefs and awareness contributed both to students’ use of metacognitive regulation as well as to their problem solving accuracy. Second, mathematics education literature may broaden the conceptualizations of metacognition by giving functional definitions of metacognition. Within the field of mathematics education, metacognition is conceptualized as thinking that takes place in specific contexts. Such definitions may help to address the lack of clarity in educational psychology definitions. As educational psychologists continue to develop their conceptualizations of metacognition, they may benefit from exploring the role of different contexts within their definitions of metacognition. Metacognition and Academic Achievement

The relationships between metacognition and academic achievement examined in this study varied based on the operationalizations of metacognition and achievement used. Metacognitive knowledge and regulation, as measured by the Jr. MAI (Sperling et al., 2002) were not significantly or meaningfully related to school achievement (i.e., GPA or math grade), mathematics content mastery (i.e., MDT score), mathematics summer course achievement, or mathematics problem solving accuracy. Similarly, metacognition, as measured by think aloud and stimulated recall during mathematics problem solving tasks, was not significantly or meaningfully related to school achievement. However, metacognition during problem solving was meaningfully related to mathematics content mastery and mathematics summer course achievement. Also, as shown by the eight problem solving trajectories presented in Study 2, there is a complex relationship between problem solving metacognition and problem solving outcomes. In future studies of metacognition and achievement, it will be important to be thoughtful about the ways in which both constructs are operationalized in order to draw meaningful conclusions about their relationships. Measuring Metacognition

Self-report questionnaires are an important methodological tool for studying psychological constructs like metacognition (Desoete & Roeyers, 2006). Findings from Study 1

62

indicated that the scores from the 14-item Jr. MAI were a structurally valid operationalization of metacognitive knowledge and metacognitive regulation as defined by Sperling et al. (2002). This questionnaire is useful because it measures both Knowledge and Regulation among adolescents. However, it is limited with regard to (a) the concurrent and predictive validity of scores with measures of academic achievement and other measures of metacognition and (b) the scope of the metacognition constructs it measures.

Further research is needed to modify the Jr. MAI (Sperling et al., 2002) or to create a new self-report questionnaire that measures aspects of metacognition that have meaningful relationships with achievement and actual metacognitive behaviors. Researchers may also wish to consider including items assessing metacognitive beliefs and metacognitive awareness on this questionnaire. The findings of this dissertation also suggest that mixed method studies may be useful in examining the validity of questionnaires assessing metacognition. Conclusion In this dissertation, I reviewed existing literature and presented three studies that examined metacognition and its relationships to academic achievement. Based on a review of educational psychology and mathematics education literatures, I identified four metacognition constructs related to the types of metacognition (i.e., knowledge, regulation, beliefs, and awareness) that students use during mathematics tasks . In Study 1, I examined the psychometric properties of Jr. MAI scores (Sperling et al., 2002) and concluded that these scores were reliable and structurally valid, but lacked a meaningful relationship with GPA, mathematics grade, MDT score, and summer course grade. In Study 2, I presented examples of how students used metacognitive regulation, metacognitive knowledge, metacognitive beliefs, and metacognitive awareness in each phase of problem solving and presented eight different problem solving trajectories that students followed based on their use of metacognition. In Study 3, I found that students’ Jr. MAI scores were not significantly or meaningfully related to their problem solving metacognition or problem solving accuracy. Students’ problem solving metacognition was not meaningfully related to GPA or mathematics grade, but it was meaningfully related to MDT score and summer course grade.

Metacognition is a complex construct that has been defined and studied in many ways by researchers in the fields of educational psychology. Theories and methodologies from mathematics education may help to clarify to the ways that educational psychologists define and measure metacognition. Using mixed methodologies in future research will allow educational psychologists to better understand metacognition and its relationships to academic achievement.

63

References

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215. doi:10.1037/0033-295X.84.2.191

Bandura, A. (1989). Human agency in social cognitive theory. American Psychologist, 44, 1175-1184. doi:10.1037/0003-066X.44.9.1175

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman and Company.

Betz, N. E., & Hackett, G. (1983). The relationship of mathematics self-efficacy expectations to the selection of science-based college majors. Journal of Vocational Behavior, 23, 329-345. doi:10.1016/0001-8791(83)90046-5

Boekaerts, M., & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and intervention. Applied Psychology, 54, 199-231. doi:10.1111/j.1464-0597.2005.00205.x

Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school (Expanded ed.). Washington, DC: National Academy Press.

Brown, A. L. (1978). Knowing when, where, and how to remember: A problem of metacognition. In R. Glaser (Ed.), Advances in instructional psychology (Vol. 1). Hillsdale, NJ: Lawrence Erlbaum Associates.

Brown, A. L. (1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition, motivation and understanding (pp. 65-116). Hillsdale, NJ: Erlbaum.

Chen, P. P. (2003). Exploring the accuracy and predictability of the self-efficacy beliefs of seventh grade mathematics students. Learning and Individual Differences, 14, 79-92. doi:10.1016/j.lindif.2003.08.003

Cheng, P.-W. (1993). Metacognition and giftedness: The state of the relationship. Gifted Child Quarterly, 37, 105-112. doi:10.1177/001698629303700302

De Corte, E., Op 't Eynde, P., & Verschaffel, L. (2002). "Knowing what to believe": The relevance of students' mathematical beliefs for mathematics education. In B. K. Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.

Desoete, A., & Roeyers, H. (2006). Current issues in the assessment and training of metacognition related to mathematical problem solving. In A. V. Mitel (Ed.), Focus on educational psychology (pp. 251-275). New York: Nova Science Publishers.

Edwards-Leis, C. (2006). Variations to stimulated recall protocols to enhance student reflection: I did, I saw, I remembered. Paper presented at the Australian Association for Research in Education Annual Conference, Adelaide, Australia.

Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (rev. ed.). Cambridge, MA: MIT Press.

Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence. Hillsdale, NJ: Lawrence Erlbaum Associates.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-development inquiry. American Psychologist, 34, 906-911. doi:10.1037/0003-066X.34.10.906

64

Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299. doi:10.1037/1040-3590.7.3.286

Francisco, J. M. (2005). Students' reflections on their learning experiences: Lessons from a longitudinal study on the development of mathematical ideas and reasoning. Journal of Mathematical Behavior, 24, 51-71. doi:10.1016/j.jmathb.2004.12.005

Garofalo, J., & Lester, F. K. (1985). Metacognition, cognitive monitoring, and mathematical performance. Journal for Research in Mathematics Education, 16, 163-176. doi:10.2307/748391

Gaskill, P. J., & Hoy, A. W. (2002). Self-efficacy and self-regulated learning: The dynamic duo in school performance. In J. Aronson (Ed.), Improving academic achievement: Impact of psychological factors on education (pp. 185-208). San Francisco: Academic Press.

Hackett, G., & Betz, N. E. (1989). An exploration of the mathematics self-efficacy/mathematics performance correspondance. Journal for Research in Mathematics Education, 20, 261-273. doi:10.2307/749515

Hammouri, H. A. M. (2003). An investigation of undergraduates' transformational problem solving strategies: Cognitive/metacognitive processes as predictors of holistic/analytic strategies. Assessment and Evaluation in Higher Education, 28, 571-586. doi:10.1080/0260293032000130225

Hayton, J. C., Allen, D. G., & Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7, 191-205. doi:10.1177/1094428104263675

Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67, 88-140. doi:10.3102/00346543067001088

Jacobs, J. E., & Paris, S. G. (1987). Children's metacognition about reading: Issues in definition, measurement, and instruction. Educational Psychologist, 22, 255-278. doi:10.1207/s15326985ep2203&4_4

Klein, A. (1991). Changing directions: A program for academically talented students re-examines its most basic assumptions. Educator, 5, 4-7.

Krathwohl. (2002). A revision of Bloom's taxonomy: An overview. Theory into Practice, 41, 212-218.

Lampert, M. (1990). When the problem is not the question and the solution is not the answer: Mathematical knowing and teaching. American Educational Research Journal, 27, 29-63. doi:10.3102/00028312027001029

Lawson, M. J., & Rice, D. N. (1987). Thinking aloud: Analysing students' mathematical performance. School Psychology International, 8, 233-243.

Li, J. (2003). U.S. and Chinese cultural beliefs about learning. Journal of Educational Psychology, 95, 258-267. doi:10.1037/0022-0663.95.2.258

Li, J. (2004). Learning as a task or virtue: U.S. and Chinese preschoolers explain learning. Developmental Psychology, 40, 595-605. doi:10.1037/0012-1649.40.4.595

Li, J. (2005). Mind or virtue: Western and Chinese beliefs about learning. Current Directions in Psychological Science, 14, 190-194. doi:10.1111/j.0963-7214.2005.00362.x

MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4, 84-99. doi:10.1037/1082-989X.4.1.84

65

Mathematics Diagnostic Testing Project. (2006). The California State University/University of California Mathematics Diagnostic Testing Project Retrieved July 12, 2007, from http://mdtp.ucsd.edu/purposes.shtml

Montague, M., & Applegate, B. (1993). Middle school students' mathematical problem solving: An analysis of think-aloud protocols. Learning Disability Quarterly, 16, 19-32. doi:10.2307/1511157

Muis, K. R. (2004). Personal epistemology and mathematics: A critical review and synthesis of research. Review of Educational Research, 74, 317-377. doi:10.3102/00346543074003317

O'Brien, J. (1993). Action research through stimulated-recall. Research in Science Education, 23, 214-221. doi:10.1007/BF02357063

Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66, 543-578. doi:10.3102/00346543066004543

Pajares, F., & Graham, L. (1999). Self-efficacy, motivation constructs, and mathematics performance of entering middle school students. Contemporary Educational Psychology, 24, 124-139. doi:10.1006/ceps.1998.0991

Pajares, F., & Kranzler, J. (1995). Self-efficacy beliefs and general mental ability in mathematical problem-solving. Contemporary Educational Psychology, 20, 426-443. doi:10.1006/ceps.1995.1029

Pajares, F., & Miller, M. D. (1994). Role of self-efficacy and self-concept beliefs in mathematical problem solving: A path analysis. Journal of Educational Psychology, 86, 193-203. doi:10.1037/0022-0663.86.2.193

Pietsch, J., Walker, R., & Chapman, E. (2003). The relationship among self-concept, self-efficacy, and performance in mathematics during secondary school. Journal of Educational Psychology, 95, 589-603. doi:10.1037/0022-0663.95.3.589

Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41, 219-225. doi:10.1207/s15430421tip4104_3

Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16, 385-407. doi:10.1007/s10648-004-0006-x

Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40. doi:10.1037/0022-0663.82.1.33

Pintrich, P. R., Smith, D. A. A., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire. Educational and Psychological Measurement, 53, 801-813. doi:10.1177/0013164493053003024

Polya, G. (1945). How to solve it: A new aspect of mathematical method. Princeton, NJ: Princeton University Press.

Puustinen, M., & Pulkkinen, L. (2001). Models of self-regulated learning: A review. Scandinavian Journal of Educational Research, 45, 269-286. doi:10.1080/00313830120074206

Randhawa, B. S., Beamer, J. E., & Lundberg, I. (1993). Role of mathematics self-efficacy in the structural model of mathematics achievement. Journal of Educational Psychology, 85, 41-48. doi:10.1037/0022-0663.85.1.41

66

Santos-Trigo, M. (1996). An exploration of strategies used by students to solve problems with multiple ways of solution. Journal of Mathematical Behavior, 15, 263-284. doi:10.1016/S0732-3123(96)90006-1

Schoenfeld, A. H. (1981). Episodes and executive decisions in mathematical problem solving. Paper presented at the Annual Meeting of the American Educational Research Association, Los Angeles, CA.

Schoenfeld, A. H. (1985). Mathematical problem solving. Orlando, FL: Academic Press, Inc. Schoenfeld, A. H. (1987). What's all the fuss about metacognition? In A. H. Schoenfeld (Ed.),

Cognitive science and mathematics education (pp. 189-215). Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Schoenfeld, A. H. (1988). When good teaching leads to bad results: The disasters of "well-taught" mathematics courses. Educational Psychologist, 23, 145-166. doi:10.1207/s15326985ep2302_5

Schoenfeld, A. H. (1989). Explorations of students' mathematical beliefs and behavior. Journal for Research in Mathematics Education, 20, 338-355. doi:10.2307/749440

Schoenfeld, A. H. (1992). Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics. In D. Grouws (Ed.), Handbook for research on mathematics teaching and learning. New York: Macmillan.

Schoenfeld, A. H. (2002). How can we examine the connections between teachers' world views and their educational practices. Issues in Education, 8, 217-227.

Schommer, M. (1994). Synthesizing epistemological belief research: Tentative understandings and provocative confusions. Educational Psychology Review, 6, 293-319. doi:10.1007/BF02213418

Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460-475. doi:10.1006/ceps.1994.1033

Schraw, G., & Graham, T. (1997). Helping gifted students develop metacognitive awareness. Roeper Review, 20, 4-8.

Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational Psychology Review, 7, 351-371. doi:10.1007/BF02212307

Sperling, R. A., Howard, B. C., Miller, L. A., & Murphy, C. (2002). Measures of children's knowledge and regulation of cognition. Contemporary Educational Psychology, 27, 51-79. doi:10.1006/ceps.2001.1091

Sternberg, R. J. (1980). Sketch of a componential subtheory of human intelligence. The Behavioral and Brain Sciences, 3, 573-614.

Swanson, H. L. (1990). Influence of metacognitive knowledge and aptitude on problem solving. Journal of Educational Psychology, 82, 306-314. doi:10.1037/0022-0663.82.2.306

Watkins, M. W. (2000). Monte Carlo PCA for parallel analysis. State College, PA: Ed & Psych Associates.

Wilson, J., & Clarke, D. (2004). Towards the modelling of mathematical metacognition. Mathematics Education Research Journal, 16, 25-48.

Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25, 3-17. doi:10.1207/s15326985ep2501_2

Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41, 64-70. doi:10.1207/s15430421tip4102_2

67

Appendix A

Jr. Metacognitive Awareness Inventory (Jr. MAI, Sperling et al., 2002)

We are interested in what learners do when they are solving math problems. Please read the following sentences and fill in the bubble for each answer that relates to you and the way you are when you are solving math problems for school work or home work. Please answer as honestly as possible. 1 = Never 2 = Seldom 3 = Sometimes 4 = Often 5 = Always 1 2 3 4 5

1. I am a good judge of how well I understand something O O O O O

2. I can motivate myself to learn when I need to O O O O O

3. I try to use strategies that have worked in the past O O O O O

4. I know what the teacher expects me to learn O O O O O

5. I learn best when I know something about the topic O O O O O

6. I draw pictures or diagrams to help me understand while learning O O O O O

7. I ask myself if I learned as much as I could have once I finish a task O O O O O

8. I ask myself if I have considered all options when solving a problem O O O O O

9. I think about what I really need to learn before I begin a task O O O O O

10. I ask myself questions about how well I am doing while I am learning

something new

O O O O O

11. I focus on the meaning and significance of new information O O O O O

12. I learn more when I am interested in the topic O O O O O

13. I use my intellectual strengths to compensate for my weaknesses O O O O O

14. I have control over how well I learn O O O O O

15. I ask myself periodically if I am meeting my goals O O O O O

16. I find myself using helpful learning strategies automatically O O O O O

17. I ask myself if there was an easier way to do thing after I finish a task O O O O O

18. I set specific goals before I begin a task O O O O O

68

Appendix B

Problem Solving Interview Script

A. Introduction My name is Adena Young, and this is my assistant Leo White. We are graduate students at UC Berkeley, and we are interested in knowing more about how students like you think while you are doing math. For your interview today, I am going to ask you to solve some math problems. As you are doing each problem, I am going to ask you to think aloud. What I mean by think aloud is that I want you to tell me everything you are thinking from the time you first see the question until you are finished with the problem. You do not need to plan out what you say, or try to explain to me what you are saying, just pretend like you are alone and speaking to yourself. I will give you one practice problem so you can practice thinking aloud while you do math, then I will ask you to solve three more math problems. After you are finished with all of the problems, we will watch your interview on the screen, and I will ask you to tell me about what you were doing, thinking, and feeling as you solved each problem. If at any time during the interview you want to stop or need to take a break, just let me know. There are just a couple more things before we get started. Do you have a cell phone? Please make sure that your cell phone is off if you have one. Since we are video taping you doing math aloud, it’s important that you speak loudly and clearly, and that you keep your paper in this area (indicate the area on the table) so we can get all of your writing on tape. Some kids get nervous around cameras, but I want you to try your best to do these math problems just like you would solve problems at school or at home for homework and to ignore the cameras. If you start to get nervous, please let me know. Do you understand what we are going to do? Do you still agree to participate in this research study? B. Concurrent Report Section B1. Practice Problem We will begin with a practice problem to help you practice thinking aloud. Please solve this problem while thinking aloud. Remember say everything you are thinking from the time you first see this question until you are finished with the problem. You may write down as much as you like on your paper, and let me know if you need more paper. [Present Practice Problem] Rope costs $1.50 per foot. How much money will Michael need to spend for rope in order to enclose a 70-foot by 30-foot garden? Thank you. Next I will have you solve three more problems. I will ask you to do the same thing that you did for this practice problem.

69

B2.1. Problem 1 Please solve this first problem while thinking aloud. Remember say everything you are thinking from the time you first see this question until you are finished with the problem. [Present Problem 1. Allow participant to solve problem.] Train A leaves UC Berkeley station travelling at 50 miles an hour on Track X. Three hours later, Train B leaves the station travelling 60 miles an hour on Track Y, which is parallel to Track X. How long does it take Train B to catch up with Train A? B2.2. Problem 2

Please solve this second problem while thinking aloud. Remember say everything you are thinking from the time you first see this question until you are finished with the problem. [Present Problem 2. Allow participant to solve problem.] Jo has three numbers which she adds together in pairs. When she does this she has three different totals: 11, 17 and 22. What are the three numbers Jo had to start with? B2.3. Problem 3

Please solve this last problem while thinking aloud. Remember say everything you are thinking from the time you first see this question until you are finished with the problem. [Present Problem 3. Allow participant to solve problem.] Find the area of DEBF.

F

A B

CD

E

70

C. Stimulated Recall Section Next I am going to play back the video recording of your interview so far. As we watch the video, I’d like you to tell me the things you were doing, thinking, and feeling from the time that you first saw the problem until when you were finished with the problem. Any time you want to tell me what you were doing or thinking just pause the video. I may also pause the video to ask you what you were doing or thinking. Do you have any questions? Initial Question: What was your first thought when you saw this problem? Questions to further thinking: What were you doing/thinking/feeling here/at this point/right then?” Why did you… How did you decide to… Final Questions: Is there anything else you were thinking or feeling when you solved this problem? What did you think about this problem? D. End of Interview That is the end of the interview. Thank you for your participation. As compensation for your time and effort, you are receiving $10. I will need you to sign a receipt saying that you have received this money. Thank you again for your participation in this study.