assessment of preservice teachers’ mathematics knowledge for teaching the middle grades margaret...

34
Assessment of Preservice Assessment of Preservice Teachers’ Mathematics Teachers’ Mathematics Knowledge for Teaching Knowledge for Teaching the Middle Grades the Middle Grades Margaret J. Mohr Texas A&M University February 23, 2006

Upload: bartholomew-ball

Post on 28-Dec-2015

218 views

Category:

Documents


0 download

TRANSCRIPT

Assessment of Preservice Assessment of Preservice Teachers’ Mathematics Teachers’ Mathematics Knowledge for Teaching Knowledge for Teaching

the Middle Grades the Middle Grades

Margaret J. MohrTexas A&M UniversityFebruary 23, 2006

IntroductionIntroduction• In relation to education reform

recommendations, a survey of selected teacher programs in the U.S. revealed few changes in program characteristics (Graham, Li, & Buck, 2000 ).

• At the middle school level, 68.5% of teachers have no major certification in mathematics and 21.9% do not have a minor in mathematics (Seastrom, Gruber, Henke, McGrath, & Cohen, 2005).

Introduction, cont’d.Introduction, cont’d.• “To deliver the kind of mathematics content in

ways that respects middle grades students as learners demands a well prepared and motivated teacher. Few existing teacher preparation programs meet this need, and certification requirements do not support adequate content and pedagogical preparation” (NRC, 2000a, p. 15).

• The improvement of middle school mathematics teacher preparation must be grounded in research that provides a theoretical understanding of what prospective mathematics teachers for middle grades need to learn, how they learn it, and how their learning can be assessed (Kulm, Li, Allen, Goldsby, & Willson, 2005).

Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching

• RAND Mathematics Science Panel (2003) report found a compelling relationship between what a teacher can do with their students and their own level of mathematics competence.

• “either teachers do not have enough content knowledge, or what they do know is not the ‘right’ content knowledge” (Sherin, 2002, p. 123)

Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching

• NCTM (2000) emphasized teachers need different kinds of knowledge– Knowledge of specific content– Curricular goals– The challenges students face in learning these ideas– Assessment– Pedagogical knowledge of effective teaching strategies

• “There is a positive connection between subject matter preparation (in both content and specific teaching methods) and teacher performance; however, for some subjects, like mathematics, current subject matter preparation (including an academic subject major) may need to be reformed to increase reasoning skills and conceptual knowledge” (ASCD, 2003, p. 1)

Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching

• Shulman (1986) introduced “pedagogical content knowledge”– A conceived complementary relationship

between the pedagogical knowledge and the content knowledge of the subject area

• Specific measures were not yet in place in mathematics education (Hill, Schilling, & Ball, 2004)– Set out to map out what elementary teachers

knew

Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching

• Hill, Schilling, and Ball (2004) found through their multiple-choice assessment that teachers’ mathematics knowledge for teaching (PCK) elementary grades was partly domain specific rather than relating to their teaching or mathematical ability

• In their study of first- and third-grade teachers, they found that teachers’ mathematical knowledge for teaching was significantly related to student achievement gains in both grade levels (Hill, Rowan, & Ball, 2005)

Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching

• “By mathematical knowledge for teaching, we mean the mathematical knowledge used to carry out the work of teaching mathematics. Examples of this “work of teaching” include explaining terms and concepts to students, interpreting students’ statements and solutions, judging and correcting textbook treatments of particular topics, using representations accurately in the classroom, and effects of teachers’ mathematical knowledge on student achievement providing students with examples of mathematical concepts, algorithms, or proofs” (Hill, Rowan, & Ball, 2005).

• The implementation of this definition may very well be the beginnings of the reform the ASCD (2003) discusses.

Statement of the Statement of the ProblemProblem

• Given the importance of mathematical content knowledge and pedagogical content knowledge in preparing preservice teachers, it is essential to know the nature of this mathematics knowledge for teaching– Four main content strands: algebra, statistics,

geometry, and number and operations

• Help improve middle grades mathematics teacher preparation programs

• Not only must the teacher understand the material they are teaching, they must be able to communicate it to the students as well.

Significance of StudySignificance of Study• Previous research has provided insights into specific areas

or parts of teacher knowledge (i.e., Carter, 2005)– No studies have been undertaken to collectively evaluate

mathematics knowledge for teaching at the middle grades levels and across different cohorts.

• The design of the study will allow for a snapshot of the development of mathematics knowledge for teaching middle grades during teacher preparation.

• The current preservice middle grades mathematics certification bachelor degree program at Texas A&M University represents a model for teacher preparation recommended by professional organizations (CBMS, 2001).

• Research on this model will produce results which are intellectually and scientifically sound; contribute to teacher preparation; and have significant implications for current and future teacher preparation programs.

Purpose of the StudyPurpose of the StudyThe purpose of this study is to:• To develop an assessment instrument that

effectively evaluates and allows prospective mathematics teachers to demonstrate their mathematics knowledge for teaching

• Examine differences in mathematics knowledge for teaching in four different content strands across five different cohorts of preservice middle grades mathematics teachers at Texas A&M University

Goal of the StudyGoal of the Study• The ultimate goal of this study is to

provide:– An effective performance assessment

instrument, – Data on preservice teachers’

mathematics knowledge for teaching, and

– Recommendations that may be used to help shape reform initiatives in teacher education programs throughout the United States

Research QuestionsResearch Questions• What is preservice middle grades

teachers’ mathematics knowledge for teaching number and operations?

• What is preservice middle grades teachers’ mathematics knowledge for teaching algebra?

• What is preservice middle grades teachers’ mathematics knowledge for teaching geometry?

• What is preservice teachers’ mathematics knowledge for teaching statistics?

Ancillary QuestionsAncillary Questions• What is the effect of various sequencing of

mathematics courses on middle grades mathematics teachers?

• What individual developmental differences are there among prospective teachers as they progress through the courses identified in the middle grades mathematics program?

• Do some types of courses (e.g. algebra, geometry, numerical, statistical or applied, theoretical) have more impact than others upon the development of a teacher’s mathematics knowledge for teaching (MKT)?

• Does development of mathematics knowledge for teaching happen at greater rates at certain stages of the middle grades mathematics program than others?

Research DesignResearch Design

• Intent is to assess the nature of mathematics knowledge for teaching middle grades across five cohorts (shown in future slide)

• Study will utilize a mixed-model design

PopulationPopulation• Include all students who indicate their intent to

be teachers by enrolling in the first specialized mathematics course at Texas A&M University– MATH 365—Structure of Math I

• MATH 365, MATH 366, MATH 367, MATH 368, MATH 403, MASC 351, MASC 450, MEFB 460, AND MEFB 497

• 586 total students Spring 2005

• 590 total enrollment for Spring 2006– Number will be less due to overlapping of students

for some courses

Cohort Course Spring ’05

Spring ‘06

1Mathematics Courses I

MATH 365: Structure of Math I

127 124

MATH 366: Structure of Math II

152 140

2Mathematics Courses II

MATH 367: Basics Concepts of Geometry

? 18

MATH 368: Introduction to Abstract Math

47 47

MATH 403: Math and Technology

64 44

3Mathematics Education

Courses

MASC 351: Problem Solving 37 28MASC 450: Integrated Mathematics

40 56

4Methods Courses

MEFB 460: Methods of Teaching Middle Grades Mathematics

66 36

5Student

Teaching

MEFB 497: Teaching Middle Grades

53 97 TOTAL Participants: 590

InstrumentationInstrumentation• Pilot tested

• Utilize ExamView Pro® Test Generator– Test given on the computer via the internet

• Create a separate bank of questions for each content strand– Number and operation, geometry, statistics, and

algebra

• Students randomly assigned a test from two different test banks – Even amount from each content area—7 questions– Example: one student will take a test consisting of

geometry and statistics problems; another, algebra and geometry; etc.

Instrumentation, cont’dInstrumentation, cont’dContent will be different for each

question; all will have the following parts:

I. Answer the above content knowledge question and explain your solution of the problem.

II. How would you explain, model, and/or demonstrate this item to someone who did not understand?

Instrumentation, cont’dInstrumentation, cont’dExample Question:

The variables a, b, c, and d each represent a different whole number. Given a = 3, use the properties of whole numbers to determine the value for each variable.

I. Determine the value for each variable and explain your solution of the problem.

II. How would you explain, model, and/or demonstrate this item to someone who did not understand?

d + d = c

a ∙ d = a

c + d = a

c ∙ b = b

Instrumentation, cont’dInstrumentation, cont’d• Content questions from:

– Balanced Assessment (1999) series

– New York State Testing Program (1998) Grade 8 Test Sampler Draft

– Learning from Assessment (Madfes & Muench, 1999)

• Rubrics from:– New York State Published State Assessment

Rubrics and Exemplar Papers– NCTM’s (2004) Mathematics Assessment: A

Practical Handbook for Grades 6-8

Procedures and Collection Procedures and Collection of the Dataof the Data

• Primary source of data will be the assessment

• Assessment will be a required part of the courses involved– Given right after mid-semester of their current courses

• Assessment delivered ExamView Pro® Software via computer via internet

• Randomly will assign 14 questions from an already-made test bank over two different content strands– Number and operation, algebra, geometry, and/or

statistics

Procedures and Collection Procedures and Collection of the Dataof the Data

• Question asked at beginning of test will identify overlapping students

• If student has already taken test, will still take it again, but the first time taken will be the only data part of the study

• Also will be asked to provide the following information: Math GPR, current courses (identified from a given list), past courses (identified from a given list), gender, and race

Analysis of the DataAnalysis of the Data• Mixed-Model design—data analyzed

quantitatively and qualitatively

• All the open-ended answers will be numerically scored according to their corresponding rubric– The first 5 assessments will be scored together in

order to obtain consistency– Intra-rater and inter-rater reliability will be calculated

and reported

• Rubrics are being adapted from current valid and reliable sources– Both the NCTM and state of New York have field-

tested their rubrics several times; are published; currently implemented across the country

Analysis of Data, cont’dAnalysis of Data, cont’d• Quantitative data will be analyzed using

multivariate analysis– Possibly MANCOVA

• Cohort 1 can be thought of as the pretest and Cohort 5 can be thought of as the posttest– TASP scores will be used in order to control for any

mathematical and/or educational differences

• Sample size should be around 450-500 with a majority of the students being found in the first two cohorts

• Determined the number of assessment questions needed to gain an adequate effect size at the end of the study based on the number of students in the last 3 cohorts– Cohorts will be restructured if after the initial analysis of

the data, there is not enough power in the last two cohorts

Analysis of the Data, Analysis of the Data, cont’dcont’d

• Analyze open-ended questions qualitatively also– Emergent themes and/or trend– Constant comparison (Denzin &

Lincoln, 2000)

• Follow-up interviews with 5-10 students– Verbally asking the same and/or similar

content questions

Definition of TermsDefinition of Terms• Algebra: the branch of mathematics that deals with

computations performed with variables, usually designated by letters which stand for numbers in arithmetic operations such as addition and multiplication. It is a “generalized arithmetic…a structured system for formulating and manipulating variables and formal mathematical statements” (Driscolli, 1988, p. 119).

• Assessment: the process of collecting, interpreting, and synthesizing information to aid in decision making (Airasian, 1991). – The purpose of assessment is to “find out what each

student is able to do, with knowledge, in context” (Wiggins, 1996/1997, p. 19)

• Content: what preservice teachers should demonstrate that they know, think, or can do (Martin-Kniep, 2005).

Definition of Terms, Definition of Terms, cont’dcont’d

• Criterion: a standard on which a judgment or decision may be based.

• Form: what perservice teachers’ products, performances, or processes should look like (Martin-Kniep, 2005).

• High-stakes test: “an assessment for which important consequences ride on the test’s results” (Popham, 2005, p. 15).

Definition of Terms, Definition of Terms, cont’dcont’d

• Mathematical achievement: the attainment and success of a student in mathematics as indicated by scores that go beyond the number of correct responses. One example of mathematical achievement is a score on a text-embedded chapter test.

• Middle grades certification: Math/Science specialist program: Middle grades certification program leading to the Bachelor of Science degree (B.S.) with a major in Interdisciplinary Studies (INST) through the Middle Grades Certification Program in the Department of Teaching, Learning and Culture at Texas A&M University. – It is a field-based program, with students spending

extensive time in middle schools. – Credit hours required for graduation in the

Mathematics/Science strand total 133-134 credit hours.

Definition of Terms, Definition of Terms, cont’dcont’d

• Mathematics knowledge for teaching: the mathematical knowledge “used to carry out the work of teaching mathematics” (Hill, Rowan, & Ball, 2005, p. 373).

• NCTM: The National Council of Teachers of Mathematics, an organization composed of classroom teachers, supervisors, educational researchers, teacher educators, university mathematicians, and administrators involved in the mathematics education of students.

• Performance Assessment: a form of testing requiring students to demonstrate their achievement of understandings and skills by completing a demanding task(s) in which they are asked to respond to the task(s) orally, in writing, or by constructing a product (Gronlund, 2003; Nitko, 2001; Popham, 2005).

• “Specialized” content knowledge: mathematical knowledge, not pedagogy (Hill, Rowan, & Ball, 2005).

DelimitationsDelimitations• Not possible to test all preservice

middle grades mathematics teachers

• Study is limited to the number of preservice teachers and topics feasibly available

• Study did not consider gender or ethnicity

Cohort Course Spring ’05

Spring ‘06

1Mathematics Courses I

MATH 365: Structure of Math I

127 124

MATH 366: Structure of Math II

152 140

2Mathematics Courses II

MATH 367: Basics Concepts of Geometry

? 18

MATH 368: Introduction to Abstract Math

47 47

MATH 403: Math and Technology

64 44

3Mathematics Education

Courses

MASC 351: Problem Solving 37 28MASC 450: Integrated Mathematics

40 56

4Methods Courses

MEFB 460: Methods of Teaching Middle Grades Mathematics

66 36

5Student

Teaching

MEFB 497: Teaching Middle Grades

53 97 TOTAL Participants: 590

Conclusions/Conclusions/ImplicationsImplications

• There are no studies to directly compare the results to– Hard to initially generalize the results to the population

• More longitudinal testing will have to be completed in order for the results to become widely accepted amongst mathematics educators.

• Some of the results may be comparable to similar studies conducted at the elementary levels such as those by Hill, Rowan, and Ball (2005).

• Other comparisons could possibly be made to assessment efforts and findings of Bill Bush, University of Louisville.

Thank You!Thank You!

Margaret J. MohrTexas A&M University

Department of Teaching, Learning and CultureMSMP Project Supervisor

Research Assistant and Assistant to Dr. Gerald Kulm

[email protected]