assessment of preservice teachers’ mathematics knowledge for teaching the middle grades margaret...
TRANSCRIPT
Assessment of Preservice Assessment of Preservice Teachers’ Mathematics Teachers’ Mathematics Knowledge for Teaching Knowledge for Teaching
the Middle Grades the Middle Grades
Margaret J. MohrTexas A&M UniversityFebruary 23, 2006
IntroductionIntroduction• In relation to education reform
recommendations, a survey of selected teacher programs in the U.S. revealed few changes in program characteristics (Graham, Li, & Buck, 2000 ).
• At the middle school level, 68.5% of teachers have no major certification in mathematics and 21.9% do not have a minor in mathematics (Seastrom, Gruber, Henke, McGrath, & Cohen, 2005).
Introduction, cont’d.Introduction, cont’d.• “To deliver the kind of mathematics content in
ways that respects middle grades students as learners demands a well prepared and motivated teacher. Few existing teacher preparation programs meet this need, and certification requirements do not support adequate content and pedagogical preparation” (NRC, 2000a, p. 15).
• The improvement of middle school mathematics teacher preparation must be grounded in research that provides a theoretical understanding of what prospective mathematics teachers for middle grades need to learn, how they learn it, and how their learning can be assessed (Kulm, Li, Allen, Goldsby, & Willson, 2005).
Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching
• RAND Mathematics Science Panel (2003) report found a compelling relationship between what a teacher can do with their students and their own level of mathematics competence.
• “either teachers do not have enough content knowledge, or what they do know is not the ‘right’ content knowledge” (Sherin, 2002, p. 123)
Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching
• NCTM (2000) emphasized teachers need different kinds of knowledge– Knowledge of specific content– Curricular goals– The challenges students face in learning these ideas– Assessment– Pedagogical knowledge of effective teaching strategies
• “There is a positive connection between subject matter preparation (in both content and specific teaching methods) and teacher performance; however, for some subjects, like mathematics, current subject matter preparation (including an academic subject major) may need to be reformed to increase reasoning skills and conceptual knowledge” (ASCD, 2003, p. 1)
Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching
• Shulman (1986) introduced “pedagogical content knowledge”– A conceived complementary relationship
between the pedagogical knowledge and the content knowledge of the subject area
• Specific measures were not yet in place in mathematics education (Hill, Schilling, & Ball, 2004)– Set out to map out what elementary teachers
knew
Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching
• Hill, Schilling, and Ball (2004) found through their multiple-choice assessment that teachers’ mathematics knowledge for teaching (PCK) elementary grades was partly domain specific rather than relating to their teaching or mathematical ability
• In their study of first- and third-grade teachers, they found that teachers’ mathematical knowledge for teaching was significantly related to student achievement gains in both grade levels (Hill, Rowan, & Ball, 2005)
Mathematics Knowledge Mathematics Knowledge for Teachingfor Teaching
• “By mathematical knowledge for teaching, we mean the mathematical knowledge used to carry out the work of teaching mathematics. Examples of this “work of teaching” include explaining terms and concepts to students, interpreting students’ statements and solutions, judging and correcting textbook treatments of particular topics, using representations accurately in the classroom, and effects of teachers’ mathematical knowledge on student achievement providing students with examples of mathematical concepts, algorithms, or proofs” (Hill, Rowan, & Ball, 2005).
• The implementation of this definition may very well be the beginnings of the reform the ASCD (2003) discusses.
Statement of the Statement of the ProblemProblem
• Given the importance of mathematical content knowledge and pedagogical content knowledge in preparing preservice teachers, it is essential to know the nature of this mathematics knowledge for teaching– Four main content strands: algebra, statistics,
geometry, and number and operations
• Help improve middle grades mathematics teacher preparation programs
• Not only must the teacher understand the material they are teaching, they must be able to communicate it to the students as well.
Significance of StudySignificance of Study• Previous research has provided insights into specific areas
or parts of teacher knowledge (i.e., Carter, 2005)– No studies have been undertaken to collectively evaluate
mathematics knowledge for teaching at the middle grades levels and across different cohorts.
• The design of the study will allow for a snapshot of the development of mathematics knowledge for teaching middle grades during teacher preparation.
• The current preservice middle grades mathematics certification bachelor degree program at Texas A&M University represents a model for teacher preparation recommended by professional organizations (CBMS, 2001).
• Research on this model will produce results which are intellectually and scientifically sound; contribute to teacher preparation; and have significant implications for current and future teacher preparation programs.
Purpose of the StudyPurpose of the StudyThe purpose of this study is to:• To develop an assessment instrument that
effectively evaluates and allows prospective mathematics teachers to demonstrate their mathematics knowledge for teaching
• Examine differences in mathematics knowledge for teaching in four different content strands across five different cohorts of preservice middle grades mathematics teachers at Texas A&M University
Goal of the StudyGoal of the Study• The ultimate goal of this study is to
provide:– An effective performance assessment
instrument, – Data on preservice teachers’
mathematics knowledge for teaching, and
– Recommendations that may be used to help shape reform initiatives in teacher education programs throughout the United States
Research QuestionsResearch Questions• What is preservice middle grades
teachers’ mathematics knowledge for teaching number and operations?
• What is preservice middle grades teachers’ mathematics knowledge for teaching algebra?
• What is preservice middle grades teachers’ mathematics knowledge for teaching geometry?
• What is preservice teachers’ mathematics knowledge for teaching statistics?
Ancillary QuestionsAncillary Questions• What is the effect of various sequencing of
mathematics courses on middle grades mathematics teachers?
• What individual developmental differences are there among prospective teachers as they progress through the courses identified in the middle grades mathematics program?
• Do some types of courses (e.g. algebra, geometry, numerical, statistical or applied, theoretical) have more impact than others upon the development of a teacher’s mathematics knowledge for teaching (MKT)?
• Does development of mathematics knowledge for teaching happen at greater rates at certain stages of the middle grades mathematics program than others?
Research DesignResearch Design
• Intent is to assess the nature of mathematics knowledge for teaching middle grades across five cohorts (shown in future slide)
• Study will utilize a mixed-model design
PopulationPopulation• Include all students who indicate their intent to
be teachers by enrolling in the first specialized mathematics course at Texas A&M University– MATH 365—Structure of Math I
• MATH 365, MATH 366, MATH 367, MATH 368, MATH 403, MASC 351, MASC 450, MEFB 460, AND MEFB 497
• 586 total students Spring 2005
• 590 total enrollment for Spring 2006– Number will be less due to overlapping of students
for some courses
Cohort Course Spring ’05
Spring ‘06
1Mathematics Courses I
MATH 365: Structure of Math I
127 124
MATH 366: Structure of Math II
152 140
2Mathematics Courses II
MATH 367: Basics Concepts of Geometry
? 18
MATH 368: Introduction to Abstract Math
47 47
MATH 403: Math and Technology
64 44
3Mathematics Education
Courses
MASC 351: Problem Solving 37 28MASC 450: Integrated Mathematics
40 56
4Methods Courses
MEFB 460: Methods of Teaching Middle Grades Mathematics
66 36
5Student
Teaching
MEFB 497: Teaching Middle Grades
53 97 TOTAL Participants: 590
InstrumentationInstrumentation• Pilot tested
• Utilize ExamView Pro® Test Generator– Test given on the computer via the internet
• Create a separate bank of questions for each content strand– Number and operation, geometry, statistics, and
algebra
• Students randomly assigned a test from two different test banks – Even amount from each content area—7 questions– Example: one student will take a test consisting of
geometry and statistics problems; another, algebra and geometry; etc.
Instrumentation, cont’dInstrumentation, cont’dContent will be different for each
question; all will have the following parts:
I. Answer the above content knowledge question and explain your solution of the problem.
II. How would you explain, model, and/or demonstrate this item to someone who did not understand?
Instrumentation, cont’dInstrumentation, cont’dExample Question:
The variables a, b, c, and d each represent a different whole number. Given a = 3, use the properties of whole numbers to determine the value for each variable.
I. Determine the value for each variable and explain your solution of the problem.
II. How would you explain, model, and/or demonstrate this item to someone who did not understand?
d + d = c
a ∙ d = a
c + d = a
c ∙ b = b
Instrumentation, cont’dInstrumentation, cont’d• Content questions from:
– Balanced Assessment (1999) series
– New York State Testing Program (1998) Grade 8 Test Sampler Draft
– Learning from Assessment (Madfes & Muench, 1999)
• Rubrics from:– New York State Published State Assessment
Rubrics and Exemplar Papers– NCTM’s (2004) Mathematics Assessment: A
Practical Handbook for Grades 6-8
Procedures and Collection Procedures and Collection of the Dataof the Data
• Primary source of data will be the assessment
• Assessment will be a required part of the courses involved– Given right after mid-semester of their current courses
• Assessment delivered ExamView Pro® Software via computer via internet
• Randomly will assign 14 questions from an already-made test bank over two different content strands– Number and operation, algebra, geometry, and/or
statistics
Procedures and Collection Procedures and Collection of the Dataof the Data
• Question asked at beginning of test will identify overlapping students
• If student has already taken test, will still take it again, but the first time taken will be the only data part of the study
• Also will be asked to provide the following information: Math GPR, current courses (identified from a given list), past courses (identified from a given list), gender, and race
Analysis of the DataAnalysis of the Data• Mixed-Model design—data analyzed
quantitatively and qualitatively
• All the open-ended answers will be numerically scored according to their corresponding rubric– The first 5 assessments will be scored together in
order to obtain consistency– Intra-rater and inter-rater reliability will be calculated
and reported
• Rubrics are being adapted from current valid and reliable sources– Both the NCTM and state of New York have field-
tested their rubrics several times; are published; currently implemented across the country
Analysis of Data, cont’dAnalysis of Data, cont’d• Quantitative data will be analyzed using
multivariate analysis– Possibly MANCOVA
• Cohort 1 can be thought of as the pretest and Cohort 5 can be thought of as the posttest– TASP scores will be used in order to control for any
mathematical and/or educational differences
• Sample size should be around 450-500 with a majority of the students being found in the first two cohorts
• Determined the number of assessment questions needed to gain an adequate effect size at the end of the study based on the number of students in the last 3 cohorts– Cohorts will be restructured if after the initial analysis of
the data, there is not enough power in the last two cohorts
Analysis of the Data, Analysis of the Data, cont’dcont’d
• Analyze open-ended questions qualitatively also– Emergent themes and/or trend– Constant comparison (Denzin &
Lincoln, 2000)
• Follow-up interviews with 5-10 students– Verbally asking the same and/or similar
content questions
Definition of TermsDefinition of Terms• Algebra: the branch of mathematics that deals with
computations performed with variables, usually designated by letters which stand for numbers in arithmetic operations such as addition and multiplication. It is a “generalized arithmetic…a structured system for formulating and manipulating variables and formal mathematical statements” (Driscolli, 1988, p. 119).
• Assessment: the process of collecting, interpreting, and synthesizing information to aid in decision making (Airasian, 1991). – The purpose of assessment is to “find out what each
student is able to do, with knowledge, in context” (Wiggins, 1996/1997, p. 19)
• Content: what preservice teachers should demonstrate that they know, think, or can do (Martin-Kniep, 2005).
Definition of Terms, Definition of Terms, cont’dcont’d
• Criterion: a standard on which a judgment or decision may be based.
• Form: what perservice teachers’ products, performances, or processes should look like (Martin-Kniep, 2005).
• High-stakes test: “an assessment for which important consequences ride on the test’s results” (Popham, 2005, p. 15).
Definition of Terms, Definition of Terms, cont’dcont’d
• Mathematical achievement: the attainment and success of a student in mathematics as indicated by scores that go beyond the number of correct responses. One example of mathematical achievement is a score on a text-embedded chapter test.
• Middle grades certification: Math/Science specialist program: Middle grades certification program leading to the Bachelor of Science degree (B.S.) with a major in Interdisciplinary Studies (INST) through the Middle Grades Certification Program in the Department of Teaching, Learning and Culture at Texas A&M University. – It is a field-based program, with students spending
extensive time in middle schools. – Credit hours required for graduation in the
Mathematics/Science strand total 133-134 credit hours.
Definition of Terms, Definition of Terms, cont’dcont’d
• Mathematics knowledge for teaching: the mathematical knowledge “used to carry out the work of teaching mathematics” (Hill, Rowan, & Ball, 2005, p. 373).
• NCTM: The National Council of Teachers of Mathematics, an organization composed of classroom teachers, supervisors, educational researchers, teacher educators, university mathematicians, and administrators involved in the mathematics education of students.
• Performance Assessment: a form of testing requiring students to demonstrate their achievement of understandings and skills by completing a demanding task(s) in which they are asked to respond to the task(s) orally, in writing, or by constructing a product (Gronlund, 2003; Nitko, 2001; Popham, 2005).
• “Specialized” content knowledge: mathematical knowledge, not pedagogy (Hill, Rowan, & Ball, 2005).
DelimitationsDelimitations• Not possible to test all preservice
middle grades mathematics teachers
• Study is limited to the number of preservice teachers and topics feasibly available
• Study did not consider gender or ethnicity
Cohort Course Spring ’05
Spring ‘06
1Mathematics Courses I
MATH 365: Structure of Math I
127 124
MATH 366: Structure of Math II
152 140
2Mathematics Courses II
MATH 367: Basics Concepts of Geometry
? 18
MATH 368: Introduction to Abstract Math
47 47
MATH 403: Math and Technology
64 44
3Mathematics Education
Courses
MASC 351: Problem Solving 37 28MASC 450: Integrated Mathematics
40 56
4Methods Courses
MEFB 460: Methods of Teaching Middle Grades Mathematics
66 36
5Student
Teaching
MEFB 497: Teaching Middle Grades
53 97 TOTAL Participants: 590
Conclusions/Conclusions/ImplicationsImplications
• There are no studies to directly compare the results to– Hard to initially generalize the results to the population
• More longitudinal testing will have to be completed in order for the results to become widely accepted amongst mathematics educators.
• Some of the results may be comparable to similar studies conducted at the elementary levels such as those by Hill, Rowan, and Ball (2005).
• Other comparisons could possibly be made to assessment efforts and findings of Bill Bush, University of Louisville.
Thank You!Thank You!
Margaret J. MohrTexas A&M University
Department of Teaching, Learning and CultureMSMP Project Supervisor
Research Assistant and Assistant to Dr. Gerald Kulm