measuring effective teaching

72
UTOP Training Mary Walker University of Texas at Austin Candace Walkington Southern Methodist University

Upload: suzy

Post on 14-Jan-2016

39 views

Category:

Documents


0 download

DESCRIPTION

UTOP Training Mary Walker University of Texas at Austin Candace Walkington Southern Methodist University. Measuring Effective Teaching. What does effective teaching look like and how can we measure it? - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Measuring Effective Teaching

UTOP Training

Mary WalkerUniversity of Texas at Austin

Candace WalkingtonSouthern Methodist University

Page 2: Measuring Effective Teaching

MEASURING EFFECTIVE TEACHING

What does effective teaching look like and how can we measure it?

Can classroom observers be trained to make key distinctions in effective teaching practices?

Can the skills involved with being an effective teacher be successfully trained through a teacher preparation program?

Page 3: Measuring Effective Teaching

SOME KEY FEATURES OF UTEACH PHILOSOPHY

Organized, well-managed, on-task classroomAttention to issues of diversity and access Incorporating inquiry/investigative learningUsing technology for teaching and learningFluid and accurate communication of contentFostering student-student collaborationFormative assessment of student progressApplications and inter-disciplinary connectionsCritical practices of self-reflectionFacilitating classroom discussion and “student talk”

Page 4: Measuring Effective Teaching

THE UTEACH PROGRAM

Steady increase in number of students with strong STEM backgrounds going into teaching

Replicated at 28 universities in 13 states92% of graduates go into teaching, 82% remain 5

years later (compared to 65% nationally)

www.nationalmathandscience.org

Page 5: Measuring Effective Teaching

BACKGROUND OF PROJECT

Persistent requests to evaluate UTeach Graduates

UTeach boosts recruitment and retention, but are UTeach graduates effective teachers?

Look towards classroom observation

Page 6: Measuring Effective Teaching

RTOP [Reformed Teaching Practices]http://physicsed.buffalostate.edu/AZTEC/RTOP/RTOP_full/about_RTOP.html

COP [Classroom Observation Protocol]www.horizon-research.com/instruments/lsc/cop.pdf

http://www.horizon-research.com/instruments/hri_instrument.php?inst_id=14

OBSERVATIONAL PROTOCOL DEVELOPMENT

Page 7: Measuring Effective Teaching

DESCRIPTION OF UTOP

Full version has 32 indicators (teaching behaviors) in 4 sectionsClassroom EnvironmentLesson Structure ImplementationMathematics/Science Content

1-5 scale, DK/NA optionsSection Synthesis Ratings

Page 8: Measuring Effective Teaching

Rating Indicator  1.1 The classroom environment encouraged students to generate ideas, questions, conjectures,

and/or propositions that reflected engagement or exploration with important mathematics and science concepts. Description Rubric Specific Rating Examples

Evidence:   1.2 Interactions reflected collegial working relationships among students. (e.g. students worked together

productively and talked with each other about the lesson). *It's possible that this indicator was not applicable to the observed lesson. You may rate NA in this case.Description Rubric Specific Rating Examples

Evidence:  1.3 Based on conversations, interactions with the teacher, and/or work samples, students were intellectually engaged with

important ideas relevant to the focus of the lesson. Description Rubric Specific Rating Examples

Evidence:  1.4 The majority of students were on task throughout the class.

Description Rubric Specific Rating ExamplesEvidence:  1.5 The teacher’s classroom management strategies enhanced the classroom environment.

Description Rubric Specific Rating ExamplesEvidence:  1.6 The classroom is organized appropriately such that students can work in groups easily, get to

lab materials as needed, teacher can move to each student of student group, etc.Description Rubric Specific Rating Examples

Evidence:  1.7 The classroom environment established by the teacher reflected attention to issues of access,

equity, and diversity for students (e.g. cooperative learning, language-appropriate strategies and materials, attentiveness to student needs). Description Rubric Specific Rating Examples

UTOP AND ONLINE MANUAL

Page 9: Measuring Effective Teaching

Rating Indicator  1.1 The classroom environment encouraged students to generate ideas, questions, conjectures,

and/or propositions that reflected engagement or exploration with important mathematics and science concepts. Description Rubric Specific Rating Examples

Evidence:   1.2 Interactions reflected collegial working relationships among students. (e.g. students worked together

productively and talked with each other about the lesson). *It's possible that this indicator was not applicable to the observed lesson. You may rate NA in this case.Description Rubric Specific Rating Examples

Evidence:  1.3 Based on conversations, interactions with the teacher, and/or work samples, students were intellectually engaged with

important ideas relevant to the focus of the lesson. Description Rubric Specific Rating Examples

Evidence:  1.4 The majority of students were on task throughout the class.

Description Rubric Specific Rating ExamplesEvidence:  1.5 The teacher’s classroom management strategies enhanced the classroom environment.

Description Rubric Specific Rating ExamplesEvidence:  1.6 The classroom is organized appropriately such that students can work in groups easily, get to

lab materials as needed, teacher can move to each student of student group, etc.Description Rubric Specific Rating Examples

Evidence:  1.7 The classroom environment established by the teacher reflected attention to issues of access,

equity, and diversity for students (e.g. cooperative learning, language-appropriate strategies and materials, attentiveness to student needs). Description Rubric Specific Rating Examples

This indicator should be rated a 1 if there is group work during the lesson, but the group work is highly unproductive. This could include behavior where the majority of the groups are socializing, off-task, arguing, or ignoring each other, as well as regular instances of students copying and/or certain group members doing all of the work.

This indicator should be rated a 2 if …

This indicator assesses the degree to which students have learned to be collegial, respectful, cooperative, and interactive when working in groups. Evidence of collegial working relationships among students includes collaborative discussions about topics relevant to the lesson and successful distributing of roles and responsibilities within each group…

Rating of 3 Example: The students were put into debate groups for this class period - one group would debate another group, while the rest of the student groups were in the audience. The groups worked together smoothly - the students were able to pick who was doing what part of the debate, coordinate their arguments, and split the time slots when necessary. The audience also would occasionally compare their notes during breaks…

UTOP AND ONLINE MANUAL

Page 10: Measuring Effective Teaching

PILOT STUDY

Test UTOP on some of our graduate’s classrooms

Conducted 83 observations of:UTeach Graduates (N=21)Non-UTeach Graduates (N=15)

Novice teachers (most 0-3 years exp)Math, science, and computer science

classes

Page 11: Measuring Effective Teaching

PILOT STUDY

Page 12: Measuring Effective Teaching

PILOT STUDY

After starting out at similar levels, UTeachers gain higher UTOP scores over time Teaching experience significant predictor of UTOP scores for UTeach group (p < .05)

Noyce Scholars rated significantly higher on UTOP than other groups, (p < .01)

Key Question: Is the UTOP a valid and reliable instrument that measures important components of effective teaching?

Page 13: Measuring Effective Teaching

NMSI/MET STUDY

UTOP study conducted in partnership with the Gates Foundation’s Measures of Effective Teaching project, and the National Math and Science Initiative

Examine reliability, consistency, factor structure

Connect teaching behaviors on UTOP to teacher value-added gains

Page 14: Measuring Effective Teaching

THE MET PROJECT

3000 teachers from 7 school districts, 7 statesVarious subjects (mathematics, English, science) and grade

levelsMultiple measures of effectiveness (observations, value-

added, student surveys, teacher exams)Multiple video lessons of each teacher Multiple classroom observation instruments

Charlotte Danielson’s FFTCLASS protocolMQI RubricUTOP

Page 15: Measuring Effective Teaching

NMSI/MET STUDY

99 raters (math and science master teachers with LTF), scored 994 video lessons of 250 teachers using UTOP

All lessons grades 4-8 mathematicsOne third of videos double-scored

Page 16: Measuring Effective Teaching

Most of the 4-8 math video lessons from this national sample did not score highly on the UTOP

Many middle school math teachers teaching problematic content; many formulaic/key word type approaches.Raters identified problematic content issues in around one half of all lessons

RESULTS

Page 17: Measuring Effective Teaching

Surface-level engagement often seen, but little emphasis on conceptual understanding

“Orderly but unambitious”

RESULTS

Page 18: Measuring Effective Teaching

INSTRUMENT RELIABILITY

Can we consistently measure teaching effectiveness, beyond the biases of individual raters, or the characteristics of particular lessons?

Goal: 60-80% of the variance in teacher scores on the instrument attributable to the stable characteristics of the individual teacher

% total variance explained

Lesson 13.3%

Teacher 32.77%

Rater 0%

Residual (Rater x Lesson) 53.9%

Page 19: Measuring Effective Teaching

INSTRUMENT RELIABILITY

Implementation Results:2 observations per year, with 2 raters present at each observation (UT Pilot study)

4 observations per year, with 1 rater present at each observation (MET Study)

Reliability Coeff

Classroom Environment 67%

Lesson Structure 62%

Implementation 64%

Mathematics Content 40%

Overall (Avg Syn) 66%

Page 20: Measuring Effective Teaching

VALUE-ADDED CORRELATIONS

Are the teaching behaviors measured on the UTOP associated with higher student learning gains, on standardized assessments and tests of conceptual understanding?

Page 21: Measuring Effective Teaching

VALUE-ADDED CORRELATIONS

This graph is copied from the released 2012 MET Report

Page 22: Measuring Effective Teaching

FACTOR ANALYSIS OF UTOP

Cluster 1: Fostering Surface

Engagement-On task & involved-Class management

-Group work-Lesson organization

Cluster 2: Fostering Deep Conceptual Understanding-Inquiry/investigation

-Higher-order questioning-Intellectual engagement

Cluster 3: Content Accuracy and

Fluidity-Verbal & written accuracy/fluidity-Effective use of

abstraction

Cluster 4: Making Content

Connections-To real world (authentic)

-To “big picture”-To history/current events

Page 23: Measuring Effective Teaching

FACTOR 1: FOSTERING SURFACE LEVEL ENGAGEMENT

Classroom managementMajority “on task”Group-work dynamicTime managementLesson OrganizationAppropriate Resources Issues of equity & accessTeacher critical of lesson

Page 24: Measuring Effective Teaching

FACTOR 2: FOSTERING DEEP, CONCEPTUAL UNDERSTANDINGStudents generate ideas/conjecturesStudents intellectually engagedStudents explore contentUse of higher-order questionsUse of inquiry/investigation

Page 25: Measuring Effective Teaching

FACTOR 3: CONTENT ACCURACY & FLUIDITY

Accurate written content informationAccurate & fluid verbal communication of contentAppropriate use of abstraction

Page 26: Measuring Effective Teaching

FACTOR 4: CONTENT CONNECTIONS

Connect content to “real world” and other disciplinesConnect content to history & current eventsConnect content to the “big picture” of the discipline

Page 27: Measuring Effective Teaching

SUMMARY & IMPLICATIONS

UTOP measures 4 factors of effective teaching

UTOP has reasonable correlations with value-added – may better detect strong teachers

Need multiple observations, multiple raters to conduct classroom observation

Multiple measures of teaching effectiveness (value-added, observations, student surveys, teacher exams, etc.)

Page 28: Measuring Effective Teaching

FUTURE DIRECTIONS

Connect specific teaching behaviors to teacher value-added – what really matters?

Investigate why the UTOP might be more effective at identifying excellent teaching

Use of UTOP to compare classrooms at project-based school (with UTeach graduates) to those at traditional school, same low income school district

Page 29: Measuring Effective Teaching

Manor ISD Working at 3 High schools – Traditional, Project- and Problem-

based and Accelerated Credit Academy Baseline 2011 – 2012 at MHS and MNTHS

6 observations per teacher Interviews of teachers Student focus groups

How to use UTOP + for Professional Development? Expand pool size Train teachers to use for self-assessment and group-assessmemt Facilitate Lesson Study in Professional Learning Groups Reliability Audits with UTeach UTOP experts Targeted PD sessions for summer 2013 with input from teachers Funding to expand model with new partners

CURRENT STUDY

Page 30: Measuring Effective Teaching

Learn to use UTOP by using the tool

UTOP Video version UTOP Manuals – Full and Video UTOP Full version

Videos at http://www.timssvideo.com/

8th grade Mathematics lesson – US3 Exponents 8th grade Science lesson – AU4 Energy Transfer 8th grade Mathematics Lesson – US1 Graphing Linear Equations High School Mathematics lesson – Lesson Lab GH 06111612.mov

Today’s training agenda

Page 31: Measuring Effective Teaching

What do you think?Need for specific examples from different domainsHow to recruit / support additional partnersFollow – up training ala Reliability Audits for Manor study

NEXT STEPS / FEEDBACK

Page 32: Measuring Effective Teaching

Rating Mathematics and Science Lessons Using the UTeach Observation Protocol (UTOP)

First Day of Training

Page 33: Measuring Effective Teaching

You will rate each indicator with a 1-5 rating, typed into box on word document

33

The UTeach Observation Protocol

4

Page 34: Measuring Effective Teaching

You also type in 1-5 sentences of supporting Evidence into the “Evidence” box to justify each numerical rating.

34

The UTeach Observation Protocol

4

Page 35: Measuring Effective Teaching

Supporting Evidence should be: Specific – based on specific quotes and interactions from video No opinions! You must justify your rating based on evidence. Somewhat brief – try to average 3-4 sentences BUT detailed enough to demonstrate how carefully and thoughtfully you watched

the video or observed and took good field notes in the classroom

35

Rating Process

Page 36: Measuring Effective Teaching

Supporting Evidence will be used: By You: To determine your numerical ratings. Read your evidence, compare it to

the rubric and examples, and then determine a rating. By Your Group Members: To understand specific comments or events that

occurred which convinced you to assign a particular score to a particular indicator By UTOP Reliability Auditors: If you participate in future research studies using the

UTOP, you will need to revalidate and recalibrate your skills Co-observations with external UTOP evaluators On-line modules with database recording your work

36

Rating Process

Page 37: Measuring Effective Teaching

Poor Supporting Evidence: Is too brief Is too vague Is clearly biased or opinionated Is not really related to the indicator’s intent Does not provide adequate detail for good teacher feedback [Assuming the use if

for ongoing professional development]

37

Rating Process

Page 38: Measuring Effective Teaching

Good Supporting Evidence: Offers some specific examples/evidence, but not so many it becomes too long Is concise, but not too brief to be uninformative Is evidence - does not reflect an opinion Is based on knowledge that all experienced teachers are expected to have of good

practices Is clearly related to your numerical score

38

Rating Process

Page 39: Measuring Effective Teaching

Several web resources are available for you to assist you in rating each indicator Description: Overview of intent of indicator and general types of appropriate

evidence (link).

39

Rating Process

Page 40: Measuring Effective Teaching

Several web resources are available for you to assist you in rating each indicator Rubric: General rating guidelines for 1-5 ratings for that indicator (link).

40

Rating Process

Page 41: Measuring Effective Teaching

Several web resources are available for you to assist you in rating each indicator Specific Rating Examples: Examples of evidence going with 1-5 ratings, for both

mathematics and science (link).

41

Rating Process

Page 42: Measuring Effective Teaching

42

Rating Process

Synthesis Ratings Each of the 4 sections concludes with a 1-5 synthesis rating. Take into account all evidence and all ratings from that section, and choose

which descriptor best characterizes the lesson.

Page 43: Measuring Effective Teaching

43

Rating Process

Synthesis Ratings Not a numerical average – “human average” where you weight the relative

importance of the different indicators Type an “x” by your choice No additional supporting evidence needed

Page 44: Measuring Effective Teaching

44

Rating Process

The final 2 sections of the UTOP are Lesson Description: Discuss purpose of lesson, content was covered, learning

activities and pedagogical approaches. (4-6 sentences) Concluding Remarks: Value judgment of the lesson & relative importance of the

ratings made - includes comments that did not fit into preceding sections. (3-4 sentences)

Page 45: Measuring Effective Teaching

45

Rating Process

The UTOP training manual contains all this information; it is available: As a complete word document (60 pages) Online with a clickable contents and interface, at

http://www.cwalkington.com/utop

Page 46: Measuring Effective Teaching

1.1 The classroom environment encouraged students to generate ideas, questions, conjectures, and/or propositions that reflected engagement or exploration with important mathematics or science concepts.

Make note of:Instances during the lesson where you observe students generating ideas, questions, conjectures, or propositions. Whether the teacher is actively encouraging these contributions

Special Considerations:Keep in mind that giving a simple response to a direct teacher question is not really “generating an idea,” and that asking a simple clarification question is not what we have in mind as reflecting engagement with mathematics or science concepts.

46

Indicators in 1: Classroom Environment

Page 47: Measuring Effective Teaching

1.2 Interactions reflected collegial working relationships among students (e.g. students worked together productively and talked with each other about the lesson). Make note of:

Whether students are talking to each other about topics relevant to the lessonWhether students are having off-topic conversations in groupsWhether the teacher is making explicit moves to promote group work skillsWhether the student groups are effectively distributing rolesWhether all students in the group are involved in solving each problem

NA Rating:Rate as NA if the lesson did not include group work. Student-student interactions greater than 3 minutes total over the entire lesson

47

Indicators in 1: Classroom Environment

Page 48: Measuring Effective Teaching

1.3 Based on conversations, interactions with the teacher, and/or work samples, students were intellectually engaged with important ideas relevant to the focus of the lesson.

Make note of:Whether students are making quality contributions during whole-class portions and small group portions of the lesson Whether students are thinking about the concepts themselves, or just following along with what the teacher is telling them to do Whether the students are giving contributions that clearly show they’re not thinking deeply about or understanding the content Contributions that evidence intellectual engagement rather than surface level engagement

48

Indicators in 1: Classroom Environment

Page 49: Measuring Effective Teaching

1.4 The majority of students were on task throughout the class.

Make note of:Whether students are actively participating (raising hands, asking questions) in whole class portions of lesson. Whether students are doing assigned work during individual/group portions of lesson Any off-task behavior evidence: socializing, head down, work for other class, etc.

Special considerations:The benchmark for a “3” is 75% on task You can only rate based on what you see/hear!

49

Indicators in 1: Classroom Environment

Page 50: Measuring Effective Teaching

1.5 The teacher’s classroom management strategies enhanced the classroom environment. Make note of:

Setting expectations & communicating objectivesForeseeing problemsManaging behaviorStudent seating & group selectionManaging student movement and participation norms How the teacher ensures positive student-student interactions Getting student attention and participation when needed

Special considerations: Some teachers will manage every hint of off-task behavior, while others will let huge violations slip by. Look not only at what the teacher is saying, but whether its appropriate based on what the students are doing.

50

Indicators in 1: Classroom Environment

Page 51: Measuring Effective Teaching

1.6 The classroom is organized appropriately such that students can work in groups easily, get to lab materials as needed, teacher can move to each student of student group, etc.•Make note of:

– How the desks are organized, and if this is appropriate for the structure of the activity

– How easily the teacher circulate– Whether students can see the board– Whether students can easily access all materials and resources

•Special Considerations:– The setup of the classroom may not always be under the teacher's

control, but we can only rate based on the information we have.– If you do not notice anything particularly positive or negative about

the classroom setup, this indicator should be rated a 3.

51

Indicators in 1: Classroom Environment

Page 52: Measuring Effective Teaching

1.7 The classroom environment established by the teacher reflected attention to issues of access, equity, and diversity for students (e.g. cooperative learning, language-appropriate strategies and materials, attentiveness to student needs). Make note of:

Use of cooperative learning, and how the teacher encourages positive student-student interactionsTeacher patience and rapport with studentsLevel of mutual respect between teacher and students, students and studentsUse of pedagogical approaches and lesson structures that reach students from different mathematical backgrounds with different learning preferences How the teacher values the things the students care about, and incorporates them into the lessonHow incorrect contributions are treated by the teacher and by other students. How much students are willing to contribute.How much the teacher lets students express themselves

52

Indicators in 1: Classroom Environment

Page 53: Measuring Effective Teaching

2.1 The lesson was well organized and structured (e.g. the objectives of the lesson were clear to students, and the sequence of the lesson was structured to build understanding and maintain a sense of purpose). Make note of:

Whether the lesson has a clear sense of purpose and clearly stated objectives How the lesson is introduced or opened, appropriateness/effectiveness What each learning activities was, and whether they were designed to promote engagement and conceptual understanding How the lesson was closed or wrapped up, appropriateness/effectivenessWhether the lesson was structured such that the objectives could be feasibly accomplished.

53

Indicators in 2: Lesson Structure

Page 54: Measuring Effective Teaching

2.2 The structure of the lesson allowed students to engage with or explore important concepts in mathematics or science (instead of focusing on techniques that may only be useful on exams).

Make note of:Whether the lesson design allowed students to uncover mathematics or science concepts, rather than the teacher simply covering them. Whether the lesson design allowed students to understand and engage with the concepts and procedures rather than simply perform the concepts and procedures. Aspects of the lesson design were there that allowed these things to be accomplished or not accomplished Parts of the lesson that were clearly targeted towards procedural knowledge, and exam preparation

54

Indicators in 2: Lesson Structure

Page 55: Measuring Effective Teaching

2.3 The structure of the lesson included opportunities for the instructor to gauge student understanding. Make note of:

Was the teacher able to circulate and see students’ thinking during conversations?Were the questions/prompts designed in such a way to capture students misconceptions, or were they close-ended?Did the teacher allow time for discussion and multiple viewpoints?Is there an appropriate amount of “student talk” structured into the lesson?

55

Indicators in 2: Lesson Structure

Page 56: Measuring Effective Teaching

2.4 The lesson included an investigative or problem-based approach to important concepts in mathematics or science.Definitions:

In a problem-based approach, the teacher presents students with realistic dilemmas. There must be a larger purpose or task that unites and gives purpose to many smaller mathematics problems or science activities.In an investigative approach, students investigate and discover important mathematical or scientific ideas, procedures, and principles, without the teacher telling them how to do it.

Make Note of:Whether problem-based or investigative approaches are used

If yes, describe the activity, its quality and lengthIf no, rate a 1

Consider not only if the activity was designed to be investigative or problem-based, but also whether it is enacted that way

56

Indicators in 2: Lesson Structure

Page 57: Measuring Effective Teaching

2.5 The teacher obtained and employed resources appropriate for the lesson. Make note of:

All resources you see being used during the lesson (PowerPoint, white board, videos, calculators, computers, textbooks, probes, lab equipment, manipulatives, construction paper, scissors, worksheets, handouts, etc.) Whether there were any resources being inappropriately used Whether resources were not prepared appropriately, or are missing Whether there were any resources that seemed to facilitate or improve student learning through their inclusion

Special considerations:Rate a “3” if resources were neither helpful for harmful (i.e. typical lesson with marker & whiteboard, worksheets completed for observing science activities or demonstrations)

57

Indicators in 2: Lesson Structure

Page 58: Measuring Effective Teaching

2.6 The teacher was critical and reflective about his/her practice after the lesson, recognizing the strengths and weaknesses of their instruction.This indicator is evidenced by the teacher’s responses to the post-interview questions Make Note of:

What the teacher identified as strengths, of the lesson and whether the UTOP identified these same strengths (rate last)What the teacher identified as weaknesses, and whether the UTOP identified these weaknesses

Special Considerations:This item is rated Not Applicable (NA) if there is no opportunity to get the teacher’s reflection on the lessonThere is no such thing as a lesson without weaknesses. There might be a lesson without strengths.

58

Indicators in 2: Lesson Structure

Page 59: Measuring Effective Teaching

3.1 The teacher used questioning strategies to encourage participation, check on skill development, and facilitate intellectual engagement and productive interaction with students about important science and mathematics content and concepts. Make note of:

How often the teacher used questioning in general to facilitate engagement/skillsAll or example instances of higher-order questions (such as analysis, synthesis, evaluation) Typical wait time Treatment of incorrect answers All questions targeted to conceptual (rather than procedural) understanding

Special Considerations:Many teachers use simple, factual, “fill-in-the-blank” type questions to promote interaction during whole-class. Teachers also use procedural questions to bring students to the next step when they help them 1 on 1. However, to get a higher score on this indicator, teachers need to be using higher-level questioning techniques as well (quality rather than quantity)

59

Indicators in 3: Implementation

Page 60: Measuring Effective Teaching

3.2 The teacher involved all students in the lesson (calling on non-volunteers, facilitating student-student interaction, checking in with hesitant learners, etc.).

Make note of:The teacher calling on non-volunteers, or giving non-volunteers a way to participate The teacher using different modes of participation (whole class, small group, individual), to allow more students to feel comfortable sharingThe teacher circulating and involving students not specifically soliciting his/her helpThe teacher making moves to involve students who are struggling or students with low prior knowledgeThe teacher encouraging all students to participate in the activity & setting norms for participation

Special Considerations:If the teacher simply calls on a lot of volunteers to give short, factual answers, this is not considered important evidence for a high score on this indicator.

60

Indicators in 3: Implementation

Page 61: Measuring Effective Teaching

3.3 The teacher used formative assessment effectively to be aware of the progress of all students and modified the lesson appropriately when formative assessment demonstrated that students did not understand.Formative assessment is anything the teacher does to check on the progress of the students, so he/she can modify instruction appropriatelyMake note of:

Written formative assessments, like quizzes, board work, journal entries Teacher monitoring student progress by looking at student work or observing student conversations Teacher monitoring student progress by asking questions to assess understanding Teacher stopping a lecture to ask if students have any questions, if students understand, using wait time, asking in such a way that students would be comfortable respondingEvidence that the lesson was modified appropriately (or inappropriately) by the teacher based on these assessmentsEvidence that appropriate modifications were NOT made

61

Indicators in 3: Implementation

Page 62: Measuring Effective Teaching

3.4 An appropriate amount of time was devoted to each part of the lesson.

Make note of:How long all major learning activities lasted (can be approx)Whether the amount of time for each activity was appropriate, given student responses

Too much time?Too little time?

Whether there was an appropriate wrap-upAny instances of wasted time

62

Indicators in 3: Implementation

Page 63: Measuring Effective Teaching

3.5 The instructional strategies and activities used in this lesson clearly connected to students’ prior knowledge and experience.

Make note of:Connections the teacher makes to concepts previously covered in that classConnections to concepts previously covered in other classesConnections to everyday experiences targeted towards helping students understand the material better through these experiencesWhether the teacher elicits prior knowledge (through questioning, assessments), in order to build on it

63

Indicators in 3: Implementation

Page 64: Measuring Effective Teaching

3.6 The teacher's instructional strategies included safe, environmentally appropriate, and ethical implementation of laboratory procedures and/or classroom activities.Make note of:

Any safety instructions teacher gives studentsHow teacher supervises/monitors students work in the labMaintenance of equipment, organization of materialsUnethical/disrespectful/dangerous behavior

Special Considerations:Even though most classroom activities require students to behave in an ethical manner with their peers and teachers, an “NA” may be chosen if a laboratory or classroom activity where this indicator would be relevant is not part of the lesson being observedIf there is inappropriate behavior, and the teacher immediately deals with it and could not have feasibly planned for it, do not penalize

64

Indicators in 3: Implementation

Page 65: Measuring Effective Teaching

4.1 The mathematics or science content chosen was significant, worthwhile, and developmentally appropriate for this course (includes content standards covered, as well as examples and activities chosen by teacher).Make note of:

What content topics were covered during the class period, and whether they were significant and worthwhile for the grade levelWhat examples/activities were covered during the class period, and whether these examples were significant and worthwhile Whether the content was developmentally appropriate (too easy, too hard)

Whether students seemed to have the prior knowledge to be successfulWhether students found the activities too easy

65

Indicators in 4: Mathematics and Science Content

Page 66: Measuring Effective Teaching

4.2 Content communicated through direct and non-direct instruction by the teacher is consistent with deep knowledge and fluency with the mathematics or science concepts of the lesson (e.g. fluent use of examples, discussions and explanations of concepts, etc.).

Make note of:Teacher “ad-hoc” explanations and discussionsTeacher making connections between different conceptsTeacher offering multiple explanations when students don’t understandTeacher understanding of student mistakes Clarity of teacher’s language and explanations Teacher’s use of examples to guide learning (for example…)

66

Indicators in 4: Mathematics and Science Content

Page 67: Measuring Effective Teaching

4.3 Teacher written and verbal content information was accurate. Make note of:

Any errors or ambiguities in the written or verbal content information (example, one variable stands for 2 things) Whether the errors were caught and corrected Whether there were any ambiguities or mistakes that clearly confused or disrupted students

Special Considerations:If no errors or ambiguities in the content, it’s a 5Often need to watch videos a couple times to catch – these can be toughOften need to work through the problems yourselves to catch the mistakes

67

Indicators in 4: Mathematics and Science Content

Page 68: Measuring Effective Teaching

4.4 Formal assessments used by teacher (if available) were consistent with content objectives (homework, lab sheets, tests, quizzes, etc.).Make note of:

What formal assessments specifically were incorporated into the lessonAnything the teacher collects for evaluation/gradeIf the assessments were correct/clearIf the assessments seemed to be assessing skills and standards that are aligned with what’s going on in the classroom

Special Considerations:This indicator can be rated NA if there are no formal assessments used during the class period

68

Indicators in 4: Mathematics and Science Content

Page 69: Measuring Effective Teaching

4.5 Elements of mathematical/scientific abstraction were used appropriately (e.g., multiple forms of representation in science and mathematics classes include – verbal, graphic, symbolic, visualizations, simulations, models of systems and structures that are not directly observable in real time or by the naked eye, etc.).Make note of:

Whether elements of abstraction are incorporated into the lesson, and what these areIf the meaning of the abstractions was made clear to students; if the abstractions were introduced sensibly; if their use was accurateTeacher using multiple representations – such as verbal, tabular, graphical, models and symbolic Teacher making connections between representationsTeacher explaining what abstractions mean

Special Considerations:This indicator can be rated NA if there are no elements of abstraction in the lesson.

69

Indicators in 4: Mathematics and Science Content

Page 70: Measuring Effective Teaching

4.6 During the observation, it was made explicit to students why the content is important to learn.

Make note of: The teacher explicitly discussing the significance of the content with the students The teacher giving the students activities that explicitly bring out the significance of the material

Special Considerations:If the teacher simply gives the students some problems that happen to be contextualized, this is not the same thing as engaging students in a discussion about why they are learning the material in the first placeSimply telling students that they need to learn the content for future classes, future topics in this class, or for a test is not what we are trying to capture.If it’s not there, it needs to be rated a 1.

70

Indicators in 4: Mathematics and Science Content

Page 71: Measuring Effective Teaching

4.7 Appropriate connections were made to other areas of mathematics or science and to other disciplines (including non-school contexts).Make note of:

The teacher making connections between mathematics/science and another discipline (science, mathematics, economics, history, geography, etc.)The teacher making connections between different areas of mathematics (algebra, geometry, statistics, etc.) or science (physics, biology, chemistry, env science)The teacher making connections to non-school (i.e. “real world”) contexts

Special ConsiderationsAlthough it may seem inappropriate to penalize a teacher for not incorporating these types of connections into every single mathematics lesson they teach, if absolutely no connections made during the class period, this indicator should be rated as a 1. It should be rated a “1” even if you think such connections are not possible.

71

Indicators in 4: Mathematics and Science Content

Page 72: Measuring Effective Teaching

4.8 During the lesson, there was discussion about the content topic’s role in history or current events.

Make note of:The teacher or the activity making connections to history or current events

Special Considerations:Although it may seem inappropriate to penalize a teacher for not incorporating these connections into every single mathematics lesson they teach, if absolutely no connections are made during the class period, this indicator should be rated as a 1. It should be rated a “1” even if you think such connections are not possible.

72

Indicators in 4: Mathematics and Science Content