1

Children First Intensive

Mathematics Best PracticesConnecting Assessment, Inquiry and the

Quality ReviewDecember 9, 2009

ESA CFN 6 (Bob Cohen) and ESO Network 19 (Elvira Barone)

Co-Facilitators: Deena Abu-Lughod, SATIF; Frederica Capshaw, Instructional Support Specialist; Karen Ames, Achievement Coach

Contact us at Dabulug@schools.nyc.gov or FCapsha@schools.nyc.gov

Electronic copies available at http://dabulughod.wikispaces.com

2

Agenda 8:00 Breakfast

8:30 Welcome, Introductions, Agenda, Rationale and Outcomes

9:00 Prediction and Analysis – Data tools and a new lens on School-Specific 2009 NYS 3-8 Mathematics Data to evaluate the effectiveness of the implemented curriculum (QR Statement 1.1)

10:00 Break

10:15 Lesson Planning and Research Lessons

11:30 Mathematics Resources, Lois Sharzer

12:15 Lunch

1:00 Breakout Groups: (A) Elementary -

(B) M/S and H/S-Donna Davis, National Consultant, Glencoe

2:30 Questions, Debrief, Evaluation

3

Rationale

The NYCDOE Progress Report uses NYS assessments to tell part of the story about how students performed and progressed in mathematics.

Item-by-item comparisons of a school’s students relative to the city’s students, and State benchmarks and “distinguishing” items identify strengths in a school’s teaching and its implemented curriculum.

Today, we will explore how teacher teams can use an inquiry approach to analyze assessment data, student work, and teacher work, to adjust curriculum, instruction, and assessments to improve learning outcomes. (QS 4.2)

4

What is Collaborative Inquiry?

Collaborative inquiry is a sustained process of investigation and action that empowers teachers to improve student achievement and close the achievement gap. Collaborative inquiry is:Focused on student outcomes, using a systematic, data-

informed approach.Conducted by teams of teachers with a focus on small

groups of students, paying close attention to those who are struggling while supporting the learning of all students.

Designed to develop and deepen rigorous, research-based instructional strategies and frameworks.

Currently, citywide, 990 of 3900 teacher teams (25%) are focusing on Mathematics.

5

Outcomes

What are we going to know at the end of the day?

1) How to analyze item data in relation to your school’s implemented curriculum (QS 1.1)

2) How to analyze summative data to create a portrait of mastery and provide meaningful and actionable feedback on the effectiveness of classroom level, curricular and instructional decisions (QS 2.2)

3) How to use these items to evaluate and adjust instructional practices and to monitor progress (QS 5.2, 5.3)

6

Guide to ResourcesItem Analyses and Test Items: Available on DOE web for 2007-2009, for evaluating and

aligning curriculum and monitoring progress. http://schools.nyc.gov/Academics/Mathematics/EducatorResources/Item+Analyses.htm

Curriculum Resources, pacing guides, strand trace, etc: Available on DOE web. http://schools.nyc.gov/Academics/Mathematics/EducatorResources/default.htm

Trend Maps and State benchmarks: Available on ARIS Connect, for identifying power standards, and for identifying items that separate the 4s from the 3s, and the 3s from the 2s. https://www.arisnyc.org/connect/node/388417

ItemData: Available in your school’s private community on ARIS Connect for viewing individual responses; available by past and current classroom (July and September versions).

Web resources: Available on ARIS Connect, for instructional and assessment support. https://www.arisnyc.org/connect/node/369071

Lesson study toolkit: http://www.lessonresearch.net/nsf_toolkit.html

Acuity ITAs, Predictives and Customized assessments: From home computer, https://nyc-acuity.mcgraw-hill.com/index.jsp

To Schedule Professional Development: http://schools.nyc.gov/Accountability/ResourcesforEducators/PeriodicAssessments/Periodic+Assessment+Professional+Development.htm

7

Activity #1: Item Analysis Predicting your outcomes

Think about the 2009 NYS assessment results in a grade you know well (you should be sitting at that table).

Look at the trend map for that grade, the Network averages by item relative to NYC, and the items themselves.

On which items do you think your students did well, relative to other students in the Network or City? On which items did they struggle?

Commit yourself to paper. On Worksheet #1, write down at least 3 strengths and 3 weaknesses and justify your predictions.

8

Trends Maps

The Trends Maps help you identify the “Power Standards”: those performance indicators that are most frequently tested on the State exams.

They show which items tested which performance indicator over the last few years and the percent of questions used to test each strand.

Let’s look at the Trend Map for Grade 8 together.

Be aware that although some indicators are not tested every year, they may involve necessary skills.

9

Gr 8 2009 Trend Map (from BOCES)

10

Priorities: NYS Math 2008 Distribution by Strand

Gr 3 Gr 4 Gr 5 Gr 6 Gr 7 Gr 8

Number Sense

52% 52% 44% 40% 32% 13%

Algebra 16% 15% 15% 20% 16% 47%

Geometry 13% 13% 26% 17% 20% 40%

Measurement

16% 19% 18% 14% 17% 14%

Statistics 10% 10% 12% 23% 32% 0%

2009 Item Analysis from Educator Resources

11http://schools.nyc.gov/Academics/Mathematics/EducatorResources/Item+Analyses.htm

2009 Items (from DOE Educator Resources)

12http://schools.nyc.gov/Academics/Mathematics/EducatorResources/Item+Analyses.htm

Activity #1, cont.: Check your predictions

Examine your school-specific item analysis chart .

How accurate were your predictions about your school’s strengths and weaknesses?

What might explain the variance on the items?

What are the implications?

Share out.

13

14

Comparing ourselves with Benchmarks

How do you know which questions are strategic (matter the most)?

How do you know which strands and indicators represent a strength or a weakness for your school?

How do you know whether a question was difficult just for your own students or for all students?

Comparing our own scores with the State Benchmarks helps us answer these questions.

Then we must ask: To work strategically, what questions should we focus on? The ones that were difficult for all students or just for some? How should we deal with infrequently tested indicators?

15

What are the State Benchmarks?

The Benchmarks will tell you what percent of Level 2, Level 3 and Level 4 students answered each item correctly. This is a more useful indicator than the p-value provided in the Item Analysis.

The benchmarks help you identify which questions distinguish the Level 4 students from the Level 3 students and the Level 3 students from the Level 2 students.

Understanding these differences may help you select what to work on if you want to push 2s to 3s, and keep 3s and 4s from slipping.

Let’s understand how to read the files

16

Item 5 distinguished the Level 3 from Level 2 students

Item 6 distinguished the Level 4 from Level 3 students

On average, there was a 34 percentage point difference between 4s and 3s, and 25 points between 3s and 2s.

17

Clarification: ARIS Taxonomy

You saw these codes in the ItemData files. Focus on the 1st, 3rd and last characters to see the grade, strand, and indicator number.

Geometry: Gr 8 Distinguishing Questions

18

Algebra: Gr 8 Distinguishing Questions

19

Activity #2: Research Lesson Modeling

The following question represents a power standard, and one that distinguishes 4s from 3s.

20

Activity #2: Research Lesson Modeling

Use Worksheet #2 to record the following (individually):

1) Sequence of understandings a student must have in order to solve the problem (use the strand trace, if desired)

2) Tasks and experiences that help students develop these understandings

3) Solve the task and record your solution.

4) Indicate additional ideas about how other students might solve the item, anticipating common errors.

5) If the tasks spark any new thoughts about student understanding and how it develops, modify your concept map as needed.

21

Problem Solving Strategies (p. 43 Van De Walle)

1. Draw a picture, act it out, use a model.

2. Look for a pattern.

3. Guess and check. (Make an attempt; reflection can lead to a better idea.)

4. Make a table or chart. (Often for problems involving ratios or measurements.)

5. Try a simpler form of the problem. (Simplify quantities so the resulting task is easier to understand and analyze.)

6. Make an organized list. (Show the number of possibilities and verify that all possible outcomes have been included.)

7. Write an equation. (Convert the story into numbers or symbols)

22

Four Step Problem Solving Template

As you plan, consider the value of the 4-Step Problem Solving Template (adapted from Bob Gyles, Math Think Tank):

23

Activity #3: Small Group Research Lesson

At your table, select ONE question that is both important and that represents a weakness at one or more of your schools.

Use your copy of Worksheet #2 to record the following:

1) Sequence of understandings a student must have in order to solve the problem (use the strand trace, if desired)

2) Tasks and experiences that help students develop these understandings

3) Solve the task and record your solutions and ideas about student solutions. If the tasks spark any new thoughts about student understanding and how it develops, modify your concept map as needed.

24

Activity #3: Small Group, Cont.

Using Worksheet 3, select a few of the tasks/activities you have identified, and indicate them and the associated questions, the anticipated student responses, the teacher supports, and the points of evaluation.

1) How can this type of planning produce better outcomes in class?

2) What else would you add or want to do?

25

Activity #4: Monitoring Progress

Group A: Assume you have developed and delivered the lesson you just drafted to the students currently in that grade.

How can you use the item or items from the 2009 State exam (and/or exams from previous years) to know whether this year’s students on that grade are performing better than last year’s students did?

Group B: Assume you have delivered the lesson you just drafted to the same students who had trouble with that indicator last year (eg, a target population).

How can you use the item or items from the 2009 State exam to know whether these students have now mastered the skill?

26

27

Both Collaborative Inquiry and Lesson Study

1) Emphasize collaboration around a common focus

2) Link directly to planning and classroom practice

3) Focus on a specific topic or problem related to goals for student learning

4) Use change strategies or research lessons as learning tools, rather than as products

5) Involve teachers’ use of existing curriculum and learning activities to improve them

6) Focus on the lesson or instruction, rather than the teacher

7) Involve team members who contribute equally

8) Involve live observation of students in the act of learning

9) Develop a common understanding by teachers of the context of student work.

28

Inquiry Cycle Lesson Study Cycle

Lesson Study and Toolkit

29

30

Important Resources for our Schools

Special Thanks to:

Lois Sharzer

Lois Sharzer and Associates

31

Teacher Actions Reference Sheet

Before:

Activate prior knowledge: begin with a simple version of the task, brainstorm solutions, estimate or use mental computation. Be sure the problem is understood. Establish clear expectations.

During:

Let go! Once students understand what the problem is asking, hold yourself back from stepping in when students struggle. Listen actively to understand the student’s approach to the problem. Ask: What ideas have you tried so far? Can you tell me more about…? Why did you …? Provide appropriate hints. Provide worthwhile extensions.

After:

Encourage student-to-student dialogue; request explanations to accompany all answers; call on students for their ideas, encouraging those who are shy; encourage students to ask questions; be certain your students also understand what you understand. Listen actively without evaluation. Summarize main ideas and identify future problems.

32

Afternoon: Breakout Groups

1) Donna Davis, National Staff Developer, Glencoe

2) Deep Dive into Lesson Study

3) Text-based Discussion: What’s Sophisticated about Elementary Mathematics?

4) Tech Tools: An Exploration of Web Resources

5) Core Standards

33

Learning Intentions: How did we do?

1) How did we analyze data? (QS 1.1)

2) How did that analysis of summative data help us create a portrait of mastery? How did we think we could provide meaningful and actionable feedback on the effectiveness of classroom level, curricular and instructional decisions? (QS 2.2)

3) How did we consider using these items to evaluate and adjust instructional practices and to monitor progress? (QS 5.2, 5.2)

Please complete the Evaluation form and leave on your table.