Supporting Teachers to make Overall Teacher Judgments

The Consortium for Professional Learning

The Purpose of OTJ workshops Moderation - To deepen the quality of professional talk in order to establish shared understandings and improve teaching and learning.

To apply evidence based decision making to buildthe consistency and validity of teacher judgments.

The aspect framework will provide the context for achieving this purpose.

Participants will have support materials to assist leading professional talk and decision making about how to consider and respond to student learning.

Workshop process• Review the current context in relation to National

Standards • Use the Aspect Frameworks and their illustrations• Make an evidence-based judgment using the

frameworks• Engage in professional discussion about judgments

(moderation process)• Discuss key considerations when moderating

2010

The New Zealand Curriculum

Literacy Learning Progressions &

The Number Framework

The National Standards

Current context• Curriculum = outcomes• Progression documents = indicators• Standards = signposts for progress

Current context• OTJs are made using multiple sources of

evidence• Evidence is collected in a variety of ways -

learning conversations, observations and assessment tools

• OTJs are made using evidence that show application of the skills, knowledge and understandings across the curriculum

• Consistency is essential to the dependability of OTJs – moderation is the process.

Monitoring and Evaluation Project

• Evidence suggests that OTJs do notyet have dependability.

• This does not mean that all OTJs are inaccurate.

• The inconsistency in teachers’ ratings is likely a result of the relatively broad nature of the National Standards scale.

• The report indicates principals and teachers perceive a need for further support around OTJs.

Reference 2013

Monitoring and Evaluation Project• Moderation – more common in

the area of writing than reading or maths.

• There has been an increase in the understanding of moderation as an ‘evaluation discussion informed by evidence of student achievement.’• No. of schools moderating effectively has

increased

Reference 2013

How does your process sit against the picture we have presented

– 2 minutes

The New Zealand Curriculum

Literacy Learning Progressions &

The Number Framework

The National

Standards

The Aspect Framework

Aspect Framework

Maths framework

The Aspect Framework – Reading and Writing• The standards for reading and writing are constructed

differently from the way the mathematics standards are. This is because they are cross-curricular competencies.

• The standards for reading and writing are established by the level of literacy expertise that students need if they are to meet the reading and writing demands in all curriculum areas.

• Therefore, the aspects reflect this by deliberately using illustrations from different learning areas. For example, a science report or a technology text.

Illustrations

• What are the common components? • What is the importance of each part?

The components of an illustration• A unique title.• An annotation that highlights the key features of

the student response.• The problem (for mathematics) or the text and

task (reading and writing), which are the focus of the illustration. This also includes the conditions under which the task was completed, for example, independently or with support.

• The student response details how and what the student did in response to the problem or task. The response may include a work sample and or a transcript of a discussion with a teacher.

Sets of Illustrations• The observable steps of learning in each aspect

are described through sets of illustrations.

• Each set has been developed to illustrate a big or important idea in the aspect progression.

• Each set describes the important features that teachers should notice and recognise about what each student knows and can do.

Activity 2

Order these illustrations to show the progression through the learning steps.

1st - Additive Thinking (white paper)

2nd - Measurement Sense (yellow paper)

Activity 2 answers – Additive thinking

Bags of wheat

Dice

Tens frames

Bears in the cave

Trusting the count

Rotten plums

The meeting

Getting off the bus

Activity 2 answers – Measurement Sense

Comparing two objects by weight

Comparing coloured strips

Coloured strips

Matchstick rulers

Make a shake

Complex area

New carpet for the library

How does an activity such as this build consistency?

Illustrations The illustrations are NOT assessment tasks to see if students can do them.

They should prompt questions such as:

• What evidence do I have?• Is this the type of level that ______ could demonstrate?• Is this the ‘kind of way’ that ______ would solve these

problems?• Is this the ‘kind of way’ that ______ would write ideas,

experiences or information• Is this the ‘kind of way’ ______ would respond to a text

Activity 3

Purpose: • Use the aspect framework to guide judgment

making

• To use professional talk as a form of moderation

Setting the scene

Judgments should be made as a result of noticing and recognising student learning in the course of a normal classroom programme.

To provide a succinct collation of student thinking we have used a one on one interview for this workshop.

Learning context – the supermarket.

Ben In pairs use the video evidence and teacher description to make a judgment in relation to the following two aspects: • Additive thinking• Measurement sense

• Use the concept of ‘best fit’• Use the illustrations – confirm with

descriptions

Moderation is the process of sharing expectations and understanding of standards in order to improve the consistency of decisions (reaching agreement).

(TKI Assessment)

Professional talk:- Planned- Informal

Principles of moderation• Evidence-based discussions• Tolerance of different views • Openness - viewing differences as an

opportunity to learn• Willingness to adapt thinking• Use agreed reference materials• Accurate judgments will better support

student learning• Multiple sources of evidence is required

Moderate your decision

• Where did you place your judgment and why?

• Is there consensus?

• How does your evidence relate to the set of illustrations?

• What do others think and why?(even if you all scored the same)

• Have we reached consensus on ‘best fit’?

Using the framework to support moderation

• Moderate what?– areas teachers are finding difficult – sample selection of students.– Students for whom making aspect

decisions was difficult (gaps in evidence or where ‘best fit’ was difficult)

Planning for moderation

When/who/how:• When does moderation need to take place?• Who is involved?• Within school / across schools?

The value of the framework

• Builds consistency of what to notice and recognize

• Highlights gaps in learning opportunities• Gives a consistent framework within and

across schools• Progress through learning steps is made

visible

Ben’s recommended OTJ

Recommended OTJ

= End of Year 5

633

5 45

33

Role of standardised tools• Standardised test information is still an

important form of evidence• For example, correlation between standardised

tool and OTJ can provide a degree of confidence.• Non-correlation does not mean the

recommended OTJ is incorrect.

Links• The Aspects Framework • The Mathematics aspects• Overall Teacher Judgements • Moderation• The National Standards School Sample Monitorin

g and Evaluation (NSSSME) Project

• Support material from this workshop will be available on the CPL website .

Reflect and evaluate

1. In what ways will this workshop impact on your moderation practice in your school?

2. What are the next steps for Teaching and Learning in your school?