the progress and consistency tool (pact) progress and consistency tool (pact) information pack june...

The Progress and Consistency Tool (PaCT) | Information Pack June 2016 1

Supporting professional judgmentsin reading, writing and mathematics

The Progress and Consistency Tool (PaCT)Information Booklet

Watch the PaCT video at: pactinfo.education.govt.nz

For more information contact [email protected]

Date: June 2016


Making judgments using PaCT

What is the PaCT?

The PaCT has been designed to help teachers make dependable judgments on students’ progress and achievement.

The PaCT is more than an online tool:

› It supports the New Zealand Curriculum. It prompts teachers to notice what students know and can do across the breadth of mathematics, reading and writing. It helps teachers to understand the learning progressions set by the Curriculum and amplified by National Standards.

The Progress and Consistency Tool – Using it in your school

› It supports moderation. It moderates individual teacher expectations through the frameworks and illustrations. It provides a common framework for teachers to discuss their judgment decisions with one another.

› It also supports assessment for learning. The breakdown of mathematics, reading and writing into aspects gives greater clarity about what to notice when making observations of students, in learning conversations and through set tasks.

Tool Outcomes

Evidence obtained from assessment tools, including standardised tools such as:

› 6 year observation study › PAT › STAR › e-asTTle

Overall Teacher Judgment

Decision made in relation to National Standards.

Observation of Process

Evidence obtained from informal assessment opportunities, incorporating the observation of process, such as:

› Focused classroom observation

› Student work books › Tasks: maths tasks, ARBs › Running records › Student peer assessment

Learning Conversations

Evidence arising from learning conversations, such as:

› Conferencing › Interviewing › Questioning › Explaining › Discussing

The Progress and Consistency Tool

Teachers make a series of judgments related to aspects of reading writing and mathematics. PaCT captures the series of judgments and makes a recommendation on how this aligns to overall curriculum expectations.


So tell me more about the tool and how it can assist us.

The frameworks

The learning progression frameworks are a principal feature of the PaCT. They break down mathematics, reading and writing into different aspects and illustrate the stages of learning in each aspect. They give teachers a clearer picture of these areas of the New Zealand Curriculum and the National Standards. They prompt teachers to consider what students know and can do across the breadth of these areas.

The frameworks were developed by curriculum experts, using the New Zealand Curriculum, the National Standards, the Literacy Learning Progressions and the Numeracy Project. About 1000 teaching professionals helped to develop and trial the sets of llustrations linked to each aspect.

The aspects

There are seven to eight aspects in each learning area (mathematics, reading and writing). For example, mathematics aspects are: additive thinking; multiplicative thinking; patterns and relationships; using symbols and expressions to think mathematically; geometric thinking; measurement sense; statistical investigations; interpreting statistical and chance situations.

The PaCT breaks each aspect into observable and distinct learning stages. The illustration below shows a PaCT mathematics framework with the stages of learning shown as ‘bubbles’ on each aspect line. It acknowledges that learning is not linear and there can be wider ‘gaps’ between one stage and another.

The mathematics framework (as seen online), showing the aspect lines and the stages of learning.

Aspects of mathematics

Ad

dit

ive

thin

kin

g

Mu

ltip

licat

ive

thin

kin

g

Pat

tern

s an

d r

elat

ion

ship

s

Usi

ng

sym

bo

ls a

nd

exp

ress

ion

s

Geo

met

ric

thin

kin

g

Mea

sure

men

t se

nse

Sta

tist

ical

inve

stig

atio

ns

Inte

rpre

tin

g s

tati

stic

al a

nd

ch

ance

sit

uat

ion

s

A set of illustrations

6

The illustrations

The stages of learning in each aspect are linked to sets of illustrations. The illustrations span curriculum level 1 to curriculum level 4, which cover primary and intermediate students (years 0 to 8). They describe students working on a range of problems and tasks in a variety of contexts.

For example there are eight sets of illustrations for additive thinking that illustrate progress and achievement as follows.

Set one: illustrates students developing early qualifying skills.

Set two: illustrates students that solve problems by counting one-to-one, demonstrating knowledge of cardinality.

Set three: illustrates students that count to solve number problems involving addition and subtraction.

Set four: llustrates students that solve problems involving addition or subtraction of single digits, by counting on or back from the larger number. The language of the problem guides the student to the operation of addition or subtraction.

Set five: illustrates students that solve problems by partitioning a single digit addend or subtrahend and uses known addition facts to ten. The language of the problem guides the student to the operation of addition or subtraction.

Set six: illustrates students that respond to two- and three-digit problems in which the mathematical operation is transparent in the wording, by applying a strategy from a limited rehearsed repertoire that is likely to include place value or compensation when the number is close to a tidy number.

Set seven: illustrates students that respond flexibly to addition and subtraction problems involving whole numbers and simple decimals by applying, exploring and explaining a range of strategies, including using inverse operations, as they seek the most efficient method.

Set eight: illustrates students that show flexibility, a strong number sense and an ability to mentally carry out multiple steps as they solve complex problems.

These five illustrations are the Set Three illustrations for additive thinking.

Collectively they illustrate students that solve problems by counting one-to-one, demonstrating knowledge of cardinality.


Teachers select the set of illustrations for each aspect that best represent what students know and can do. If, for example, they decide set two of the illustrations for additive thinking is the ‘best fit’ for a particular student, they tick the second ‘bubble’ on the additive thinking aspect line.

To watch an introductory video on the PaCT and for more information go to: pactinfo.education.govt.nz

The online engine

The engine of the PaCT is a psychometric calibration that captures teacher judgments on the aspects and recommends an overall judgment that teachers can confirm or review. It helps teachers with the dilemma that many face when making their judgments. Without the PaCT, teachers have to figure out for themselves how much weight to give to particular aspects of reading, writing or maths as they make an overall judgment and it’s hard to work out how this balance might change as a child’s learning progresses over time.

8

Examples of illustrations from Set Four in Measurement Sense.

A practical example: Making judgments about student progress using the measurement aspect.

The mathematics framework has eight aspects. Each aspect has sets of illustrations that describe the important features that teachers should notice about what each student knows and can do. The PaCT asks teachers to choose illustrations that they judge best fit the student’s current skills and understanding. The sets of illustrations are numbered. Below illustrations from Set Four for ‘measurement sense’.

Set Four illustrates a student that:

› estimates and measures using the commonly used standard metric units of measure - centimetre, metre, litre and kilogram; and

› is developing an understanding of the concept of a scale, for example by constructing a measuring device and accurately marking the device with numbers for all units in that measure, including positioning a 0

› is able to partition a unit of measure and measure a half unit.

When a teacher chooses the set that ‘best fits’ the student, the PaCT captures their judgment. When the teacher has judged all aspects the PaCT recommends an overall judgment.

The illustration on the facing page shows how a teacher’s classroom observations and knowledge of their students assist them to make ‘best fit’ judgments when using the PaCT.


Making judgments in the PaCT efficiently

1. Think about your programme in relation to the aspect. Which set reflects the concepts/skills/strategies you have taught? (Most single level classrooms will only span 2-3 sets)

2. Think about your students. Can you rank them from the least able in relation to the aspect? As you do, your students will naturally cluster. Which sets are best fit for each of the clusters of students?

1. In our recent measurement unit I introduced metre rulers and litres and most of the class used them appropriately. I did some focussed teaching on how to measure length using a number of different non-standard lengths with a small group. This means that the third or fourth sets are likely to be the best fit for my students.

2. The majority of my class are able to use metres and litres for measuring so the fourth set is best fit for them. Two children were particularly confident but the fourth set is still best fit for them. The third set is the best fit for the small group who had additional teaching. I have one student with special education needs and the first set is the best fit for him.

1

2

5

6

7

Measurement sense

3

4

10

If we want to use the PaCT, how do we get started?

Think about the uses of the PaCT. You can use the tool to:

› Track the progress of a student or a class throughout the year.

› Identify areas where groups of students need extra teaching support or where the classroom programme needs adjustment.

› Inform decisions about teaching, learning and professional development.

› Plan your school’s future and set targets for student achievement.

Decide if you want to start small and scale up. For example, start with one PaCT framework and a small number of teachers and students.

Discuss the implementation of the PaCT with your staff and Board; ensure they are on board with the decisions.

Identify a ‘PaCT Leader’. This person will lead the implementation in the school. The role of the PaCT Leader is to:

› Work with other leaders and teachers in the school to plan implementation

› Work with leaders and teachers to make decisions about the best way to setup the PaCT to support the school’s judgment and moderation processes

› Co-ordinate the setup of the PaCT with the PaCT Administrator.

Identify at least one ‘PaCT Administrator’. The role of the PaCT Administrator is to:

› Configure the PaCT as planned by the school leaders.

› Set up the PaCT for the school (updating Education Sector logins and managing teacher access).

› Manage the student profile data within the PaCT including updating class lists/groups at the start of the school year or when students arrive or leave.

› Extract the information for the end of year teaching Overall Teacher Judgment (OTJ) results.

Sign the agreement and submit it to the Ministry. If you have any questions about implementation contact the Ministry before you sign up.

You can contact the PaCT programme team via [email protected]

You can find more informatio at pactinfo.education.govt.nz

There is a four step process to setting up the PaCT in your school.

Steps one and two - Planning and sign up1&2

› Ensure all teachers who will be using the PaCT have Education Sector Logons.

› To obtain Education Sector Logons you first sign up to the Education Sector Logon Provisioning Application (EPA).

› For further information on the EPA go to: http://www.education.govt.nz/school/managing-and-supporting-students/managing-student-attendance/education-sector-lpa/

› Or, for a guide to the Education Sector Logon process and useful links go to: pactinfo.education.govt.nz/gettingstarted/step-3-obtain-education-sector-logons/

› Setting up the PaCT for your school is a one-off task for the PaCT Administrator.

› For a guide to the set up process go to pactinfo.education.govt.nz/setup-and-administration/

› Once the PaCT Administrator has set up the tool and invited school leaders and teachers to join the PaCT, you can start using the tool to make judgments and create reports.

› Give your teachers time to practice before they use the tool for interim and end-of-year Overall Teacher Judgments (OTJs).

3 Step three - Education Sector Logons 4 Step four -

Set Up the PaCT

12

How will the PaCT help teachers and schools?

› Teachers make judgments on student progress and achievement, moment by moment, day by day. The PaCT will give teachers confidence that their interim and end of year judgments for National Standards are consistent and dependable.

› Teachers can use the PaCT to make judgments at various times of the year (out of the interim and end of year cycles) to build a clear and consistent understanding of student progress and achievement.

› The PaCT illustrations help to give teachers confidence that their judgments, based on professional knowledge and a range of classroom activities, are valid.

› The PaCT online tool helps overcome the difficulties in making overall judgments for students who achieve highly in some aspects of mathematics, reading or writing but not so highly in others.

› Because teachers will be basing their judgments on the same sets of frameworks and illustrations they, and their colleagues, will have more confidence that their judgments are consistent and dependable.

› The frameworks give teachers a common tool for moderation discussions with other schools.

› The PaCT also helps teachers give specific and constructive feedback to students and parents. For example, a teacher may say that a student is progressing well in addition and multiplication but needs more support with geometry.

The benefits of PaCT over time

› Teachers will be able to track students’ and class progress; use the data to inform teaching programmes; and use it to guide decisions about how to improve students’ learning.

› School leaders will have dependable information on progress in their school and areas where extra support might be needed.

› Boards of Trustees will be better able to track progress towards their school’s goals and targets and plan for the school’s future.

Dependable information is important for everyone: › Students need good feedback so they

can plan their own learning goals. › Parents need dependable information

so they can support their child’s learning. › A student’s next teacher needs to be

confident about the information they are receiving about the student’s progress and achievement.

› School leadership teams need dependable information to inform their self-review and planning.

› Trustees need dependable information to make resourcing decisions.


Questions and Answers about the PaCT

How has the tool been developed?

The PaCT has been developed by the Ministry of Education working with teachers, school leaders, and curriculum, research, and technical experts. There has been extensive consultation and about 1000 teaching professionals have either given feedback on the tool or helped with calibration trials.

Who has contributed to the development of the PaCT?

In addition to feedback from schools, several leading educational organisations, curriculum experts and suppliers have contributed to the development of the PaCT.

Maths Technology Limited and Borderfields Consulting were responsible for coordinating the writing and refinement of the framework.

The frameworks were developed by curriculum experts using the New Zealand Curriculum, the Standards, the Literacy Learning Progressions and the Number Framework.

Experts include Dr Gill Thomas, Sue Douglas, Dr Jenny Ward, Dr Chris Linsell, Dr Fiona Ell, Lisa Darragh, Denise Carter, Lynn Tozer, Lyn Bareta, Kath Jones, Pat Johnstone, Pip Arnold and Melanie Winthrop.

The New Zealand Council for Educational Research (NZCER) was contracted to trial the framework with teachers and schools to ensure it is psychometrically sound.

Catalyst IT Limited developed the software for the PaCT application.

A Teacher Advisory Group has contributed knowledge, advice, and expertise on educational and pedagogical aspects.

Where does the tool fit in the wider picture of assessment for learning?

› The primary purpose of National Standards policy is to support assessment for learning in the classroom.

› It is a standards-based system that relies on the professional judgments of teachers rather than one-off national tests.

› Information to make judgments is gathered through a range of assessment activities that normally occur in an effective classroom environment such as classroom observations, teaching and learning conversations and tests such as e-asTTle and PAT. To be effective, standards-referenced assessment must be valid (i.e. draw on a range of information) and teacher judgments consistent.

› The PaCT will support dependable (valid and consistent) teacher judgments of student achievement and progress.

14

How long will it take me to make a judgment using the PaCT?

The time it takes teachers to make judgments varies, whether they are using PaCT or not. Survey results indicate the time involved in making an OTJ, without using PaCT, can range from five minutes to an hour. Most teachers that have trialled PaCT find the time involved in making judgments reduces as they gain experience with the tool and the PaCT frameworks and illustrations help to build confidence in the judgments they make. The frameworks also give teachers a common tool for moderation discussions and will help with specific and descriptive feedback to students and parents.

Is the PaCT being designed to work in tandem with the SMS?

The PaCT has been designed to interface with SMS so that a teacher’s class groupings can be loaded automatically. This will reduce “double handling” of data and information. Schools can choose to export their final overall teacher judgments to their SMS if they wish to do so. Check if your SMS provider offers automated file uploads. However, not all schools use a SMS, so the PaCT can also be used as a “stand alone” tool.

Who owns the data captured by the PaCT?

PaCT data and the other data sets the Ministry of Education hold are owned by the Government so effectively belong to the New Zealand public. The Ministry is the steward of the data. This is the case across government and the Ministry’s draft Data Management and Use policy reaffirms the principles for managing the data and information the Government holds.

When data is captured by the PaCT in a school, what can the Ministry see/do with the data?

The Ministry is not able to access data in a school’s SMS. All exchange of data between the PaCT and SMS systems is under the control of the PaCT Administrator in the school.

The PaCT data will sit in a secure central repository just like other data sets the Ministry holds, such as PISA data.

Ministry IT staff engaged in support and maintenance will have access to the PaCT dataset to support users of the system, to investigate reported issues and maintain the system (e.g., back-up databases and installing application updates). This is very similar to the ongoing support for e-asTTle.

The Ministry may use the data for statistical or research purposes. In such cases no individual student, teacher or school will be identified. In addition, it will be subject to the same research procedures, ethics and protocols our other data sets are subject to, again like PISA data.

Will parents be able to access the data?

An individual student report will be available that may be useful for discussions with parents. Teachers and schools can choose to use this report at their discretion. At the moment, direct parental access to the tool itself is not planned.

16

www.education.govt.nz

Lifting aspiration and raising educational achievement for every New Zealander

the progress and consistency tool (pact) progress and consistency tool (pact) information pack june...

Documents