Assessing Early Years Writing

Sandy Heldsinger and Melanie Osborne

Research partners• Dr Stephen Humphry (Australian Research Council Linkage

grant with the Australian Curriculum and Reporting Authority and Western Australian Curriculum Council as Industry Partners, on which Stephen Humphry and David Andrich are chief investigators.)

• Western Australian Primary Principals’ Association (WAPPA), Comet Bay PS and Noranda PS

• Association of Independent Schools of Western Australia (AISWA), Moerlina School

Research

• Is it possible that given the right method of assessment, teacher judgements could be as reliable and comparable as large-scale tests?

• Yes• Two stages of research– Calibrated 60 exemplars (ordered them from

easiest to hardest) and developed a rubric– Assessed 29 performances using the calibrated

exemplars

Stage 1 – Calibrating the exemplars and developing the rubric

• Over 300 samples were collected from 11 schools; 60 samples were selected to represent the range: pre-primary to year 2.

• 15 ECE teachers (from DoE) were asked to compare pairs of performances, and select the better performance in pair.

• The person separation index was 0.994 (very high agreement of what constituted a better performance)

• Allowed us to generate an Early Writing Scale

• The 15 teachers reviewed the ordering and began to articulate why one performance was better than another.

• Based on that work we devised a rubric. We trialled the rubric and refined it.

• Marked 118 year 1 and year 2 performance using the NAPLAN guide and the Early Writing guide – The correlation was 0.9

Stage 2 – Assessing using calibrated exemplars

• 15 AISWA teachers • 29 samples (pre-primary to

year 2)• The participants were asked to

compare each of the samples with the calibrated exemplars and judge which exemplar each sample was closest to.

• Once they selected the closest exemplar they allocated the scale score to the sample performance.

• Inter-rater reliability: Cronbach’s alpha - 0.992. (This indicates a very high level of inter-rater reliability.)

• The judges’ estimates were compared with the mean from all other judges. The correlations range from 0.897 to 0.984

• The correlation of the early writing scale score to the NAPLAN total score was 0.895