Feedback from CEM Assessments:

Individual Pupil Records & Predictions

Belfast, March 6th 2013Neil Defty

Business & Development ManagerCEM

Neil@cem.dur.ac.uk

• To inform professional judgement

• To start a conversation

Baseline Assessment Data

USING BASELINE DATA FOR TEACHING AND LEARNING

IF FEEDBACK DATA IS TO BE TRUSTED THEN PROBABLY:

• The students understood the purpose of the assessments

• And each student did their best at the time…….

• As a year group, how able are they?

• What strengths and weaknesses does the group have?

• How did each student perform?

Bands, percentiles, standardised scores…

0

50 60 70 80 90 100 110 120 130 140 150

Standardised scores

DC B

A

105 20 301 40 90 958070 996050Percentiles:

25 75

Alis: year 12 students, two baseline profiles for the same school

Nationally, 25% in each band

Band Profile Graph: all MidYIS cohort

Checking this graph each year will give you an immediate overview of your intake.

A school with a ‘completely average’ intake would have 25% of pupils within each band.

Band D Band A

3526

21 17

0

10

20

30

40

50

60

70

80

90

100

D C B A

Perc

ent

Maths: Percent in each BandYear 7 MidYIS Test 2011/2012

17

37

2025

0

10

20

30

40

50

60

70

80

90

100

D C B A

Perc

ent

Vocabulary: Percent in each BandYear 7 MidYIS Test 2011/2012

Band A Band ABand D Band D

1121

33 35

0

10

20

30

40

50

60

70

80

90

100

D C B A

Perc

ent

Non-Verbal: Percent in each BandYear 7 MidYIS Test 2011/2012

2924 26

21

0

10

20

30

40

50

60

70

80

90

100

D C B A

Perc

ent

Skills: Percent in each BandYear 7 MidYIS Test 2011/2012

2924 26

21

0

10

20

30

40

50

60

70

80

90

100

D C B A

Perc

ent

Skills: Percent in each BandYear 7 MidYIS Test 2011/2012

38

2619 17

0

10

20

30

40

50

60

70

80

90

100

D C B A

Perc

ent

Proof Reading: Percent in each BandYear 7 MidYIS Test 2011/2012

24 23 2330

0

10

20

30

40

50

60

70

80

90

100

D C B A

Perc

ent

Perceptual Speed & Accuracy: Percent in each BandYear 7 MidYIS Test 2011/2012

Vocabulary Maths Non Verbal Skills MidYIS Score

St. S

core

Ban

d

St. S

core

Ban

d

St. S

core

Ban

d

St. S

core

Ban

d

St. S

core

Ban

d

Average 100.6 95.9 105.3 99.2 98.1

St. error 1.1 1.1 1.1 1.1 1.1

On another day, with 95% certainty, the score of 100.6 (+/- 1.1 x 2) • would not be higher than 100.6 + 2.2 = 102.8• would not be lower than 100.6 – 2.2 = 98.4

Standardised Scores Sort Data By...

Mean Score: 100

Standard Deviation: 15

Scores Standardised On A Nationally Representative Sample of Schools

Vocabulary Maths Non Verbal Skills MidYIS ScoreSt

. Sco

re

Ban

d

St. S

core

Ban

d

St. S

core

Ban

d

St. S

core

Ban

d

St. S

core

Ban

d

100.6 95.9 105.3 99.2 98.1

1.1 1.1 1.1 1.1 1.1

134 A 119 A 116 A 87 D 130 A95 C 97 C 88 D 85 D 95 C99 C 104 B 112 A 110 B 101 B93 C 94 C 120 A 91 C 93 C

116 A 99 C 118 A 84 D 109 B93 C 95 C 120 A 113 A 93 C92 C 83 D 112 A 94 C 87 D

102 B 102 B 112 A 108 B 102 B

Proof Reading PSA

St. S

core

Ban

d

St. S

core

Ban

d

93.7 102.2

1.1 1.1

101 B 83 D76 D 93 C97 C 114 A80 D 99 C

102 B 78 D119 A 107 B80 D 102 B

106 B 107 B

50 60 70 80 90 100 110 120 130 140 150 Standardised Test Score

Stanine 1

2 3 4 5 6 7 8 9

4% 4 % 7% 7% 12% 12% 17% 17% 20%

(See www.cem.org/midyisiprbooklet for report: “Using MidYIS Individual Pupil Records to Inform Teaching and Learning”)• To start a conversation

Individual Pupil Records (IPRs)

Individual Pupil Record 75Date Of Birth: 24/12/97 Student Number:Sex: F Form:

Band Stanine Percentile Standard Score

Vocabulary B 5 53 101

Maths B 5 52 101

Non-Verbal C 5 46 98

Skills C 5 46 99

MidYIS Score B 5 52 101

50

60

70

80

90

100

110

120

130

140

150

Vocabulary Maths Non-Verbal Skills MidYIS Score

Natio

nally

Sta

ndar

dise

d Sc

ores

Nationally Standardised Scores with 95% Confidence Band

Individual Pupil Record LAUREN BAKERDate Of Birth: 28/11/99 Student Number: J870396003050Sex: F Form: 7C

Band Stanine Percentile Standard Score

Vocabulary C 4 27 91

Maths B 6 68 107

Non-Verbal B 6 75 110

Skills D 2 10 81

MidYIS Score C 5 45 98

50

60

70

80

90

100

110

120

130

140

150

Vocabulary Maths Non-Verbal Skills MidYIS Score

Natio

nally

Sta

ndar

dise

d Sc

ores

Nationally Standardised Scores with 95% Confidence Band

INSIGHT Pupil IPR

Band Stanine Percentile Standard Score

KS3 Equivalent

Speed Reading B 6 69 107 6c

Text Comprehension B 5 60 104 5a

Passage Comprehension C 5 45 98 5b

Overall Reading B 5 59 103 5a

Number & Algebra D 3 19 87 4a

Handling Data B 6 61 104 6a

Space, Shape & Measures D 4 23 89 5c

Overall Mathematics C 4 31 93 5b

Biology A 7 82 114 6a

Chemistry A 9 96 127 7a

Physics A 8 89 118 7c

Overall Science A 8 93 122 7b

Vocabulary B 6 65 106

Non Verbal B 6 70 108

Skills A 8 92 121

Overall Ability C 4 36 94

N2a

c3bac

4bac

5bac

6bac

7bac

8ba

Spee

d R

eadi

ng

Text

Com

preh

ensi

on

Pass

age

Com

preh

ensi

on

Ove

rall

Rea

ding

Num

ber &

Alg

ebra

Han

dlin

g D

ata

Spac

e, S

hape

& M

easu

res

Ove

rall

Mat

hem

atic

s

Biol

ogy

Che

mis

try

Phys

ics

Ove

rall

Scie

nce

KS3

Leve

lKS3 Equivalent Levels with 95% Confidence Band

IPRs in KS3 Levels

NATIONAL AVERAGE

The code:

j: reduces size

k: infills

m: inverts

“Predictions”

BASELINE SCORE

G

RA

DE

G

RA

DE

* ** *** *******************

* ** *** ********************************

* ** *** ********************************* *** ** *

* ** *** ************************************ *** ** *

* ** *** ******************************** **** ** *

* ** ***************************** *** ** *

* ** *** ******************* ** * *

* ** *********** ** *

How is a ‘prediction’ generated?

C

A*

A

B

50 100 150

3 key points are:

• The higher the baseline score the higher the final grade

• Any one grade is achievable from a range of baseline scores

• From any baseline score, a range of grades are possible

BASELINE SCORE

G

RA

DE

G

RA

DE

* ** *** *******************

* ** *** ********************************

* ** *** ********************************* *** ** *

* ** *** ************************************ *** ** *

* ** *** ******************************** **** ** *

* ** ***************************** *** ** *

* ** *** ******************* ** * *

* ** *********** ** *

Subject National trend line (regression line)

How is a ‘prediction’ generated?50% on or above the trend line

50% on or below the trend line

‘PREDICTION’ (expected grade)C

A*

A

B

2008 GCSE Maths (representative sample)

0

1

2

3

4

5

6

7

8

9

0 20 40 60 80 100

baseline score

Res

ult

‘Predictions’…...are based on

Average performance by similar pupils in past examinations

The problem with the word ‘prediction’ is…?

An alternative is ‘expected’ grade

Predictions

4 points = D

6.6 points = A/B Trend line

FACTORS THAT WILL INFLUENCE RELIABILITY OF PREDICTIONS:

• Knowledge of student • Parental support/home life• Peer influences/social life• Student attitude, interest, language• Expectations of staff• Department/institution ethos • Resources • Quality of teaching and learning: pace of lessons• Understanding how children learn………• And the reliability of the predictions......

29

Correlation = 1

Res

ult

Correlation = 0

Correlation = 0.7

subjectMIDYIS( Yr 7)

to GCSEART 0.49BIOL 0.56ENG LANG 0.7FRENCH 0.64HISTORY 0.63MATHS 0.74MUSIC 0.57

CORRELATIONS

SUBJECTGCSE to A2

ART and DESIGN 0.54BIOLOGY 0.69ECONOMICS 0.65ENG LIT 0.71FRENCH 0.6HISTORY 0.69MATHS 0.6MUSIC 0.65PHYS EDUC 0.65PSYCHOLOGY 0.62

50 60 70 80 90 100 110 120 130 140 1501

2

3

4

5

6

7

8

MidYIS Year 7 to GCSE 2011 Value Added Analysis Art & Design

Biology

English

French

History

Mathematics

Science

MidYIS Test Score

GC

SE P

oint

s Sc

ore

The graph below shows the middle 2/3 of some subject trend lines

0 0 15

19

39

27

9

10

5

10

15

20

25

30

35

40

45

U G F E D C B A A*

Perc

ent

Grade

Individual Chances Graph for Student 4- GCSE EnglishMidYIS Score 101 MidYIS Band B

Teacher's Adjustment : 0 grades / levels / points

Prediction/expected grade: 5.1 grade C

Most likely grade

Not a label for life...just another piece of information

• The Chances graphs show that, from almost any baseline score, students come up with almost any grade - - -there are just different probabilities for each grade depending on the baseline score.

• In working with students these graphs are more useful than a single predicted or target grade

• Chances graphs show what can be achieved: – By students of similar ability– By students with lower baseline scores

Predictions & Chances Graphs to GCSE subjects based on Year 9 nationally standardised scores

Please refer to the Information Sheet for details of the method used to make these predictions

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Student Surname Forename Sex Form CEM Id Cust Mid

YIS

Scor

e

Mid

YIS

Band

Addi

tiona

l App

lied

Scie

nce

Addi

tiona

l Sci

ence

Art &

Des

ign

Biol

ogy

Busi

ness

Stu

dies

Che

mis

try

Des

ign

& Te

chno

logy

Dra

ma

Engl

ish

Engl

ish

Lite

ratu

re

Fren

ch

Geo

grap

hy

1 F 9140005J 105 B 4.4 5.3 5.8 5.9 5.1 5.9 5.3 5.2 5.4 5.5 5.0 5.12 F 9140005K 111 A 4.7 5.8 6.1 6.3 5.6 6.3 5.7 5.6 5.8 5.9 5.4 5.63 F 91400042 108 B 4.6 5.6 6.0 6.1 5.4 6.1 5.5 5.4 5.6 5.7 5.2 5.44 F 91400043 101 B 4.2 5.0 5.6 5.7 4.8 5.7 5.0 5.0 5.1 5.3 4.7 4.7

Teac

her's

Ad

just

men

t (G

rade

s)

Predictions based on 2011 Regression Analysis

MidYIS Year 9 Test Academic Year 2011/2012

GCSE Point PredictionsPoints To Grades Adjust

CEM Centre Scale

QCA Scale

Display Legend

Upper Quartile of Schools

Prior Value added

Generate File for MIS Import

to place school at 75th percentile of value added results

Predictions & Chances Graphs to GCSE subjects based on Year 9 nationally standardised scores

Please refer to the Information Sheet for details of the method used to make these predictions

0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2

Student Surname Forename Sex Form CEM Id Cust Mid

YIS

Scor

e

Mid

YIS

Band

Addi

tiona

l App

lied

Scie

nce

Addi

tiona

l Sci

ence

Art &

Des

ign

Biol

ogy

Busi

ness

Stu

dies

Che

mis

try

Des

ign

& Te

chno

logy

Dra

ma

Engl

ish

Engl

ish

Lite

ratu

re

Fren

ch

Geo

grap

hy

1 F 9140005J 105 B 4.6 5.5 6.0 6.1 5.3 6.1 5.5 5.4 5.6 5.7 5.2 5.32 F 9140005K 111 A 4.9 6.0 6.3 6.5 5.8 6.5 5.9 5.8 6.0 6.1 5.6 5.83 F 91400042 108 B 4.8 5.8 6.2 6.3 5.6 6.3 5.7 5.6 5.8 5.9 5.4 5.64 F 91400043 101 B 4.4 5.2 5.8 5.9 5.0 5.9 5.2 5.2 5.3 5.5 4.9 4.9

Teac

her's

Ad

just

men

t (G

rade

s)

Predictions based on 2011 Regression Analysis

MidYIS Year 9 Test Academic Year 2011/2012

GCSE Point PredictionsPoints To Grades Adjust

CEM Centre Scale

QCA Scale

Display Legend

Reset Adjustments

Prior Value Prior Value addedadded

Generate File for MIS Import

Predictions & Chances Graphs to GCSE subjects based on Year 9 nationally standardised scores

Please refer to the Information Sheet for details of the method used to make these predictions

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Student Surname Forename Sex Form CEM Id Cust Mid

YIS

Scor

e

Mid

YIS

Band

Addi

tiona

l App

lied

Scie

nce

Addi

tiona

l Sci

ence

Art &

Des

ign

Biol

ogy

Busi

ness

Stu

dies

Che

mis

try

Des

ign

& Te

chno

logy

Dra

ma

Engl

ish

Engl

ish

Lite

ratu

re

Fren

ch

Geo

grap

hy

1 F 9140005J 105 B 4.4 5.3 5.8 5.9 5.1 5.9 5.3 5.2 5.4 5.5 5.0 5.12 F 9140005K 111 A 4.7 5.8 6.1 6.3 5.6 6.3 5.7 5.6 5.8 5.9 5.4 5.63 F 91400042 108 B 4.6 5.6 6.0 6.1 5.4 6.1 5.5 5.4 5.6 5.7 5.2 5.44 F 91400043 101 B 4.2 5.0 5.6 5.7 4.8 5.7 5.0 5.0 5.1 5.3 4.7 4.7

Teac

her's

Ad

just

men

t (G

rade

s)

Predictions based on 2011 Regression Analysis

MidYIS Year 9 Test Academic Year 2011/2012

GCSE Point PredictionsPoints To Grades Adjust

CEM Centre Scale

QCA Scale

Display Legend

Upper Quartile of Schools

Prior Value added

Generate File for MIS Import

Or insert own valuesand click Adjust

Predictions & Chances Graphs to GCSE subjects

based on Year 9 nationally standardised scores

Please refer to the Information Sheet for details of the method used to make these predictions

0.0 0.3 1.4 0.7 0.0 0.7 0.0 0.9 1.3 0.8 0.7 1.4

Student Surname Forename Sex Form CEM Id Cust Mid

YIS

Scor

e

Mid

YIS

Band

Addi

tiona

l App

lied

Scie

nce

Addi

tiona

l Sci

ence

Art &

Des

ign

Biol

ogy

Busi

ness

Stu

dies

Che

mis

try

Des

ign

& Te

chno

logy

Dra

ma

Engl

ish

Engl

ish

Lite

ratu

re

Fren

ch

Geo

grap

hy

1 F 9140005J 105 B 4.4 5.6 7.2 6.6 5.1 6.6 5.3 6.1 6.7 6.3 5.7 6.52 F 9140005K 111 A 4.7 6.1 7.5 7.0 5.6 7.0 5.7 6.5 7.1 6.7 6.1 7.03 F 91400042 108 B 4.6 5.9 7.4 6.8 5.4 6.8 5.5 6.3 6.9 6.5 5.9 6.84 F 91400043 101 B 4.2 5.3 7.0 6.4 4.8 6.4 5.0 5.9 6.4 6.1 5.4 6.1

Teac

her's

Ad

just

men

t (G

rade

s)

Predictions based on 2011 Regression Analysis

MidYIS Year 9 Test Academic Year 2011/2012

GCSE Point PredictionsPoints To Grades Adjust

CEM Centre Scale

QCA Scale

Display Legend

Upper Quartile Upper Quartile of Schoolsof Schools

Reset Adjustments

Generate File for MIS Import

Predictions & Chances Graphs to GCSE subjects based on Year 9 nationally standardised scores

Please refer to the Information Sheet for details of the method used to make these predictions

0.0 0.3 1.4 0.7 0.0 0.7 0.0 0.9 1.3 0.8 0.7 1.4

Student Surname Forename Sex Form CEM Id Cust Mid

YIS

Scor

e

Mid

YIS

Band

Addi

tiona

l App

lied

Scie

nce

Addi

tiona

l Sci

ence

Art &

Des

ign

Biol

ogy

Busi

ness

Stu

dies

Che

mis

try

Des

ign

& Te

chno

logy

Dra

ma

Engl

ish

Engl

ish

Lite

ratu

re

Fren

ch

Geo

grap

hy

1 F 9140005J 105 B C/D B/C A A/B C A/B B/C B A/B A/B B/C A/B2 F 9140005K 111 A C/D B A*/A A B/C A B/C A/B A A/B B A3 F 91400042 108 B C/D B A*/A A B/C A B/C A/B A A/B B A4 F 91400043 101 B D B/C A A/B C A/B C B A/B B B/C B

Teac

her's

Ad

just

men

t (G

rade

s)

Predictions based on 2011 Regression Analysis

MidYIS Year 9 Test Academic Year 2011/2012

GCSE Grade PredictionsGrades To Points Adjust

CEM Centre Scale

QCA Scale

Display Legend

Upper Quartile Upper Quartile of Schoolsof Schools

Reset Adjustments

Generate File for MIS Import

Prediction/expected grade: 6.4 grade A/B

Most likely grade

Predictions & Chances Graphs to GCSE subjects based on Year 9 Independent Sector standardised scores

Please refer to the Information Sheet for details of the method used to make these predictions

Student Surname Forename Sex Form CEM Id Cust Mid

YIS

Scor

e

Mid

YIS

Ban

d

Addi

tiona

l Sci

ence

Art &

Des

ign

Biol

ogy

Busi

ness

Stu

dies

Che

mis

try

Cla

ssic

al C

ivilis

atio

n

Des

ign

& Te

chno

logy

Dra

ma

Engl

ish

Engl

ish

Lite

ratu

re

Fren

ch

Geo

grap

hy

Ger

man

1 F 9140005J 87 D 5.8 6.6 6.4 5.7 6.4 5.8 6.1 6.1 6.1 6.2 5.7 6.1 5.82 F 9140005K 94 C 6.1 6.8 6.7 6.0 6.7 6.1 6.4 6.4 6.5 6.4 6.1 6.5 6.23 F 91400042 90 C 6.0 6.7 6.6 5.9 6.6 6.0 6.3 6.3 6.3 6.3 5.9 6.3 6.04 F 91400043 81 D 5.5 6.5 6.1 5.4 6.1 5.6 5.8 5.9 5.9 5.9 5.4 5.8 5.6

Predictions based on 2011 Independent Sector Regression Analysis

MidYIS Year 9 Test Academic Year 2011/2012

GCSE Point PredictionsPoints To Grades

CEM Centre Scale

QCA Scale

Display Legend

Generate File for MIS Import