Heading

Using the NPD: Some pitfalls and issues

Presentation to PLUG13 September 2006

Andrew RaySchools Analysis and Research Division, DfES

Heading

Statistical bulletins based on

analysis of the NPD

3

PLASC variablesPLASC

• FSM

• Ethnicity

• SEN

• Looked after children

• EAL

• Not covered here: variables linked to postcode

4

FSM is limited as a proxy for social class

FSM

Proportions achieving 5+ A*-C at GCSE in 2003

0

10

20

30

40

50

60

70

80

PR

OF

ES

SIO

NA

L

Hig

her

prof

essi

onal

Low

erpr

ofes

sion

al

NO

N-

PR

OF

ES

SIO

NA

L

Inte

rmed

iate

Low

ersu

perv

isor

y

Rou

tine

Oth

er: n

eith

erpa

rent

em

ploy

ed

Oth

er: 1

+ pa

rent

empl

oyed

ALL

PR

OF

ES

SIO

NA

L

Hig

her

prof

essi

onal

Low

erpr

ofes

sion

al

NO

N-

PR

OF

ES

SIO

NA

L

Inte

rmed

iate

Low

ersu

perv

isor

y

Rou

tine

Oth

er: n

eith

erpa

rent

em

ploy

ed

Oth

er: 1

+ pa

rent

empl

oyed

ALL

Non-FSM

Non-FSM

Non-FSM

Non-FSM

Non-FSM

Non-FSM

Non-FSM

Non-FSM

Non-FSM

Non-FSM

FSM FSM FSM FSM FSM FSM FSM FSM FSM FSM

Grey lines are 95% confidence intervals. Source: YCS linked to the NPD. See Table 5.7 in DfES Bulletin 'Trends in Attainment Gaps: 2005' (June 2006)

Non-FSM FSM

5

Code changes 2002 03Ethnicity

2002 classifications of the 2003 PLASC 'any other mixed background' ethnic group

WHITEUK

WHITEEURO PEAN

WHITE OTHER

WHITE NOT KNOWN

INDIAN

PAKI STANI

BANGLADESHI

BLACKCARIBBEAN

BLACKAFRICAN

BLACKOTHER

CHINESE

OTHER

NOT GIVEN

NOT OBTAINED

UNCLASSIFIED

Data taken from Ray Godfrey’s Changes in Ethnicity Codes in the Pupil Level Annual Schools Census 2002-2003

6

Local Authority variation in special school provision

SEN

Statemented Pupils in Local Authorities2005 Key Stage 2 cohort

0%

10%

20%

30%

40%

50%

60%

70%

0% 1% 2% 3% 4% 5% 6% 7%

Overall percentage with statements

Pro

port

ion

of s

tate

men

ted

SE

N in

spe

cial

sc

hool

s

7

Under-reporting of looked after children

Looked after children

Comparison of data sources on looked after children

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

Year 2 Year 6 Year 9 Year 11

Nu

mb

er

of

loo

ked

aft

re p

up

ils

Data from OC2

Data from PLASC

Source: PLASC quality report on pupils in care

8

Complex relationship with ethnicity

EAL

Pupils under the PLASC heading of 'Black African' by EAL status(2006 key stage 4, all schools)

64 327 748 67 1349 36 176114

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Angolan Congolese Ghanaian Nigerian Sierra Leonian Somali Sudanese Other BlackAfrican

EAL

Non-EAL

9

Attainment dataAttainment

• Changing test structures

• Changing qualifications

• Changing point systems

• Ages and Key Stages

• Ceiling effects

• Use of marks data for sublevels

10

Changes to KS3 English levelsLevels and

tests

Available levels in years before current NPDAvailable levels in years within current NPD

Year A D T B N W 1 2 3 4 5 6 7 8 EP19961997199819992000200120022003200420052006

W, 1 and 2 dropped in 1997 and replaced by B

Extension paper and Level 8 dropped in 2003

L3 dropped in 2004 but reintroduced in

2005

Dispplied replaced

by 'T' in 2004

11

Key Stage 2: artificial increase in ‘Bs’

Levels and tests

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

1998 1999 2000 2001 2002 2003 2004 2005

% a

chie

vin

g le

vel

A T/D B N Level 2

Pupils achieving below Level 3 at Key Stage 2 English

12

Key Stage 1: the move to teacher assessment

Levels and tests

Key Stage 1 time series

0

10

20

30

40

50

60

70

80

90

100

1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Pe

rce

nta

ge

ach

ievi

ng

leve

ls

Maths (L2+)

Maths (L2B+)

Maths (L3+)

Trial TA dataTest data

13

Additional qualificationsQuals

Some of the new qualifications only collected from 2004

-

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

90,000

2002 2003 2004 2005

Nu

mb

er

take

n

Entry Level Qualifications

Level 1 VRQ

Level 2 Key Skills

Level 3 AS Level?

14

New GCSE points system introduced in 2004

Points

Grade Old Points New Points

A* 8 58

A 7 52

B 6 46

C 5 40

D 4 34

E 3 28

F 2 22

G 1 16

U/X/Q 0 0

Gap between “fail” and G has increased

New Points = (Old Points x 6) + (10 x number of passes at G or above)

(but points not awarded for all qualifications under the old system)

15

Age 15 or KS4Ages and

stages

56

.3%

44

.3%

89

.0%

96

.4%

57

.1%

44

.9%

90

.2%

97

.4%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

5+ A*-C 5+ A*-C inc E&M 5+ A*-G Any Passes

At age 15 At end of KS4

• End of KS4 reporting allows for more flexible rates of learning.

• 98.8% of pupils at end of KS4 are 15, 0.3% were 14 and 0.9% were 16.

16

Test scores are truncatedCeilings

0

0.25

0.5

0.75

1

1.25

1.5

15 17 19 21 23 25 27 29 31 33 35

Key Stage 2 Average Point Score

% o

f p

up

ils

usi

ng

fin

e g

rad

es

0

5

10

15

20

25

30

% o

f p

up

ils

usi

ng

wh

ole

lev

els

17

Marks and levelsSublevels

2006 Key stage 2 thresholds

Marks

Levels Points lower upper lower upperN 15 0 19 0 152 15 20 22 16 183 21 23 42 19 454 27 43 69 46 775 33 70 100 78 100

For example: Level 3 marks range: 20 for English, 27 for Maths

English Maths

18

Sublevels don’t directly correspond to marks

Sublevels

Comparison of marks and fine grade points

15.0

20.0

25.0

30.0

35.0

40.0

20 30 40 50 60 70 80 90 100

Mark

Fin

e gr

ade

poin

ts

English

Maths

19

Coverage of the NPDThe NPD

• Pupils with incomplete records (2 examples)

Not covered here:

• the extent of coverage of the independent sector

• pitfalls in choosing coverage, e.g. when to include selective schools or schools that are semi-independent (e.g. City Technology Colleges)

• whether or not to impute missing values

20

Pupils with no PLASC or KS4 records

The NPD

8

26

48

1719

31

38

11

0

10

20

30

40

50

60

Below Level 3 Level 3 Level 4 Level 5 or above

KS2 English Level

Per

cen

tag

e o

f p

up

ils

(%

)

1998 national KS2 figures Pupils with no PLASC in 2002 or 2003 and no GCSEs

Source: Trends in Attainment Gaps bulletin Appendix D

21

Pupils with no PLASC recordsThe NPD

8

26

48

1716

3739

9

3

9

47

42

0

10

20

30

40

50

60

Below level 3 Level 3 Level 4 Level 5 or above

KS2 English level

Per

cen

tag

e o

f p

up

ils

(%)

1998 national KS2 figures Non Independent schools Independent schools

Source: Trends in Attainment Gaps bulletin Appendix D

22

Issues for PLUG

• Future speakers: experts on PLASC or test data?

• Documentation: what improvements can be made?

• Further investigation: are there useful analyses that could illuminate some data issues?

•Can any common standards be agreed, or should everything be left to individual analyses?

23

Useful links

Bulletins:

Trends in Attainment Gaps http://www.dfes.gov.uk/rsgateway/DB/SBU/b000665/index.shtml

Characteristics of Low Attainers http://www.dfes.gov.uk/rsgateway/DB/SBU/b000588/index.shtml

Variation in pupil progress http://www.dfes.gov.uk/rsgateway/DB/SBU/b000481/index.shtml

Progress by pupil characteristics http://www.dfes.gov.uk/rsgateway/DB/SBU/b000402/index.shtm

Other key publications:

Results by pupil characteristics 2005 http://www.dfes.gov.uk/rsgateway/DB/SFR/s000640/index.shtml

Ethnicity and education topic note http://publications.teachernet.gov.uk/default.aspx?PageFunction=productdetails&PageMode=publications&ProductId=DFES+0208+2006&