Severe Grading in MFL- the people

• Helen Myers (Past President, ALL & Asst

Head, The Ashcombe School)

• David Blow (Head, The Ashcombe School)

• Duncan Byrne (Chair ISMLA)

• Geoffrey Plow (Exams, ISMLA)

• John Dunford (Gen. Secretary ASCL)

• Peter Downes (former Chief ML examiner, Head, President ALL & SHA)

draft Sat 18th

ISMLA = Independent Schools ML Assoc

ALL = Assoc. for Language Learning

• Long-standing concern - research by Nuttall in 1974; Peter Downes raised over time, leading to recommendation in the Nuffield Report

• Series of presentations by Helen Myers and David Blow using DfES and other data

• Continuing concern in ML community, esp as Languages is no longer compulsory and numbers go into freefall

• Series of meetings by ALL, ISMLA etc with QCA and exam boards (‘06)

Severe Grading in MFL- brief history

GCSE MFL grading• Our outline principle is that students studying

a particular GCSE subject should have a reasonable expectation that they will get comparable grades across a range of subjects.

• The PANDA Relative Performance Indicator measures the performance of students in their other subjects compared with the particular one. By the principle above, these should be negligible. They are not.

“Severe grading”Terminology is important. Words like “hard”,

“standards”, “demand” carry multiple connotations.

Restate principle that pupils of comparable prior

attainment should attain comparable grades across

a range of subjects whether Business Studies

which will be only started at GCSE, Maths, Drama,

etc all of which have very varying situations for

different pupils

thanks to Peter Tymms for this

phrase

Depressing exams (NB separate issue from grading)

As professionals, we know (or should know) that

a “difficult” paper with low scores does not need

to lead to low grades as the grade boundaries

/conversion to UMS will be adjusted to

compensate...

BUT pupils leaving that exam will perceive that

they have performed badly, which compounds the

“severe grading” problem

Issues for pupils

• Perceive that they are performing worse

in languages in other subjects

• Reduces the take-up at KS4 and A-level

• In each case, pupils are doing relative

comparisons on grades (note key issue

at AS in French as compare with other

subjs - at A2 Univs can make subj. by

subj. adjustments)

Issues for staff & managers

• Heads and Governors may think that

languages are “under-performing”

• Mixed messages re school targets and

DCFS targets

• Ritual debate each year about whether

standards are slipping / pupils are

choosing “soft” subjects........

• National outcry at drop in numbers GCSE Aug 06

- Alan Johnson sets up Dearing Review

• “Mr Johnson wants to see what more can be done to

encourage14-16 year olds to study GCSE or other

language courses leading to a recognised qualification.”

DfES Press Notice 2006/0144 Oct 06

• Meeting Nov 06 with Lord Dearing–Idea of broad comparability with Maths grading

–Awareness of public and political sensitivity

Dearing Review

Numbers - GCSE French

German - similar pattern but 40% as many, e.g 120,000 (French 300,000)

‘06‘02

• drop of 30,000 in number gaining A*-C from ‘02 to ‘06

• drop of 40,000 in number gaining A*-C from ‘02 to ‘08

Grades‘02

‘08

German - similar pattern but 10,000 (‘02 to ‘06) and

18,000 (‘02 to ‘08)

The mandate of QCA and the examination boards is to ensure that "standards are maintained over time". This is an enormously challenging task and one which has a high public profile.

Comparability of grading

It is now accepted by QCA and the exam boards that pupils of similar prior/concurrent attainment will gain a lower grade in languages than most other subjects.

The presented documentation is available at http://www.all-london.org.uk/severe_grading.htm

Because this difference is embedded historically, and their mandate is to maintain standards over time, a decision from the DfES is needed to change this.

Joint Proposal to Dearing (1)

Opportunity for change

In the light of the crisis at KS4, there is now the opportunity for a change.

The very strong experience of modern language teachers, advisers and senior managers is that the discrepancy in grading is one significant negative factor in the decline.

Making changes here will not in itself solve the overall problem, but, to use a metaphor, will remove having a hand tied behind one's back.

Joint Proposal (2)

ProposalThat QCA, and thus the exam boards are mandated to

ensure that the grades awarded in Modern Languages move to become comparable, on average, to those awarded in Mathematics. Different methodologies exist to measure the comparability (described in the papers), and these should all be used, so that the pupils of similar prior/concurrent attainment have a similar average grade in Modern Languages to that in Mathematics.

This proposal should be treated independently of the many other ideas and proposals to deal with the issue of KS4 M.L.

Joint Proposal

• Led to recommendation in Consultation Report (Jan 07) confirmed in Final Report (Mar 07) to have definitive study published on “perception” of severe grading (with implicit corollary that there should be action...) - the consultation had “found strong confirmation of the view that award of grades is more demanding than for most other subjects”

Dearing Review

• Dr Coe, of Durham's curriculum, evaluation and management centre, believed such trends were repeated in other years, but insisted "the question of difficulty is not about content of the subject ... It is purely about the examination and grading process."

• David Willetts, Conservative education spokesman, said: "If there is evidence modern languages is tougher than other GCSEs, then that is something that has to be corrected. They should be the same level of challenge as

traditional academic GCSEs." http://education.guardian.co.uk/gcses/story/0,,2031769,00.html

James Meikle, education correspondent

Monday March 12, 2007

Dearing Review - press

• “ I would like to wait until I have this advice before making any decisions about changing grade boundaries” Alan Johnson, Secretary of State, May 08 referring to the QCA study set up in response to Dearing recommendation when replying to joint letter from ASCL, ALL, ISMLA

• Meetings with Jim Knights, David Laws (LibDem) and Nick Gibb (Con) to ensure cross-party awareness

Political context

• We do not underestimate the sensitivity of this topic

• Remember Feb 08 –“get GCSE ML without speaking”–“ML Orals abolished!”

• Through all of our involvement, have sought to make politically realistic proposals - e.g Lord Dearing liking the idea of comparability with Maths “no-one will say that Maths is easy”

Sensitivity

• Introduced two definitions of comparability:

1. “Attainment–based comparability” - § 8

• “each boundary must be set using professional judgement. The judgement must reflect the quality of candidates’ work, informed by relevant technical and statistical evidence.' - § 8

• “Fundamentally, awarding meetings are required to ensure that the quality of work describing a particular grade this year is the same as that of the previous year.” - § 9

QCA report - Feb 08

2. Aptitude–based comparabilityThe ‘aptitude-based’ conception of comparability holds that two examinations may be seen as comparable if students of a certain ‘calibre’ have an equal chance of achieving a particular grade in any examination (either within subject areas or between them).

One examination is harder than another if the results of a (representative) group of students taking both examinations are worse in it. This is the approach to comparability on which the ALL proposal is based.

QCA report - Feb 08

• What if French grades based on candidates’ key stage 3 test scores and their results in GCSE mathematics? The analyses indicated that the changes would be marked at the higher grades.

• About half the candidates presently awarded a grade B would gain a grade A as the threshold mark or performance standard for a grade A would have to move down by about half a grade width. There would be a similar effect at grade C.

QCA report - Feb 08statistical case of “severe

grading” now accepted - see joint press release

evidence suggests that for D grade in French 70% gain higher grade in Maths

QCA report had said that only 123 schools (excluding grammar schools) had maintained a high rate of GCSE languages take-up (over 90 per cent entered

in each year from 2003 to2006).

• Conclusion: “The present examination system is not based on an

aptitude-based conception of comparability and its adoption would

create a major threat to public confidence in students’ results. We

have similar concerns about applying a new performance standard to

languages but not to any other subjects, some of which might also

want to claim special status.

We do not have evidence that there have been significant changes to

grade standards in GCSE languages in recent years. In conclusion,

we do not therefore, recommend any adjustment to national grade

standards in GCSE languages.”

• unfortunate implication in conclusion of report that T&L special

problem with MFL :

“Instead we should focus on improving levels of teaching and learning

in modern languages in order to gain students' commitment and raise

performance.”

QCA report - Feb 08

• 2007 GCSE results: – 53% 5A*-C; 40% 5A*-C inc En + Ma–24% A*-C in MFL i.e. 59 out of 243

• comparing MFL results with Maths:– 70% lower grade than Maths– 23% same grade in Maths– 7% higher grade than Maths

• so in Year 11 those not in line for C switched to Asset Languages (138 of 240) of which expect 46 to get equiv grade C or higher. So (46 +102) / 240 = 60% A*-C in MFL for Jun ‘08

Brigshaw School & Language College

presented at Language College

Conference May 08 and Oct 08

but is it T&L issue?

“What is most striking about the school is the pervasive sense that it is a place where learning languages matters. The school has extensive provision for languages, with good links across subjects so that pupils can see how and why they can apply their language skills and knowledge. Pupils appreciate the very good opportunities to get involved in visits abroad. New courses are ensuring that, whilst everyone has to study a language throughout their time at school, there are courses to suit different needs. Extensive use is made of the expertise in the language college to support other schools and the community.”

Ofsted Nov 06

The school had also called in the examiners to look at the papers, and they agreed that the school had properly prepared the candidates

Brigshaw School & Language College

• We believe it is unrealistic to move to comparable grading for all subjects

• Currently 4 main bands – ML– Maths, Sci, His, Geo– English– Art, Drama, PE,

• We are proposing simply that ML moves into the 3rd band - tiny changes in grade boundaries involved, similar to those which have taken place - “one-off step change”

Way forward

data from CEM

SCORE paper

Situation of MFL - A/L

from post-16 PANDA national data for Relative Performance Indicator (RPI)

20 points = 1 grade

nearly 1 grade

on average

1 2 3

subject

ave

score in

subject

ave score in

OTHER

subjects

differ-

ence

G.S. 73.0 87.8 -14.8Biology 79.5 90.8 -11.3Physics 82.7 92.2 -9.5Chemistry 85.8 95.0 -9.2German 86.4 95.3 -8.9French 89.0 96.2 -7.2Music 79.5 84.9 -5.4History 85.2 90.0 -4.8Mathematics 89.7 93.5 -3.8Economics 89.9 92.4 -2.5Other Soc St 81.8 84.0 -2.2Eng Lit 85.6 86.5 -0.9Geography 85.3 83.7 1.6R.S. 87.5 85.6 1.9DT 78.0 75.6 2.4Sport St/PE 75.5 72.8 2.7Business St 81.8 78.2 3.6Drama 85.1 80.2 4.9Sociology 84.7 79.4 5.3Comm St (Media)83.5 75.8 7.7

To asses the relative performance of subjects in the context of uneven grading at a national level, there is a calculation in the PANDA of

1) the average pts score of candidates in the subject

2) the average points score of those candidates in the OTHER subjects they are taken.

The difference between the two figures shows how the grading for the subject compares to other subjects. Because the second figure is calculated ONLY for the candidates doing the first subject, it automatically takes account of the ability profile of the candidates doing the subject

This establishes the national context, and if the same calculation is done for the school, it removes the "severe grading" issue from the consideration of performance

Situation of MFL - A/LFrench - blueGerman - black

from post-16 PANDA national data for Relative Performance Indicator (RPI)

20 points = 1 grade

nearly 1 grade

on average

relative grading of subjects - A/L 2004

-20

-15

-10

-5

0

5

10

G.S

.

Biol

ogy

Phys

ics

Chem

istry

Ger

man

Fren

ch

Mus

ic

Hist

ory

Mat

hem

atics

Econ

omics

Oth

er S

oc S

t (e.

g. P

sych

)

Eng

Lit

Geo

grap

hy

R.S. DT

Spor

t St/P

E

Busin

ess

St

Dram

a

Socio

logy

Com

m S

t (M

edia

) Art

subject

diff

eren

ce in

UC

AS

ptsCol. 3

"difference" as in

previous graph

average pt v "severe grading" A/L 2004

G.S.

Biology

Physics

Chemistry

MusicHistory

German

French

MathematicsEconomicsOther Soc St

Eng Lit

GeographyR.S.DT

Sport St/PE Business StDrama

Sociology

Comm St (Media)Art

-20

-15

-10

-5

0

5

10

70 75 80 85 90 95 100

ave pts per subject

rela

tiv

e g

rad

ing

of

su

bje

ct

Situation of MFL - A/LFrench - blueGerman - black

20 points = 1 grade

Col. 3"difference"

as in previous

graph

Col. 1 - average score in subject

BC 89.0

-7.2

0

average pt in OTHER subjects v "severe grading" A/L 2004

French

German

Sport St/PE

DT

Comm St (Media)

Business St

SociologyArt

Drama

Geography

Other Soc St (e.g. Psych)

Music

R.S.

Eng Lit

G.S.

History

Biology

Physics

EconomicsMathematics

Chemistry

-20

-15

-10

-5

0

5

10

70 75 80 85 90 95 100

ave pts per subject in OTHER subjects = "ability profile"

rela

tive

grad

ing

of s

ubje

ct

Situation of MFL - A/LFrench - blue

German - black

20 points = 1 grade

Col. 3"difference"

as in previous graph

Col. 2 - average score in OTHER subjects (related to ability profile)

0C B96.2

-7.2

7.2

Issues for MFL - A/L• Note that French & German have high “severe

grading factor” at A/L.• French & German have the highest average grades

in OTHER subjects• So

– will suffer in comparison with other subjects, – are only left with the most able

• Compounded by situation at AS: for these 2004 A/L grades,in 2003 at AS French had: subject ave 22.9

OTHER subject ave 29.9 (diff = - 7.7)

A virtuous circle???• Language grading is fair, so...

• Pupils choose MFL at KS4, feel good after their exams and get grades in line with other subjects so...

• More choose at AS and get grades in line with other subjects so...

• More continue to A2 and so...

• Everyone is happy!