Feedback on HKDSE Practice Papers

School Leaders Briefing Session5 March 2012

HKDSE Practice PapersPapers for 24 Category A subjects produced to familiarise teachers and students with the format and requirements of the HKDSE examsFunded by EDBDelivered to schools under secure conditions in early January, with provisional marking schemes uploaded the next day for teachers' referenceAll papers and marking schemes available on HKEAA website from mid February (www.hkeaa.edu.hk/tc/HKDSE/Practice_Papers/index.html)

*

*Research Study (Core Subjects)20 schools selected based on 2010 HKCEE results; stratified sampling but on voluntary basisAbout 2,800 students' scripts collectedCEML practice papers taken as school mock exams under standard exam conditions, with HKEAA observer Answers scripts scanned and marked onscreen Marking statistics and samples of performance analysed

Research Study (Elective Subjects)About 100 to 400 scripts collected from a range of schoolsMarked and samples of performance analysedValuable information about actual performance of studentsGood learning opportunity for HKEAA staff and exam personnelTrial run of marking and grading system

*

Feedback to SchoolsOver 50 briefing sessions conducted for subject teachers between mid January and 29 FebruaryExam requirements, assessment criteria and standards, and annotated samples of performance providedComments on strengths and weaknesses of candidates and advice givenInformation available on HKEAA website

*

*Limitations of the Research StudySample sizeStudent readinessMarking done by a small proportion of experienced CE/AL examiners/markersMarks for speaking papers collected from schools for reference, not standardisedOnly paper-level statistics available at this stageStatistical data to be viewed with caution

* (20%)

(20%)

(10%)

(14%)

(16%)

*

(%) (%)18434.111.2220653.812.438450.010.9510038.69.6

* :

*:

*

*:

*0-132-425-718

*:

(%) (%) (%)49.656.212.747.951.411.52.551.714.1

*:

*

*:

* *

*:

English Language

*Assessment Framework

ComponentWeightingDurationPublic examPaper 1 Reading

Paper 2 Writing

Paper 3 Listening & Integrated Skills

Paper 4 Speaking 20%

25%

30%

10%1 hr 30 mins

2 hours

2 hours

20 minutesSchool-based assessment15%

*Summary Statistics

PaperMax markMean (%)S.D. (%)18638.219.2221044.021.0310442.722.5

*Paper 2 WritingPart A (10%)Short, guided taskAbout 200 wordsSituation, purpose & relevant information providedPart B (15%)Longer, more open-ended taskAbout 400 wordsChoice of one out of 8 questionsEach based on one of the 8 modules in the Elective Part

*MarkingDouble marking by questionThree domains for assessment: Content, Language and Organisation, with equal weighting for each

*Paper 2 Raw Mark Statistics

PaperMarks awarded per marker Mean (%)S.D. (%)2A218.8 (41.9)4.5 (21.4)2B219.8 (46.7)4.4 (21.0)

*Paper 2B Raw Mark Statistics

ThemePopularityMean(out of 21)S.D.(out of 21)No. of cands.%Drama 2107.396.965.54Short stories2097.356.334.72Poems & songs1013.5510.775.13Popular culture 1746.127.294.8

*Paper 2B Raw Mark Statistics

ThemePopularityMean(out of 21)S.D.(out of 21)No. of cands.%Sports comm.47916.8511.024.08Debating33411.7512.054.2Workplace English85229.979.474.08Social issues48417.0510.914.55

*Advice for Candidates (1)Read the questions carefully and try to fulfil all the requirements of the taskDevelop ideas with examples and supporting detailsPay attention to who the writer and the intended readers areDont use prepared sentences, paragraphs and stock phrases Memorised paragraphs / essays will be disregarded and not awarded any marks. Zero marks will be given to totally memorised essays

*Candidates should not waste time counting the no. of words written. As a general guide, + or 10% is acceptable (so between 360 to 440 for about 400 words). Quality is more important than quantity.

Advice for Candidates (2)

*Paper 1 ReadingConsists of two parts: Part A - Compulsory Part B - Section 1 (easier) Section 2 (more difficult) Candidates choose either Section 1 or 2 Candidates attempting Section 2 can attain up to Level 5** Candidates attempting Section 1 can attain up to Level 4 only

*Paper 1 ReadingNumber of scriptsPart A 2,839Part B1 1,431 (~50%)Part B2 1,419 (~50%)

* 11 students answered both B1 and B2

*Results in Part A

Part AFull MarkMean (%)SD (%)ReliabilityMCQ957.1823.330.59Non-MCQ3144.8322.340.90Total:4047.61

*Results in Part B1

Part B1Full MarkMean (%)SD(%)ReliabilityMCQ735.3820.140.27Non-MCQ3635.3720.100.89Total:4335.37

*Results in Part B2

Part B1Full MarkMean (%)SD(%)ReliabilityMCQ1248.8320.330.56Non-MCQ3439.4818.040.86Total:4641.92

*Paper 3 Listening and Integrated SkillsConsists of two parts: Part A (compulsory) Part B - Section 1 (easier)- Section 2 (more difficult) Candidates choose to do either Section 1 or 2Candidates attempting Section 2 can attain up to Level 5** Candidates attempting Section1 can attain up to Level 4 only

*Paper 3 Listening and Integrated Skills

Number of candidates:Part 3A: 2,845Part 3B1: 1,332 (46.74%)Part 3B2: 1,513 (53.26%)

*Paper 3A Results

Full MarkMean (%)SD (%)Reliability5049.0424.530.95

*Part 3A Score Distribution

*Paper 3 Part B1 Results

TaskFull MarkMean (%)SD (%)Reliability51240.2524.830.8361834.1324.380.8971829.2224.840.87

*Paper 3 Part B2 Results

TaskFull MarkMean (%)SD (%)Reliability81854.7118.930.8191853.914.830.69101844.8720.310.81

*Papers 1 and 3: Graded ApproachNeed to convert B1 scores to the same scale as B2 scores After equating, scores can be compared across these two parts of the paper, even though they differ in number of items, difficulty, etc.Paper score = Part A score + Part B score

*Part 3A Score Distribution

*How 3B1 Students Performed in 3A

*How 3B2 Students Performed in 3A

*Equating (e.g. Paper 3)

3B1 Scores3B2 scores5448.............00

*Equating

3B1 Scores3B2 scores5448.............00

*Equating

3B1 Scores3B2 scores5448.............00

*Equating

3B1 Scores3B2 scores5448.............00

*Part B1 to Part B2 Conversion Factor

PaperB1 converted to B2 score1Multiply by 0.6523Multiply by 0.735

*Part B1 to Part B2 Conversion FactorExample: Paper 3Candidates score in 3B1 = 26Candidates score after conversion= 26 x 0.735 = 19

*Part B1 to Part B2 Conversion FactorIn live exam Carried out with 70,000+ candidatesMore sophisticated method used (see page 19 of Grading Procedures and SRR booklet)Conversion factor will vary from paper to paper and from year to year, depending on the difficulty of the items in each partMaximum level a candidate who attempted B1 can get is Level 4 in the relevant component

-

*: : 215 65

*

10542.6%23.2%

**: : 115 35 2/3 1/3

**:

*

4551.7%17.5%

*

*

Liberal Studies

**Paper StructurePaper 1 (50%): Data-response questions 3 questionsAll compulsory

Paper 2 (30%): Extended-response questions3 extended-response questionsAnswer one question only

**MarkingMarked by questionDouble marking Third marking and fourth marking whenever necessaryCriterion for 3rd marking: >3 marks difference between the first two markersCriterion for 4th marking: after third marking, no closest pair occurs4th marking by Chief/Assistant Examiner

**Paper 1: Data-response Questions All questions compulsoryFull marks of each question: Q1 (18 marks), Q2 (20 marks), Q3 (20 marks)Mean: Q.1 50.0%, Q.2 44.4%, Q.3 44.2%; paper 46.0%S.D. of paper: 12.5%

39 students (1.4%) got 0 marks in the last question which may be due to their problem with time allocation

**Paper 2: Extended-response QuestionsChoose ONE question out of threeEach question has the same full marks (20 marks)Mean: Q.1 42.2%, Q.2 39.0%, Q.3 52.0%; paper 46.6%S.D. of paper: 16.0%Question popularity:Q.1 (gender stereotyping): 15%Q.2 (Chinese economic growth and energy consumption): 30%Q.3 (ethnic minorities in HK): 55%

Marking GuidelinesSuggested marking criteria for each question for markers' referenceNot to be regarded as model answers Alternative answers are acceptable as long as they are reasonableCandidates are entitled to take any position they wish on an issue. They are not to be marked on the correctness of their views' or on how close their views correspond to those of the markers. What counts is whether they can put forward a well-reasoned argument for their view.A script scoring the top mark range need not be perfect. It may contain minor flaws in content, approach or presentation.

*

**

Thank YOU!

Statistical moderation will not change rank order