NSW Education Standards Authority

Marking Operations Manual

Marker Handbook

Updated August 2020

2

Contents

A message from the Executive Director, Assessment and Curriculum Standards, on COVID-19 and the HSC .......................................................................................................................... 3

1 Introduction ......................................................................................................................... 4 1.1 NESA Examinations ......................................................................................................4

2 Roles and Responsibilities .................................................................................................. 4 2.1 Marker..........................................................................................................................4 2.2 Senior Marker ...............................................................................................................4 2.3 Supervisor of Marking ...................................................................................................5 2.4 Chief Examiner .............................................................................................................5

3 Goals of marking................................................................................................................. 5

4 The process: examination development to marking ........................................................... 6 4.1 Developing the examination and marking guidelines........................................................6 4.2 Developing the marking kit ............................................................................................6 4.3 Test Marking - testing the marking kit ............................................................................6 4.4 Practice Marking - practising using the marking kit .........................................................6 4.5 Marking ........................................................................................................................6

5 Standards-setting ................................................................................................................. 7

6 Marking administration ....................................................................................................... 7 6.1 Online reference materials..............................................................................................7 6.2 Marker briefings ............................................................................................................7 6.3 Marking ........................................................................................................................7 6.4 Confidentiality and security ...........................................................................................8 6.5 Administrative matters ...................................................................................................8 6.6 Double-marking ............................................................................................................8 6.7 Filling in marksheets .....................................................................................................9 6.8 Capturing marks ............................................................................................................9 6.9 Non-Attempts ...............................................................................................................9 6.10 Non Serious Attempts ....................................................................................................9 6.11 Question Restrictions ................................................................................................... 10 6.12 Responses of concern .................................................................................................. 10 6.13 Other issues ................................................................................................................ 10 6.14 Pay claims .................................................................................................................. 10 6.15 Marker Appraisal Process ............................................................................................ 10

Glossary of Marking Terms ..................................................................................................... 14

7 Appendices ........................................................................................................................ 17 Appendix I: Tally sheet ....................................................................................................... 17 Appendix IV: Marker Appraisal Form .................................................................................. 18

3

A message from the Executive Director, Assessment and Curriculum

Standards, on COVID-19 and the HSC

2020 has been a challenging year for HSC students and their teachers.

The COVID-19 Emergency Response Committee was established in March to make decisions about

the 2020 HSC due to the impact of COVID-19. The Committee is made up of the Heads of each of the

three education sectors in NSW, the Chairman of the NESA Board and the Chairs of the Assessment

and Curriculum Committees.

The Committee’s decisions have been published progressively on the NESA website. They include

changes to practical and performance examinations and a delayed timetable for the oral and written

examinations.

It is very important that HSC marking is undertaken this year using the same processes as in previous

years. Please make sure you mark all responses according to the marking kit, and do not be tempted to make individual adjustments. This will ensure that all students are given the same opportunity to be

rewarded for what they demonstrate in the exams.

This Marking Operations Manual provides a background to the HSC and the processes you will follow

at marking. Please read it carefully and talk to your senior marker or the Supervisor of Marking if you

have any questions.

Thank you for being an HSC marker, and best of luck for the 2020 HSC Marking Operation.

Yours sincerely

Sofia Kesidou

4

1 Introduction

1.1 NESA Examinations The Higher School Certificate (HSC) examinations are high profile, world class assessments that are

developed, marked and reported within a standards-referenced framework. They are subject to close

scrutiny by schools, students, parents, the community and the media.

Each year teachers and suitably qualified academics are employed to mark HSC examinations. Each

examination is divided into parts so that a marker is marking a manageable portion of the examination

at any one time. The markers are trained and given instructions on how to mark that portion of the

examination. The instructions are supplemented by a ‘marking kit’ which consists of a set of marking

guidelines that were devised by the examination committee, as well as any other materials that the

leader of the marking team decides will help markers. These materials sometimes include actual

student responses and senior marker annotations.

The marking kit is designed to be something that markers can refer to at all times to ensure that the

correct mark is given to each response. It is also designed to appropriately discriminate between

responses so that the better performances receive higher marks, and that this discrimination is linked back to the standards and syllabus outcomes.

Markers are monitored at all times, from training to the last day of marking, to ensure that they are

marking reliably, that is, that they are correctly and consistently following the instructions they were

given.

2 Roles and Responsibilities

2.1 Marker Your role as a marker is to:

Assess responses and assign marks consistently according to the marking kit;

Bring to the attention of the senior marker cases where responses appear to be acceptable

alternatives to the marking kit, or are unusual, irrelevant or incomplete;

Enter the correct mark awarded to each response;

Regard all responses, marks and the marking kit as confidential, and not divulge information on

such matters to anyone other than authorised persons.

2.2 Senior Marker A senior marker is an experienced marker appointed to lead and manage a team of markers during the

marking operation. The primary focus of the senior marker’s role is to ensure that the goals of marking

are met in a timely and efficient manner. There are a number of tools available to assist senior markers

in monitoring quality assurance. Senior markers:

Use the guidance provided by the Chief Examiner to develop the marking kits and the select

appropriate materials to be used in the marking kit;

Brief the team of markers in how to use the marking kit consistently;

Monitor the quality of marking;

Collect and analyse evidence and data to show that markers are consistently applying the marking

kit.

5

2.3 Supervisor of Marking The Supervisor of Marking is the person responsible for managing the marking operation. By the

application and monitoring of quality assurance procedures, Supervisors of Marking ensure the timely,

efficient and effective completion of marking. The Supervisor of Marking takes responsibility for:

The overall management of all aspects of the marking operation;

Ensuring that the marking operation produces valid results and that appropriate discrimination is

achieved on each item;

Ensuring the reliable application of the marking guidelines;

Preparing the HSC Marking Feedback;

Providing feedback to the examination committee after marking.

2.4 Chief Examiner The Chief Examiner is the person appointed by NESA to chair the examination committee which sets

the examination paper and prepares the marking guidelines. The Chief Examiner reviews data from

the trialling of the marking guidelines and marking kit prior to the start of marking.

3 Goals of marking Underpinning the NSW HSC programs is the goal of providing reliable and fair measures of students’

achievements in the courses they have undertaken. To this end, the goal of marking is to validly and

reliably award marks to student responses. Quality assurance measures are carried out in an effort to

achieve this goal so that stakeholders can have confidence in the accuracy of the marks awarded to

every student’s response to every item, task or component.

Figure 1: The goal of marking

The Supervisor of Marking and the Chief Examiner share the responsibility for ensuring that the

marking operation produces valid results and that appropriate discrimination is achieved on each

item.

The Supervisor of Marking and senior markers take responsibility for ensuring the reliable

application of the marking guidelines by the markers.

Higher marks are awarded to responses displaying superior knowledge or skill

Marks are used to appropriately reflect variations in quality and the full range of marks is used A particular quality

of response is always allocated the same mark, regardless of who marked it and when it was marked

6

NESA staff support the Supervisor of Marking and Chief Examiner in ensuring effective

application of quality assurance procedures.

4 The process: examination development to marking

4.1 Developing the examination and marking guidelines Examination questions are closely linked to syllabus outcomes and content. The examination

committee is responsible for developing the marking guidelines, which are written at the same time as

the questions. The marking guidelines and sample answers (where they exist) show markers the intent

of each question.

4.2 Developing the marking kit Before markers arrive at the marking centre, the Chief Examiner works with the Supervisor or

Marking and senior markers to ensure that the marking of responses will be consistent with the

examination committee’s intentions and that the implementation of the marking guidelines will allow

for appropriate rewarding of the range of responses. The senior markers then develop any further

material required to support markers in interpreting the marking guidelines and applying them

consistently.

The marking kit consists of the marking guidelines and any material developed in the marking centre

to supplement, elaborate, illustrate or explain the marking guidelines. While the marking guidelines

identify and describe performances at the various mark levels, the marking kit adds detail and

elaborates on particular components of anticipated student responses. The marking kit supports the

marking guidelines; it does not replace them.

4.3 Test Marking - testing the marking kit Marking kits are refined by test marking. The guidelines and marking kits are tested by senior markers

and/or pilot markers against a representative range of responses before markers arrive. This provides

an initial test of the marking guidelines and assists in refining the marking kit. Test marking concludes

with the Chief Examiner and Supervisor of Marking approving and signing off the marking kit.

4.4 Practice Marking - practising using the marking kit When markers arrive at the marking centre, they are briefed by senior markers. The marking kit is

used in the training of markers and to maintain consistency throughout marking.

In practice marking, markers apply the marking kit to a range of responses drawn from a broad range

of examination centres. Typically, markers then discuss in small groups with their senior marker the

marks they have awarded. This discussion provides an opportunity to further explain the meaning of

the marking guidelines and to achieve a shared understanding of them. Practice marking continues

until the senior marker is confident that their markers are able to consistently apply the marking kit.

Some markers may start real marking earlier than others.

4.5 Marking Markers award marks to responses using the marking kit. Senior markers use a variety of quality

assurance tools such as common responses, check-marking and statistical reports to monitor the

consistency of markers. Regular feedback, support and assistance will ensure that the marking

operation is of the highest standard.

Markers are monitored at all times, from training to the last day of marking, to ensure that they are

marking reliably, that is, that they are correctly and consistently following the instructions they were

given.

7

5 Standards-setting Standards-setting is an important process that involves interpreting students’ examination

performances in terms of the HSC performance bands for the course. In a 2-unit course, Band 1

represents a level of achievement below the minimum standard expected in the course. Bands 2, 3, 4, 5

and 6 represent increasing levels of achievement in the course, and are described in terms of the kinds

of knowledge, skills and understanding typically displayed by students achieving at those levels.

Each year, the standards-setting process is used to determine the total mark on that year’s examination

that corresponds to the borderlines between each of the performance bands. For example, the

standards-setting process determines the minimum examination mark a student must achieve to be

assigned Band 6. This raw examination mark will not necessarily be the same in different years

because examinations differ slightly in difficulty from year to year.

The standards-setting process is conducted for each examination by a group of ‘judges’ who are

usually experienced markers for that course. They make judgements about how students at various

points in relation to the course achievement standards are likely to perform on each of the examination

questions.

6 Marking administration

6.1 Online reference materials Supervisors of Marking, senior markers and markers all have access to the NESA web portal, Markers

Online, using their marker ID and a secure PIN. Supervisors of Marking, senior markers and markers

are able to do things like apply for marking and change their personal details when logged-in to

Markers Online, but they are also able to access other reference materials (such as this manual and

training presentations) under the various menu items. Supervisors of Marking, senior markers and

markers are expected to have read the documents that are relevant to them before the start of marking.

6.2 Marker briefings Markers are trained to make judgements about the merit of students’ responses (in accordance with the

marking guidelines), and to award appropriate marks to every response. In all situations, markers

award whole marks only (ie half marks are not used). The Supervisor of Marking and the senior

markers brief markers on administrative procedures, as well as instructions on how to mark the

question or task that has been allocated.

6.3 Marking Markers award marks according to the marking kit. When marking extended responses, markers first

determine the mark range of the response (A, B, C, D or E) before using their professional judgement

to determine the specific mark within that range. The marks awarded reflect the extent to which a

response addresses the criteria described in the marking guidelines.

A prose response is only one means of written communication. Annotated maps, diagrams, tables, and

headings with summaries of information or ideas listed as points may be appropriate ways of

presenting a response.

Individual student marks should not be recorded on documents other than tally sheets or mark sheets.

Markers should refer to senior markers any responses that:

- do not appear to be accommodated by the marking guidelines;

- are obviously incomplete (eg part of the answer detached);

8

- seem irrelevant (eg a response to another question);

- are atypical or unusual (eg a possible non-serious attempt or possible malpractice).

6.4 Confidentiality and security Confidentiality is a condition of marker employment. Markers must not communicate information

about student responses, marking guidelines, marking kits or marks to any person outside the marking

operation. Student responses and marking materials are not to be removed from the marking centre.

Markers must not seek marks of known candidates or centres. Markers are not permitted to mark

responses from their own school or from students known to them. If you recognise a student or centre,

bring it to the attention of your senior marker. Student responses must not be discussed with other

markers, apart from necessary consultations.

Markers are not permitted to advertise their appointment as a marker for commercial gain. For security

reasons, marker identification cards must be worn in the marking centre at all times.

6.5 Administrative matters Attendance at every marking session is a condition of employment. In case of illness or unavoidable

delay, markers must contact the Supervisor of Marking as soon as possible. Markers are required to

regularly check any notices issued.

Mobile phones and other electronic devices must be switched off during marking sessions. Markers

may provide the marking centre emergency telephone number to their family members in case urgent

contact is necessary. Electronic devices such as radios, CD/MP3 players and headsets or earbuds are

not to be used during marking except when provided by NESA for the purposes of marking. NESA

cannot accept responsibility for equipment brought to marking or left in vehicles.

Please bring your own cup or mug to the marking centre. Tea, coffee and milk are provided. Please

consider other markers; avoid talking or walking around the marking centre except when necessary.

Marking is an alcohol-free operation and smoking is not permitted in any marking centre. Markers

must not consume, or be under the influence of, alcohol during marking.

Markers should update their pay claims each day. Correcting fluid is not to be used to amend pay

claims. If a mistake is made, a line should be drawn through the error and the change should be

accompanied by the marker’s initials.

6.6 Double-marking Double-marking is used by NESA for examination items where the nature of the responses is such that

experienced, competent and well-trained markers may validly award different marks because they

value different aspects of the response differently. For such items, the ‘true’ mark can be thought of as

the average of all the marks that would be awarded if the response were marked by a large number of

qualified and well-trained markers. For these types of items, NESA uses double-marking. That is, each

response is marked twice independently. The mark awarded to the response is the average of the two

individual marks awarded, and can be thought of as representing a synthesis of the two markers’

independent judgements of the merit of the response in relation to the marking guidelines. Double-

marking is not a ‘safety net’ to minimise the effects of any uncertainty a marker may have in making a

judgement.

In pen-and-paper marking operations, clerical officers are responsible for putting the marksheets onto

bundles of scripts prior to distribution. The ‘A’ and ‘B’ marksheets are usually printed on different coloured paper. At any one time in the marking centre, there will be a combination of ‘A’ and ‘B’

marksheets. Markers are to mark the bundles of scripts as they are presented to them; neither markers

9

nor senior markers are to prioritise particular bundles. In onscreen marking, scripts are randomly

presented to markers. In double-marked operations, the second marker is presented with the script as

soon as possible after the first mark has been entered.

6.7 Filling in marksheets In pen-and-paper operations, markers use a pre-printed marksheet with a student ID number and the

details of which part of the examination the response comes from to record their marks. Markers must

identify themselves by entering their group number and marker number in the boxes at the top of the

marksheet.

A document entitled How to Use Marksheets will be distributed to all markers in pen-and-paper

operations on the first day of marking. It is essential that markers read and understand this document.

Incorrect or badly completed marksheets significantly slow the marks capture process, resulting in

delayed or inaccurate statistics.

6.8 Capturing marks In pen-and-paper operations, after the marks are written on the marksheet by the marker, the

marksheets are then scanned by clerical officers and the marks are recorded against the relevant

student.

Markers are trained to write clearly, but if the computer scanner cannot read the mark awarded – if it is

untidy or unclear – the system alerts a human ‘verifier’ who reads the sheet and enters the correct

mark into the computer. During marking, a variety of checks and quality control processes are carried

out to ensure the scanner reads the marks correctly.

In onscreen marking, responses are scanned or uploaded into the system and delivered via a secure

internet connection to a marker who views it on their computer screen and awards a mark. In these

cases, the marks are entered directly into the examination system by the marker.

6.9 Non-Attempts A non-attempt to a question or item is when a student has not responded to the question or there is no

evidence of academic engagement in the response they have made. For example, merely rewriting the

question is considered a non-attempt. In such cases the marker records ‘non-attempt’ rather than zero.

Directions on how to record non-attempts in written marking can be found on the sheet entitled How to Use Marksheets. For courses marked onscreen, markers should refer to the onscreen marking

manual.

One aspect of a Higher School Certificate results check verifies that ‘where any question is recorded

as a non-attempt, the student has not attempted the question’. For this reason, anything that could be

construed as an attempt at the question must be awarded a mark of zero, and NOT be recorded as a

non-attempt.

6.10 Non Serious Attempts A student must demonstrate that they have engaged with a range of question types in an examination

for it to be considered a serious attempt. As such, answering only the multiple choice questions

constitutes a non-serious attempt. Markers should bring to the attention of their senior marker any

frivolous or objectionable responses that may be non-serious attempts. Any suspected non-serious

attempts should then be referred to the Supervisor of Marking who will notify the Student Records and

Support branch of the responses.

10

6.11 Question Restrictions In some courses there are rules regarding the combination of questions that a student may answer. For

example, in Studies of Religion, a student must focus on a different religious tradition for Section III

than they focussed on in Section II. These restrictions are coded into the NESA examinations system.

If a student was to answer questions on the same religious tradition in both these sections, both

responses would be marked, and the student would be awarded the higher mark of the two. A mark of

zero would be awarded for a question in the other section.

6.12 Responses of concern If a marker sees evidence of a troubled student (blood on the paper, threats of self-harm etc), they

should bring the response to the attention of the Supervisor of Marking. The Supervisor of Marking

should contact the Student Records and Support branch.

6.13 Other issues Markers are encouraged to bring to the attention of their senior marker any issues that relate to the

marking of a response including instances where a marker may have difficulty reading a response due

to illegible writing.

6.14 Pay claims Pay claims are distributed to markers on the first day of marking. Markers are responsible for

completing their own pay claims correctly – this is not the responsibility of the senior markers. Senior

markers are only responsible for checking and signing the pay claims each day, and again at the end of

marking. Completed and signed pay claims are to be given to the Supervisor of Marking when

marking has concluded.

6.15 Marker Appraisal Process The Marker Appraisal process assists Supervisors of Marking and senior markers in monitoring and

assessing senior marker and marker performance during the marking period.

6.15.1 Markers’ Award

The Markers’ Award states:

‘13. Reporting of Performance A system of appraisal of performance of Employees will be undertaken during HSC marking … This

will involve provision of an individual report at the conclusion of marking to Employees showing:

i) an overall grading of his/her performance; ii) supporting information relating to accuracy and rate of marking;

iii) an indication of his/her attendance record throughout the marking; and iv) a written comment describing his/her performance throughout the marking.’

6.15.2 The appraisal process

Supervisors of marking and senior markers monitor and assess marker performance throughout

marking. An electronic marker appraisal is used to record a summary of the marker’s performance,

and is available for markers after the conclusion of marking. An example of the online appraisal form

can be found in the Appendix.

Markers are assessed on the following criteria:

6.15.2.1 Marking accuracy If a marker is not marking accurately, the senior marker should establish why this is happening and

then take appropriate steps to fix the problem. Any conversations/steps taken to address the issue

11

should be documented. If marking consistency does not improve within a reasonable timeframe then a

warning must be given to the marker by the senior marker, the conversation must be documented.

If the marker’s consistency continues to show no signs of improvement despite counselling and

intervention by the senior marker, the Supervisor of Marking should have a conversation with the

marker and issue an unsatisfactory appraisal, ask the marker to electronically sign the appraisal and to

discontinue marking.

6.15.2.2 Marking rate If a marker is marking too slowly, the senior marker needs to ascertain the reason why (eg lack of

understanding, lack of confidence) and then take the appropriate steps to address the situation. Any

conversations/steps taken to address the issue should be documented. If the marking rate does not

improve within a reasonable time then a warning must be given to the marker by the senior marker,

and the conversation must be documented.

If the marker’s rate continues to show no signs of improvement despite counselling and intervention

by the senior marker, the supervisor of marking must speak with the marker and make a final decision

about whether or not to issue an unsatisfactory appraisal. If an unsatisfactory appraisal is issued, the

marker should be asked to sign the form, and discontinue marking.

6.15.2.3 Punctuality/attendance Markers should be warned at briefing that they must arrive at the marking centre in time to start

marking at the specified time and should be available for all marking sessions. If they cannot do this,

they should not continue marking. If a marker continues to arrive at the marking centre late or does not

present for marking on certain days and has not been given permission to do so, then a warning should

be given to the marker by the senior marker, and a record of the conversation kept.

Not being punctual or not attending without good reason can be classed as unsatisfactory attendance at

the marking centre and the marker concerned can be issued with an unsatisfactory appraisal and then

asked to leave the marking centre. Please note this issue should be dealt with on a case-by-case basis,

however, in all such cases, pay sheets should be scrutinised and markers should only be paid for the time they attend.

6.15.2.4 Overall recommendation/performance A marker’s overall performance needs to be flagged as satisfactory or unsatisfactory taking into

account their marking consistency, rate and punctuality/attendance. A person who receives an

unsatisfactory appraisal would be excluded from future appointments as a marker.

6.15.3 Completion of the marker appraisal form

In addition to assessing employee performance according to the criteria as either unsatisfactory or

satisfactory, the Supervisor of Marking/senior marker should also ensure that a comment and the

appropriate signatures are included.

6.15.3.1 Comments A comment on the marker’s performance should be provided, including areas of improvement, and, in

the case of an unsatisfactory appraisal, documentation should be attached detailing the process that the

Supervisor of Marking/senior marker went through, including a record of any conversations/steps

taken and confirmation, that clear communication had been given to the marker throughout the

process.

6.15.3.2 Signatures It is expected that the senior marker or the Supervisor of Marking speak to all markers individually

about their performance at the end of the marking period (or earlier if there are any issues with the

marker’s performance) and then complete a marker appraisal for each marker. All appraisal should be

signed electronically by the senior marker, and the Supervisor of Marking and the marker concerned.

12

Senior marker appraisals need to be signed electronically by both the Supervisor of Marking and the

senior marker.

A copy of a marker’s appraisal will be available on their Markers Online profile.

6.15.4 Other Concerns/Issues

6.15.4.1 Attitude A marker should be counselled if they display a discourteous or disruptive attitude, and any

conversations/steps taken should be recorded. A marker cannot be given an unsatisfactory appraisal

based on their attitude alone if their marking consistency, rate and punctuality/attendance are all

satisfactory. However, their behaviour should be noted in the Comments section on the appraisal.

If the behaviour continues to be problematic, it should be reported to the Leader, Marker Support.

6.15.4.2 Discrimination Discriminatory behaviour is unacceptable in any work place. If any marker displays discriminatory

behaviour whilst at marking, whether it is in relation to sex, age, race etc, please contact People and

Culture at NESA on (02) 9367 8122 for further advice and ensure the issue is raised with the Head, Examination and Marker Support.

6.15.4.3 Confidentiality If a marker (of any classification including senior marker) is caught breaching confidentiality or

security, such as talking to the public about the marking process or removing scripts or any marking

materials (hard copy or PDFs) used during briefing/ marking sessions, or taking photocopied,

photographed, downloaded, recorded or printed scripts or any marking material from the marking

centre, external operation and itinerant operation.

Any situations like this should be reported to the Leader, Marker Support immediately.

6.15.5 What is an unsatisfactory appraisal?

An unsatisfactory appraisal is where a marker has been deemed as unsatisfactory in their overall

performance. If a marker is deemed unsatisfactory in one area but satisfactory overall, that person has

not received an unsatisfactory appraisal. If a senior marker is deemed unsatisfactory as a senior

marker, they may still be deemed satisfactory as a marker.

If a marker receives a satisfactory appraisal for overall performance, they cannot be omitted from the

marking operation the following year based on their performance, and should be appointed to mark,

provided they satisfy the marker selection criteria. The only way a marker can be omitted from future

marking based on performance, is if they have received an unsatisfactory appraisal overall. This means

that the Supervisor of Marking and the senior marker have discussed the issues with the marker, and

an online appraisal has been completed for the marker, and electronically signed by the senior marker

and Supervisor of Marking. If the marker is not made aware they are being issued an unsatisfactory

appraisal, then the appraisal is not valid. If a marker has been deemed unsatisfactory but refuses to

electronically sign it, the appraisal is still valid, provided it is electronically signed by the senior

marker and Supervisor of Marking, and the marker’s refusal to sign it is reported to the Marker

Support Unit. This applies to all subjects including those that are marked itinerantly or onscreen.

In a number of cases, an unsatisfactory marker may be asked to leave the marking operation before the

completion of marking. However, it is important that time is spent with the person concerned and that

an appraisal is completed before they leave.

6.15.6 What happens if a marker receives an unsatisfactory appraisal?

If a marker receives an unsatisfactory appraisal, it is recorded on the marker’s record on the NESA

system. They can still apply for marking in future years, however, they will need to provide a written

13

statement to support their application and to address the issues outlined on the unsatisfactory appraisal.

This statement is then passed on to the Supervisor of Marking who will need to consider the additional

information together with the application form and, if the applicant satisfies the selection criteria,

make a decision as to whether they should be re-appointed.

If the Supervisor of Marking decides not to appoint the applicant, they need to document their reasons

and send them to their allocated Marker Support Officer.

If, based on the information provided, the Supervisor of Marking is happy to appoint the applicant as a

marker, and they perform satisfactorily, then the marker will be able to apply for marking the

following year.

14

Glossary of Marking Terms Term Definition

Check-marking Check-marking is the process whereby the senior marker re-marks responses

that have already been marked by a marker. The senior marker compares the

mark that they gave the response to the mark that the marker gave it. This

comparison provides valuable feedback to the senior marker about the

reliability of their markers. The senior marker will then use this information to

further inform their quality assurance monitoring tools. There are two types of

check-marking: random and targeted. Random check-marking is undertaken

by a senior marker when there is little evidence for them to know which

marker needs this attention and as such usually only occurs in the first session

of real marking. Targeted check-marking is a more powerful tool that

enables a senior marker to focus on markers that have been identified as

inconsistent by common responses or the marker accuracy report.

Chief Examiner The Chief Examiner chairs the examination committee and has further

responsibilities in the marking and standards-setting processes.

Common

response

Common responses are responses that have been pre-marked by the senior

marker. They are then provided to the teams of markers to monitor and test

their application of the marking kits. The information gathered from this

provides valuable feedback to the senior marker and Supervisor of Marking

about how consistently markers are applying the marking kits.

Discrepancy

resolution

The task in discrepancy resolution is to determine the best estimate of the true

mark. This process is usually undertaken by the senior marker.

Judge Suitably qualified teachers and academics employed within the standards-

setting operation. They make judgements about how students at the

borderlines between the course achievement standards are likely to perform on

each of the examination questions and recommend final cut scores for each

performance band to the Consultative Committee.

Marker Suitably qualified teachers and academics employed to mark the Higher

School Certificate examinations. They are responsible for applying the

marking kit accurately and consistently, and are guided by a senior marker, to

whom they report.

Marking kit Consists of a set of marking guidelines that were devised by the examination

committee, as well as any other materials that the leader of the marking team

decides will help markers award the appropriate mark consistently. These

materials sometimes include actual student responses and annotations.

Marking plan The marking plan helps the Supervisor of Marking to plan their operation. It

details numbers of markers needed to mark the responses, the ratio of senior

markers to pilot markers to markers, the starting and finishing times of the

various stages of the marking process, the proposed standards-setting schedule

and marking guidelines sign off, test marking and practice marking. Time for

the discussion of wide reading with the Chief Examiner, the senior marker

Quality Assurance briefing and the standards-setting operation has been

factored into the marking plan.

15

Term Definition

Marksheet In pen-and-paper operations, markers use a pre-printed marksheet with a

student ID number and the details of which part of the examination the

response comes from to record their marks.

Moderation The moderation of assessment marks is a statistical procedure that uses the

external examination marks for the whole school group in a course to adjust

the school’s assessment marks for those students. For each course the

procedure adjusts the average of the school assessments to be equal to the

average of the examination marks obtained by the group. It also sets the top

school assessment to be equal to the top examination mark, and sets the

bottom assessment mark to be equal to (or close to) the bottom examination

mark wherever possible. Once this is done all the assessment marks awarded

by a school for a course will have been adjusted (moderated).

Onscreen

marking

Onscreen marking is where student responses are presented to markers as

electronic files on a computer via a secure internet site. Onscreen marking can

be done corporately or externally.

Optional

question scaling

If an examination has optional parts, the process of optional question scaling

may take place. It is a statistical procedure that ensures students are not

unfairly advantaged or disadvantaged by choosing an easier or more difficult

optional question in an examination. The final marks awarded to the students

who have answered each optional question are statistically adjusted by taking

into account the pattern of marks gained by that group of students on the

compulsory questions.

Parallel double-

marking

During double marking, the two markings happen in parallel. That is the

second marking of the response happens as soon as possible after the first

marking.

Practice marking Markers carry out practice marking to ensure that they understand and can

implement the marking kit reliably. The senior marker monitors the markers’

performance in reliably assigning the appropriate mark.

Senior marker Senior markers are appointed on the basis of their marking experience and

demonstrated leadership skills. Their primary role is to ensure the quality of

marking. Senior markers lead and supervise groups of markers in the marking

of one section or question of an examination, depending on the size of the

candidature and the nature of the examination. Senior markers are responsible for briefing and monitoring markers, check-marking, analysing group results

and the resolving of discrepant markings. They report to the Supervisor of

Marking.

Startup response A startup response is a common response delivered at the beginning of a

marking session in onscreen marking. Startup responses ensure that the marker

is on-task and ready to mark.

16

Term Definition

Supervisor of

Marking

Supervisors of Marking (SOMs) have administrative responsibility for the

marking operation for a course or course component. They are responsible for

recommending the appointment of markers and senior markers, for

implementing a marking plan, for managing the operation of the marking

centre, and for ensuring the accurate and timely marking of all examination

responses. Supervisors of Marking usually have extensive experience both as a

marker and senior marker. They report directly to NESA.

Test marking Test marking is the process of the senior marker (and possibly some pilot

markers) testing the marking kit and marking guidelines to provide evidence to

the Supervisor of Marking and Chief Examiner about the how well the

marking kit is working.

True mark Underpinning NESA marking operations is a belief that there is a single ‘true’

mark that captures the essence of each student’s response. The goal of marking

operations is for the awarded mark to be equal to that true mark.

17

7 Appendices

Appendix I: Tally sheet

18

Appendix IV: Marker Appraisal Form

Name of Marker: ___________________________________________________________

Name of Appraiser: _________________________________________________________

(Senior marker)

Subject: _________________________________ Section marked: _________________ (if applicable)

PERFORMANCE

FACTORS (at all sessions):

Marking Accuracy: Satisfactory throughout marking Unsatisfactory

Marking Rate: Satisfactory throughout marking Unsatisfactory

Punctuality/attendance: Satisfactory throughout marking Unsatisfactory

OVERALL RECOMMENDATION:

Satisfactory throughout marking Unsatisfactory

NB: Where an unsatisfactory appraisal is given, a detailed comment is to be provided

including the specific nature of the unsatisfactory performance. The comments

section of this form must be completed where the marker has been deemed

overall unsatisfactory.

COMMENTS:

Signature of Appraiser (senior marker)

______________________________________________ Date: ____ / ____ / 2020

Signature of Supervisor of Marking

______________________________________________ Date: ____ / ____ / 2020

Marker’s Declaration

I have read and noted the above appraisal.

______________________________________________ Date: ____ /____ / 2020