consistency of assessment (validation) webinar – part 1 renae guthridge wa training institute...

Consistency of Assessment(Validation)

Webinar – Part 1

Renae GuthridgeWA Training Institute (WATI)

What is your understanding of validation?

Describe an assessment validation process in which you have been

involved.

– What was the purpose?– Who was involved?– What was validated?– Were any problems identified?– How were the findings reported and

followed-up?

What is assessment validation all about?

• What are the purposes of assessment

validation?

Contribute to RTO quality assurance

Address any identified area of risk in assessment practice and qualityDemonstrate

compliance against Standards for RTOs

PURPOSESEnsure assessments meet the evidence requirements of the competency standards or other assessment benchmarks

Quality review of the assessment process

Improve assessment practices

Evaluate the quality of assessment tools

Provide professional development

Increase assessor confidence

Determine whether different assessors using the same tools collect the same types and levels of evidence

Determine whether different assessors interpret the same evidence similarly

Determine whether assessment decisions reflect the principles of assessment and rules of evidence.

To ensure our assessment judgements are consistent


• What kinds, types and methods of

assessment documentation do we need

to validate?

Learning based assessment

Direct

RPL

KINDS, TYPES & METHODS OF ASSESSMENT DOCS

SupplementaryIndirect

Instruments

Assessment StrategiesMarking Guide

Instructions for CandidateObservation; Q&A;

Case Studies; Portfolios; Projects…

Instructions for Assessor

Mapping Documents

Assessment Plans


• Who wants assurance of quality?

Learners / Candidates

WHO WANTS QUALITY?

Assessors / Trainers

Technical / Subject Matter Experts

Industry / Employers

Registering / Course Accreditation Bodies


• What are our benchmarks of quality?

Standards for RTOs

BENCHMARKS OF QUALITY

AQF

Principles of Assessment

TAE10 Training and Education Training Package…assessment units.

Training Packages / Accredited Courses

Rules of Evidence


• What aspects of the assessment process

need to be validated?

Planning assessment processes (industry consultation, links with delivery, TP/AC requirements, recognition of client needs)

ASPECTS TO VALIDATE

Delivery of assessment services (modes, plans, resources)

Processes for the review of assessment services (validation and continuous improvement)

Judgement making procedures

Evidence gathering processes and tools (on-the-job, workplace, simulation)

Support for learners / candidates (Information, feedback, flexibility, reasonable adjustment, appeals)

Management of assessors / evidence gatherers (selection, supervision, professional development)


• How do we assess quality and make

improvements?

Assessors working in collaboration to review, compare and evaluate assessment processes and outcomes

ASSESSING QUALITY

It’s systematically planned and implemented

Verifying consistency of evidence and judgements made against benchmark

Foxes and Vixen

Can occur before, during and after assessment

Validating methods, tools and interpretation of evidence

Foxes and a Vixen


Rules of Evidence

FourVeryRampantFoxes

ASatisfiedCunningVixen

Foxes and a Vixen


Rules of Evidence

Flexibility

Validity

ReliabilityFairness

Authenticity

Sufficiency

CurrencyValidity

Foxes and a Vixen

Flexibility

Foxes and a Vixen

Validity

Foxes and a Vixen

Reliability

Foxes and a Vixen

Fairness

Foxes and a Vixen

Authenticity

Foxes and a Vixen

Sufficiency

Foxes and a Vixen

Currency

Foxes and a Vixen

Validity

Standards for RTOs 2015What do you know?What is different to AQTF / VET Quality Framework?

Standards for RTOs 2015

Validation is the quality review of the assessment process.

Validation involves checking that the assessment tool/s produce/s valid, reliable, sufficient, current and authentic evidence to enable reasonable judgements to be made as to whether the requirements of the training package or VET accredited courses are met.

It includes reviewing a statistically valid sample of the assessments and making recommendations for future improvements to the assessment tool, process and/or outcomes and acting upon such recommendations.

Standards for RTOs 2015Standard 1 – The RTO’s training and assessment strategies and practices are responsive to industry and learner needs and meet the requirements of training packages and accredited courses

• comprehensive plan of systematic validation• risk-based approach to developing the plan• Trainers and assessors involved in training and assessment may also

participate in the validation process

1.9. The RTO implements a plan for ongoing systematic validation…for each training product on the RTO’s scope of registration:a) when assessment validation will occur;b) which training products will be the focus of the validation;c) who will lead and participate in validation activities; andd) how the outcomes…will be documented and acted upon.

Standards for RTOs 20151.10. …each training product is validated at least once every five years, with at least 50% of products validated within the first three years of each five year cycle, taking into account… risks…on the RTO’s scope…including those risks identified by the VET Regulator.

1.11. …systematic validation…is undertaken by one or more persons who are not directly involved in the particular instance of delivery and assessment of the training product being validated, and who collectively have:a) vocational competencies and current industry skills relevant to the assessment being validated;b) current knowledge and skills in vocational teaching and learning; andc) the training and assessment qualification or assessor skill set referred to in Item 1 or 3 of Schedule 1.

Industry experts may be involved in validation to ensure there is the combination of expertise set out in (a) to (c) above.

Standards for RTOs 2015Independent validation of training and assessment qualifications1.25. From 1 January 2016, to deliver any AQF qualification or assessor skill set from the Training and Education Training Package (or its successor), the RTO must have undergone an independent validation…

Independent validation means the validation is carried out by a validator/s who:a) are not employed or subcontracted by the RTO to provide training and assessment; andb) have no other involvement or interest in the operations of the RTO.

Standard 2. The operations of the RTO are quality assured.2.2. The RTO:b) systematically evaluates and uses the outcomes of the evaluations to continually improve the RTO’s training and assessment strategies and practices. Evaluation information includes but is not limited… validation outcomes...

Standards for RTOs 2015What do you think?What impact is this going to have on consistency of assessment?What will need to change in your RTO?

RQI - 321

Write down three things you recall about today’s webinar.

Write down two questions you have about today’s webinar.

Write down one interesting point about today’s webinar.

Tomorrow…Part 2…

Tips and techniques for:• Meeting principles of assessment• Ensuring rules of evidence are met• Developing criteria for reviewing, comparing

and evaluating assessment processes

All pictures in these slides were taken from http://search.creativecommons.org/