integrating lln and the acsf & assessment lln... · literacy and numeracy (lln) in learning and...
TRANSCRIPT
Integrating LLN and the ACSF
into our Training & Assessment
To be effective as trainers/assessors we need to have an understanding of the
significance of language, literacy and numeracy (LLN) in learning and assessment
processes.
Numeracy
LLN Reading Writing
Oral communication
The LLN demands of the workplace for which
the learner is being prepared
The LLN demands we impose through
our training and assessment
materials and resources
The LLN skills of the learner
Three intersecting sets of LLN demands
on the learner
What do we need to know about LLN?
The session will also consider how aspects of the very complex Australian Core Skills Framework can be used by the general trainer/assessor to analyse, determine
and cater for each of these demands.
How do we establish the level of these demands?
A complex framework, designed primarily for “LLN specialists”
BUT, also a framework that can help the “ordinary”
trainer/assessor
ACSF Overview
The ACSF describes each of the five core skills [learning, reading, writing, oral communication, numeracy] across three interactive dimensions.
5 Core Skills
Five levels of performance ranging from 1 (low level performance) to 5 (high level performance) The five levels of performance represent milestones along a continuum of performance.
ACSF PERFORMANCE INDICATORS
Core Skill Indicator
Number Description
Learning
.01 Active awareness of self as a learner, planning and management of
learning
.02 Acquisition and application of practical strategies that facilitate
learning
Reading .03 Audience, purpose and meaning-making
.04 Reading strategies
Writing .05 Audience, purpose and meaning-making
.06 The mechanics of writing
Oral
Communication
.07 Speaking
.08 Listening
Numeracy
.09 Identifying mathematical information and meaning in activities and
texts
.10 Using and applying mathematical knowledge and problem solving
processes
.11 Communicating and representing mathematics
Indicator numbers
are consistent across core skill levels
e.g. Reading
1.03 / 1.04 2.03 / 2.04 3.03 / 3.04 4.03 / 4.04 5.03 / 5.04
Core skills are colour coded
ACSF Overview
Applying the ACSF
What are important reading tasks carried out by an assessor?
FOR EXAMPLE Reading and interpreting competency standards Reading attendance sheets Reading and interpreting assessment tools Reading and analysing documents supplied as evidence Reading and analysing evidence presented in different media Reading learner records sheets
ACSF Reading Levels
• The first Reading Indicator addresses audience, purpose and meaning-making. • The second Reading Indicator addresses reading strategies. • Specific Indicator statements describe exit performance at each level.
Reading attendance sheets 2.03 Reading learner records sheets 2.03
Reading strategies
Audience, purpose and
meaning making
Reading and interpreting competency standards 3.03 Reading and analysing documents supplied as evidence 3.03 Reading and interpreting assessment tools 3.03 Reading and analysing evidence presented in different media 3.04
Applying the ACSF
What are important writing tasks carried out by an assessor?
FOR EXAMPLE Completing learner records Filling out previously prepared checklists Writing judgements about evidence provided Using various communication strategies to record outcomes (e.g. text, charts, electronic)
The first Writing Indicator addresses audience, purpose and meaning-making. The second Writing Indicator addresses the mechanics of writing. Specific Indicator statements describe exit performance at each level.
ACSF Writing Levels
Completing learner records 1.05
Filling out previously prepared checklists 2.05/2.06
Writing judgements about evidence provided 4.05 Using various communication strategies to record outcomes (e.g. text, charts, electronic) 4.06
Establishing LLN Skill Levels
oral communication skills,
and numeracy skills,
required in your job.
We could do a similar mapping of the learning skills,
Note that we will probably end up with a “spiky profile”.
“Spiky” Profiles
This concept of “spiky profiles” draws our attention to the lack of correspondence
between AQF levels and ACSF levels.
ACSF AQF
More Profiles
We could do an analysis of the LLN demands of the training and assessment materials and processes we provide as part of our training context.
Hopefully we would find some match between these “spiky profiles” and the “spiky profile” of the LLN demands of the actual job.
Having established the LLN demands of the work environment and of the training and assessment context, we now need some understanding of the LLN skills of our learner – another “spiky profile”.
Learner LLN Skills
In particular, we need to be aware of mismatches between the learner “spiky profile” and the workplace and training contexts “spiky profiles”.
As “non-specialists” in LLN we are not able to carry out a
fine-grained analysis of each learner. However, we can use LLN assessment tools that will
help us make useful judgements.
Identifying Learner Skills
These tools need to better focused than generalised
reading, writing and maths tests.
Such tests don’t tell us if a learner will be able to deal with
the LLN demands of the workplace and training contexts.
Go back to the list of reading tasks we identified as being required by an assessor.
Let us assume that one of these was reading a report to gain some required information.
How could we establish if someone else would have the necessary reading skill to complete this task?
Developing a Better LLN Test
So we could build up a test of reading tasks directly related to the workplace and training contexts.
We could do the same thing with writing tasks, oral communication tasks, and
numeracy tasks.
We would then have a set of assessment tasks that would allow us to make
judgements of the extent to which our learners could cope with these tasks.
The outcomes of these tests would identify where the LLN
skills of learners are adequate
and where there are gaps in these skills.
Developing a Better LLN Test
So What?
This information might help us make adjustments
to our delivery and assessment processes so that
we could include the development of some of the
“gap” skills, or refer the learner to more
specialised help.
It might also help us identify where our own training and assessment materials and/or
processes do not match the LLN demands of the workplace/unit of competency.
Validated LLN Assessment Tools
Some of these will exist in your industry area. If not, existing tools may be able to be contextualised to address the LLN demands of your industry area.
If these LLN assessment tools are to have real value it is important that they are validated (with our peers and/or with specialists and/or with industry.)
Fortunately, a considerable amount of work has already been done in developing sets of such validated tools.
Some of these will work in your industry area. If not, existing tools may be able
to be contextualised to address the LLN demands
of your industry area.
Validated LLN Tools - Precision
http://www.precisionconsultancy.com.au/acs_framework
Validated LLN tools - VET-FEE HELP
'The testing tool must be an approved testing tool. The approved testing tool, available since 1 January 2016 and specified in the VET Guidelines is the Core Skills Profile for Adults (CSPA).’ Department of Education and Training, Update on VET FEE−HELP Reforms, 8 December 2015
Assessment of LLN testing instruments and processes for VET FEE-HELP providers Report (Stage 2)
• The provider must arrange to have their LLN assessment tool independently assessed. • An independent assessor who undertakes the audits of any LLN testing instruments
should be an independent body or approved independent and specialist personnel, such as an auditor with VET experience for details.
• The independent assessment must directly respond to the specifications and requirements for approval.
• Completed assessment must be sent to the Secretary, DET, care of the Branch Manager, VET FEE-HELP Branch for approval. This could be in electronic or paper form.
Very detailed requirements are included in pages 31 - 43
Validated Tools - VET-FEE HELP
Criterion 1: Validity • The assessment is based on quality assessment development processes • The learner undertakes the assessment independently (Also in Criterion 2) • The tasks and questions are validly mapped against the different ACSF levels (Also in Criteria 2, 3) • Information is provided about the performance or scores required to decide that a learner has
independently proved their competence at ACSF Exit Level (x) Criterion 2: Reliability
• Administration guidelines include details about who can mark the assessment if it is to be manually marked and a specified set of marking information and scoring rubrics are provided
• The marking and scoring guides or rubrics can be applied consistently and reliably across different assessors/markers
• Administration guidelines include clear details of the conditions under which the assessment should take place (Also in Criteria 3 and 4, plus reasonable adjustment)
Criterion 3: Fairness • The assessment tasks were tested and trialled with different groups of learners and checked for
performance and feedback gathered from different learners Criterion 4: Well-constructed
• The LLN assessment tool is well constructed with quality presentation and quality and clear sets of instructions for both the learners being assessed and for the administrators of the assessment
This list of criteria has been adapted from the ACER list for developing tools for VET FEE-HELP purposes. It is provided as a guide for developing less formal tools.
NOTE: VET FEE-HELP tools MUST meet ALL of the original requirements.
Validating your own LLN Assessment Tools
1
2
3
We have discussed LLN and
ACSF
We have identified the LLN demands of workplaces and
training materials and training processes.
We have looked at ways of identifying learner LLN levels,
using validated tools.
We have put all of this together
Dave Tout
Numeracy: A Core and Crucial Skill in VET