localisation of assessment: theory & practice · answer key or rating scale ... aptis [bc - uk]...
TRANSCRIPT
![Page 1: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/1.jpg)
Localisation of Assessment:
Theory & Practice
Barry O’Sullivan
![Page 2: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/2.jpg)
Validity & Validation
A Validation Model
Localisation in Theory
Localisation in Practice
1
Overview
![Page 3: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/3.jpg)
2
Language Testing = Validity
the title that never was
![Page 4: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/4.jpg)
3
A theory of validity is only of practical value if it is translatable into a coherent
theory or ‘model’ of validation which can then be operationalised through a set of
validation procedures. O’Sullivan & Weir, 2011
We cannot understand assessment if
we do not understand validity
![Page 5: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/5.jpg)
4
THE SOCIO-COGNITIVE MODEL
focusing on the test-taker
![Page 6: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/6.jpg)
5
PERFORMANCE Task Language
Target Language
Task Performance Parameters
Social Factors
Personal Characteristics
Cognitive Characteristics
TEST TASK
TEST TAKER
![Page 7: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/7.jpg)
6
Machine or Human
Answer Key or Rating Scale
Selection & Training Consistency
Accuracy
Interpretation
Scoring Systems
![Page 8: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/8.jpg)
7
Test Taker Test Task [including examiner]
Scoring
![Page 9: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/9.jpg)
8
Weir (2005)
![Page 10: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/10.jpg)
9
![Page 11: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/11.jpg)
10
Weir (2005)
?
![Page 12: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/12.jpg)
11
O’Sullivan (2011)
![Page 13: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/13.jpg)
12
LOCALISATION - THEORY
refocusing on the test-taker
![Page 14: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/14.jpg)
12
The process of ensuring that all predicted construct-irrelevant
variance related to the test taking population is controlled for
in the design and delivery of a test
![Page 15: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/15.jpg)
13
Parameters related to the test population are known and fixed
Inferences or decisions are limited to a specific local context
Testing a specific population within a specific context in order to make
decisions that will apply specifically to that context
![Page 16: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/16.jpg)
14
Neutral
Linguistically
Visually
Conceptually
No curriculum No local language No cultural references
No local references No cultural references
No complex cognitive No local references No cultural references
General Purpose Examinations
![Page 17: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/17.jpg)
15
Localised Examinations
Test Population
Specific
Linguistically
Visually
Conceptually
Curriculum Local language Cultural references
Local references Cultural references
Possible complex cognitive Local references Cultural references
![Page 18: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/18.jpg)
16
LOCALISATION - PRACTICE
the effect on the test
![Page 19: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/19.jpg)
17
Exaver [Mexico] - 2005 - Word-lists
Task design
Aptis [BC - UK] - 2012 -
Test Concept Localised to:
Domain Culture
Age L1
![Page 20: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/20.jpg)
18
Scoring
System
Appropriate Content
Appropriate Task
Appropriate Language
Appropriate Scoring
Appropriate Topic
Test
Taker
Test
Task
![Page 21: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/21.jpg)
19
Case Study [India] Population 15 & 17 yrs Secondary School Similar Culture Similar Education Mixed ability Mixed gender Mixed L1
Localisation Language Reviewed [inc. instructions] Reading Topics Listening Topics Writing Topics/Tasks Speaking Topics Speaking Images
![Page 22: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/22.jpg)
20
Speaking Task
Images
![Page 23: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/23.jpg)
21
Cost Change means cost in terms of external input, training, quality assurance, systems development, delivery and monitoring
People Identification, training and nurturing of local expertise [assessment; technology; management; administration etc.]
Resource Technical – hardware and software + skills; financial; political/social
![Page 24: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/24.jpg)
22
THE FUTURE
where to next?
![Page 25: Localisation of Assessment: Theory & Practice · Answer Key or Rating Scale ... Aptis [BC - UK] - 2012 - Test Concept Localised to: Domain ... Writing Topics/Tasks Speaking Topics](https://reader034.vdocuments.net/reader034/viewer/2022051602/5b2165517f8b9aa4128b4669/html5/thumbnails/25.jpg)
23
Fuller theoretical definition
Broader understanding of how it works in practice
Operational checklists
Research on impact of localisation