-
Gyngyvr MolnrUniversity of Szeged http://www.staff.u-szeged.hu/~gymolnar
Technology-based Assessment: Challenges and Promises
6th EDEN Research WorkshopBudapest, Hungary, 24-27 October, 2010
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
Developing Diagnostic Assessments Project
-
Outline
Need for change Possibilities and promises of TBA Implementing TBA in international projects
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
The need for change
Technology abounds Change - Live, learn,
communicate New insight into the
learning process Potential to transform
education
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
The need for change
New expectation, lifestyle - the Net Generation digitalnatives => calls for different type of teaching
Schifting from generationgap to generation lap
New demands on the education
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
The need for change Existing models of
assessment fail New assessments are
required
Multi-sensory feature Retention rates Technology alone will not
result in higher achievement
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
Technology is not the issue - > catalyst
-
The need for change
Data about education quality and its impact on economic output
After 4 cycles of PISA Economic modeling E.g. how the GDP would
change according to the change in educational output
Result: relatively small improvements large impact on future
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
0
200
400
600
800
1000
1200
Mex
ico
Turk
eyG
reec
eP
ortu
gal
Italy
Luxe
mbo
urg
Spa
inU
nite
d S
tate
sP
olan
dN
orw
ayS
lova
k R
epub
licO
EC
DH
unga
ryD
enm
ark
Ger
man
yIc
elan
dFr
ance
Irela
ndS
wed
enA
ustri
aS
witz
erla
ndB
elgi
umC
zech
Rep
ublic
Uni
ted
Kin
gdom
Aus
tralia
New
Zea
land
Can
ada
Net
herla
nds
Japa
nK
orea
Finl
and
Catching up with Finland(in percent of GDP)
% currentGDP
-
Why is assessment so important?
You can not improve what you can not measure(Schleicher, 2010)
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
The feedback in education ICT growing in every segment of education International tendencies (OECD, ETS, NCES) in
educational assessment PPT reached its barrier Human resources, cost of money and time TBA solve this problem NEED to use technology to improve the efficiency of
assessment Improved quality of data and increased speed of
assessment Better conditions to improve the educational system
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Possibilities and promises of TBA(1) Reach stimulus material (multimedia tools) New item formats New construct (PS in technology reach environment) New areas (dynamic assessment) assessing dynamic
MicroDYN approach example
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Possibilities and promises of TBA
(2) TBA rises new issues in assessment:
background data testing prozess - e.g. educational data mining log file analyses - issues of reaction time, sequence
annotations regarding eye and face tracking
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Possibilities and promises of TBA(3) offers new assessment methods (adaptive testing)
the difficulty of the test tailors dynamically to the students ability level
items are selected from an item bank dependent at a difficulty appropriate for the student
provides more time-efficient and accurate assessments.
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
-4
-3
-2
-1
0
1
2
3
4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
logi
tsca
le
item
Adaptive testing
Person ability level
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
(4) increases motivation - students with high and low ability
level - flow experience (5) changes the whole assessment process (item generation,
scoring, data-processing, information flow, feedback and the speed of assessment)
rapid and precise feedback (6) poses new questions and problems if TBA replace PP
improved quality and increased quantity of data and increased speed of assessment
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
Possibilities and promises of TBA
-
Implementing TBA in internationalprojects
Europe: growing attention in several European countriesand the European Union also facilitates these attempts
At national level Luxemburg leads the way: able to assess an entire cohort simultaneously automatic generation of the feedback to the teachers University of Luxembourg and the Public Research
Center Henri Tudor TAO (Testing Assist par Ordinateur)
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Implementing TBA in internationalprojects
Germany DIPF TBA project : adapts and further develops the TAO platform establish a national standard for technology-
assisted testing
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Implementing TBA in internationalprojects
In Hungary 2008 first media effect study: Inductive reasoning 5th grader (PP + CB) TAO platform Result: global achievement highly correlate items,
persons behave differently 2009 - a large-scale project was launched by the Center
for Research on Learning and Instruction - University of Szeged - develop an online diagnostic assessment system for the first six grades of primary school in reading, mathematics and science
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Developing DiagnosticAssessments Project
the development of assessment frameworks devising a large number of items both on paper and on
computer - building item banks research on comparing the achievements on the tests
using different media. long-term project aims: an online formative assessment
system for the first six grades of primary school First phase: pilot the system in 200 schools study the related technological and methodological
issues in detail
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Implementing TBA in internationalprojects
US - many instances CBA and online assessment so wide spread - testing
industry - engaged in writing and developing task and different testing software
Despite of this in 2009 Microsoft, Cisco and Intel -transforming global educational assessment and improving learning outcomes
Teaching of 21st Century Skills project
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Teaching of 21st Century Skills project
Aim: resolve the gap - exists between what goes on in schools and what goes on in the real world
better measure the skills - needed for the 21st-century
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Teaching of 21st Century Skills project
Specify high-priority skills, competencies, - 21st century -turn these into measurable standards and assessment framework
Examine innovative learning environments and formative assessments - draw implications for ICT-based summative assessments
Identify methodological and technological barriers support solutions
implementation of these standards and methodologies, pilot test, make recommendations for broader educational assessment reform.
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
OECD PISA
Started to implement CBA 2006 CBAS 2009 - ERA 2012 - problem solving 2015 - completely to CBT still a lot of open questions
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Despite of the advantages - in large-scale national and international assessment - faces several technological challenges
complexity of issues - no simple and clear solution Several technological means available in schools diversity, compatibility, connectivity require further
considerations
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
But the future is more promising
Research Group on the Development of Competencies, Hungarian Academy of Sciences
Institute of Education, University of Szeged
-
Thank you for your kind attention!http://www.staff.u-szeged.hu/~gymolnar
Research Group on the Development of Competencies, Hungarian Academy of Sciences
and remember:Without data, you are just another person with an Opinion (Schleicher, 2010)
Slide Number 1OutlineThe need for changeThe need for changeThe need for changeThe need for changeSlide Number 7Why is assessment so important?The feedback in educationPossibilities and promises of TBASlide Number 11Possibilities and promises of TBASlide Number 13Possibilities and promises of TBAAdaptive testingPossibilities and promises of TBAImplementing TBA in international projectsImplementing TBA in international projectsImplementing TBA in international projectsDeveloping Diagnostic Assessments Project Implementing TBA in international projectsTeaching of 21st Century Skills projectTeaching of 21st Century Skills projectOECD PISA Slide Number 25But the future is more promising http://www.staff.u-szeged.hu/~gymolnar