TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63
TEASIG NEWSLETTERI s s u e 6 3D E C E M B E R 2 0 1 7
3 Editorial
4 >ĞƩ ĞƌĨƌŽŵƚŚĞŽŽƌĚŝŶĂƚŽƌƐ
5 dŚĞd^/' ĐŽŵŵŝƩ ĞĞ
6 t ĂƐŚďĂĐŬŽĨWƌĂĐƟĐĂůŶŐůŝƐŚĞdžĂŵŝŶĂƟŽŶƐŽŶƚŚĞƉƌŽĐĞƐƐŽĨƚĞĂĐŚŝŶŐĂŶĚůĞĂƌŶŝŶŐ Elżbieta Zawadowska-<ŝƩ Ğů
11 ĚƵĐĂƟŽŶŽƌĞǀ ĂůƵĂƟŽŶ Ilya Denisenko
15 D ĂdžŝŵŝƐŝŶŐĐŽůůĂďŽƌĂƟŽŶŝŶŽƌĂůŶŐůŝƐŚƚĞƐƟŶŐĂŶĞdžĂŵƉůĞĨƌŽŵŚŝŶĂ Susanna Wickes
19 ZĞĐĞƉƟǀ ŝƚLJƚŽůĞĂƌŶĞƌ-driven feedback Clare Maas
24 ' ŝǀ ŝŶŐƚĞƐƟŶŐƐŽŵĞŐŽŽĚƉƌĞƐƐĨŽƌĂĐŚĂŶŐĞ Assessment as learning check - ĂŵĞĂŶŝŶŐĨƵůƉĂƌĂĚŝŐŵƐŚŝŌĨŽƌƚƌĂŝŶĞĞƚĞĂĐŚĞƌƐ
Lynn Williams
28 ŶŵŝŚƵŵŝůĚĞŽƉŝŶŝſ Ŷ ϭ ůŝƐƚĞŶŝŶŐƚŽD ĞdžŝĐĂŶƐƚƵĚĞŶƚƐ ƉĞƌĐĞƉƟŽŶƐŽĨĂŶŶŐůŝƐŚlanguage proficiency test
David Ewing Ryan
35 IATEFL TEASIG Webinar(24 October 2017): Developing a test: Where do you start?When should you stop?
Jo Tomlinson
38 TEASIG Member Spotlight ŚƌŝƐƟŶĞŽŽŵďĞ
40 TEASIG Member Spotlight Gladys Quevedo Camargo
Price £ 4.50 Free for TEASIG members ISSN 2414-6307 www.iatefl.org
T h e N e w s l e t t e r o f t h e T e s t i n g , E v a l u a t i o n a n d A s s e s s m e n t S p e c i a l I n t e r e s t G r o u p
2
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 2
Editorial
t ĞůĐŽŵĞƚŽ/ƐƐƵĞϲϯŽĨƚŚĞd^/' EĞǁ ƐůĞƩ Ğƌ dŝŵĞŚĂƐƐŝŵƉůLJƐƉĞĚďLJƐŝŶĐĞŽƵƌůĂƐƚƉƵďůŝĐĂƟŽŶ ƚŚĞƵƚƵŵŶEĞǁ ƐůĞƩĞƌĂŶĚŝƚŝƐŝŶĚĞĞĚƚŚĂŶŬƐƚŽƚŚĞĐŽŶƚƌŝďƵƟŽŶƐŽĨŽƵƌŝŶĚƵƐƚƌŝŽƵƐd^/' ŵĞŵͲbers that we are able to bring out another News-ůĞƩĞƌƐŽƐŽŽŶ /ǁ ŽƵůĚůŝŬĞƚŽƚŚĂŶŬƚŚĞŵ ďŽƚŚTESIG members and non-members, for helping tomake this possible, and hope that they will inspireŵŽƌĞŽĨLJŽƵƚŽƐĞŶĚŝŶĂƌƟĐůĞƐ;ǁ ŚŝĐŚĚŽŶŽƚŶĞĞĚto be long) or take part in a TEASIG Member Spot-light interview. Further details can be found atŚƩ ƉƐ ƚĞĂ ŝĂƚĞŇŽƌŐ ŶĞǁ ƐůĞƩĞƌ
dŚŝƐŝƐƐƵĞƐƚĂƌƚƐŽī ǁ ŝƚŚƚŚƌĞĞĐŽŶƚƌŝďƵƟŽŶƐƐƵďͲŵŝƩ ĞĚďLJƉƌĞƐĞŶƚĞƌƐĂƚƚŚĞ/d&>ŽŶĨĞƌĞŶĐĞŝŶGlasgow last April. The first is based on a talk given at the joint ESPSIG/TEASIG Pre-Conference Event;WͿ /ŶƚŚŝƐĂƌƟĐůĞůǏ ďŝĞƚĂĂǁ ĂĚŽǁ ƐŬĂ-<ŝƩ ĞůĚĞͲscribes a new exam for BA students together with aƐƚƵĚLJǁ ŚŝĐŚǁ ĂƐƵƐĞĚƚŽŝŶǀ ĞƐƟŐĂƚĞŝƚƐǁ ĂƐŚďĂĐŬrecalling the power of tests to influence the teach-ing and learning process that precedes them. TheƐĞĐŽŶĚƚǁ ŽĂƌƟĐůĞƐĂƌĞďĂƐĞĚŽŶƉƌĞƐĞŶƚĂƟŽŶƐĂůƐŽgiven in Glasgow, both at the TEASIG day. Ilya Den-isenko’s thought-ƉƌŽǀ ŽŬŝŶŐĐŽŶƚƌŝďƵƟŽŶĞŶĚŽƌƐĞƐ ĂƵƚŚĞŶƟĐĂƐƐĞƐƐŵĞŶƚ ĂŶĚĞŶĐŽƵƌĂŐĞƐƚĞĂĐŚĞƌƐƚŽƐĞĞŽƉƉŽƌƚƵŶŝƟĞƐĂŶĚďĞŶĞĮ ƚƐŝƚƉƌĞƐĞŶƚƐ ǁ ŚŝůĞ^ƵƐĂŶŶĂt ŝĐŬĞƐ ĂƌƟĐůĞƉƌŽǀ ŝĚĞƐĂŶŝŶƚĞƌĞƐƟŶŐĂĐͲĐŽƵŶƚŽĨƚŚĞŝŶƚƌŽĚƵĐƟŽŶŽĨĂŐƌŽƵƉŽƌĂůƚĞƐƚŝŶŚŝͲŶĂ ĂůƐŽĐŽŶĐĞƌŶĞĚǁ ŝƚŚĂƵƚŚĞŶƟĐŵĞĂŶŝŶŐĨƵůĂƐͲsessment.
/ŶƚŚĞƌĞŵĂŝŶŝŶŐĐŽŶƚƌŝďƵƟŽŶƐ ůĂƌĞD ĂĂƐƐŚĂƌĞƐher experience of the LDF (Learner-Driven Feed-ďĂĐŬͿĂƉƉƌŽĂĐŚĂŶĚƐƚƵĚĞŶƚƐ Ăƫ ƚƵĚĞƐƚŽǁ ĂƌĚƐŝƚ The findings are promising, and Clare kindly pro-ǀ ŝĚĞƐƟƉƐĨŽƌƚƌLJŝŶŐŽƵƚƚŚŝƐƵƐĞĨƵůƚĞĐŚŶŝƋƵĞ>LJŶŶt ŝůůŝĂŵƐƚĂŬĞƐĂƉŽƐŝƟǀ Ğǀ ŝĞǁ ŽĨĂƐƐĞƐƐŵĞŶƚŝŶŚĞƌwork as a teacher trainer, reminding us that as as-sessment is not necessarily a test and sharing someof the strategies she uses with her trainee teachers.dŽĐŽŶĐůƵĚĞƚŚŝƐƐĞĐƟŽŶ Ăǀ ŝĚǁ ŝŶŐZLJĂŶ ƐĂƌƟĐůĞmakes intriguing reading as he describes a study inD ĞdžŝĐŽƚŚĂƚŝŶǀ ĞƐƟŐĂƚĞĚƚŚĞĂƫ ƚƵĚĞƐĂŶĚŽƉŝŶŝŽŶƐŽĨƚŚĞŵŽƐƚŝŵƉŽƌƚĂŶƚ ďƵƚŽŌĞŶŶĞŐůĞĐƚĞĚ ƐƚĂŬĞͲ
holders in a language test, that is, the test-takersthemselves.
/ŶƚŚĞĮ ŶĂůƐĞĐƟŽŶŽĨƚŚĞEĞǁ ƐůĞƩĞƌ:ŽdŽŵůŝŶƐŽŶĂŶƐǁ ĞƌƐŽƵƚƐƚĂŶĚŝŶŐƋƵĞƐƟŽŶƐŝŶĂĨŽůůŽǁ -up to theŝŶĨŽƌŵĂƟǀ Ğǁ ĞďŝŶĂƌƐŚĞŐĂǀ ĞŽŶƚĞƐƚĚĞǀ ĞůŽƉŵĞŶƚon 24 October 2017. (Check out the next webinaron the TEASIG website!). We round off this issue by turning the TEASIG spotlight on two of our mem-ďĞƌƐĨƌŽŵĚŝī ĞƌĞŶƚĐŽƌŶĞƌƐŽĨƚŚĞŐůŽďĞŚƌŝƐƟŶĞCoombe, who is based in Dubai, and Gladys Queve-do Camargo from Brasilia.
Thanks again to everyone who contributed to thisEĞǁ ƐůĞƩĞƌĂŶĚĨŽƌƐŚĂƌŝŶŐƚŚĞŝƌŝĚĞĂƐǁ ŝƚŚƚŚĞTEASIG community. Very best wishes to you all forthe New Year – hoping to hear from you or see youat a TEASIG event in 2018!
Maggi Lussi Bell
TEASIG Joint Editor
3
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 3
Letter from the Coordinators
Dear TEASIG members
ƐƚŚŝƐŵĂLJǁ ĞůůďĞŵLJůĂƐƚůĞƩ ĞƌƚŽLJŽƵŝŶƚŚĞ
d^/' EĞǁ ƐůĞƩ ĞƌĂƐd^/' ŽŽƌĚŝŶĂƚŽƌEĞŝůŚĂƐ
asked me to write and say something about myself.
Some of you may have read in the latest issue of
ƚŚĞ/d&>ĞƵůůĞƟŶƚŚĂƚ/ŚĂǀ ĞďĞĞŶĞůĞĐƚĞĚƚŽ
represent all the 16 IATEFL SIGs on the Board of
Trustees. I feel very honoured to have been en-
ƚƌƵƐƚĞĚǁ ŝƚŚƚŚŝƐƉŽƐŝƟŽŶ ĂŶĚŚŽƉĞƚŚĂƚ/ĐĂŶůŝǀ Ğ
ƵƉƚŽƚŚĞĞdžƉĞĐƚĂƟŽŶƐƚŚĂƚŵLJĨĞůůŽǁ /' ŽŽƌĚŝͲ
nators have of me.
/ǁ ŝůůďĞĐŽŵĞ /' ZĞƉƌĞƐĞŶƚĂƟǀ ĞĨƌŽŵƉƌŝůϮϬϭϴĂƚ
the Annual Conference, which means that I will be
stepping down as TEASIG Coordinator. I will, how-
Ğǀ ĞƌƌĞŵĂŝŶŽŶƚŚĞd^/' ŽŵŵŝƩ ĞĞĂŶĚĐŽŶƟŶͲ
ƵĞƚŽǁ ŽƌŬǁ ŝƚŚD ĂŐŐŝŽŶƚŚĞEĞǁ ƐůĞƩ Ğƌ/ŚĂǀ Ğ
ƚǁ ŽƌĞĂƐŽŶƐĨŽƌǁ ĂŶƟŶŐǀ ĞƌLJŵƵĐŚƚŽĐŽŶƟŶƵĞ
with this work, connected with TEASIG itself and
ǁ ŝƚŚƉƵďůŝĐĂƟŽŶƐ
I became a member of IATEFL in 1980 because I
wanted to become part of a community of EFL
teachers. Almost the only contact with the associa-
ƟŽŶĂŶĚǁ ŝƚŚŽƚŚĞƌ/d&>ŵĞŵďĞƌƐǁ ĂƐƚŚƌŽƵŐŚ
ƉƌŝŶƚĞĚƉƵďůŝĐĂƟŽŶƐƐƵĐŚĂƐs ŽŝĐĞƐ ǁ ŚŝĐŚǁ ĂƐƐĞŶƚ
by post. I lurked happily in the background of
/d&>ĨŽƌĂǀ ĞƌLJůŽŶŐƟŵĞĨŽůůŽǁ ŝŶŐŝƚƐĂĐƟǀ ŝƟĞƐ
with great interest. However, for a number of per-
sonal and professional reasons, such as small chil-
ĚƌĞŶĂŶĚĨƌĞĞůĂŶĐĞǁ ŽƌŬ/ǁ ĂƐŶ ƚĂďůĞƚŽĂƩ ĞŶĚĂ
ĐŽŶĨĞƌĞŶĐĞƵŶƟůϮϬϬϱ ĂůƚŚŽƵŐŚŝƚŚĂĚďĞĞŶŽŶĞŽĨ
ŵLJĂŵďŝƟŽŶƐĨŽƌƐŽŵĞƟŵĞ/ǁ ĂƐǁ ŽƌŬŝŶŐĂƐĂ
ƚĞƐƚĚĞǀ ĞůŽƉĞƌĂŶĚŵLJƉƌĞƐĞŶƚĂƟŽŶǁ ĂƐƐĞůĞĐƚĞĚ
as part of the TEASIG Day, which turned out to be
ƚŚĞƐƚĂƌƚŽĨĂŚĂƉƉLJĂŶĚĨƌƵŝƞƵůƌĞůĂƟŽŶƐŚŝƉ ĂŶĚ
one which I now no longer want to forego.
Not only do I belong to the strange breed of test-
lovers, but I am a self-designated manic proof-
ƌĞĂĚĞƌ/ĚŝƐĐŽǀ ĞƌĞĚƚŚĞũŽLJƐŽĨĞĚŝƟŶŐĂŶĚƉƌŽŽĨ-
reading through my work as a test developer, and
my main professional responsibility is now quality
control of assessment procedures, which also in-
ǀ Žůǀ ĞƐĂŐƌĞĂƚĚĞĂůŽĨĞĚŝƟŶŐ/ĂĐƚƵĂůůLJĮ ŶĚŝƚŚĂƌĚ
ƚŽƌĞĂĚĂŶLJƚŚŝŶŐǁ ŝƚŚŽƵƚŵĞŶƚĂůůLJĞĚŝƟŶŐŝƚŽƌĮ ŶĚͲ
ŝŶŐƉƌŝŶƟŶŐŵŝƐƚĂŬĞƐ ďĞŚĂǀ ŝŽƵƌƐŽŵĞƟŵĞƐĚĞͲ
scribed as obsessive!
D LJŝŶƚĞƌĞƐƚŝŶƉƵďůŝĐĂƟŽŶƐŝƐǁ ŚĂƚ/ĨĞĞů/ŚĂǀ Ğ
brought to TEASIG. TEASIG produces a regular
EĞǁ ƐůĞƩ ĞƌĂŶĚŚĂƐƉƌŽĚƵĐĞĚĂŶƵŵďĞƌŽĨŽŶĨĞƌͲ
ĞŶĐĞWƌŽĐĞĞĚŝŶŐƐĂƐǁ ĞůůĂƐĐŽŶƚƌŝďƵƟŶŐƚŽƚŚĞ
/d&>ĞƵůůĞƟŶĂŶĚs ŽŝĐĞƐ dŚĞƌĞĂƌĞŵĂŶLJŽƚŚĞƌ
ŝĚĞĂƐĨŽƌƉƵďůŝĐĂƟŽŶƐŝŶƚŚĞĂŝƌƐƵĐŚĂƐĂd^/'
peer-reviewed journal, a “Best of TEASIG” publica-
ƟŽŶ Ăd^/' ďůŽŐĂŶĚŵŽƌĞ, Žǁ Ğǀ ĞƌĂůůŽĨ
ƚŚĞƐĞĞdžĐĞůůĞŶƚŝĚĞĂƐƌĞƋƵŝƌĞŵŽƌĞƟŵĞĂŶĚĞŶĞƌŐLJ
ƚŚĂŶŽŶĞŽƌƚǁ ŽĐŽŵŵŝƩĞĞŵĞŵďĞƌƐĐĂŶƉƌŽǀ ŝĚĞ
So I am going to end this message with a call for
volunteers to work in the area of TEASIG publica-
ƟŽŶƐ ŶŽƚŽŶůLJǁ ŝƚŚŝĚĞĂƐĨŽƌĚĞǀ ĞůŽƉŵĞŶƚďƵƚĂůƐŽ
with a commitment to do something to make
d^/' ƐƐĞƌǀ ŝĐĞƐĞǀ ĞŶďĞƩĞƌĨŽƌĂůůŝƚƐŵĞŵďĞƌƐ
Please contact me if you wish to be a part of our
dynamic and progressive team.
ZĞŵĞŵďĞƌĞǀ ĞƌLJůŝƩ ůĞŚĞůƉƐ
Wishing you all the very best for
2018,
Judith
4
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 4
Mehtap InceEvents Coordinator
Neil BullockJoint Coordinator
Webinar Coordinator
Sharon HartleWebinar Coordinator
Thom KiddleWebmaster
Mehvar Ergun TurkkanMembership Officer
Maggi Lussi Bell:ŽŝŶƚEĞǁ ƐůĞƩ ĞƌĚŝƚŽƌ
Judith MaderJoint Coordinator
:ŽŝŶƚEĞǁ ƐůĞƩ ĞƌĚŝƚŽƌ
Ceyda MutluSocial Media Manager
Dave AllanD ĞŵďĞƌǁ ŝƚŚŽƵƚWŽƌƞŽůŝŽ
The TEASIG committee
dŽĮ ŶĚŽƵƚŵŽƌĞĂďŽƵƚd^/' ĐŽŵŵŝƩĞĞŵĞŵďĞƌƐŐŽƚŽŚƩ Ɖ ƚĞĂ ŝĂƚĞŇŽƌŐ ŝŶĚĞdžƉŚƉ ĂďŽƵƚ-teasig/teasig-ĐŽŵŵŝƩ ĞĞ
People involved in this issue:Editors: Maggi Lussi Bell, Judith MaderPhotos: TEASIG-CRELLA Conference, Luton 2017Design and Layout: elc – European Language Competence
5
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 5
ŽŶƚƌŝďƵƟŽŶƐƚŽƚŚĞd^/' E Ğǁ ƐůĞƩ Ğƌ
dŚĞ/d&>d^/' ;dĞƐƟŶŐǀ ĂůƵĂƟŽŶĂŶĚƐƐĞƐƐŵĞŶƚ ƉĞĐŝĂů/ŶƚĞƌĞƐƚ' ƌŽƵƉͿEĞǁ ƐͲůĞƩ ĞƌŝƐƉƵďůŝƐŚĞĚƚŚƌĞĞƟŵĞƐĂLJĞĂƌ– firstly, a post-IATEFL Annual Conference issue, between July and September, secondly, at the end of the calendar year, and finally, at the beginning of the calendar year, before the IATEFL Annual Conference. All issues are indigital form.
dŚĞŶĞdžƚƐƵďŵŝƐƐŝŽŶĚĂƚĞƐĨŽƌĂƌƟĐůĞƐĂŶĚĂĚǀ ĞƌƟƐĞŵĞŶƚƐĂƌĞϯϭ:ĂŶƵĂƌLJϮϬϭϴĂŶĚ31 May 2018..
ŽŶƚƌŝďƵƚŽƌƐĚŽŶŽƚŶĞĞĚƚŽďĞŵĞŵďĞƌƐŽĨ/d&>Žƌd^/' ŚŽǁ Ğǀ ĞƌĐŽŶƚƌŝďƵƟŽŶƐĨƌŽŵd^/' ĂŶĚ/d&>ŵĞŵďĞƌƐ ĂŶĚŽŶŵĂƩ ĞƌƐŽĨŝŶƚĞƌĞƐƚƚŽd^/' ŵĞŵďĞƌƐ ǁ ŝůůďĞgiven priority.
d^/' ƌĞƐĞƌǀ ĞƐƚŚĞƌŝŐŚƚƚŽĂĐĐĞƉƚŽƌƌĞũĞĐƚĂƌƟĐůĞƐĂƐŝƚĚĞĞŵƐŶĞĐĞƐƐĂƌLJ/ĨĂŶĂƌƟĐůĞŝƐĂĐĐĞƉƚĞĚĨŽƌƉƵďůŝĐĂƟŽŶ ŝƚŵĂLJďĞƐƵďũĞĐƚƚŽĞĚŝƟŶŐĂŶĚĂůƚĞƌĂƟŽŶƐŵĂLJďĞŵĂĚĞŝŶƟͲtles, headings, length and other aspects. Minor editorial changes in the text may be madefor reasons of space, style, clarity, acceptability and correctness of language. If more thanminor changes are considered necessary, the author(s) will be consulted by the Editors ofƚŚĞEĞǁ ƐůĞƩ ĞƌĨŽƌƚŚĞŝƌĂƉƉƌŽǀ Ăů
/ŶƐŽŵĞĐĂƐĞƐ ĂƌƟĐůĞƐŵĂLJŶŽƚďĞŝŶĐůƵĚĞĚŝŶƚŚĞŶĞdžƚƉƵďůŝƐŚĞĚŝƐƐƵĞŽĨƚŚĞd^/' EĞǁ ƐůĞƩ ĞƌďƵƚƌĞƐĞƌǀ ĞĚĨŽƌĂůĂƚĞƌŝƐƐƵĞdŚŝƐǁ ŝůůďĞĂƚƚŚĞƐŽůĞĚŝƐĐƌĞƟŽŶŽĨƚŚĞEĞǁ ƐͲůĞƩ ĞƌĚŝƚŽƌƐ
Guidelines for contributors
ƌƟĐůĞƐĚŽŶŽƚŶĞĞĚƚŽďĞůŽŶŐdŚĞLJŵĂLJďĞĂƐƐŚŽƌƚĂƐϲϬϬǁ ŽƌĚƐďƵƚƐŚŽƵůĚgenerally be no longer than 2000 words.
Photographs, graphics or diagrams should be sent as separate .jpg files. Please mark clearly in the text where they should be placed and ensure you have permission forƌĞƉƌŽĚƵĐƟŽŶŽĨĂŶLJƐƵĐŚŝƚĞŵƐ /ĨŝŶĚŽƵďƚ ƚŚĞEĞǁ ƐůĞƩ ĞƌĚŝƚŽƌƐŵĂLJĂƐŬĂĐŽŶƚƌŝďƵƚŽƌƚŽǀ ĞƌŝĨLJƚŚĞƐŽƵƌĐĞĂŶĚĂƵƚŚŽƌŝƐĂƟŽŶŽĨĂŶLJƐƵĐŚŵĂƚĞƌŝĂů
Only 5 key references should be given (where required). Other references should beavailable if requested.
WŝĞĐĞƐƐŚŽƵůĚŶŽƚŚĂǀ ĞďĞĞŶƉƵďůŝƐŚĞĚŽƌďĞƵŶĚĞƌŐŽŝŶŐĐŽŶƐŝĚĞƌĂƟŽŶĨŽƌƉƵďůŝĐĂͲƟŽŶĞůƐĞǁ ŚĞƌĞ/ĨƚŚŝƐŝƐƚŚĞĐĂƐĞƉůĞĂƐĞŝŶĐůƵĚĞŝŶĨŽƌŵĂƟŽŶŽŶǁ ŚĞƌĞĂŶĚǁ ŚĞŶƚŚĞĂƌƟĐůĞŚĂƐƉƌĞǀ ŝŽƵƐůLJďĞĞŶŽƌǁ ŝůůďĞƉƵďůŝƐŚĞĚ ĂŶĚĞŶƐƵƌĞLJŽƵŚĂǀ ĞƚŚĞƌŝŐŚƚƚŽƌĞƉƵďůŝƐŚĨƌŽŵƚŚĞĚŝƚŽƌŽĨƚŚĞŽƚŚĞƌƉƵďůŝĐĂƟŽŶ
D ŽƌĞŝŶĨŽƌŵĂƟŽŶŽŶŚŽǁ LJŽƵĐĂŶĐŽŶƚƌŝďƵƚĞƚŽƚŚĞd^/' ŶĞǁ ƐůĞƩ ĞƌĐĂŶďĞĨŽƵŶĚŽŶŚƩ Ɖ ƚĞĂ ŝĂƚĞŇ ŽƌŐ ŝŶĚĞdžƉŚƉ ŶĞǁ ƐůĞƩ Ğƌ
6
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 6
Washback of Practical English examinationson the process of teaching and learning
Elżbieta Zawadowska-<ŝƩ Ğů
is currently working at the Uni-
versity of Social Sciences in
Warsaw. Her PhD thesis centred
on the washback of a school
leaving exam in English on the
teaching and learning process.
Her research interests include
ƚĞƐƟŶŐĂŶĚĂƐƐĞƐƐŵĞŶƚ ŝŶŶŽǀ ĂͲ
ƟŽŶƐŝŶǀ ĂƌŝŽƵƐĮ ĞůĚƐŽĨŵĞƚŚŽĚͲ
ology and the concept of
learning outcomes.
This paper is based on a presen-
ƚĂƟŽŶŐŝǀ ĞŶĂƚƚŚĞ^W/'
TEASIG Pre-Conference Event at
the IATEFL Conference in Glas-
gow in April 2017.
ϭ džĂŵŝŶWƌĂĐƟĐĂůŶŐůŝƐŚĂƐĂƌĞŇĞĐƟŽŶŽĨƚŚĞŝŵƉůĞŵĞŶƚĂƟŽŶŽĨƚŚĞE ĂƟŽŶĂůY ƵĂůŝĮ ĐĂƟŽŶƐFramework
dŚĞŝŶƚƌŽĚƵĐƟŽŶŽĨĂŶĞǁ ĞdžĂŵ
was connected with the implemen-
ƚĂƟŽŶŽĨƚŚĞEĂƟŽŶĂůY ƵĂůŝĮ ĐĂƟŽŶƐ
Framework (NQF) and new learning
outcomes for BA students at the
college where the research was
conducted. Before the change to
ƚŚĞŶĞǁ ĞdžĂŵ ĂƚƌĂĚŝƟŽŶĂůĞdžĂŵ
was used, composed of two parts:
ǁ ƌŝƩ ĞŶĂŶĚŽƌĂůdŚĞǁ ƌŝƩ ĞŶƉĂƌƚ
included reading comprehension,
ůŝƐƚĞŶŝŶŐĐŽŵƉƌĞŚĞŶƐŝŽŶ ǁ ƌŝƟŶŐ
and grammar competence. The oral
exam was taken in pairs and con-
sisted of a discussion on a given
subject. The format of the examina-
ƟŽŶŵĂƚĐŚĞĚƚŚĞǁ ĂLJƚŚĞĐůĂƐƐĞƐĂƚ
ƚŚĞĐŽůůĞŐĞǁ ĞƌĞĨŽƌŵĞĚ ĂƐWƌĂĐƟͲ
cal English classes were divided into
ĐŽŶǀ ĞƌƐĂƟŽŶĐŽŵďŝŶĞĚǁ ŝƚŚƌĞĂĚͲ
ing skills, audio-visual classes, gram-
ŵĂƌĂŶĚǁ ƌŝƟŶŐ
dŚĞŝŶƚƌŽĚƵĐƟŽŶŽĨƚŚĞE Y &ĐĂƚĂͲ
lysed the changes in the process of
teaching and learning. The classes
ĚĞǀ ŽƚĞĚƚŽƚĞĂĐŚŝŶŐWƌĂĐƟĐĂůŶŐͲ
lish (PE) were divided into blocks of
ŝŶƚĞŐƌĂƚĞĚƐŬŝůůƐĂŶĚ ĂĚĚŝƟŽŶĂůůLJ
ŝŶƚŽĐůĂƐƐĞƐĚĞǀ ŽƚĞĚƚŽƉƌŽĚƵĐƟǀ Ğ
skills. The new division is intended
to facilitate and accelerate the
achievement of learning outcomes,
especially as far as speaking and
ǁ ƌŝƟŶŐĂƌĞĐŽŶĐĞƌŶĞĚ
1.1. Washback of exams on the
process of teaching and learning
The research that has been con-
ducted so far (A. Hughes, J. Alder-
son, Li Cheng, E. Zawadowska-<ŝƩ ĞůͿ
shows that it is the manner of
ƚĞƐƟŶŐƚŚĂƚĐŚĂŶŐĞƐƚŚĞƉƌŽĐĞƐƐŽĨ
teaching and learning both as far as
ƉƌŝŽƌŝƟĞƐĂŶĚƚĞĐŚŶŝƋƵĞƐĂƌĞĐŽŶͲ
cerned.
This concept is also reflected in the
ƉƵďůŝĐĂƟŽŶ , Žǁ ƚŽƉƌĞƉĂƌĞĞĚƵĐĂͲ
ƟŽŶĂůƉƌŽŐƌĂŵŵĞƐŝŶĂĐĐŽƌĚĂŶĐĞ
with the NQF” by Andrzej Kraśniew-
ski (2011) who writes: “Proper plan-
ŶŝŶŐŽĨƚŚĞŵĞƚŚŽĚƐŽĨǀ ĞƌŝĮ ĐĂƟŽŶ
of learning outcomes may to a large
extent decide what outcomes have
been, in fact, achieved by the stu-
dent. This results from the fact that
ĂůĂƌŐĞƉĂƌƚĂŶĚƐŽŵĞƟŵĞƐƚŚĞŵĂͲ
jority of students adapts their way
of learning to the content and form
of tests. Thus, these are the meth-
ŽĚƐŽĨǀ ĞƌŝĮ ĐĂƟŽŶƐŽĨůĞĂƌŶŝŶŐŽƵƚͲ
comes which determine what stu-
dents know to a larger extent than
the teaching methods.”
The new exam was designed based
on this view. Summing up the differ-
ĞŶƚŽƉŝŶŝŽŶƐŽŶĂŶĚĚĞĮ ŶŝƟŽŶƐŽĨ
washback, washback may be de-
fined as the influence of exams, in-
ƚĞŶƟŽŶĂůŽƌƵŶŝŶƚĞŶƟŽŶĂůďŽƚŚ
ƉŽƐŝƟǀ ĞĂŶĚŶĞŐĂƟǀ ĞŽŶƚŚĞůĞĂƌŶͲ
ing and teaching process, as this is
widely understood, and on all its
ƉĂƌƟĐŝƉĂŶƚƐĂƐǁ ĞůůĂƐŽŶƚŚĞĞŶƟƌĞ
ĞĚƵĐĂƟŽŶĂůƐLJƐƚĞŵ; Ăǁ ĂĚŽǁ ƐŬĂ
7
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 7
-<ŝƩ ĞůϮϬϭϯ Ϳ /ŶƚŚĞƉƌĞƐĞŶƚƉĂƉĞƌ
two major aspects of the washback
of the PE exam will be presented. It
is worth adding that in order to
ĂĐŚŝĞǀ ĞƉŽƐŝƟǀ Ğǁ ĂƐŚďĂĐŬƐƚƵĚĞŶƚƐ
need to be acquainted with the pur-
pose of the exam and regard its re-
sults as credible and fair, and they
also need to receive a detailed re-
port on their exam results, rather
than only a mark or a percentage
ƐĐŽƌĞĚĚŝƟŽŶĂůůLJƚŚĞƚĂƐŬƐƐŚŽƵůĚ
ďĞĂƵƚŚĞŶƟĐ
2. The concept of the PE exam
The new concept of the exam
ƐƚƌĞƐƐĞƐƚŚĞƌŽůĞŽĨƉƌŽĚƵĐƟǀ ĞƐŬŝůůƐ
in the teaching and learning process
and for this reason both speaking
ƐŬŝůůƐĂŶĚǁ ƌŝƟŶŐƐŬŝůůƐĂƌĞƚĞƐƚĞĚ
ƚǁ ŝĐĞ/ŶĐĂƐĞŽĨǁ ƌŝƟŶŐƐŬŝůůƐ ƚŚĞ
progress students have made during
the whole semester is assessed (by
ŵĞĂŶƐŽĨĂƉŽƌƞŽůŝŽͿ ǁ ŚŝĐŚƐƵƉͲ
ports both autonomy and system-
ĂƟĐůĞĂƌŶŝŶŐĂŶĚĂůƐŽŚĞůƉƐƚŽŝŶͲ
ĐƌĞĂƐĞŵŽƟǀ ĂƟŽŶĨŽƌůĞĂƌŶŝŶŐĂƐ
frequent feedback is given to stu-
ĚĞŶƚƐ ůƐŽ ĐƌŝƟĐĂůƚŚŝŶŬŝŶŐĂďŝůŝƚLJ
and the ability to express opinions
ŝŶǁ ƌŝƟŶŐŝƐƚĞƐƚĞĚ;ǁ ŝƚŚƌĞĨĞƌĞŶĐĞ
to reading).
Speaking skills are also assessed
twice – Į ƌƐƚ ŝŶĐŽŶŶĞĐƟŽŶǁ ŝƚŚůŝƐͲ
tening skills to increase the authen-
ƟĐŝƚLJŽĨƚŚĞƚĞƐƚ ĂƐŝŶƌĞĂůůŝĨĞ
ƐƉĞĂŬŝŶŐŝƐŵŽƐƚŽŌĞŶĐŽŶŶĞĐƚĞĚ
with listening, and then for the sec-
ŽŶĚƟŵĞŝŶĐŽŶŶĞĐƟŽŶǁ ŝƚŚĂ
WŽǁ ĞƌWŽŝŶƚƉƌĞƐĞŶƚĂƟŽŶĂŶĚĚŝƐͲ
cussion of a selected subject (as the
ĂďŝůŝƚLJƚŽƐƉĞĂŬŝŶƉƵďůŝĐŝƐŽŌĞŶ
expected of the students by pro-
ƐƉĞĐƟǀ ĞĞŵƉůŽLJĞƌƐͿ
dŚĞŶĞǁ ĨŽƌŵĂƚŽĨƚŚĞĞdžĂŵŝŶĂƟŽŶ
makes use of modern teaching tech-
niques – film, IT and also (thanks to
ƚŚĞƉŽƌƞŽůŝŽĂŶĚƉƌĞƐĞŶƚĂƟŽŶƐͿƌĞͲ
duces to some extent the risk of
ƵŶƌĞůŝĂďůĞƚĞƐƟŶŐĂƐ ŝĨŽŶůLJŽŶĞƉĞƌͲ
formance is assessed, the quality of
this may be affected by, for in-
stance, stress. The new format of
the exam has been evaluated both
by students and academic teachers
ŝŶƐƉĞĐŝĂůůLJĐŽŶƐƚƌƵĐƚĞĚƋƵĞƐƟŽŶͲ
naires.
dŚĞƌĞƐƵůƚƐŽĨƚŚĞƐĞƋƵĞƐƟŽŶŶĂŝƌĞƐ
will be used to modify both the con-
cept of the exam and the way it is
graded. The conclusions may serve
to change the concept of teaching
ƚŚƌŽƵŐŚƚŚĞŵĞƚŚŽĚƐŽĨǀ ĞƌŝĮ ĐĂƟŽŶ
of teaching concepts.
3. The study
dŚĞĂŝŵŽĨƚŚĞƐƚƵĚLJŝƐƚŽŝŶǀ ĞƐƟͲ
gate the washback of a new exam
on students and teachers, and find
out their opinion of the test format.
ϯ ϭ dŚĞƋƵĞƐƟŽŶŶĂŝƌĞ
dŚĞƐƵƌǀ ĞLJƋƵĞƐƟŽŶŶĂŝƌĞǁ ĂƐĚĞǀ ĞůͲ
oped specifically for the study and
administered in Polish. Nineteen of
ƚŚĞƋƵĞƐƟŽŶƐǁ ĞƌĞĐůŽƐĞĚƋƵĞƐƟŽŶƐ
and one open-ended, the aim of
which was to allow the respondents
to voice their opinions of the new
exam in a more precise way. The
surveys of teachers and students
ǁ ĞƌĞĂƐŝĚĞŶƟĐĂůĂƐƉŽƐƐŝďůĞŝŶ
terms of their content.
3.2. Profile of the respondents
A total of 70 valid responses from
the students of philology at a pri-
vate university and 10 from their
teachers were received from those
respondents who volunteered to
take part in the study. All the stu-
dents are 1st year BA students; 70%
of them study English as their ma-
jor. More than half of the teachers
teach English and almost all are in-
ƚĞƌĞƐƚĞĚŝŶƚĞƐƟŶŐŝƐƐƵĞƐ dŚĞŝƌ
teaching experience ranges from 2
ƚŽϭϬLJĞĂƌƐ , Žǁ Ğǀ ĞƌŶŽĐŽƌƌĞůĂƟŽŶ
between their opinions and experi-
ence has been found.
4. Discussion of results
A comparison of teachers’ and stu-
dents’ most significant answers to
ƚŚĞƋƵĞƐƟŽŶƐŝŶƚŚĞƐƵƌǀ ĞLJŝƐƉƌĞͲ
sented in Table 1.
Table 1 Comparison of opinions
“The new concept ofthe exam stresses the
role of productive skills inthe teaching and
learning process and forthis reason both speak-ing skills and writing skills
are tested twice.”
8
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 8
The majority of both teachers and
students realize that the manner of
ƚĞƐƟŶŐŝŶŇƵĞŶĐĞƐƚŚĞƉƌŽĐĞƐƐŽĨ
teaching and learning, which is re-
ŇĞĐƚĞĚŝŶƚŚĞĂĐƟǀ ŝƟĞƐŵŽƐƚĨƌĞͲ
quently used during the classes.
dŚĞƚĂďůĞĚŽĞƐŶŽƚŝŶĐůƵĚĞĂĐƟǀ ŝƟĞƐ
perceived as most frequent by less
than 10% of the respondents
(listening to CDs, projects, transla-
ƟŽŶƐͿ dŚĞƚĂďůĞƐŚŽǁ ƐƚŚĂƚƚĞĂĐŚͲ
ĞƌƐĨŽĐƵƐŽŶĂĐƟǀ ŝƟĞƐƚŚĂƚĂƌĞƐŝŵŝͲ
ůĂƌŽƌŝĚĞŶƟĐĂůƚŽƚŚĞĞdžĂŵŝŶĂͲ
ƟŽŶƚĂƐŬƐ WŽǁ ĞƌWŽŝŶƚƉƌĞƐĞŶͲ
ƚĂƟŽŶƐƐĞĞŵƚŽďĞƚŚĞĂĐƟǀ ŝƚLJ
perceived as even more fre-
quently used by students than
by the teachers. It is noteworthy
ƚŚĂƚŶĞǁ ŵĞƚŚŽĚƐŽĨƚĞƐƟŶŐ
ƌĞĐĞŝǀ ĞĚŚŝŐŚƌĂƟŶŐƐĨƌŽŵďŽƚŚ
teachers and students.
ZĞƐƉŽŶƐĞƐƚŽƚŚĞƋƵĞƐƟŽŶĐŽŶͲ
cerning the use of L1 may sug-
gest that the format of the ex-
am is too complicated and leads
to overuse of L1 on the part of
the teachers, who explain the
format of the exam and its rules
in L1 and this, in turn, provokes
students to use L1 as well. It
must be noted, however, that
talking about the exam is not
ƚŚĞŽŶůLJĐůĂƐƐƌŽŽŵƐŝƚƵĂƟŽŶŝŶ
which L1 is used, so it is rather a
ŵĂƩĞƌŽĨƚĞĂĐŚĞƌƐ ŚĂďŝƚƐĂŶĚ
these may gradually change.
The teachers are of the opinion
that learners made the most
ƐŝŐŶŝĮ ĐĂŶƚƉƌŽŐƌĞƐƐŝŶƌĞĐĞƉƟǀ Ğ
skills: listening and reading, as
well as vocabulary, which is di-
ƌĞĐƚůLJĐŽŶŶĞĐƚĞĚǁ ŝƚŚƚŚĞŝƌĞƐƟͲ
mate of their achievements in
these skills. Progress in speaking
ĂŶĚǁ ƌŝƟŶŐŝƐůĞƐƐƐƉĞĐƚĂĐƵůĂƌ
even though the aim of the changes
ǁ ĂƐƚŚĞĚĞǀ ĞůŽƉŵĞŶƚŽĨƉƌŽĚƵĐƟǀ Ğ
skills. The least progress was
achieved in grammar competence,
which most probably results from
the fact that grammar is not sepa-
9
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 9
rately tested on the exam. The stu-
dents share the teachers’ opinion
that they made the most significant
ƉƌŽŐƌĞƐƐŝŶƌĞĐĞƉƟǀ ĞƐŬŝůůƐ ůŝƐƚĞŶŝŶŐ
and reading, as well as in vocabu-
ůĂƌLJĂĐƋƵŝƐŝƟŽŶ ŝŵŝůĂƌůLJƚŽƚŚĞ
teachers, they feel they made less
ƉƌŽŐƌĞƐƐŝŶƐƉĞĂŬŝŶŐǁ ƌŝƟŶŐĂŶĚŝŶ
grammar competence.
/ƚŝƐŝŶƚĞƌĞƐƟŶŐƚŚĂƚƚĞĂĐŚĞƌƐĐŽŶͲ
sider students’ progress to be
higher than the students them-
ƐĞůǀ ĞƐĚŽ ǁ ŚŽƐĞĞƐƟŵĂƚĞŝƐ
closer to the actual results of
the exam. It seems likely that
teachers observe students’
achievements from the per-
ƐƉĞĐƟǀ ĞŽĨƚŚĞǁ ŚŽůĞĂĐĂĚĞŵŝĐ
year and students see the quali-
ƚLJŽĨƉĂƌƟĐƵůĂƌƉĞƌĨŽƌŵĂŶĐĞƐŽƌƚŚĞ
results of individual tests. The actu-
ĂůŐƌĂĚĞƐĂƌĞůŽǁ ĞƌĞĚďLJƵŶƐĂƟƐĨĂĐͲ
tory marks.
Some discrepancies exist in the
teachers’ and students’ views on
the pros and cons of the exam.
dĞĂĐŚĞƌƐƉŽŝŶƚŽƵƚƚŚĂƚƉŽƌƞŽůŝŽƐ
ĂƌĞƟŵĞ-consuming and in the stu-
dents’ opinion, make the exam eas-
ier to pass. Teachers complain
about the fact that grammar is not
tested and that accuracy is not
sufficiently stressed in the criteria,
whereas students approve of the
fact that the exam focuses on com-
ŵƵŶŝĐĂƟŽŶ
Consequently, teachers and stu-
dents share the view that exam
ƚĂƐŬƐƌĞƐĞŵďůĞƐŝƚƵĂƟŽŶƐĞŶĐŽƵŶͲ
tered in real life and may be regard-
ĞĚĂƐĂƵƚŚĞŶƟĐdŚĞLJĂƌĞĂůƐŽŽĨ
the opinion that the exam is valid
and more just.
Conclusions
As it turns out, the concept of
washback of important exams on
the teaching and learning process is
no less true in case of the PE exam
that has been discussed here. The
ŵŽƐƚŝŵƉŽƌƚĂŶƚƉŽƐŝƟǀ Ğǁ ĂƐŚďĂĐŬ
is that teachers focus more on pro-
ĚƵĐƟǀ ĞƐŬŝůůƐ ĂƐŝŶĐůĂƐƐƚŚĞLJŵŽƐƚ
ĨƌĞƋƵĞŶƚůLJƵƐĞĂĐƟǀ ŝƟĞƐƌĞƐĞŵďůŝŶŐ
ƚŚĞĞdžĂŵŝŶĂƟŽŶƚĂƐŬƐ , Žǁ Ğǀ Ğƌ
progress in those skills is not auto-
ŵĂƟĐĂůůLJŚŝŐŚĞƌƚŚĂŶŝŶƌĞĐĞƉƟǀ Ğ
skills, an aspect which needs to be
improved. Insufficient progress may
result from the overuse of L1 by
teachers, something which needs
ƚŽďĞĐŚĂŶŐĞĚ EĞŐůĞĐƟŶŐĂĐĐƵƌĂĐLJ
seems to be another important
problem, which is why including
grammar tasks in a new exam is
postulated by the teachers. Most
probably, the students will not ap-
ƉƌŽǀ ĞŽĨƚŚĂƚƐŽůƵƟŽŶ ĂƐƚŚĞLJǀ ĂůƵĞ
ŵŽƐƚƚĂƐŬƐƐƵĐŚĂƐƚŚĞƉŽƌƞŽůŝŽĂŶĚ
ƉƌĞƐĞŶƚĂƟŽŶ ǁ ŚŝĐŚƐĞĞŵƚŽƚŚĞŵ
ĨĂƌƐĂĨĞƌƚŚĂŶƚƌĂĚŝƟŽŶĂůƚĞƐƚƐ K Ŷ
the whole, the new exam is ap-
proved of both by the students and
the teachers as more valid, reliable
and contemporary. The teachers
ĂĚĚŝƟŽŶĂůůLJǀ ĂůƵĞƚŚĞƉŽƐƐŝďŝůŝƚLJŽĨ
Ğǀ ĂůƵĂƟŽŶŽĨƚŚĞĞdžĂŵ &ŽƌƚŚŝƐ
ƌĞĂƐŽŶĂŶĞǁ ƐƚƵĚLJŝƐƉůĂŶŶĞĚĂŌĞƌ
the changes in the exam are intro-
duced.
References
Alderson, J.C./ Wall, D (1993), Does
washback exist? (w:) “Applied Linguis-
ƟĐƐ 14, 115-129.
ŚĞŶŐ> t ĂƚĂŶĂďĞz ƵƌƟƐ
(2004). t ĂƐŚďĂĐŬŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐ
Research context and methods.
Mahwa.
Hughes, A (1994). dĞƐƟŶŐĨŽƌůĂŶŐƵĂŐĞ
teachers. Cambridge.
Kraśniewski, A (2011). Jak przygotować
programy kształcenia zgodnie z
wymaganiami Krajowych Ram
Kwalifikacji dla Szkolnictwa Wyższego?
Warszawa.
Zawadowska-<ŝƩ Ğů ;ϮϬϭϯ Ϳ Nowa
matura z języków obcych: szanse i
zagrożenia. Efekt zwrotny egzaminu
z języka angielskiego. Piaseczno.
“It is interesting thatteachers consider students’progress to be higher than
the students themselves do,whose estimate is closer to
the actual results of theexam.”
10
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 10
IATEFL Testing, Evaluation and AssessmentSpecial Interest Group (TEASIG)
Pre-Conference Event9 April 2018
IATEFL Annual Conference, Brighton
Assessing Listening Why do we want to test listening? What listening skills do we want to test and how? , Žǁ ĐĂŶǁ ĞďĞƩĞƌŝĚĞŶƟĨLJĂŶĚĂƐƐĞƐƐƚŚĞƐĞƐŬŝůůƐ Where can we source appropriate material? Should it be scripted or taken from the real world? t ŚĂƚŝƐƚŚĞŝŵƉŽƌƚĂŶĐĞŽĨƐƚĂƟƐƟĐƐŝŶƚŚĞĂƐƐĞƐƐŵĞŶƚŽĨůŝƐƚĞŶŝŶŐƐŬŝůůƐ
In this 2-ƉĂƌƚWǁ Ğǁ ŝůůďĞĂƩ ĞŵƉƟŶŐƚŽƚĂŬĞƚŚĞůŝĚŽī ƚŚŝƐŵLJƐƚĞƌŝŽƵƐǁ ŽƌůĚǁ ŝƚŚƚŚĞŚĞůƉŽĨĞdžƉĞƌƚƐĂŶĚƉƌĂĐƟƟŽŶĞƌƐŝŶƚŚŝƐĚŽŵĂŝŶ
dŚĞWŝƐĂŝŵĞĚĂƚĂůůƚĞĂĐŚĞƌƐ ǁ ŚĞƚŚĞƌƚŚĞLJĂƌĞŶĞǁ ƚŽƚŚŝƐĂƐƉĞĐƚŽĨƚĞƐƟŶŐĞǀ ĂůƵĂƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚŽƌǁ ŝƐŚƚŽƌĞĨƌĞƐŚƚŚĞŝƌŬŶŽǁ ůĞĚŐĞĂŶĚƉƵƚŝƚƚŽƉƌĂĐƟĐĂůƵƐĞdŚĞƌĞǁ ŝůůĂůƐŽďĞƌŽŽŵĨŽƌƋƵĞƐƟŽŶƐƚŽĞdžƉĞƌƚƐĂŶĚĚŝƐĐƵƐƐŝŽŶŽĨďĞƐƚƉƌĂĐƟĐĞ
Morning – Plenary Speakers
John FieldThe University of Bedfordshire, UK
ŽŐŶŝƟǀ ĞƉƌŽĐĞƐƐĞƐŝŶůŝƐƚĞŶŝŶŐ
Sheila ThornThe Listening Business
ŶĞǁ ŵĂƚƌŝdžĨŽƌƚŚĞƚĞƐƟŶŐŽĨůŝƐƚĞŶŝŶŐ
Rita GreenTest Development Training & Analysis Ltd ƚĂƟƐƟĐĂůĞǀ ŝĚĞŶĐĞƚŽƐƵƉƉŽƌƚĂƐƐĞƐƐŵĞŶƚŽĨůŝƐƚĞŶŝŶŐ
ŌĞƌŶŽŽŶ– Hands-on workshopsůĞĚďLJE />ĂŶĚ>dϭϮϯǁ ŝƚŚŽƉƉŽƌƚƵŶŝƟĞƐƚŽŵĂƚĐŚƚŚĞƚŚĞŽƌLJĂŶĚŝĚĞĂƐƚŽLJŽƵƌǁ ŽƌůĚ
For more information and registration
https://tea.iatefl.org/upcoming-teasig-events/
https://secure.iatefl.org/registration/conf_reg_login.php
This event is being generously sponsored by Cambridge English Language Assessment.
11
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 11
Education or evaluation?
Imagine that a graduate English ex-
am is obligatory for all your stu-
dents and it is taking place tomor-
row. Will they pass? Imagine your
brightest students, will they have
any problems? Think of the student
that worries you most. Do you think
they will be lucky enough to get
through the test with mostly correct
answers? Is it the speaking and
ǁ ƌŝƟŶŐƉĂƌƚƐƚŚĂƚĐŽŶĐĞƌŶLJŽƵ Ž
you agree that what we teach, what
students need, and what is being
tested are the same thing? When a
student gets top marks in an exam,
does it mean they are well prepared
for life and their career?
t Ğůůǁ ĞƐƟůůŚĂǀ ĞŵĂŶLJƋƵĞƐƟŽŶƐ
ĂďŽƵƚĞǀ ĂůƵĂƟŽŶ D ŽƐƚƚĞĂĐŚĞƌƐ
usually agree that students should
be taught relevant skills but others
may claim that they themselves lack
the tools for developing such skills.
Normally, an EL teacher has to up-
grade students’ grammar and vo-
cabulary on different topics as well
as develop students’ skills. Further-
ŵŽƌĞƐƵďƐĞƋƵĞŶƚĞdžĂŵŝŶĂƟŽŶƐǁ ŝůů
add some more uncertainty, so the
task formats used in exams should
be explained and thoroughly in-
ƐƟůůĞĚ D ĂLJďĞĂůůƚŚŝƐĐƌŝƟĐĂůƚŚŝŶŬͲ
ŝŶŐ/dĂŶĚĐŽŵŵƵŶŝĐĂƟǀ ĞĐŽŵƉĞͲ
ƚĞŶĐĞƐŚŽƵůĚďĞůĞŌƚŽƐŽŵĞďŽĚLJ
ĞůƐĞďĞĐĂƵƐĞǁ ĞĚŽŶ ƚŚĂǀ ĞƟŵĞĨŽƌ
it?
Can you put an equals sign between
what you teach, what your students
need, and what the exam will test?
If you consider these competences
important, you no doubt spend con-
ƐŝĚĞƌĂďůĞƟŵĞĂŶĚĞī ŽƌƚŽƌŐĂŶŝƐŝŶŐ
ƉƌĂĐƟĐĞŽĨƚŚĞŵ zŽƵŵŝŐŚƚďĞ
struggling with the syllabus, trying
ƚŽĮ ŶĚĞŶŽƵŐŚƟŵĞĂŶĚƐĞĂƌĐŚŝŶŐ
for sufficient materials. You do it
simply because there is a real world
outside the classroom and correct
grammar alone won’t suffice. We,
as teachers, just need to suggest
something more applicable to office
or freelance work in business.
School is not just a place where
knowledge and skills are learned.
Every subject should contribute to
cross-subject competence building –
‘should’, but doesn’t ‘have to’. How-
Ğǀ ĞƌƚŚĞƌĞĂƌĞƐƟůůƚŚŝŶŐƐƚŚĂƚĂƌĞĂ
ŵƵƐƚ ƚŚĞƐLJůůĂďƵƐ ĞĚƵĐĂƟŽŶĂůƉƌŽͲ
gramme, year plan, and even the
teacher’s book guide.
So, will an EL teacher risk the lim-
ited hours available on building up
language skills to develop some-
thing that neither a test nor an ex-
am can really assess? What if they
ĚŽĮ ŶĚƚŚĞĞdžƚƌĂƟŵĞ
Ilya Denisenko
Ilya Denisenko is Head of theLanguages Department at theState Academic University for, ƵŵĂŶŝƟĞƐ;^h , D ŽƐĐŽǁ Russia) and a founder of Au-ƚŚĞŶƟĐ>dƐƐŽĐŝĂƟŽŶ dŚŝƐyear he has completed a studyon assessment and given atalk at the IATEFL Conference.
This paper is based on aƉƌĞƐĞŶƚĂƟŽŶŐŝǀ ĞŶĂƚƚŚĞTEASIG Day at the IATEFL Con-ference in Glasgow in April2017.
12
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 12
English as a subject makes it possi-
ble to speak about anything from
ancient history to space flight; it
ĚŽĞƐŶ ƚŚĂǀ ĞĐŽŶƚĞŶƚƌĞƐƚƌŝĐƟŽŶƐ
(that depends on the cultural tradi-
ƟŽŶƐŽĨLJŽƵƌĐŽƵŶƚƌLJŽĨĐŽƵƌƐĞͿ
This means that an EL lesson can be
built on various contexts, related to
interests, experience and the pro-
ĨĞƐƐŝŽŶĂůŽƌĨƵƚƵƌĞĞĚƵĐĂƟŽŶĂů
needs of the students. This objec-
Ɵǀ ĞŵĂŬĞƐĂŶŶŐůŝƐŚůĞƐƐŽŶ
unique in that it prepares stu-
dents for real-life problems
ŵƵĐŚďĞƩĞƌƚŚĂŶĂŶLJƚŚŝŶŐĞůƐĞ
in the school curriculum.
An English lesson, moreover, is
ĂŶŝĚĞĂůƟŵĞĂŶĚƉůĂĐĞƚŽďƵŝůĚ
up the crucial skills needed to-
day. A teacher doesn’t have to
ƌĞĚƵĐĞŽƌĐĂŶĐĞůƚƌĂĚŝƟŽŶĂů
ŐƌĂŵŵĂƌŽƌǀ ŽĐĂďƵůĂƌLJƉƌĂĐƟĐĞ
(if they can’t do without it) to
ƐƉĂƌĞƐŽŵĞƟŵĞĨŽƌĂŶŽƚŚĞƌƚĂƐŬ
dŚĞƌĞŝƐĂŶŽƚŚĞƌƐŽůƵƟŽŶ
There is no need to go too deeply
ŝŶƚŽƚŚĞĐĂůĐƵůĂƟŽŶƐ ƚŚĞůĂƌŐĞƐƚ
ĂŵŽƵŶƚŽĨƟŵĞƐƉĞŶƚŝŶĞǀ ĞƌLJĐůĂƐƐͲ
room around the world is used for
Ğǀ ĂůƵĂƟŽŶ dŚĞƟŵĞ>ƚĞĂĐŚĞƌƐ
ƐƉĞŶĚŽŶƚĞƐƟŶŐĐĂŶďĞƵƐĞĚŵŽƌĞ
efficiently if it’s used for teaching
purposes as well.
Assessment is a valuable part of the
ĞĚƵĐĂƟŽŶĂůƉƌŽĐĞƐƐƚŚĂƚŝƐŽŌĞŶŶŽƚ
used enough. Many scholars have
agreed that assessment has a dou-
ble-duty and works as a scaling and
a teaching tool. However, many
ƉƌĂĐƟĐŝŶŐƚĞĂĐŚĞƌƐĮ ŶĚŝƚƚĂŬĞƐƚŽŽ
ŵƵĐŚĞī ŽƌƚƚŽƐƉĞŶĚĂůŽƚŽĨƟŵĞ
Ğǀ ĂůƵĂƟŶŐƚŚĞŝƌƐƚƵĚĞŶƚƐ ĂƚůĞĂƐƚ
ĨŽƌŵĂƟǀ ĞůLJ/ŶĂĚĚŝƟŽŶ ŵŽƐƚƚĞĂĐŚͲ
ĞƌƐĚŽŶ ƚƚƌLJƚŽĚĞǀ ĞůŽƉƚĞƐƟŶŐŵĂͲ
terials themselves. A well-organised
assessment system could solve such
problems.
Imagine such a system exists. What
ĂƌĞŝƚƐĐŚĂƌĂĐƚĞƌŝƐƟĐƐ &ŝƌƐƚŽĨĂůůŝƚ
should be built on students’ perfor-
mance. This means that students
need to be evaluated on how they
ĚĞĂůǁ ŝƚŚĂƌĞĂůƉƌĂĐƟĐĂůƚĂƐŬ/ŶƚŚŝƐ
ĐĂƐĞŽďƐĞƌǀ ĂƟŽŶŽĨƚŚĞƐƚƵĚĞŶƚƐ
ǁ ŽƌŬƐŚŽǁ ƐƌĞĂůĂĐƋƵŝƐŝƟŽŶŽĨĐŽŵͲ
petences.
Another vital thing is to make per-
formance meaningful for students,
which can be done through the con-
text of a task. For this reason, tasks
should be correlated with the stu-
ĚĞŶƚƐ ƉŽƚĞŶƟĂůŬŶŽǁ ůĞĚŐĞĂŶĚƉƌŽͲ
vide them with relevant feedback.
D ĞĂŶŝŶŐĨƵůŵĞĂŶƐŝŶƚĞƌĞƐƟŶŐĐŽŶͲ
nected with their experience, as
ǁ ĞůůĂƐƚŚŽƵŐŚƚĂŶĚĐƌĞĂƟǀ ŝƚLJƉƌŽͲ
voking.
Such meaningful contexts bolster
ŵŽƟǀ ĂƟŽŶ t ŚĞŶĂƐƚƵĚĞŶƚĐĂŶ
apply their own personal experi-
ĞŶĐĞŝƚƉƌŽǀ ŝĚĞƐƐĐŽƉĞĨŽƌĐƌĞĂƟǀ ŝƚLJ
and readiness to work in the pro-
ƉŽƐĞĚƐĞƫ ŶŐ
I’d like to illustrate this with one
example. A school boy was consid-
ered to be a poorly performing pupil
by all his teachers. Even those who
didn’t actually teach him were
Ăǁ ĂƌĞŽĨŚŝƐůŽǁ ƉŽƚĞŶƟĂů, ĞŽďǀ ŝͲ
ously behaved in a way that was
consistent with how he was seen,
and his teachers usually didn’t ex-
pect much of him. Yet, on an English
ƉƌŽũĞĐƚĂďŽƵƚĐĞƌƚĂŝŶƚŽƵƌŝƐƚĂƩ ƌĂĐͲ
ƟŽŶƐŚĞǁ ĂŶƚĞĚƚŽƐĞĞŝŶŚŝƐĂƌĞĂ
he really did his best. It turned out
that he was into extreme sports, for
which there are many English words
describing tricks, moves and equip-
ment. So, English lessons can reach
Ğǀ ĞƌLJŽŶĞD ĂƚŚƐŽƌŶĂƟǀ ĞůĂŶŐƵĂŐĞ
ƚĞĂĐŚĞƌƐƐƉĞŶĚĂůŽƚŽĨƟŵĞǁ ŝƚŚ
students but they don’t normally
discuss personal things with them
and, if they do, they are unable to
apply this in the lessons. English les-
sons, however, are more suited to
this.
“Another vital thing is tomake performance
meaningful for students,which can be done
through the context of atask. For this reason, tasksshould be correlated with
the students’ potentialknowledge and provide
them with relevantfeedback.”
13
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 13
A task can be ŵƵůƟŽƌĐƌŽƐƐ-subject
because, in real life, we rarely come
across isolated tasks – think of con-
ǀ ĞƌƐĂƟŽŶƐ ƐĞŶĚŝŶŐĞ-mails, working
on projects, travelling and so on.
We base our work on all our skills
and experience, so teachers should
ƚƌLJƚŽƐĞƚƚŚĞƐĂŵĞĐŽŶĚŝƟŽŶƐĨŽƌ
their students. Knowing what topics
are being discussed in history or
ŵĂƚŚƐĂƚƚŚĞƐĂŵĞƟŵĞĐĂŶŚĞůƉŝŶ
developing such tasks.
ůƚŚŽƵŐŚĂƐƐĞƐƐŵĞŶƚŝƐŽŌĞŶƐƚƌĞƐƐͲ
ĨƵůĨŽƌƐƚƵĚĞŶƚƐ ƐƚƌĞƐƐĚŽĞƐŶ ƚŽŌĞŶ
ŚĂǀ ĞŶĞŐĂƟǀ ĞƌĞƐƵůƚƐ ƐƚƌĞƐƐĨƵů
ƐŝƚƵĂƟŽŶĂůƐŽŵĂŬĞƐŝƚŵŽƌĞƌĞĂů
that is, closer to true foreign lan-
ŐƵĂŐĞĐŽŵŵƵŶŝĐĂƟŽŶ
By combining the above ideas, we
can turn assessment into ĂƵƚŚĞŶƟĐ
ĂƐƐĞƐƐŵĞŶƚ ƚŚĂƚŝƐ Ğǀ ĂůƵĂƟŽŶŝŶĂ
ƐŝƚƵĂƟŽŶǁ ŚĞƌĞĨŽƌĞŝŐŶůĂŶŐƵĂŐĞ
competence is applied when it is
ĞƐƐĞŶƟĂůĂŶĚŶĂƚƵƌĂů
There are several ways to make as-
ƐĞƐƐŵĞŶƚƉƌĂĐƟĐĂůKŶĞƚŚĂƚĐĂŶďĞ
controlled at all stages is a project.
A project calls for students to per-
form a variety of tasks and check
different goals. There can be several
stages, each with different aims and
ŽďũĞĐƟǀ ĞƐ
The first stage works as an introduc-
ƟŽŶƚŽƚŚĞƚŽƉŝĐt ƌŝƩ ĞŶĂŶĚĂƵĚŝŽ
tasks are given. If there is a need,
reading and listening skills can also
be tested here. The materials set
the context for all subsequent work.
At the second stage, students re-
ceive the project task, which usually
involves some research and produc-
Ɵǀ ĞƉĂƌƚƐ ƐƚƵĚLJƉůĂŶŝƐĞŝƚŚĞƌŐŝǀ Ͳ
en to the students or they develop
one themselves, with some points
ĂƐƋƵĞƐƟŽŶƐŽƌƐƚĂƚĞŵĞŶƚƐ dŚĞŶ
ƚŚĞLJĐƌĞĂƚĞĂƉƌĞƐĞŶƚĂƟŽŶ ƚƚŚĞ
ŶĞdžƚƐƚĂŐĞƚŚĞLJŐŝǀ ĞĂƉƌĞƐĞŶƚĂƟŽŶ
ĞƉĞŶĚŝŶŐŽŶƚŚĞŽďũĞĐƟǀ ĞƐ ƚŚĞ
teacher can add different tasks to
the project: group research, collec-
Ɵǀ ĞŽƌĂůƉƌĞƐĞŶƚĂƟŽŶ ǁ ƌŝƩ ĞŶƚĂƐŬ
ƉŽƐƚĞƌŽƌůĞƩ ĞƌdŚĞŵĂŝŶĂŝŵŽĨ
this stage is to see the language
competence the students have truly
acquired, and analyse their perfor-
mance. The teacher plays the role of
ŝŶƚĞƌůŽĐƵƚŽƌĂŶĚƐƟŵƵůĂƚĞƐĚŝƐĐƵƐͲ
ƐŝŽŶĂƌŽƵŶĚƚŚĞƉƌĞƐĞŶƚĂƟŽŶ ĂƐŬͲ
ŝŶŐĂĚĚŝƟŽŶĂůƋƵĞƐƟŽŶƐĂŶĚďƌŝŶŐͲ
ing up points on some details that
can be upgraded and improved. This
provides considerable speaking
ƉƌĂĐƟĐĞĂŶĚƉƌŽĚƵĐĞƐƌĞůĞǀ ĂŶƚŝŶͲ
ĨŽƌŵĂƟŽŶĂďŽƵƚƐŬŝůůƐůĞǀ ĞůƐ
The feedback stage is crucial for au-
ƚŚĞŶƟĐĂƐͲ
sessment.
The teach-
er can
ŝĚĞŶƟĨLJ
common
mistakes
more easily
when viewing students working to-
gether.
To sum up, I’d like to emphasise
that, once the teacher decides to
develop and assess competence
ĂĐƋƵŝƐŝƟŽŶŚŽůŝƐƟĐĂůůLJĂƵƚŚĞŶƟĐ
assessment features can be very
ŚĞůƉĨƵůƵƚŚĞŶƟĐĂƐƐĞƐƐŵĞŶƚĐĂŶ
be beneficial for both teachers and
students. The former obtain an op-
portunity to see to what extent
their students have really acquired
various competences, and to draw
conclusions about the efficacy of
ƚŚĞŝƌƚĞĂĐŚŝŶŐdŚĞůĂƩĞƌĐĂŶƚƌLJŽƵƚ
ƚŚĞŝƌƐŬŝůůƐƵŶĚĞƌŶĂƚƵƌĂůĐŽŶĚŝƟŽŶƐ
and understand their weaknesses
ĂŶĚĂďŝůŝƟĞƐďĞĨŽƌĞůĞĂǀ ŝŶŐƚŚĞƐĂĨĞͲ
ty of the classroom.
ǀ ĂůƵĂƟŽŶŝƐĂƚƌĞĂƐƵƌĞĐŚĞƐƚǁ ŚĞŶ
it comes to teaching languages, and
teachers should be familiar with it
ĂŶĚƵƐĞŝƚ D ƵůƟƉůĞĐŚŽŝĐĞƚĞƐƚƐ
ƐŚŽƵůĚďĞůĞŌĨŽƌŐĞŶĞƌĂůƚĞƐƟŶŐ
and other less fortunate subjects.
The EL teacher should try to make
ƚŚĞŵŽƐƚŽĨĞǀ ĂůƵĂƟŽŶ ĞŶƐƵƌŝŶŐ
Ğǀ ĞƌLJĂƐƐĞƐƐŵĞŶƚƐĞƐƐŝŽŶŝƐĂƉƌĂĐƟͲ
cal one.
14
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 14
What you can do for TEASIGIATEFL TEASIG members all have one thing in common: their interest in issues connected withƚĞƐƟŶŐĞǀ ĂůƵĂƟŽŶ ĂŶĚĂƐƐĞƐƐŵĞŶƚ ŽŵĞŵĞŵďĞƌƐĂƌĞĞdžƉĞƌƚƐŝŶƚŚĞĮ ĞůĚĂŶĚĐŽŶƚƌŝďƵƚĞƚŽƚŚĞ^/' ďLJƐŚĂƌŝŶŐƚŚĞŝƌĞdžƉĞƌƟƐĞǁ ŝƚŚŽƚŚĞƌŵĞŵďĞƌƐŝŶǀ ĂƌŝŽƵƐǁ ĂLJƐ ŚŽůĚŝŶŐǁ ĞďŝŶĂƌƐ ƐƉĞĂŬŝŶŐĂƚĐŽŶĨĞƌĞŶĐĞƐĂŶĚĞǀ ĞŶƚƐ ĂŶĚǁ ƌŝƟŶŐĨŽƌƚŚĞEĞǁ ƐůĞƩ ĞƌƐ/d&>ŝƐĂƚĞĂĐŚĞƌƐ ĂƐƐŽĐŝĂƟŽŶ ŵĂŶLJŵĞŵďĞƌƐĂƌĞƚĞĂĐŚĞƌƐ ƐŽŵĞũƵƐƚƐƚĂƌƟŶŐŽŶƚŚĞŝƌƚĞĂĐŚŝŶŐĐĂƌĞĞƌĂŶĚŽƚŚĞƌƐĂůŽŶŐǁ ĂLJŝŶƚŽŝƚ ŚŝŐŚŶƵŵďĞƌŽĨƚĞĂĐŚĞƌƐŚĂǀ ĞĂŐƌĞĂƚĚĞĂůŽĨĞdžƉĞƌŝĞŶĐĞŽĨƚĞƐƟŶŐǁ ŚŝůĞŶŽƚĐŽŶƐŝĚĞƌŝŶŐƚŚĞŵͲselves experts, whereas others are keen to get into what they feel is an important and dynamic areaof teaching EFL. Many teachers hope to learn something from experts and from other teachers, andƚŽŚĂǀ ĞƚŚĞŽƉƉŽƌƚƵŶŝƚLJƚŽĂƐŬƋƵĞƐƟŽŶƐĂŶĚƐŚĂƌĞƚŚĞŝƌŽǁ ŶŝĚĞĂƐ t ŚŝĐŚĞǀ ĞƌŐƌŽƵƉLJŽƵďĞůŽŶŐƚŽ ƚŚĞƌĞĂƌĞŵĂŶLJǁ ĂLJƐŝŶǁ ŚŝĐŚLJŽƵĐĂŶĐŽŶƚƌŝďƵƚĞƚŽd^/' ĂŶĚŽƵƌĞī ŽƌƚƐƚŽƉƌŽǀ ŝĚĞĞī ĞĐƟǀ ĞƐĞƌͲvices to all our members. Here are a few ideas.
E Ğǁ ƐůĞƩĞƌContribute
ďLJĂŶƐǁ ĞƌŝŶŐŵĞŵďĞƌƐƉŽƚůŝŐŚƚƋƵĞƐƟŽŶƐĂŶĚŐŝǀ ŝŶŐĂůůŵĞŵďĞƌƐĂŶŝĚĞĂŽĨǁ ŚŽĞůƐĞŝƐŝŶƚŚĞ /'
ďLJƐĞŶĚŝŶŐƵƐĂƐŚŽƌƚĂŶƐǁ ĞƌƚŽƚŚĞD ŽƐĂŝĐƋƵĞƐƟŽŶĨŽƌƚŚĞŶĞdžƚŝƐƐƵĞ
ďLJƌĞƉŽƌƟŶŐŽŶĂŶLJ;ŚŽǁ Ğǀ ĞƌƐŵĂůůͿƌĞƐĞĂƌĐŚŝŶƚŽĂƐƐĞƐƐŵĞŶƚŽƌƚĞƐƟŶŐLJŽƵŚĂǀ ĞĚŽŶĞ
ďLJƌĞƉŽƌƟŶŐŽŶĂŶĂƐƐĞƐƐŵĞŶƚŽƌƚĞƐƟŶŐĞǀ ĞŶƚŽƌĐŽŶĨĞƌĞŶĐĞLJŽƵŚĂǀ ĞĂƩ ĞŶĚĞĚHelp the Editors by
ĂƉƉƌŽĂĐŚŝŶŐƉŽƚĞŶƟĂůĐŽŶƚƌŝďƵƚŽƌƐ
ĂƉƉƌŽĂĐŚŝŶŐƉŽƚĞŶƟĂůĂĚǀ ĞƌƟƐĞƌƐ
proof-reading
Őŝǀ ŝŶŐĨĞĞĚďĂĐŬŽŶƚŚĞEĞǁ ƐůĞƩ Ğƌ;ǁ ŚĂƚLJŽƵůŝŬĞǁ ŚĂƚLJŽƵǁ ŽƵůĚůŝŬĞƚŽƐĞĞĂŶĚǁ ŚĂƚLJŽƵĚŽŶ ƚůŝŬĞ– ďƵƚƉůĞĂƐĞƉƌŽǀ ŝĚĞĂŶĂůƚĞƌŶĂƟǀ Ğǁ ŚĞƌĞƉŽƐƐŝďůĞ Ϳ
KƚŚĞƌƉƵďůŝĐĂƟŽŶƐ>ĞƚƵƐŬŶŽǁ ǁ ŚĂƚŽƚŚĞƌƉƵďůŝĐĂƟŽŶƐǁ ĞĐŽƵůĚŽī Ğƌ
WebsiteSend
ŝŶĨŽƌŵĂƟŽŶŽŶĞdžƚĞƌŶĂůĞǀ ĞŶƚƐ ƐĞŵŝŶĂƌƐ ǁ ŽƌŬƐŚŽƉƐ ĂŶĚĐŽŶĨĞƌĞŶĐĞƐ;ƐƚƌŝĐƚůLJĐŽŵŵĞƌĐŝĂůĂĐƟǀ ŝƟĞƐ including fee-ďĞĂƌŝŶŐĐŽƵƌƐĞƐ ǁ ŝůůŶŽƚďĞĂĚǀ ĞƌƟƐĞĚĨƌĞĞŽĨĐŚĂƌŐĞͿ ĚĞĐŝƐŝŽŶŽŶǁ ŚĞƚŚĞƌƚŽŝŶĐůƵĚĞĂŶLJĞǀ ĞŶƚŝƐĂƚƚŚĞĚŝƐĐƌĞƟŽŶŽĨƚŚĞ /' Ž-ordinator(s).
feedback on the website (what you like, would like to see, and don’t like)
WebinarsD ĂŬĞƐƵŐŐĞƐƟŽŶƐĨŽƌƐƉĞĂŬĞƌƐĂŶĚĨŽƌŵĂƚƐ
Events Help to organize an event in your area. Provide technical support with live-streaming and recording of sessions. Act as a helper at a TEASIG event.
Social media Encourage colleagues to join TEASIG social media groups. Start and contribute to discussions.
15
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 15
/ŶƚƌŽĚƵĐƟŽŶ
The teaching and assessment ofspoken English has arguably beenneglected by China’s exam-focusedĞĚƵĐĂƟŽŶĐƵůƚƵƌĞ&ŽƌŚŝŶĞƐĞƐƚƵͲdents, becoming confident users of ŶŐůŝƐŚŽī ĞƌƐŵLJƌŝĂĚŽƉƉŽƌƚƵŶŝƟĞƐ ďƵƚƐŝŶĐĞƐƵĐĐĞƐƐ ƐŽŽŌĞŶŵĞƌĞůLJequates to passing exams, many ofƚŚĞƐĞLJŽƵŶŐƉĞŽƉůĞŚĂǀ ĞůŝƩ ůĞĐŽŶͲĐĞƉƟŽŶŽĨůŝĨĞůŽŶŐůĞĂƌŶŝŶŐ/Ɛƚƌŝǀ ĞĚƚŽĐŚĂŶŐĞƚŚŝƐĂƫ ƚƵĚĞǁ ŝƚŚŵLJƐƚƵͲdents by designing a speaking testwhich aimed to be both useful andmeaningful: a spontaneous groupĐŽŶǀ ĞƌƐĂƟŽŶĂĐƟǀ ŝƚLJdŚŝƐĂƌƟĐůĞbegins with a few background de-tails and then explains the test’sdesign and marking procedures.
The learners and the course
My class, at a public university inInner Mongolia, China, consisted ofĮ ŌĞĞŶD ĂŶĚĂƌŝŶ>ϭůĞĂƌŶĞƌƐǁ ŚŽĂůůstruggled with oral English to somedegree. As English majors, theircompulsory modules included gram-ŵĂƌůŝƐƚĞŶŝŶŐƌĞĂĚŝŶŐǁ ƌŝƟŶŐĂŶĚƚƌĂŶƐůĂƟŽŶ ŵŽƐƚůLJƚĂƵŐŚƚďLJŶĂƟǀ ĞŚŝŶĞƐĞƚĞĂĐŚĞƌƐŝŶƉƌĞƉĂƌĂƟŽŶĨŽƌŶĂƟŽŶĂůĞdžĂŵƐůŝŬĞdD ;dĞƐƚĨŽƌEnglish Majors) and CET (CollegeEnglish Test). Since oral fluency is not integral to these exams, speak-ŝŶŐĐůĂƐƐĞƐĂƌĞŶŽƌŵĂůůLJůĞŌƚŽĨŽƌͲĞŝŐŶƚĞĂĐŚĞƌƐ ǁ ŚŽƐĞŝŶƚĞƌĂĐƟǀ ĞƚĞĂĐŚŝŶŐŵĞƚŚŽĚƐŽŌĞŶĐŽŶƚƌĂĚŝĐƚƚŚĞƚƌĂĚŝƟŽŶĂůƚĞĂĐŚĞƌ-centred cur-riculum. Because of these factors,most of my learners were anxiousabout speaking spontaneously inĐůĂƐƐĂŶĚŽŌĞŶŵĞŵŽƌŝƐĞĚĂŶĚƌĞͲĐŝƚĞĚƚĞdžƚƐĚƵƌŝŶŐƉƌĞƐĞŶƚĂƟŽŶƐŽƌ
ŽƚŚĞƌĂĐƟǀ ŝƟĞƐ ŵĂŬŝŶŐŝƚĐŚĂůůĞŶŐͲing to observe or obtain samples of‘natural’ speech to monitor, andƵůƟŵĂƚĞůLJĂƐƐĞƐƐ ƚŚĞŝƌƉƌŽŐƌĞƐƐ
Developing the confidence, knowledge and skills required forengaging in informal discussion wasƚŚĞĐĞŶƚƌĂůůĞĂƌŶŝŶŐŽďũĞĐƟǀ ĞŽĨŽƵƌ18-week course. Our weekly lessonsfocused on various themes – ŽŌĞŶsuggested by the learners – and in-cluded related vocabulary andŐƌĂŵŵĂƟĐĂůƐƚƌƵĐƚƵƌĞƐǁ ŝƚŚƌĞŐƵůĂƌƉĂŝƌĂŶĚŐƌŽƵƉĚŝƐĐƵƐƐŝŽŶĂĐƟǀ ŝƟĞƐƚŽŵĂdžŝŵŝƐĞƐƉĞĂŬŝŶŐŽƉƉŽƌƚƵŶŝƟĞƐas well as build knowledge and skillsƚŚƌŽƵŐŚĂĐƟǀ ĞƉƌĂĐƟĐĞdŽĨŽƐƚĞƌ>ϮĐŽŶĮ ĚĞŶĐĞƚŚĞĨŽĐƵƐǁ ĂƐƐŚŝŌĞĚfrom accuracy to fluency and I en-couraged the learners to speak asmuch as possible without worryingabout mistakes. They had good re-ĐĞƉƟǀ ĞŬŶŽǁ ůĞĚŐĞŽĨŐƌĂŵŵĂƌƐŽ/ŬŶĞǁ ǁ ŝƚŚƉƌĂĐƟĐĞƚŚĞLJĐŽƵůĚŐƌĂĚͲually begin to noƟce and correcttheir errors.
Test design
ŽŶĚƵĐƟŶŐĂŶĞŶĚ-of-term achieve-ment exam was a course require-ment, but I was given the freedomto design, administer and mark theƚĞƐƚ ĐƌƵĐŝĂůĐŽŶƐŝĚĞƌĂƟŽŶĨŽƌŶŽŶ-ƚƌĂĚŝƟŽŶĂůĂƐƐĞƐƐŵĞŶƚŝƐƚŚĂƚƚĞƐƚƐshould be “integrated with the goalsof the curriculum and … have a con-ƐƚƌƵĐƟǀ ĞƌĞůĂƟŽŶƐŚŝƉǁ ŝƚŚƚĞĂĐŚŝŶŐand learning” (McNamara, 2000,Ɖ ϳ Ϳ ĐŽŶǀ ĞƌƐĂƟŽŶ ƚĞƐƚĂĐƟǀ ŝƚLJclearly reflected the course objec-Ɵǀ ĞĂŶĚƚŚĞĐŽŵŵƵŶŝĐĂƟǀ ĞŇƵĞŶĐLJ-based classroom content, thusfirmly linking the test to the learning
Maximising collaboration in oral English testing:an example from China
Susanna Wickes
began teaching in her hometown of Edinburgh in 2009.Since then she has taught inAustralia, India, Ireland and -most extensively - China. Dur-ing her years in this countryshe has developed researchinterests in assessment, learn-er autonomy, and collabora-Ɵǀ ĞůĞĂƌŶŝŶŐƉĂƌƟĐƵůĂƌůLJŝŶthe university context. Shecurrently teaches EAP at thehŶŝǀ ĞƌƐŝƚLJŽĨE Žƫ ŶŐŚĂŵNingbo China.
This paper is based on aƉƌĞƐĞŶƚĂƟŽŶŐŝǀ ĞŶĂƚƚŚĞTEASIG Day at the IATEFLConference in Glasgow inApril 2017.
16
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 16
goal. The test results then enabledthe learners to gauge their progress,diagnose their needs, and set newgoals.
The individual competences re-ƋƵŝƌĞĚĨŽƌĐŽŶǀ ĞƌƐĂƟŽŶĂŶĚŝŶĨŽƌͲmal discussion were adapted fromthe CEFR (Council of Europe, 2001)and are shown below with a de-ƐĐƌŝƉƟŽŶŽĨƚŚĞĂďŝůŝƚLJůĞǀ Ğů/ĞdžͲpected my learners to reach in eachcategory. These competences wereƚĂƵŐŚƚĂŶĚƉƌĂĐƟƐĞĚƚŚƌŽƵŐŚŽƵƚƚŚĞterm and provided the basis for themarking rubrics.
>ŝŶŐƵŝƐƟĐĐŽŵƉĞƚĞŶĐĞƐ
Phonological
Lexical
' ƌĂŵŵĂƟĐĂů
^ĞŵĂŶƟĐ
The learner can use ap-propriate and intelligiblewords, expressions andstructures, with clear, intel-ůŝŐŝďůĞƉƌŽŶƵŶĐŝĂƟŽŶ ƚŽexpress his/her ideas.
WƌĂŐŵĂƟĐĐŽŵƉĞƚĞŶĐĞƐ
Discourse competence
&ƵŶĐƟŽŶĂůĐŽŵƉĞƚĞŶĐĞ
The learner can use ap-propriate techniques to ini-ƟĂƚĞĞŶŐĂŐĞŝŶĂŶĚŵĂŝŶͲƚĂŝŶĐŽŶǀ ĞƌƐĂƟŽŶ ĐĂŶĂƐŬƋƵĞƐƟŽŶƐ ĂŐƌĞĞĚŝƐĂŐƌĞĞĞƚĐ ĐĂŶŐŝǀ ĞĚĞƐĐƌŝƉƟŽŶƐ ĞdžƉůĂŶĂƟŽŶƐ ĞƚĐǁ ŚĞƌĞappropriate, and can per-form all of these in a confi-dent manner without exces-Ɛŝǀ ĞƉĂƵƐŝŶŐŽƌŚĞƐŝƚĂƟŽŶ
Test method and content
ůƚŚŽƵŐŚĐŽŶǀ ĞƌƐĂƟŽŶ-style testscan take place between two people– a test taker and an assessor/interlocutor or two test takers – Iorganised my learners into groupsof three as talking in groups can re-duce anxiety (He & Dai, 2006). PeerŝŶƚĞƌĂĐƟŽŶĂůƐŽƉƌŽŵŽƚĞƐŵŽƌĞƌĞͲĂůŝƐƟĐĐŽŵŵƵŶŝĐĂƟŽŶƚŚĂŶĂƐƐĞƐƐŽƌ-ĞdžĂŵŝŶĞĞĐŽŶǀ ĞƌƐĂƟŽŶĂƐůĞĂƌŶĞƌƐare in an “equal power posi-ƟŽŶ ;>ƵŽŵĂϮϬϬϰ Ɖϭϴϳ ͿĂŶĚĐĂŶĚĞŵŽŶƐƚƌĂƚĞŵŽƌĞŝŶƚĞƌĂĐƟǀ ĞĨƵŶĐͲƟŽŶƐƚŚĂŶƐŝŵƉůLJĂŶƐǁ ĞƌŝŶŐĂŶŝŶͲƚĞƌůŽĐƵƚŽƌƐƋƵĞƐƟŽŶƐ dŚŝƐĞŶĂďůĞƐthe assessor to observe, rather than
ƉĂƌƟĐŝƉĂƚĞŝŶƚŚĞĂĐƟǀ ŝƚLJĂŶĚƚŚĞƌĞͲĨŽƌĞƉĂLJŵŽƌĞĂƩĞŶƟŽŶ ĚĚŝƟŽŶĂůͲůLJŐƌŽƵƉĂƐƐĞƐƐŵĞŶƚŝƐƉƌĂĐƟĐĂůĂŶĚĞĸ ĐŝĞŶƚ ƚŚĞĮ ŌĞĞŶƚĞƐƚƚĂŬĞƌƐworked in five groups, allowing all the tests to be completed duringone two-hour lesson.
Open-ĞŶĚĞĚĐŽŶǀ ĞƌƐĂƟŽŶĐŽŶƚĞŶƚwas prompted using topic cards.Before the test week, six broadthemes covered in the course wereĐŚŽƐĞŶďLJƐƚƵĚĞŶƚǀ ŽƚĞĂŶĚǁ ƌŝƩ ĞŶon the backs of six cards. During thetest, one card was randomly select-ĞĚƚŽŝŶƐƉŝƌĞĂƐŚŽƌƚĐŽŶǀ ĞƌƐĂƟŽŶ The topics – personality, technology,
ƌĞůĂƟŽŶƐŚŝƉƐ ĞŶǀ ŝƌŽŶŵĞŶƚ ƚƌĂǀ Ğůand ĞĚƵĐĂƟŽŶ – were all familiar tothe learners and were consideredequally ‘difficult’ as they could all be discussed at any level from basicand concrete to more abstract andcomplex. Knowing the six topics inadvance allowed learners to prac-ƟƐĞĂŶĚƌĞǀ ŝƐĞƌĞůĞǀ ĂŶƚǀ ŽĐĂďƵůĂƌLJand grammar, thus reducing nervesand maximising fairness, but notƌĞǀ ĞĂůŝŶŐƚŚĞƐƉĞĐŝĮ ĐƚŽƉŝĐƵŶƟůƚŚĞtest ensured that learners could not ĐŚĞĂƚ ďLJƐĐƌŝƉƟŶŐĂŶĚŵĞŵŽƌŝƐŝŶŐƚŚĞŝƌĐŽŶǀ ĞƌƐĂƟŽŶƐ
dĞƐƚĂĚŵŝŶŝƐƚƌĂƟŽŶ
ŌĞƌĞdžƉůĂŝŶŝŶŐĂŶĚƉƌĂĐƟƐŝŶŐthe assessment format, the testwas administered during classƟŵĞŝŶƚŚĞƐĞĐŽŶĚ-last week ofterm, leaving the final week free for a peer/self-Ğǀ ĂůƵĂƟŽŶƐĞƐͲƐŝŽŶ /ŝŶŝƟĂƚĞĚĂƐŚŽƌƚ ƵŶĂƐͲsessed ‘warm up’ to help the testtakers relax, then a group mem-
ber was asked to randomly selectone of the six cards, which providedƚŚĞƚŽƉŝĐĨŽƌƚŚĞĐŽŶǀ ĞƌƐĂƟŽŶ /ƐƚĂƌƚĞĚƚŚĞƌĞĐŽƌĚĞƌĂŶĚƐĞƚĂƟŵĞƌǁ ŚŝĐŚďĞĞƉĞĚĂŌĞƌƐĞǀ ĞŶŵŝŶƵƚĞƐto let the group know they hadreached the minimum limit for com-ƉůĞƟŶŐƚŚĞƚĂƐŬdŚŝƐŵĞƚŚŽĚĂůͲlowed the learners to end their dis-cussion when and how they wished,thus producing a more representa-Ɵǀ ĞƵŶĐŽŶƚƌŝǀ ĞĚĐŽŶǀ ĞƌƐĂƟŽŶƐĂŵͲƉůĞdŚĞƟŵĞƌǁ ĂƐƐĞƚƚŽďĞĞƉĂŐĂŝŶĂŌĞƌƚĞŶŵŝŶƵƚĞƐ– the upper limit –to tell the group to round things off ŝĨƚŚĞLJǁ ĞƌĞƐƟůůƚĂůŬŝŶŐE ŽƚĞƐǁ ĞƌĞnot allowed before or during theĐŽŶǀ ĞƌƐĂƟŽŶĂŶĚƚĞƐƚƚĂŬĞƌƐǁ ĞƌĞ
“Although conversation-styletests can take placebetween two people
I organised my learners intogroups of three as talking in
groups can reduce anxiety.”
17
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 17
ŶŽƚŐŝǀ ĞŶƟŵĞƚŽƉƌĞƉĂƌĞdŚŝƐǁ ĂƐa deliberate decision to maximiseĂƵƚŚĞŶƟĐŝƚLJĂƐƌĞĂůĐŽŶǀ ĞƌƐĂƟŽŶƐare spontaneous. I only intervenedŝĨƚŚĞĐŽŶǀ ĞƌƐĂƟŽŶŚĂĚĐŽŵƉůĞƚĞůLJhalted or if the learners directlyĂƐŬĞĚŵĞĂƋƵĞƐƟŽŶ ǁ ŚŝĐŚƚŚĞLJcould choose to do.
Teacher marking
/ŶĂƵƚŚĞŶƟĐƚĞƐƚƐŝƚŝƐǀ ŝƚĂůƚŽŝŶǀ Žůǀ Ğlearners in the assessment processas it enables them to developawareness and take responsibilityfor their own learning (O’Malley &Pierce, 1996), however peer andself-ĂƐƐĞƐƐŵĞŶƚĐĂŶŽŶůLJƌĞĂůŝƐƟĐĂůͲly supplement the expert evalua-ƟŽŶŽĨƚŚĞƚĞĂĐŚĞƌ;>ƵŽŵĂϮϬϬϰͿ For these reasons, I made markingĂĐŽůůĂďŽƌĂƟǀ ĞĞŶĚĞĂǀ ŽƵƌǁ ŝƚŚĂŶĂŶĂůLJƟĐƌĂƟŶŐƐĐĂůĞĨŽƌƚĞĂĐŚĞƌĂƐͲƐĞƐƐŵĞŶƚĂŶĚĂƐĞƉĂƌĂƚĞŚŽůŝƐƟĐscale for peer and self-assessment,ƚŚĞůĂƩ ĞƌĚĞǀ ŝƐĞĚďLJƚŚĞůĞĂƌŶĞƌƐ dŚĞƚĞĂĐŚĞƌƐƐĐĂůĞǁ ĂƐǁ ƌŝƩ ĞŶƚŽrate and score individual test takers’ƉĞƌĨŽƌŵĂŶĐĞƐŝŶƚŚĞĐŽŶǀ ĞƌƐĂƟŽŶƐ and was adapted from the commu-ŶŝĐĂƟǀ ĞůĂŶŐƵĂŐĞĐŽŵƉĞƚĞŶĐĞƐŵĞŶƟŽŶĞĚĞĂƌůŝĞƌ
1. >ŝŶŐƵŝƐƟĐĐŽŵƉƌĞŚĞŶƐŝďŝůŝƚLJ
2. Phonological comprehensibility
3. Fluency
4. ŽŶǀ ĞƌƐĂƟŽŶĂůŝŶƚĞƌĂĐƟŽŶ
5. Evidence of learning
>ŝŶŐƵŝƐƟĐĐŽŵƉƌĞŚĞŶƐŝďŝůŝƚLJǁ ĂƐincluded to discourage test takersfrom overly focusing on accuracyand thereby threatening fluency. It was defined as being able to use
appropriate and intelligible wordsand construct intelligible phrasesand sentences. Learners weremarked down if their errors imped-ed understanding, but mistakes andslips were disregarded. Pronuncia-ƟŽŶǁ ĂƐĂůƐŽĚĞĮ ŶĞĚŝŶƚĞƌŵƐŽĨintelligibility, though a high score inthis category also involved goodĐŽŶƚƌŽůŽĨŝŶƚŽŶĂƟŽŶĂŶĚƐLJůůĂďůĞstress. Fluency was defined accord-ing to the learning goal of speaking
spontaneously and confidently with-out unnatural pauses and excessiveŚĞƐŝƚĂƟŽŶ /ŶƚĞƌĂĐͲƟŽŶŝŶĐůƵĚĞĚƚƵƌŶͲtaking, co-ŽƉĞƌĂƟŶŐĂŶĚƉĂƌƟĐŝƉĂƟŽŶ Finally, ‘evidence oflearning’ was includ-ed to assess learn-ers’ use and under-standing of taughtcontent, rated ac-cording to its fre-quency, accuracyand appropriate us-age.
I adapted five de-scriptors from theCEFR and wrotethem as five-point ƐĐĂůĞƐŝŶĂƐƉŽƐŝƟǀ Ğwording as possible.
During the test, I simplified this into a basic 5x5 grid so I could makeƋƵŝĐŬŶŽƚĞƐĂŌĞƌĞĂĐŚĐŽŶǀ ĞƌƐĂƟŽŶand then use these with the full ru-bric when listening to the record-ings.
Peer and self-assessment
Another rubric was created for peerand self-Ğǀ ĂůƵĂƟŽŶ ĂŶĚƚŽŐŝǀ ĞĂgroup score. This five-ƉŽŝŶƚŚŽůŝƐƟĐscale made for an efficient, ƐƚƌĂŝŐŚƞŽƌǁ ĂƌĚŵĂƌŬŝŶŐƉƌŽĐĞƐƐthat promoted discussion and nego-ƟĂƟŽŶĂŵŽŶŐƐƚůĞĂƌŶĞƌƐ dŽŵĂdžͲimise learner involvement in thisprocess, the class was responsibleĨŽƌĐƌĞĂƟŶŐƚŚĞƐĐĂůĞĚƵƌŝŶŐŽŶĞŽĨtheir lessons. In small groups, theywere asked write a short descrip-ƟŽŶŽĨǁ ŚĂƚƚŚĞLJƚŚŽƵŐŚƚĐŽŶƐƟͲtuted an ‘excellent’ and a ‘not so
good’ performance, and eventually,ĂŌĞƌƐŽŵĞƌĞƉŽƌƟŶŐĂŶĚĐŽŵƉĂƌͲing, we wrote a five-point scale that
“To maximise learner
involvement in this process,
the class was responsible
for creating the scale
during one of their lessons.”
Teacher’s marking rubric (adapted from CEFR)
18
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 18
everyone agreed on.
ǁ ĞĞŬĂŌĞƌƚŚĞƚĞƐƚ ƚŚĞůĞĂƌŶĞƌƐreturned to their groups and Iplayed the recordings. Each groupagreed on a number from one tofive to score each recording, includ-ing their own, and wrote some con-ƐƚƌƵĐƟǀ ĞĐŽŵŵĞŶƚƐŽŶĂŶĂŶŽŶLJͲŵŽƵƐĞǀ ĂůƵĂƟŽŶƐŚĞĞƚ /ĨĂŶƵŵďĞƌcould not be agreed on, groupmembers could award a half marksuch as 3.5. I recorded the scores
and calculated an average for eachgroup, adding it to all the membersof that group on top of their individ-ual scores. At the end of class theĞǀ ĂůƵĂƟŽŶƐŚĞĞƚƐǁ ĞƌĞĚŝƐƚƌŝďƵƚĞĚto the learners to be read and dis-cussed.
Group marks encourage learners towork together and take joint re-sponsibility for their performancewhile individual marks make learn-ers accountable for their own con-ƚƌŝďƵƟŽŶƐ;K D ĂůůĞLJΘWŝĞƌĐĞϭϵϵϲͿ dŚƵƐ ĂůůŽĐĂƟŶŐďŽƚŚŵĂdžŝŵŝƐĞƐƚŚĞďĞŶĞĮ ƚƐŽĨĐŽůůĂďŽƌĂƟǀ ĞůĞĂƌŶŝŶŐyet remains fair for everyone.
A note on validity
Success in the test provided me and
learners with evidence that thelearning goal had been achieved;the students were able to haveƐƉŽŶƚĂŶĞŽƵƐŶŐůŝƐŚĐŽŶǀ ĞƌƐĂƟŽŶƐ However, the nature of open-endedŐƌŽƵƉĐŽŶǀ ĞƌƐĂƟŽŶƚĞƐƚƐƉŽƐĞƐsome validity issues. Assessinglearners in groups introduces nu-merous variables, making it difficult to isolate individual performances.t ŚŝůĞƚŚŝƐŝƐƉƌŽďůĞŵĂƟĐŝƚŵƵƐƚďĞƌĞŵĞŵďĞƌĞĚƚŚĂƚĐŽŶǀ ĞƌƐĂƟŽŶŝƐ
inherent-ly collab-oraƟveand con-textbound,and soŝƐŽůĂƟŶŐspeechand testitems
would invalidate the whole test.
Another issue is that learners inevi-tably act differently when being as-sessed (Luoma, 2004), most likelydue to test anxiety and the desire todemonstrate their best perfor-mance (He & Dai, 2006). Of courseneither of these would be present in ƌĞĂůǁ ŽƌůĚ ĐŽŶǀ ĞƌƐĂƟŽŶƐ dŽŵŝŶŝͲmise such threats I did everythingpossible to ensure my learners feltĂƚĞĂƐĞŝŶĐůƵĚŝŶŐŐŝǀ ŝŶŐƚŚĞŵƟŵĞƚŽƉƌĞƉĂƌĞĂŶĚƉƌĂĐƟĐĞĂŶĚĂůůŽǁ Ͳing them to choose their owngroups.
Conclusion
dŚŝƐĂƌƟĐůĞŚĂƐŽƵƚůŝŶĞĚĂƐŝŵƉůĞLJĞƚŝŶŶŽǀ ĂƟǀ ĞƐƉĞĂŬŝŶŐƚĞƐƚ ĚĞƐŝŐŶĞĚto help a class of Chinese university
students develop the skills,knowledge and confidence to en-gage in English language conversa-ƟŽŶƐ dŚĞĐůĂƐƐƌŽŽŵĐŽŶƚĞŶƚƚŚĂƚprovided the basis of the test facili-tated a meaningful and valuablelearning experience, and the collab-oraƟve evaluaƟon process promot-ed inter-peer dialogue and encour-aged learners to take control oftheir learning. Though group testsintroduce inevitable threats to valid-ity I took acƟon to minimise these,ĐƌĞĂƟŶŐĂƚĞƐƚƚŚĂƚ/ĐŽŶƚĞŶĚŝƐĂƵͲƚŚĞŶƟĐĨĂŝƌĂŶĚƵƐĞĨƵů
References
Council of Europe. 2001. Common Euro-pean Framework of Reference for Lan-guages: learning, teaching, assessment.Cambridge: Cambridge University Press.
He, Lianzhen & Dai, Ying. 2006. A cor-pus-ďĂƐĞĚŝŶǀ ĞƐƟŐĂƟŽŶŝŶƚŽƚŚĞǀ ĂůŝĚŝƚLJof the CET-SET group discus-sion. >ĂŶŐƵĂŐĞdĞƐƟŶŐ, 23(3): 370-401.
Luoma, Sari. 2004. Assessing speaking.Cambridge: Cambridge University Press.
McNamara, Tim. 2000. LanguageƚĞƐƟŶŐ. Oxford: Oxford University Press.
O’Malley, Michael & Pierce, Lorraine V.1996. ƵƚŚĞŶƟĐĂƐƐĞƐƐŵĞŶƚĨŽƌŶŐůŝƐŚůĂŶŐƵĂŐĞůĞĂƌŶĞƌƐ ƉƌĂĐƟĐĂůĂƉƉƌŽĂĐŚĞƐfor teachers. New York: Longman.
Peer and self-assessment rubric
19
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 19
/ŶĂƌĞĐĞŶƚũŽƵƌŶĂůĂƌƟĐůĞ/ƉƌĞƐĞŶƚͲ
ĞĚŵLJŝŶŝƟĂůƌĞƐĞĂƌĐŚŝŶƚŽƐƚƵĚĞŶƚƐ
openness to a new approach to giv-
ŝŶŐĨĞĞĚďĂĐŬŽŶǁ ƌŝƩ ĞŶǁ ŽƌŬ–
Learner-Driven Feedback (LDF).
Here is the reference for the full
ĂƌƟĐůĞ;Ăǀ ĂŝůĂďůĞŽƉĞŶůLJĂƐĚŝƚŽƌƐ
Choice on the ELTJ website): Maas,
ZĞĐĞƉƟǀ ŝƚLJƚŽ>ĞĂƌŶĞƌ-Driven
Feedback in EAP’, ELT Journal, 71/2
(2017), pp. 127-140.
The LDF approach understands that
ůĞĂƌŶĞƌƐŽŌĞŶĨĞĞůŚĂƉƉŝĞƌƌĞĐĞŝǀ ŝŶŐ
feedback from a teacher than
through peer review or self-ĞĚŝƟŶŐ
In LDF, learners ‘drive’ feedback by
ĂƐŬŝŶŐƋƵĞƐƟŽŶƐĂďŽƵƚƚŚĞŝƌǁ ŽƌŬ
and then re-ĚƌĂŌďĂƐĞĚŽŶƚŚŝƐ
feedback. The teacher gives the
feedback, but the learners decide
how and on what they receive com-
ments: they can choose between
various formats (e.g. hand-ǁ ƌŝƩ ĞŶ
email, audio recording), and are re-
ƋƵŝƌĞĚƚŽƉŽƐĞƋƵĞƐƟŽŶƐƚŽǁ ŚŝĐŚ
the teacher responds (e.g. about
grammar, vocabulary/register, ref-
erencing, text structure).
Some Background
I devised this approach by combin-
ing insights from other fairly recent
studies on giving feedback on L2
ǁ ƌŝƟŶŐƉĂƌƟĐƵůĂƌůLJŝŶWĐŽŶƚĞdžƚƐ
I looked into two key areas:
Making feedback more dialogic
Using technology to deliver
feedback
With regard to involving students in
a feedback dialogue, Bloxham and
ĂŵƉďĞůů;ϮϬϭϬͿƚƌŝĂůůĞĚ/ŶƚĞƌĂĐƟǀ Ğ
Coversheets’, which students use to
ƉŽƐĞƋƵĞƐƟŽŶƐĂďŽƵƚƚŚĞŝƌĞƐƐĂLJƐ
ǁ ŚĞŶƐƵďŵŝƫ ŶŐĚƌĂŌƐ ůŽdžŚĂŵ
and Campbell found that most stu-
dents in their study were grateful to
receive feedback that was more in-
dividualised than usual, which then
ĂůƐŽŵŽƟǀ ĂƚĞĚƚŚĞŵƚŽĞǀ ĂůƵĂƚĞ
their own work more thoroughly.
Moreover, the teachers involved
reported being able to provide feed-
ďĂĐŬŽŶĚƌĂŌƐŵŽƌĞƋƵŝĐŬůLJďLJŽŶůLJ
ĐŽŵŵĞŶƟŶŐŽŶĂƌĞĂƐŚŝŐŚůŝŐŚƚĞĚŝŶ
ƐƚƵĚĞŶƚƐ ƋƵĞƐƟŽŶƐ
Campbell and Schumm-Fauster
(2013) also required their EAP stu-
ĚĞŶƚƐƚŽƉŽƐĞĨĞĞĚďĂĐŬƋƵĞƐƟŽŶƐ
ŽŶƚŚĞŝƌǁ ŽƌŬǁ ŚĞŶƐƵďŵŝƫ ŶŐĞƐͲ
ƐĂLJĚƌĂŌƐ , ĞƌĞƐƚƵĚĞŶƚƐŶŽƚĞĚ
ƚŚĞŝƌƋƵĞƐƟŽŶƐŝŶĨŽŽƚŶŽƚĞƐŽƌŝŶ
the text’s margins. The students
were surveyed and reported finding
ƚŚĞĨĞĞĚďĂĐŬƉĞƌƐŽŶĂůĂŶĚŵŽƟͲ
ǀ ĂƟŶŐĂŶĚĞī ĞĐƟǀ ĞŝŶŵĞĞƟŶŐ
their individual needs regarding aca-
ĚĞŵŝĐǁ ƌŝƟŶŐ
Studies into using technology to de-
liver feedback have explored, for
example, using audio recordings or
emails. Johanson (1999) lists some
advantages of recording audio feed-
Clare Maas
holds post-ŐƌĂĚƵĂƚĞƋƵĂůŝĮ ĐĂƟŽŶƐfrom the University of Wales andTrinity College London. BeforeŵŽǀ ŝŶŐŝŶƚŽƚĞƌƟĂƌLJĞĚƵĐĂƟŽŶ she taught English at Germangrammar schools, and English forSpecific Purposes at several lan-guage academies in the UK andGermany. Her professional inter-ests include EAP, materials devel-opment, and CPD for teachers.Her most recent research hasďĞĞŶŝŶƚŽĨĞĞĚďĂĐŬŽŶ>Ϯǁ ƌŝƟŶŐespecially in EAP and higher edu-ĐĂƟŽŶ ŚĞĂůƐŽďůŽŐƐĂƚClaresELTCompendi-um.wordpress.com and is a mem-ber of the team behind EL-TResearchBites.com.
.
Receptivity to Learner-Driven Feedback
20
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 20
back for ESL students, which include
ŝƚƐƉŽƐƐŝďůLJƐĂǀ ŝŶŐƚĞĂĐŚĞƌƐƟŵĞ
making comments clearer through
ŝŶƚŽŶĂƟŽŶ ĂŶĚĨĞĞůŝŶŐŵŽƌĞƉĞƌͲ
sonal to students. These ideas are
supported by studies outside ELT,
such as Brearley and Cullen (2012),
who found three minutes of audio
recording could include ~500 words
of feedback and did not take more
ƟŵĞƚŚĂŶŵĂƌŬŝŶŐĞƐƐĂLJƐďLJŚĂŶĚ
ůƐŽŝŶǀ ĞƐƟŐĂƟŶŐĚŝŐŝƚĂůĨĞĞĚďĂĐŬ
Farshi and Safa (2015) compared
grades of EFL students who re-
ceived hand-ǁ ƌŝƩ ĞŶ ĞŵĂŝůĞĚ Žƌ
ŶŽĐŽƌƌĞĐƟǀ ĞĨĞĞĚďĂĐŬŽŶǁ ƌŝƩ ĞŶ
tasks for one term, and found
that emailed feedback led to sig-
nificantly greater improvement
on post-test grades than hand-
ǁ ƌŝƩ ĞŶĨĞĞĚďĂĐŬ
My Research
To put this summary in context,
readers need to know that I used
>&ŽŶƚŚƌĞĞĚƌĂŌƐŽĨĂŶĞƐƐĂLJ
that my undergraduate EAP stu-
dents (B2 level on CEFR) wrote dur-
ing one semester at a German uni-
versity. The students could:
choose between various (not
mutually exclusive!) modes of
feedback: in-ƚĞdžƚĐŽƌƌĞĐƟŽŶƐ
ĐŽƌƌĞĐƟŽŶƐLJŵďŽůƐ ŚĂŶĚǁ ƌŝƩ ĞŶ
feedback, email, audio recording,
or face-to-ĨĂĐĞĐŽŶƐƵůƚĂƟŽŶ
ĂƐŬƋƵĞƐƟŽŶƐĂďŽƵƚĂŶLJaspect
ŽĨƚŚĞŝƌǁ ƌŝƟŶŐĞ ŐƚĞdžƚƐƚƌƵĐͲ
ture, referencing, vocabulary,
grammar, etc.
pose both specific and general
ƋƵĞƐƟŽŶƐ
ŝŶĐůƵĚĞƚŚĞŝƌƋƵĞƐƟŽŶƐĂƐĨŽŽƚͲ
notes or in margins, or at the end
of the text.
At the end of semester, I surveyed
the 40 students to discover their
Ăƫ ƚƵĚĞƐƚŽǁ ĂƌĚƐƚŚĞ>&ĂƉƉƌŽĂĐŚ
dŚĞƋƵĞƐƟŽŶŶĂŝƌĞŝŶĐůƵĚĞĚƋƵĞƐͲ
ƟŽŶƐŽŶƐƚƵĚĞŶƚƐ ƉĞƌĐĞƉƟŽŶŽĨƚŚĞ
usability of various delivery modes,
ƚŚĞƉĞƌĐĞŝǀ ĞĚĞī ĞĐƟǀ ĞŶĞƐƐŽĨ>&
for improving their language accura-
cy and academic skills related to
essays, and any problems they ex-
perienced with LDF.
Some Findings
The delivery modes most students
requested were audio recording (by
67% of students) and email (60%).
dŚĞĮ ŐƵƌĞƐĨŽƌŚĂŶĚǁ ƌŝƩ ĞŶĐŽŵͲ
ments (13%) and in-ƚĞdžƚĐŽƌƌĞĐƟŽŶƐ
(20%) were much lower, which
seems to show that students are
happy to move away from
ƚƌĂĚŝƟŽŶĂůŚĂŶĚ-ǁ ƌŝƩ ĞŶĨĞĞĚďĂĐŬ
Students’ comments to open-ended
ƐƵƌǀ ĞLJƋƵĞƐƟŽŶƐŚŝŐŚůŝŐŚƚĞĚƐŽŵĞ
reasons why audio recordings and
email were so popular, for exam-
ƉůĞŵĞŶƟŽŶŝŶŐƚŚĂƚĨĞĞĚďĂĐŬǁ ĂƐ
ŵŽƌĞĚĞƚĂŝůĞĚĂŶĚƟŵĞůLJĂŶĚĨĞůƚ
more individualized than the feed-
back they were used to, and that,
ŝŶĂƵĚŝŽƌĞĐŽƌĚŝŶŐƐ ŝŶƚŽŶĂƟŽŶ
ŚĞůƉĞĚƚŚĞŵƚŽŝĚĞŶƟĨLJƉŽƐŝƟǀ Ğ
ĂŶĚŶĞŐĂƟǀ ĞĐŽŵŵĞŶƚƐ
Y ƵĞƐƟŽŶƐĂůƐŽĂƐŬĞĚǁ ŚŝĐŚĂƐƉĞĐƚƐ
of general language accuracy stu-
dents felt had significantly or
somewhat improved by working
with LDF. High numbers of students
felt they had improved their use of
ƚƌĂŶƐŝƟŽŶƐŝŐŶĂůƐ;ϰϮй ƐŝŐŶŝĮ ĐĂŶƚůLJ
33% somewhat), general grammar
(42% significantly, 40% somewhat)
and sentence structure (60% signifi-
cantly, 33% somewhat). Naturalness
of expression, text structure and
general vocabulary were also per-
ceived as having improved at least
somewhat by most students. The
“Students’ comments to
open-ended survey
questions highlighted some
reasons why audio record-
ings and email were so
popular, for example,
mentioning that feedback
was more detailed and
timely, and felt more indi-
vidualized than the feed-
back they were used to.”
21
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 21
ƌĞĂƐŽŶƐĨŽƌƚŚĞƐĞƉĞƌĐĞƉƟŽŶƐ ĂƐ
reported in open-ended survey
ƋƵĞƐƟŽŶƐ ĐĞŶƚƌĞĚĂƌŽƵŶĚƚŚĞĨĞĞĚͲ
back being more specific and placed
throughout the text, in contrast to
ƌĂƚŚĞƌŵŽƌĞƐƵŵŵĂƟǀ ĞĨĞĞĚďĂĐŬ
ĐŽŵŵĞŶƚƐƵƐƵĂůůLJǁ ƌŝƩĞŶĂƚƚŚĞ
ends of texts.
dŚĞĂĐĂĚĞŵŝĐǁ ƌŝƟŶŐƐŬŝůůƐŽĨĐƌŝƟĐĂů
thinking and self-ĞĚŝƟŶŐǁ ĞƌĞƐĞĞŶ
as having significantly improved by
over half of the students surveyed,
and as having somewhat improved
ďLJĂƌŽƵŶĚϰϬй >ĂƌŐĞŵĂũŽƌŝƟĞƐŽĨ
students also reported perceiving
improvement in logical argumenta-
ƟŽŶƐŬŝůůƐ;ϴϳ й ͿĂŶĚƚŚĞŝƌĂďŝůŝƚLJ
to engage in academic discourse
(80%). Overall, over three quar-
ters of the students surveyed
felt that LDF had been more
helpful in improving their aca-
demic skills related to essay
ǁ ƌŝƟŶŐƚŚĂŶƚŚĞŚĂŶĚ-ǁ ƌŝƩ ĞŶ
ĨĞĞĚďĂĐŬƚŚĞLJƚƌĂĚŝƟŽŶĂůůLJƌĞͲ
ceive.
Comparisons of students’ grades on
the essays for which they received
LDF and their grades on other essay
assignments supported the self-
ƌĞƉŽƌƚĞĚƉĞƌĐĞƉƟŽŶƐ ĂŶĚĐŽŶͲ
firmed that there was some real
improvement. On a grading system
out of 15 points, students’ essay
scores on the LDF assignment were
on average 2.22 points higher than
those received one year earlier and
on average 1.7 points higher than
those received on an assignment
ǁ ŝƚŚƚƌĂĚŝƟŽŶĂůĨĞĞĚďĂĐŬŝŶƚŚĞ
same term. The students’ essay
scores on an assignment with tradi-
ƟŽŶĂůĨĞĞĚďĂĐŬŝŶƐĂŵĞƚĞƌŵĂƐƚŚĞ
LDF assignment were on average
just 0.64 points higher than those
received one year earlier – smaller
improvements here would seem to
indicate that LDF had a role to play.
Discussion
These results show that learners
ƌĞƐƉŽŶĚĞĚƉŽƐŝƟǀ ĞůLJƚŽƚŚĞ>&ĂƉͲ
proach and that it was beneficial for
ƚŚĞŝƌǁ ƌŝƩ ĞŶǁ ŽƌŬThe findings thus
highlight LDF as a viable feedback
ƉƌĂĐƟĐĞŝŶƚŚŝƐĐŽŶƚĞdžƚ ǁ ŝƚŚĂĚͲ
vanced-level EAP students. Stu-
dents’ comments in response to
open-ĞŶĚĞĚƐƵƌǀ ĞLJƋƵĞƐƟŽŶƐŚŝŶƚ
at concepts which may underpin the
efficacy of the approach: LDF ad-
dresses issues of intelligibility,
‘authority’ over the texts, learner
ĂƵƚŽŶŽŵLJĂŶĚƉĞƌƐŽŶĂůŝƐĂƟŽŶ
These issues are not unique to EAP,
and thus the LDF approach may
ǁ ĂƌƌĂŶƚƉŝůŽƟŶŐŝŶŽƚŚĞƌĐŽŶƚĞdžƚƐ
too.
Students’ willing use of digital deliv-
ery modes for feedback can be prac-
ƟĐĂůĨŽƌƚĞĂĐŚĞƌƐ ǁ ŚŽĐĂŶŽŌĞŶ
provide more, and more detailed,
feedback more quickly than by
ǁ ƌŝƟŶŐĐŽŵŵĞŶƚƐďLJŚĂŶĚ /ƚĐĂŶ
also be efficient for tutors to give
feedback on selected aspects of the
ǁ ƌŝƟŶŐĚƵƌŝŶŐƚŚĞĚƌĂŌŝŶŐƐƚĂŐĞ
ƌĂƚŚĞƌƚŚĂŶĂƩĞŵƉƟŶŐƚŽĂĚĚƌĞƐƐ
all of the issues at once. Involving
students in the decisions about
the delivery of their feedback and
content may also help remove
some urgency from teachers to
agree on one ‘correct’ feedback
procedure, and can moreover
ĞŶŚĂŶĐĞĚŝī ĞƌĞŶƟĂƟŽŶďLJĂůůŽǁ Ͳ
ing for students to request feed-
back that suits their individual
strengths and weaknesses.
There are, of course, a few caveats
to these promising findings. For ex-
ample, students may not know
which aspects of their work to ask
ĂďŽƵƚ ŽƌŵĂLJŶŽƚďĞĂďůĞƚŽŝĚĞŶƟͲ
fy their own weaknesses, and lower-
level learners may not have suffi-
cient meta-language to pose effec-
Ɵǀ ĞĨĞĞĚďĂĐŬƋƵĞƐƟŽŶƐ /ĂĚĚƌĞƐƐ
some of these drawbacks below.
“Students’ willing use of digital
delivery modes for feedback
can be practical for teachers,
who can often provide more,
and more detailed, feedback
more quickly than by writing
comments by hand.”
22
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 22
dŝƉƐĨŽƌŝŵƉůĞŵĞŶƟŶŐ>&
LDF is easiest (I find) if students
submit their work electronically.
Decide which delivery modes you
wish to offer, and how students
ƐŚŽƵůĚƉŽƐĞƚŚĞƋƵĞƐƟŽŶƐ;Ğ Ő
footnotes, comments in margins,
or cover-sheet).
Allow any/only feedback delivery
modes that are workable in your
context. (i.e. perhaps not face-to-
ĨĂĐĞĐŽŶƐƵůƚĂƟŽŶŝĨLJŽƵŚĂǀ ĞϯϬϬ
students!)
Talk through the LDF process and
provide a handout to support stu-
ĚĞŶƚƐƚŚĞĮ ƌƐƚƟŵĞ;ƐͿƚŚĞLJǁ ŽƌŬ
with it.
ŽŵďŝŶĞ>&ǁ ŝƚŚǁ ƌŝƟŶŐǁ ŽƌŬͲ
shops, peer review, and other
ǁ ĂLJƐŽĨŐĞƫ ŶŐƐƚƵĚĞŶƚƐƚŚŝŶŬŝŶŐ
more carefully about their work,
to scaffold the LDF process.
Aim to provide feedback in a
ƟŵĞůLJŵĂŶŶĞƌĂŶĚĂůůŽǁ ĞŶŽƵŐŚ
ƟŵĞďĞƚǁ ĞĞŶĚƌĂŌƐƵďŵŝƐƐŝŽŶƐ
for students to thoroughly en-
gage with the feedback.
Give students guidance and/or
training on what makes for a
ŐŽŽĚƌĞƐƉŽŶƐĞƚŽƚŚĞǁ ƌŝƟŶŐƚĂƐŬ
at hand so that they can ask
about these points in their own
ǁ ƌŝƟŶŐ
If possible, provide students with
a grading matrix so they know
ǁ ŚĂƚĂƐƉĞĐƚƐŽĨƚŚĞŝƌǁ ƌŝƟŶŐƚŚĞLJ
ƐŚŽƵůĚƉĂLJĂƩĞŶƟŽŶƚŽ
Encourage students to pose both
ƐƉĞĐŝĮ ĐƋƵĞƐƟŽŶƐ;Ğ Őt ŚŝĐŚ
word is most appropriate here?)
ĂŶĚŐĞŶĞƌĂůƋƵĞƐƟŽŶƐ;Ğ Ő/ƐƚŚĞ
vocabulary I’m using formal
enough?).
If necessary and feasible, allow
ĨĞĞĚďĂĐŬƋƵĞƐƟŽŶƐŝŶƐƚƵĚĞŶƚƐ
L1.
For audio recordings or emailed
feedback, insert line numbers to
be able to refer to
specific parts of the
text.
For audio recordings,
you can email stu-
dents an .mp3 file, or
use free recording
websites such as
www.vocaroo.com
and send students the
links to their feed-
back.
Remind students to
ŚĂǀ ĞƚŚĞŝƌǁ ƌŝƩ ĞŶ
text at hand whilst
listening to / reading
the feedback.
Note down common
problems that stu-
dents do not request feedback
on, and use these to plan future
teaching or workshops.
Sit back and enjoy a cup of tea, as
>&ǁ ŝůůƉƌŽďĂďůLJƐĂǀ ĞLJŽƵƟŵĞ
(You might even want to throw
out your red pens!)
Foto
:Sh
aro
nH
artl
e
23
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 23
Date Presenter Title
19.12.17
Felicity O’Dell andRussell Whitehead
' Ğƫ ŶŐŝƚƌŝŐŚƚ ŐĞƫ ŶŐŝƚǁ ƌŽŶŐĂƚŽƵƌŽĨƚŚĞ>dϭϮϯƚĞƐƟŶŐƉƌŝŶĐŝƉůĞƐ
This webinar with Felicity O’Dell and Russell Whitehead willĚŝƐĐƵƐƐ>dϭϮϯ ƐůŝƐƚŽĨƚĞŶƉƌŝŶĐŝƉůĞƐĨŽƌŐŽŽĚ ƚĞƐƟŶŐƐŚĂƌŝŶŐplenty of examples of successful and unsuccessful items andtasks. This promises to be a useful round-up for newcomersand experienced testers alike.
Gudrun EricksondŚĞƵĂů&ƵŶĐƟŽŶŽĨƐƐĞƐƐŵĞŶƚ–Enhancing Learning and Equity
dŚŝƐǁ ĞďŝŶĂƌǁ ŝůůĨŽĐƵƐŽŶƚŚĞĚƵĂůĨƵŶĐƟŽŶŽĨĂƐƐĞƐƐŵĞŶƚ whether in classrooms or large scale contexts, namely toenhance learning as well as fairness and equity. Gudrun willdiscuss the balancing act between the two, emphasizingĐŽŵŵŽŶƉƌŝŶĐŝƉůĞƐƚŚĂƚŶĞĞĚƚŽŐƵŝĚĞƉůĂŶŶŝŶŐƉƌĂĐƟĐĞƐĂŶĚmaterials at individual as well as structural levels.
26.02.18
The webinar can be accessed on the day at: ŚƩ Ɖ ŝĂƚĞŇĂĚŽďĞĐŽŶŶĞĐƚ ĐŽŵ ƚĞĂƐŝŐǁ ĞďŝŶĂƌƐand is open to everybody. It will be recorded, and the recording will be accessible to everybody for aǁ ĞĞŬŽŶƚŚĞd^/' ǁ ĞďƐŝƚĞĂŌĞƌǁ ŚŝĐŚŝƚǁ ŝůůŽŶůLJďĞĂǀ ĂŝůĂďůĞƚŽ/d&>ŵĞŵďĞƌƐ d^/' ǁ ĞďŝŶĂƌƐǁ ŝůůďĞĐŽŶƟŶƵŝŶŐŝŶϮϬϭϳ ĂŶĚĚĞƚĂŝůƐǁ ŝůůďĞĂŶŶŽƵŶĐĞĚŽŶƚŚĞd^/' ǁ ĞďƐŝƚĞ
TEASIGwebinars
2017
24
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 24
ŌĞƌƐĞǀ ĞƌĂůLJĞĂƌƐŝŶƚĞĂĐŚĞƌĞĚƵͲĐĂƟŽŶ/ĐŽŶƟŶƵĞƚŽďĞĂŵĂnjĞĚďLJthe trainee teachers in my seminargroups. They consistently bring withthem a huge amount of knowledge,ǁ ŚĞƚŚĞƌůŝŶŐƵŝƐƟĐůŝƚĞƌĂƌLJŽƌĐƵůƚƵƌͲal. They also bring a wealth of skillsin areas ranging from tutoring toĂĚŵŝŶŝƐƚƌĂƟŽŶ &ŝŶĂůůLJĂŶĚŝŶŵLJview crucially, for the purposes oftraining new teachers who are alertand aware, they have of courseclocked up thousands of hours aslearners in school contexts them-selves. These vital experiences(good lessons and bad, inspiringtopics and less inspiring ones) canvery usefully be put to work in arange of ways in our seminars –from examininglearner (andƚĞĂĐŚĞƌ ͿŵŽƟǀ ĂͲƟŽŶ ĨŽƌĞdžĂŵƉůĞƚŽƌĞŇĞĐƟŶŐŽŶclassroom meth-ods which learnersappreciate. Thisprovides an idealƐŝƚƵĂƟŽŶĨŽƌƉƌŽͲĚƵĐƟǀ ĞƐĞŵŝŶĂƌwork which focuses on classroomƉƌĂĐƟĐĞ;ƐͿ
dŚŝƐĨŽĐƵƐŽŶĐůĂƐƐƌŽŽŵƉƌĂĐƟĐĞŶĞĐĞƐƐŝƚĂƚĞƐĂƉƌĂŐŵĂƟĐĂƉƉƌŽĂĐŚto learning how to teach, and weƚŚĞƌĞĨŽƌĞǁ ŽƌŬǀ ĞƌLJƉƌĂĐƟĐĂůůLJŝŶour seminar sessions. It goes with-out saying that, as a universityƚĞĂĐŚĞƌĞĚƵĐĂƟŽŶĚĞƉĂƌƚŵĞŶƚ ǁ Ğconsistently draw on research find-ŝŶŐƐĂŶĚĂĐĐĞƉƚĞĚďĞƐƚƉƌĂĐƟĐĞŝŶour work, but we nevertheless al-ǁ ĂLJƐŚĂǀ ĞŝŶŵŝŶĚƚŚĞƵůƟŵĂƚĞtransfer to the classroom and possi-ďůĞŝŵƉůĞŵĞŶƚĂƟŽŶǁ ŝƚŚƐĞĐŽŶĚĂƌLJ
school learners. To this end, train-ee teachers are regularly asked todesign a task or sequence of tasksfor one or more lessons in whichƚŚĞLJƉƵƚƚŚĞŝƌƚŚĞŽƌĞƟĐĂůƌĞĂĚŝŶŐƐŝŶƚŽƉƌĂĐƟĐĞ
Against this backdrop we engagewith many key issues of upper sec-ondary (language) learning, lookinginto principles of lesson-planningand curricula, the four skills, andtask- and competence-orientedlearning. Within this demandingmenu of important angles, one ma-jor area it's easy to feel over-whelmed by as a beginning teacheris surely that of assessment andƚĞƐƟŶŐdĞƐƟŶŐŐĞƚƐĂůŽƚŽĨďĂĚ
press: pupilshave on oc-casion beenknown tocomplainabout thevolume ofmaterial tobe learned,and teachersabout all the
marking theyhave to do. For this reason, as wellas many others, I feel it's importantƚŽƉƵƚĂƉŽƐŝƟǀ ĞƐƉŝŶŽŶĂƐƐĞƐƐŵĞŶƚand to show trainee teachers thatassessment can be a manageable,ƌĞǁ ĂƌĚŝŶŐĂŶĚƵůƟŵĂƚĞůLJŝŶƚĞƌͲĞƐƟŶŐĂƐƉĞĐƚŽĨƚĞĂĐŚŝŶŐĂŶĚůĞĂƌŶͲing.
We typically start by looking at clas-ƐŝĐƚĞƐƚĚĞƐŝŐŶĂŶĚĨŽƌŵĂƚƐ ŝĚĞŶƟĨLJͲing the markers of a successful test,ĂŶĚĂƚƚŚĞƐĂŵĞƟŵĞĐŽŶƚĞdžƚƵĂůŝƐͲŝŶŐŽƵƌŝŶƚƵŝƟǀ ĞƌĞĂĐƟŽŶƐĂŐĂŝŶƐƚĂďĂĐŬŐƌŽƵŶĚŽĨƚŚĞƚŚĞŽƌĞƟĐĂůƵŶͲ
Lynn Williams
grew up and went to school nearManchester, England. She studiedfor a language degree at universi-
ty and later added a teachingƋƵĂůŝĮ ĐĂƟŽŶ ŚĞŶŽǁ ƚĞĂĐŚĞƐ
English language and literature ata Swiss grammar school and alsoworks as a teacher trainer at theW, &, E t ŝŶĂƐĞů ŚĞŝƐƉĂƌƟĐƵͲ
larly interested in assessmentƉƌĂĐƟĐĞƐ ƚŚĞƚĞĂĐŚŝŶŐŽĨůŝƚĞƌĂͲƚƵƌĞĂŶĚŽƉƉŽƌƚƵŶŝƟĞƐĨŽƌŝŶĚŝͲ
ǀ ŝĚƵĂůŝƐĂƟŽŶůLJŶŶ ǁ ŝůůŝĂŵƐΛ Ĭ Ŷǁ ĐŚ
Giving testing some good press for a changeAssessment as learning check - a meaningful paradigm shift
for trainee teachers
“The link between
teaching and assessment
is not always clear at first
sight and sometimes
needs to be made more
explicit.”
25
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 25
ĚĞƌƉŝŶŶŝŶŐƐŽĨƚĞƐƚĚĞƐŝŐŶ ŌĞƌĐŽŶƐŝĚĞƌŝŶŐƚŚĞŽƌĞƟĐĂůĂƐƉĞĐƚƐƐƵĐŚas validity and reliability, we moveon to look at seemingly banal butƵůƟŵĂƚĞůLJĞƐƐĞŶƟĂůƉƌĂĐƟĐĂůĂƐƉĞĐƚƐlike concise and precise rubric for-ŵƵůĂƟŽŶŽƌĞǀ ĞŶŶƵŵďĞƌŝŶŐůŝŶĞƐŝŶa reading text for ease of orienta-ƟŽŶǁ ŚĞŶĚŝƐĐƵƐƐŝŶŐĂƚĞƐƚŝŶĐůĂƐƐat a later date.
Furthermore, I encourage traineeteachers to engage with principlesof backwash, my message beingĞƐƐĞŶƟĂůůLJ
KŶĐĞLJŽƵĂƌĞƐĂƟƐĮ ĞĚǁ ŝƚŚƚŚĞtest, use it to inform students /shape your teaching / studentƉƌĞƉĂƌĂƟŽŶǁ ŝƚŚŽƵƚĚŝƐĐůŽƐŝŶŐŬĞLJaspects.
^ŽŵĞƚŚŝŶŐŝŶƚĞƌĞƐƟŶŐĂŶĚƉĞƌŚĂƉƐ
ƵŶƐƵƌƉƌŝƐŝŶŐƚŚĂƚ/Ζǀ ĞŶŽƟĐĞĚŽǀ Ğƌthe years is that trainee teachersfrequently associate assessmentwith tests. In response to this obser-ǀ ĂƟŽŶ /ŚĂǀ ĞƚLJƉŝĐĂůůLJƐƚĂƌƚĞĚĂseminar session on assessment andƚĞƐƟŶŐǁ ŝƚŚĂĨƵŶĚĂŵĞŶƚĂůĚŝƐĐƵƐͲsion on the nature of assessmentwith a task like this:
A test is an assessmentbut an assessment is notnecessarily a test!
Discuss what you under-stand by this and be pre-pared to feedback in theseminar group.
As we know, the range ofŽƉƟŽŶƐǁ ŚŝĐŚŐŽďĞLJŽŶĚthe classic 'test' scenariois huge, ranging from quizƋƵĞƐƟŽŶƐĂƚƚŚĞĞŶĚŽĨĂlesson to self-Ğǀ ĂůƵĂƟŽŶ
according to prescribed learningŽďũĞĐƟǀ ĞƐ dƌĂŝŶĞĞƚĞĂĐŚĞƌƐĂƌĞtherefore encouraged to trawlthrough their (language) learningďŝŽŐƌĂƉŚŝĞƐƚŽŝĚĞŶƟĨLJƉŽƐƐŝďůĞĂƐͲƐĞƐƐŵĞŶƚƐĐĞŶĂƌŝŽƐ ƉĂƌƟĐƵůĂƌůLJthose that explode the classic canonŽĨĐŽŶƚƌŽůůĞĚŐƌĂŵŵĂƌƉƌĂĐƟĐĞĞdžͲercises. In a further step, the train-ees devise an appropriate assess-ment strategy for a teaching unit
they are developing andƌĞŇĞĐƚŽŶƚŚĞƌĂƟŽŶĂůĞbehind their choice.
However, in my experi-ence, the link betweenteaching and assessmentis not always clear at first ƐŝŐŚƚĂŶĚƐŽŵĞƟŵĞƐneeds to be made more
explicit. For example, trainee teach-ĞƌƐĨƌĞƋƵĞŶƚůLJ;ĂŶĚũƵƐƟĮ ĂďůLJͿƚĂŬĞĂƚŽƉŝĐŽƌƉĂƌƟĐƵůĂƌĂƌƟĐůĞŽƌĮ ůŵĐůŝƉĂƐƚŚĞƐƚĂƌƟŶŐƉŽŝŶƚĨŽƌĂƉŽƚĞŶͲƟĂůƚĞĂĐŚŝŶŐƵŶŝƚǁ ŝƚŚŽƵƚŶĞĐĞƐƐĂƌŝͲly having any special learning objec-Ɵǀ ĞŝŶŵŝŶĚ– they simply trust theirŝŶƐƟŶĐƚƐ ŬŶŽǁ ŝŶŐƚŚĂƚŝƚŝƐůŝŬĞůLJƚŽŵŽƟǀ ĂƚĞƵƉƉĞƌƐĞĐŽŶĚĂƌLJůĞĂƌŶĞƌƐor that it serves to exemplify a par-ƟĐƵůĂƌůĂŶŐƵĂŐĞƉŽŝŶƚǁ ŝƚŚŝŶƚŚĞĐŽŶƚĞdžƚŽĨĂŶĂƵƚŚĞŶƟĐƚĞdžƚ– bin-go! And good for them!
In my seminars, therefore, as well asŝŶŝŶĚŝǀ ŝĚƵĂůĐŽĂĐŚŝŶŐƐĞƫ ŶŐƐĂŶĚŝŶǁ ƌŝƩ ĞŶĨĞĞĚďĂĐŬŽŶĚƌĂŌƵŶŝƚƐ /seek to guide trainee teachers to-wards examining the underlyingƌĂƟŽŶĂůĞĨŽƌƚŚĞƵŶŝƚ ƚŚĞƌĞĂƐŽŶƐǁ ŚLJƚŚĞLJŚĂǀ ĞĐŚŽƐĞŶƚŚĞƉĂƌƟĐƵůĂƌtopic, text or task. Having estab-lished the competences orknowledge they want their uppersecondary learners to acquire, itbecomes easier for them to thinkĂďŽƵƚƚŚĞŝƌ;ŝŵƉůŝĐŝƚͿŽďũĞĐƟǀ ĞƐĨŽƌthe unit and, consequently, alsoabout how they can then check thatƚŚĞƐĞŽďũĞĐƟǀ ĞƐŚĂǀ ĞďĞĞŶachieved. This is the point at whichǁ ĞƚƵƌŶŽƵƌĂƩĞŶƟŽŶƚŽĂƐƐĞƐƐŵĞŶƚprocedures. Trainees realise –ƐŽŵĞƟŵĞƐǁ ŝƚŚĂũŽůƚ ƐŽŵĞƟŵĞƐwith a degree of disappointment,ďƵƚŵŽƌĞŽŌĞŶƚŚĂŶŶŽƚǁ ŝƚŚƌĞůŝĞĨ– that a classic 'test' might not benecessary to check their learners'progress. Or it might not be feasible,plausible or manageable within cer-tain constraints. Or, quite simply, itmight not be the best way of check-ing learning.
"So what is the best way?" my train-
26
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 26
ees then start to wonder, and I'malways nothing short of thrilled atthis point. It means we are breakingaway from the mindset that seesteachers replicate the kind of testthey themselves sat at school orthose that they might find through external searches for material,ŽŌĞŶĚĞƐŝŐŶĞĚƚŽĮ ƚĂůůƚƚŚŝƐpoint, let it be said that a good testis a good test, no quibbles there(and I've taken, set and correctedenough of them by now!), but weĂƌĞŶŽǁ ĂƚĂũƵŶĐƚƵƌĞƐĞƫ ŶŐŽƵƚinto new territory – namely the re-ĂůŝƐĂƟŽŶƚŚĂƚĂŐŽŽĚĂƐƐĞƐƐŵĞŶƚdoesn't need to be a test at all…
In my opinion, the 'best' way to as-sess a unit of work is at the sameƟŵĞĂŶŽďũĞĐƟǀ ĞĂŶĚƐƵďũĞĐƟǀ ĞĚĞĐŝƐŝŽŶ KďũĞĐƟǀ ĞďĞĐĂƵƐĞŵƵĐŚŽĨƚŚĞůŝƚĞƌĂƚƵƌĞŽŶƚĞƐƟŶŐƌŝŐŚƚůLJargues that an assessment shouldmirror the learning as far as thismakes sense (i.e. it would be odd tofollow up a series of discussion les-sons to develop fluency with a dis-crete-item grammar test, and byƚŚĞƐĂŵĞƚŽŬĞŶ ǁ ƌŝƟŶŐĂĨŽƌŵĂůůĞƩ ĞƌŵĂLJŶŽƚŶĞĐĞƐͲsarily be the best way tocheck learners' understand-ing of a range of new gram-mar points). Once traineeƚĞĂĐŚĞƌƐŚĂǀ ĞŝĚĞŶƟĮ ĞĚwhat they consider to bethe best way to check learn-ing of the unit they have puttogether – i.e. that whichbest aligns with the learningthey are striving for and theoverall competence-oriented outcome they areaiming at – they are encour-
aged to engage with a variety ofƋƵĞƐƟŽŶƐƚŽŚĞůƉƚŚĞŵƌĞĮ ŶĞƚŚĞŝƌassessment strategy. These mightƉƌŽĚƵĐƟǀ ĞůLJŝŶĐůƵĚĞƚŚĞĨŽůůŽǁ ŝŶŐ
Does the assessment methodreflect the style of work under-taken? (Does it assess thelearning which has gone be-fore? Are learners well-prepared to respond to thetask? In what way will itdemonstrate learners' grasp ofƚŚĞƚŽƉŝĐŝŶƋƵĞƐƟŽŶ Ϳ
Is the assessment task manage-able? (Timing, level, resources,teacher feedback)
/ƐƚŚĞĂƐƐĞƐƐŵĞŶƚƚĂƐŬŵŽƟͲǀ ĂƟŶŐ ;ŽĞƐŝƚŽī ĞƌůĞĂƌŶĞƌƐchoices? Does it appeal todifferent interests? Does it take up a relevant topic?)
KŶĐĞƚƌĂŝŶĞĞƚĞĂĐŚĞƌƐŚĂǀ ĞΖǀ ĞƩ ĞĚΖtheir assessment procedure tocheck its viability in these areas,ƚŚĞLJĂƌĞŝŶĂƉŽƐŝƟŽŶƚŽŵĂŬĞĮ ŶĂůadjustments and refine their assess-ment task. In a final step, we typi-
cally share teaching units and therelated assessments at the end ofthe semester. There is no small de-gree of pride during this phase and,having had the privilege of guidingƚƌĂŝŶĞĞƚĞĂĐŚĞƌƐŝŶƚŚĞƌĞĂůŝƐĂƟŽŶof their proposed assessment tasks,I understand their pride! I also hopethat a few key tenets will stay withƚŚĞŵĂŌĞƌŽƵƌũŽŝŶƚǁ ŽƌŬŽŶĂƐƐĞƐƐͲment strategies:
^ĞĞƚŚĂƚƚĞƐƟŶŐŝƐĂďŽƵƚŵŽƌĞƚŚĂŶũƵƐƚĂƩ ĂŝŶŝŶŐƐĐŽƌĞƐ Make tests worthwhile to allŝŶǀ Žůǀ ĞĚ ŵŽƟǀ ĂƟŶŐĨŽĐƵƐĞĚ ƌĞĂůŝƐƟĐŝŶĨŽƌŵĂƟǀ Ğ
>ŝŶŬƚĞƐƟŶŐƚŽĐůĂƐƐƌŽŽŵǁ ŽƌŬas far as possible scores. Maketests worthwhile to all class-ƌŽŽŵǁ ŽƌŬĂŶĚƚĞƐƟŶŐ
These are principles which guide myown classroom work, leading tolearning which I hope is all the rich-er, sustainable and more enjoyablefor it.
27
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 27
Assessment in the upper secondary ELT classroom
(introductory handout for trainee teachers)
&ŽƌŵƐŽĨƚĞƐƟŶŐĂŶĚĂƐƐĞƐƐŵĞŶƚ
It is important to note that an assessment does not necessarily need to be a test. It is equally possibleƚŽĐŚĞĐŬǁ ŚĞƚŚĞƌƐƚƵĚĞŶƚƐŚĂǀ ĞĂĐŚŝĞǀ ĞĚƚŚĞƌĞůĞǀ ĂŶƚůĞĂƌŶŝŶŐŽďũĞĐƟǀ ĞƐĨŽƌĂƵŶŝƚŝŶĂĚŝī ĞƌĞŶƚĂƐͲsessment set-ƵƉƚŚĂŶĂĨŽƌŵĂůǁ ƌŝƩ ĞŶƚĞƐƚ;ŽƌƚŚĞĞƋƵŝǀ ĂůĞŶƚƐƉĞĂŬŝŶŐĂƐƐĞƐƐŵĞŶƚͿ ŽŵĞŝĚĞĂƐŝŶͲclude:
- ĐŽŶĐĞƉƚƋƵĞƐƟŽŶƐ;ǁ ƌŝƩ ĞŶŽƌŽƌĂůͿŝŶĚŝǀ ŝĚƵĂůůLJŝŶŐƌŽƵƉƐŽƌŝŶƚŚĞƉůĞŶĂƌLJ;Ăůǁ ĂLJƐǁ ŝƚŚĨĞĞĚďĂĐŬĂŶĚƚĞĂĐŚĞƌĐůĂƌŝĮ ĐĂƟŽŶ ĞdžƉĂŶƐŝŽŶǁ ŚĞƌĞŶĞĐĞƐƐĂƌLJĂŶĚĂƉƉƌŽƉƌŝĂƚĞͿ
- self-Ğǀ ĂůƵĂƟŽŶ;ďĂƐĞĚŽŶƚĞĂĐŚĞƌƉƌŽŵƉƚƐͿ
- ƉĞĞƌĨĞĞĚďĂĐŬ;ĨŽůůŽǁ ĞĚďLJĚŝƐĐƵƐƐŝŽŶŝŶƉůĞŶĂƌLJĐŽŶƐƵůƚĂƟŽŶǁ ŝƚŚƚŚĞƚĞĂĐŚĞƌͿ
- ǁ ƌŝƩ ĞŶŽƌŽƌĂůĨĞĞĚďĂĐŬĐŽŵŵĞŶƚƐĨƌŽŵƚŚĞƚĞĂĐŚĞƌ
- ĂŶĂƉƉůŝĐĂƟŽŶŽĨƚŚĞĐŽŵƉĞƚĞŶĐĞƐĂĐƋƵŝƌĞĚŝŶĂŶĞǁ ƐĞƫ ŶŐ;ƚƌĂŶƐĨĞƌͿ
- a product (poster, mind map, reader's guide, handout, flyer, theatre programme…)
Principles of assessment
ůůĨŽƌŵƐŽĨĂƐƐĞƐƐŵĞŶƚŵƵƐƚĚĞŵŽŶƐƚƌĂƚĞŐĞŶƵŝŶĞǀ ĂůŝĚŝƚLJĂŶĚƐƚĂŶĚƵƉƚŽƐĐƌƵƟŶLJdŚĞƌĞĨŽƌĞ
- ĂĚŚĞƌĞƚŽƚŚĞƐƚĂŶĚĂƌĚƉƌŝŶĐŝƉůĞƐŽĨƚĞƐƟŶŐ;ƌĞůŝĂďŝůŝƚLJǀ ĂůŝĚŝƚLJĨĂŝƌŶĞƐƐͿĂŶĚŽī ĞƌŽƉƉŽƌƚƵŶŝƟĞƐĨŽƌƉŽƐŝƟǀ ĞďĂĐŬǁ ĂƐŚ
- communicate test requirements, format and criteria early on
- ďĞƉƌĞƉĂƌĞĚƚŽĂŶƐǁ ĞƌĂŶLJƋƵĞƐƟŽŶƐ;ĂŶĚƌĞ-think as and when necessary)
WƌĂĐƟĐĂďŝůŝƚLJ
dĞƐƚƐƐŚŽƵůĚďĞƐƚƌĂŝŐŚƞŽƌǁ ĂƌĚĨŽƌƚĞĂĐŚĞƌƐƚŽĂĚŵŝŶŝƐƚĞƌĂŶĚƐƚƵĚĞŶƚƐƚŽƚĂŬĞĂŶĚƐŚŽƵůĚŶŽƚƉƌĞͲsent unnecessary challenges in marking.
t ƌŝƩ ĞŶƚĞƐƚƐƐŚŽƵůĚ
- test what has been learned
- ƌĞŇĞĐƚƉƌĞǀ ŝŽƵƐƚĞĂĐŚĞƌŝŶĨŽƌŵĂƟŽŶŽŶƚŚĞƚĞƐƚ
- Žī ĞƌĐůĞĂƌŝŶƐƚƌƵĐƟŽŶƐĂŶĚŝŶĚŝĐĂƚĞŵĂƌŬƐƉĞƌƋƵĞƐƟŽŶ
- ideally, present a variety of tasks
- give all students a fair chance, and strong students the chance to excel
&ĞĞĚďĂĐŬ ĐŽƌƌĞĐƟŽŶƐĂƌĞĞƐƐĞŶƟĂůĂŶĚƐŚŽƵůĚ
- clearly show students where they went wrong, help address these points, and offer students paths for development
- report on performance in content terms, insofar as this is being tested
Suggested further reading ůĚĞƌƐŽŶ : ůĂƉŚĂŵ Θt Ăůů ;ϭϵϵϱͿ >ĂŶŐƵĂŐĞdĞƐƚŽŶƐƚƌƵĐƟŽŶĂŶĚǀ ĂůƵĂƟŽŶ ĂŵďƌŝĚŐĞhW
ŽƵŐůĂƐ ;ϮϬϭϬͿ hŶĚĞƌƐƚĂŶĚŝŶŐůĂŶŐƵĂŐĞƚĞƐƟŶŐ>ŽŶĚŽŶ , ŽĚĚĞƌĚƵĐĂƟŽŶ
&ƵůĐŚĞƌ' ;ϮϬϭϬͿ WƌĂĐƟĐĂůůĂŶŐƵĂŐĞƚĞƐƟŶŐ>ŽŶĚŽŶ , ŽĚĚĞƌĚƵĐĂƟŽŶ
, ƵŐŚĞƐ ;ϮϬϬϯ Ϳ dĞƐƟŶŐĨŽƌ>ĂŶŐƵĂŐĞdĞƐƟŶŐ;ϮŶĚĞĚ ͿĂŵďƌŝĚŐĞĂŵďƌŝĚŐĞhŶŝǀ ĞƌƐŝƚLJWƌĞƐƐ
D ĐEĂŵĂƌĂ d ;ϮϬϬϬͿ >ĂŶŐƵĂŐĞdĞƐƟŶŐKdžĨŽƌĚ KdžĨŽƌĚhŶŝǀ ĞƌƐŝƚLJWƌĞƐƐ
/ƚŝƐĂůƐŽǁ ŽƌƚŚĐŽŶƐƵůƟŶŐƚŚĞŐƵŝĚĞůŝŶĞƐĨŽƌŐŽŽĚƉƌĂĐƟĐĞŽŶůĂŶŐƵĂŐĞƚĞƐƟŶŐĂŶĚĂƐƐĞƐƐŵĞŶƚŝƐƐƵĞĚďLJƚŚĞƵƌŽƉĞĂŶƐƐŽĐŝa-ƟŽŶĨŽƌ>ĂŶŐƵĂŐĞdĞƐƟŶŐĂŶĚƐƐĞƐƐŵĞŶƚ;>dͿ ƐĞĞwww.ealta.ue.org .
28
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 28
/ŶƚƌŽĚƵĐƟŽŶ
/ŶŚŝƐϮϬϬϰĂƌƟĐůĞŝŶLanguage As-sessment Quarterly ĂĚǀ ŽĐĂƟŶŐƚŚĞneed to “broaden, deepen and con-solidate” many of our ideas aboutůĂŶŐƵĂŐĞƚĞƐƟŶŐƵŵŵŝŶŐŵĂŬĞƐthe convincing argument that moreresearch is needed on the role ofƐƚĂŬĞŚŽůĚĞƌƐŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐcontexts and areas that have tradi-ƟŽŶĂůůLJďĞĞŶŽǀ ĞƌůŽŽŬĞĚ;ƵŵŵŝŶŐ2004, p.3). It can be successfullyargued that one of these neglectedareas is Mexico. Mexico seems tofind itself in the paradoxical situa-ƟŽŶŽĨŵĂŶLJ>ĂƟŶŵĞƌŝĐĂŶĐŽƵŶͲtries which, while witnessing a pro-nounced increase in demand forhigh quality English language in-ƐƚƌƵĐƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚ ŚĂǀ Ğfailed to produce a significant body ŽĨƌĞƐĞĂƌĐŚŝŶǀ ĞƐƟŐĂƟŶŐƚŚĞƐƉĞĐŝĮ Đvariables that help to define the uniqueness of their contexts.
Without doubt, one of these varia-bles is the students or candidateswho actually take language tests.Their story is largely untold, and inƌĞĐĞŶƚůĂŶŐƵĂŐĞƚĞƐƟŶŐůŝƚĞƌĂƚƵƌĞŝƚis difficult to find an issue that more scholars seem to agree on than theidea that candidates are among themost important – yet neglected –ƐƚĂŬĞŚŽůĚĞƌƐŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐShohamy (2001) perceives that “it isthrough the voices of test takerswho report on their experiences andconsequences that the features ofƚŚĞƵƐĞŽĨƚĞƐƚƐĐĂŶďĞŝĚĞŶƟĮ ĞĚ zĞƚ ŝŶƚŚĞƚĞƐƟŶŐůŝƚĞƌĂƚƵƌĞƚĞƐƚƚĂŬĞƌƐĂƌĞŽŌĞŶŬĞƉƚƐŝůĞŶƚ ƚŚĞŝƌpersonal experiences are not heardor shared” (p.7). Cumming (2004)
maintains that “serious considera-ƟŽŶŽĨƚŚĞƵƐĞƐŽĨůĂŶŐƵĂŐĞĂƐƐĞƐƐͲŵĞŶƚƌĞƋƵŝƌĞƐĂĚŽƉƟŶŐƌĞƐĞĂƌĐŚŵĞƚŚŽĚƐƚŚĂƚŝŶǀ ĞƐƟŐĂƚĞƉĞŽƉůĞƐĂƫ ƚƵĚĞƐ ďĞůŝĞĨƐ ĐƵůƚƵƌĂůǀ ĂůƵĞƐ ĂŶĚǁ ĂLJƐŽĨŝŶƚĞƌĂĐƟŶŐ; Ϳ ƵĐŚinquiry is indispensable for under-standing why people perform theways they do in language assess-ment, and thus necessary for valida-ƟŽŶ ;Ɖ ϵͿ
The purpose of this study, there-fore, was to give free rein to theneglected voices of test candidatesŝŶŽŶĞƉĂƌƟĐƵůĂƌĐŽŶƚĞdžƚ ƐMcNamara and Roever (2006) insist, ůĂŶŐƵĂŐĞƚĞƐƟŶŐŚĂƐĂƌĞĂůŝŵƉĂĐƚon real people's lives” (p.8). Thisimpact starts with the stakeholderswho are immediately affected by the test, i.e. test candidates and testdevelopers, and extends outward tosociety at large. This impact alsoimplies a significant amount of re-sponsibility on the part of test de-velopers to ensure that the teststhey write and administer are asvalid and reliable as possible.
One of the most valuable tech-niques for helping test developersto measure test validity is by listen-ing to candidates’ voices. CandidateƉĞƌĐĞƉƟŽŶƐ ĨĞĞůŝŶŐƐ ƉŽŝŶƚƐŽĨǀ ŝĞǁ Ăƫ ƚƵĚĞƐ ŽƉŝŶŝŽŶƐĂŶĚƐƵŐͲŐĞƐƟŽŶƐ ƚĂŬĞŶƚŽŐĞƚŚĞƌĐĂŶƐĞƌǀ ĞĂƐĞǀ ŝĚĞŶĐĞŽĨƚŚĞƉŽƐŝƟǀ ĞĂŶĚŶĞŐͲaƟve consequences of tests. In ad-ĚŝƟŽŶ ĨĞĞĚďĂĐŬĨƌŽŵĐĂŶĚŝĚĂƚĞƐcan serve as the impetus for discus-sions that can, and should, be hap-pening among a variety of stake-holders (Madaus, in press, as citedin Shohamy, 2001, p.149). Enlarging
David Ewing Ryan
is a full-ƟŵĞƉƌŽĨĞƐƐŽƌĂƚƚŚĞUniversidad Veracruzana (UV),Mexico, where he conductsƌĞƐĞĂƌĐŚŽŶůĂŶŐƵĂŐĞƚĞƐƟŶŐand serves as Chief Editor ofys Z ƚŚĞhs ƐƟĞƌĞĚ-suiteŽĨŶŐůŝƐŚůĂŶŐƵĂŐĞĐĞƌƟĮ ĐĂƟŽŶtests.
“En mi humilde opinión”1 …listening to Mexican students’perceptions of an English language proficiency test
29
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 29
the dialogue in this way can helpfurther promote not just the validityof individual tests, but also of thetest system ǁ ŚŝĐŚŶĞĞĚƐƚŽĐŽŶƟŶͲually “encourage testers, teachers,test takers, and the public at largeƚŽƋƵĞƐƟŽŶƚŚĞƵƐĞƐŽĨƚĞƐƚƐ ƚŚĞmaterials they are based on, and toĐƌŝƟƋƵĞƚŚĞǀ ĂůƵĞƐĂŶĚďĞůŝĞĨƐŝŶŚĞƌͲent in them” (Shohamy, p.131).
dŚĞĂƌƟĐůĞŚĂƐƐŝdžƉĂƌƚƐ WĂƌƚϭŽƵƚͲlines the goal of the study. Part 2summarises the EXAVER English lan-ŐƵĂŐĞĐĞƌƟĮ ĐĂƟŽŶƚĞƐƚƐ ǁ ŚŝĐŚƐĞƌǀ ĞĚĂƐƚŚĞƉƌĂĐƟĐĂůĐŽŶƚĞdžƚŽĨthe study. Part 3 explains the meth-odology of the study. Parts 4 and 5Žī ĞƌƌĞƐƉĞĐƟǀ ĞůLJĂŶŽǀ Ğƌǀ ŝĞǁ ĂŶĚa discussion of the findings. Finally, Part 6 offers some general conclu-sions.
1. Goal of the study
The study fo-cused specifi-cally on whatBachman andPalmer (1996)consider asone of thethree waysthat languagetests have adirect impact on test candidates,namely, the consequences that can-didates experience as a result ofpreparing for and taking these tests(p.31). In order to measure this im-pact, it was necessary to liberatethe voices of the candidates whoƉĂƌƟĐŝƉĂƚĞĚŝŶƚŚĞƐƚƵĚLJĂŶĚƚŚŝƐbecame the study’s primary goal.This was accomplished, first, by so-
ůŝĐŝƟŶŐĐĂŶĚŝĚĂƚĞŽƉŝŶŝŽŶƐĂďŽƵƚƚŚĞprocesses of preparing for and tak-ŝŶŐƚŚĞƚĞƐƚ ĂŶĚƚŚĞŶďLJƐŽůŝĐŝƟŶŐƚŚĞŝƌƐƵŐŐĞƐƟŽŶƐŽŶŚŽǁ ƚŚĞƐĞƉƌŽͲcesses might be improved.
Ϯ WƌĂĐƟĐĂůĐŽŶƚĞdžƚŽĨƐƚƵĚLJƚŚĞEXAVER English LanguageĞƌƟĮ ĐĂƟŽŶdĞƐƚƐ
Ϯ ϭ' ĞŶĞƌĂůĚĞƐĐƌŝƉƟŽŶ
EXAVER is the name ofthe tests used as the ba-sis of the study, and re-ĨĞƌƐƚŽĂƟĞƌĞĚ-suite ofŶŐůŝƐŚůĂŶŐƵĂŐĞĐĞƌƟĮ ĐĂͲƟŽŶƚĞƐƚƐĚĞǀ ĞůŽƉĞĚĂŶĚadministered by the UniversidadVeracruzana (UV) in the southeast-ern Mexican state of Veracruz. Thefirst suite was developed in the year 2000 by a small group of Englishlanguage teachers at the UV, as wellĂƐďLJƌĞƉƌĞƐĞŶƚĂƟǀ ĞƐĨƌŽŵƚŚĞ
ƌŝƟƐŚŽƵŶĐŝůĂŵďƌŝĚŐĞƐƐĞƐƐͲment, and Roehampton University’sCenter for Language Assessmentand Research (CLARe). The con-struct behind the EXAVER tests is tomeasure three language proficiency ůĞǀ ĞůƐŝĚĞŶƟĮ ĞĚŝŶƚŚĞŽƵŶĐŝůŽĨEurope's Common European Frame-work of Reference for Languages(CEFR), summarised in Table 1. The
EXAVER tests are administeredtwice a year at 11 language centersthroughout Veracruz.
Table 1: Levels of EXAVER tests andtheir corresponding CEFR Levels(adapted from Abad et al., 2011)
2.2 Test Structure
Each EXAVER test contains threeseparate papers. The structure ofeach paper is described below.
Table 2: EXAVER test structure;ĂŌĞƌƵŶŶĞϮϬϬϳ Ϳ
2.3 dĞƐƚůŽĐĂůŝnjĂƟŽŶ
According to O’Sullivan (2011), oneŽĨƚŚĞĚĞĮ ŶŝŶŐĐŚĂƌĂĐƚĞƌŝƐƟĐƐŽĨƚŚĞys ZĞdžĂŵŝŶĂƟŽŶƐŝƐƚŚĂƚƚŚĞLJƌĞƉƌĞƐĞŶƚ ƚŚĞĮ ƌƐƚƐLJƐƚĞŵĂƟĐĂƩ ĞŵƉƚƚŽĐƌĞĂƚĞĂůŽĐĂů Ăī ŽƌĚĂͲble, and sustainable language testsystem” (O’Sullivan, p.10). In focus-
EXAVER CEFR Council of Europe
1Upper Beginner
A2 Waystage
2Lower Intermediate
B1 Threshold
3Upper Intermediate
B2 Vantage
Paper 1ZĞĂĚŝŶŐĂŶĚt ƌŝƟŶŐ
Paper 2Listening
Paper 3Speaking
5 parts
Variety of tasks: matching,ŵƵůƟƉůĞĐŚŽŝĐĞŵŽĚŝĮ ĞĚcloze text
/ŶĚŝƌĞĐƚŵĞĂƐƵƌĞŽĨǁ ƌŝƟŶŐ
4 parts
Range from comprehension ofƌĞůĂƟǀ ĞůLJƐŚŽƌƚŝŶĨŽƌŵĂůĐŽŶǀ ĞƌͲƐĂƟŽŶƐƚŽĐŽŵƉƌĞŚĞŶƐŝŽŶŽĨŵŽƌĞĨŽƌŵĂůĂŶĚƐƵďƐƚĂŶƟĂůůLJůŽŶŐĞƌĐŽŶǀ ĞƌƐĂƟŽŶƐ
3 parts
Combine some type of inter-view task (interlocutor to candi-date), discussion task(between a pair of candidates)and a long-turn task(interlocutor to candidate)
30
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 30
ŝŶŐƚŚĞŝƌĂƩ ĞŶƟŽŶŽŶƚŚĞůŽĐĂůŐĞŽͲŐƌĂƉŚŝĐĐŽŶƚĞdžƚŽĨƚŚĞĞdžĂŵŝŶĂƟŽŶƐ(southeastern Mexico) and the par-ƟĐƵůĂƌŶĞĞĚƐŽĨƚŚĞĐĂŶĚŝĚĂƚĞƐǁ ŝƚŚͲin that context (students, primarily,of the Universidad Veracruzana),EXAVER’s test developers havehelped create a process now knownĂƐ ůŽĐĂůŝnjĂƟŽŶ K ^Ƶůůŝǀ ĂŶĚĞĮ ŶĞƐƚŚŝƐĂƐ ƚŚĞƉƌĂĐƟĐĞŽĨƚĂŬŝŶŐŝŶƚŽaccount those learning-focused fac-ƚŽƌƐƚŚĂƚĐĂŶŝŵƉĂĐƚŽŶůŝŶŐƵŝƐƟĐƉĞƌĨŽƌŵĂŶĐĞ ĂŶĚƚŚĞƌĞĐŽŐŶŝƟŽŶof the importance of test context ontest development…” (O’Sullivan,p.6).
Economic affordability was one of the first local variables that EXA-VER’s test developers considered.As the majority of EXAVER’s candi-ĚĂƚĞƐĐŽƵůĚŶŽƚ;ĂŶĚƐƟůůĐĂŶŶŽƚͿafford the cost of more reputable ŝŶƚĞƌŶĂƟŽŶĂůŶŐůŝƐŚůĂŶŐƵĂŐĞĐĞƌƟͲĮ ĐĂƟŽŶƚĞƐƚƐ ys Z ƐƚĞƐƚĚĞǀ ĞůͲopers decided to create a suite ofeconomically affordable tests, in line with median to lower incomebrackets based on the Mexican min-imum wage.2 Table 3 shows the cur-rent costs (as of September 2017) oftaking an EXAVER test, with approxi-mate equivalents in Euros.3
As of January 2017, the Mexicanminimum wage was approximately80 pesos per day.
For more details on the EXAVER ex-ĂŵŝŶĂƟŽŶƐĂŶĚƚŚĞys ZƚĞƐƚsystem, especially as they relate toůŽĐĂůŝnjĂƟŽŶ ƐĞĞďĂĚĞƚĂů“Developing affordable, ‘local’ tests: the EXAVER Project” in LanguagedĞƐƟŶŐdŚĞŽƌŝĞƐĂŶĚWƌĂĐƟĐĞƐ Ě Barry O’Sullivan (Palgrave Macmil-lan, 2011) pages 228-243.
dĂďůĞϯ ŽŵƉĂƌĂƟǀ ĞĐŽƐƚŽĨƚĂŬŝŶŐĂŶEXAVER test
3. Methodology
A mixed methods (quan → QUAL) ĂƉƉƌŽĂĐŚĨŽƌĚĂƚĂĐŽůůĞĐƟŽŶĂŶĚanalysis was used for the study. TheƋƵĂŶƟƚĂƟǀ ĞĚĂƚĂĐĂŵĞĨƌŽŵƚŚĞresponses of 245 EXAVER candi-dates who completed a web-basedƋƵĞƐƟŽŶŶĂŝƌĞǁ ŚŝĐŚǁ ĂƐĂĚŵŝŶŝƐͲtered in the summer of 2010 follow-ŝŶŐƚŚĞƐƉƌŝŶŐϮϬϭϬĂĚŵŝŶŝƐƚƌĂƟŽŶof EXAVER’s three levels. The ques-ƟŽŶŶĂŝƌĞŝŶĐůƵĚĞĚϰϮĐůŽƐĞĚ-format,ŵƵůƟƉůĞ-ĐŚŽŝĐĞƋƵĞƐƟŽŶƐ ĂŶĚƚǁ Žopen-ĞŶĚĞĚƋƵĞƐƟŽŶƐ K ĨƚŚĞclosed-ĨŽƌŵĂƚƋƵĞƐƟŽŶƐ ϭϬĞŵͲƉůŽLJĞĚĂ>ŝŬĞƌƚ ĐĂůĞǁ ŝƚŚŽƉƟŽŶƐspanning from 1 to 5, to ascertaincandidates' opinions about severaltopics related to the test. Excel Ver-sion 2003 was used to analyze theĚĂƚĂ dŚĞƋƵĂůŝƚĂƟǀ ĞĚĂƚĂĐĂŵĞfrom the author’s research journalfrom March to October 2010, andfrom semi-structured interviewsconducted in October 2010 withĨŽƵƌŽĨƚŚĞƋƵĞƐƟŽŶŶĂŝƌĞΖƐƌĞƐƉŽŶĚͲents.
4. Findings 4
Out of the 964 candidates who tookan EXAVER test in May 2010, 245 ofthem (or 25%) responded to theweb-based survey. Of these, 99 (orϰϬй ͿƟĐŬĞĚƚŚĞďŽdžĂƚƚŚĞĞŶĚŽĨthe survey, signifying their desire to
ƉĂƌƟĐŝƉĂƚĞŝŶĂƐĞŵŝ-structured in-ƚĞƌǀ ŝĞǁ dŚŝƐƌĞůĂƟǀ ĞůLJŚŝŐŚƌĂƚĞŽĨ
response was the first ŝŶĚŝĐĂƟŽŶŽĨƚŚĞĂƉƉĂƌͲent strong desire of EXA-VER’s candidates to havetheir voices heard.
Web-ďĂƐĞĚƋƵĞƐƟŽŶŶĂŝƌĞƐƵƌǀ ĞLJ
ƐƌĞƐĞĂƌĐŚŝŶƐƚƌƵŵĞŶƚƐ ƋƵĞƐƟŽŶͲŶĂŝƌĞƐŚĂǀ ĞƚŚĞŝƌĚŝƐƟŶĐƚĂĚͲvantages and disadvantages. WithƌĞŐĂƌĚƚŽƚŚĞůĂƩ ĞƌƌĞƐĞĂƌĐŚĞƌƐƐŽŵĞƟŵĞƐĐŽŵƉůĂŝŶĂďŽƵƚƚŚĞůĂĐŬŽĨĚĞƉƚŚĂŶĚƌŝĐŚŶĞƐƐŝŶŵƵůƟƉůĞ-choice responses (Dörnyei, 2007,p.115). For this reason, the re-searcher included two open-endedƋƵĞƐƟŽŶƐŝŶƚŚĞƐƵƌǀ ĞLJĂůŽŶŐǁ ŝƚŚƚŚĞϰϮŵƵůƟƉůĞ-ĐŚŽŝĐĞƋƵĞƐƟŽŶƐ While the responses to all of theƐƵƌǀ ĞLJƐƋƵĞƐƟŽŶƐƉƌŽǀ ŝĚĞĚŝŵͲportant feedback, the responses tothe two open-ĞŶĚĞĚƋƵĞƐƟŽŶƐ(numbers 17 and 30) are notewor-thy, due both to the high number ofcandidates who responded to them(well over half of the total 245 can-didates who took part in the sur-vey), as well as to the diversity oftheir answers. Summaries of theseresponses follow.5
Y ƵĞƐƟŽŶϭϳ “Do you feel that thereis anything we could include on theEXAVER website that might helpfuture candidates to feel less anx-ious and/or more confident before taking the test? If so, please writeyour comment(s) below, taking allthe space that is necessary.” Ques-ƟŽŶϭϳ LJŝĞůĚĞĚϭϰϰƌĞƐƉŽŶƐĞƐ ŽƌŐĂͲnized into the following categories:
ϮϯƉŽƐŝƟǀ ĞƌĞƐƉŽŶƐĞƐ ƐƵĐŚĂƐ
LEVEL Cost in MX Pesos Cost in Euros
EXAVER 1 500 Approx. 23
EXAVER 2 550 Approx. 25
EXAVER 3 600 Approx. 27
31
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 31
“I didn’t hire a tutor or use anybooks to prepare for the test, as/ĨŽƵŶĚƚŚĞŝŶĨŽƌŵĂƟŽŶŽŶƚŚĞwebsite very useful.”
ϮϰŶĞŐĂƟǀ ĞƌĞƐƉŽŶƐĞƐ ƐƵĐŚĂƐ
dŚĞǁ ĂŝƟŶŐƟŵĞƚŽŐĞƚLJŽƵƌgrade is too long...you reallyneed to find a way to make it go faster.”
“I would have benefited from a greater variety, and greaterscale of difficulty, of test prep-ĂƌĂƟŽŶŵĂƚĞƌŝĂůƐ– the Sam-ple Tests on the website werereally easy and not very help-ful.”
ϵϳ ƐƵŐŐĞƐƟŽŶƐ ŶŽƚĂďůLJĨŽƌĨƵƌƚŚĞƌĚŽĐƵŵĞŶƚĂƟŽŶŽƌmaterials to include on theEXAVER website, such as:
a video of a sample speakingtest
a bibliography of literature toconsult to help prepare forthe test
ĂĚĞƐĐƌŝƉƟŽŶŽĨŚŽǁ ŐƌĂĚĞƐare calculated
Y ƵĞƐƟŽŶϯϬ“Do you have any oth-ĞƌĐŽŵŵĞŶƚƐ;ƉŽƐŝƟǀ ĞŽƌŶĞŐĂƟǀ ĞͿĂŶĚ ŽƌƐƵŐŐĞƐƟŽŶƐƚŚĂƚLJŽƵ ĚůŝŬĞƚŽadd regarding the specific test you ƚŽŽŬŽƌĂďŽƵƚƚŚĞys ZĞƌƟĮ ĐĂͲƟŽŶdĞƐƚƐŝŶŐĞŶĞƌĂů /ĨƐŽ ƉůĞĂƐĞwrite them below, taking all thespace that is necessary.” Y ƵĞƐƟŽŶ30 yielded 127 responses.
ϯϴƉŽƐŝƟǀ ĞƌĞƐƉŽŶƐĞƐ ƐƵĐŚĂƐ
“The EXAVER staff appeared to be very knowledgeable andwhen they gave the instruc-ƟŽŶƐŝŶŶŐůŝƐŚ ŝƚǁ ĂƐǀ ĞƌLJclear, which set me at easeand made me feel more confi-dent.”
ϲϭŶĞŐĂƟǀ ĞĐŽŵŵĞŶƚƐ ƐƵĐŚas:
t ŚŝůĞǁ ĂŝƟŶŐŝŶůŝŶĞƚŽĞŶͲter the test center, I was toldthat my name was not on thelist even though I had my reg-ŝƐƚƌĂƟŽŶƌĞĐĞŝƉƚ /ŶƚŚĞĞŶĚ/was able to take the test, butI felt very nervous.”
ϮϴƐƵŐŐĞƐƟŽŶƐ ƐƵĐŚĂƐ
“It would be nice to have amore detailed report on how Ifared in the test, such asknowing how I performed oneach part of the test, maybein terms of percentages.”
5. Discussion
5.1 Specific concerns
dŚĞƉŚƌĂƐŝŶŐŽĨY ƵĞƐƟŽŶϭϳ ǁ ŝƚŚspecial emphasis on the words“more confident” and “less anx-ŝŽƵƐ ǁ ĂƐŝŶƚĞŶƟŽŶĂůŝŶŽƌĚĞƌƚŽreflect the researcher’s premise that the less anxious and more confident candidates feel before taking a test,the more likely they are to performďĞƩĞƌ6 dŚĞƌĞůĂƟǀ ĞůLJůŽŶŐůŝƐƚŽĨƐƵŐŐĞƐƟŽŶƐ;ϵϳ ŝŶƚŽƚĂůͿƚŚĂƚĐĂŶĚŝͲdates gave in response to this ques-ƟŽŶŚĂǀ ĞƉƌŽǀ ĞŶƵƐĞĨƵůŝŶŚĞůƉŝŶŐEXAVER’s test developers improvethe quality of the website’s prepara-ƟŽŶŵĂƚĞƌŝĂůƐƐŽƚŚĂƚĐĂŶĚŝĚĂƚĞƐ
can indeed feel more confident and less anxious before taking a livetest.
Y ƵĞƐƟŽŶϯϬƐŚŽƵůĚƐĞĞŵĨĂŵŝůŝĂƌƚŽƋƵĂůŝƚĂƟǀ ĞƌĞƐĞĂƌĐŚĞƌƐ ƐŝŶĐĞŝƚŝƐthe classic “Do you have anythingelse to add?” that usually appears atthe end of an interview. It was con-sidered necessary as a ‘safety net’to ensure that candidates had theopportunity to state anything andeverything they wished to aboutpreparing for and taking an EXAVERtest.
KŶĞŽĨƚŚĞŶĞŐĂƟǀ ĞƌĞƐƉŽŶƐĞƐƚŽY ƵĞƐƟŽŶϯϬƌĞĨĞƌƌŝŶŐƚŽĂĐĂŶĚŝͲdate’s sense of anxiety over theirname not being found on the official list of test registrants, relates to theƚŚĞŵĞŽĨY ƵĞƐƟŽŶϭϳ /ƚƐŚŽƵůĚƌĞͲmind testers of the importance ofĂĚŽƉƟŶŐŵĞĂƐƵƌĞƐƚŚĂƚŚĞůƉƚŽavoid circumstances that can poten-ƟĂůůLJĐĂƵƐĞƵŶŶĞĐĞƐƐĂƌLJƐƚƌĞƐƐĂŶĚanxiety for candidates. Test examin-ers and administrators should devel-op a list of things that could possiblygo wrong on the day of the test,ŝĚĞŶƟĨLJŝŶŐĂŶĞī ĞĐƟǀ ĞƐŽůƵƟŽŶĨŽƌeach of them. Such a list shouldthen be printed and given to testinvigilators.
LJĐŽŶƚƌĂƐƚ ŽŶĞŽĨƚŚĞƉŽƐŝƟǀ ĞƌĞͲƐƉŽŶƐĞƐƚŽY ƵĞƐƟŽŶϯϬŝůůƵƐƚƌĂƚĞƐŚŽǁ ĂƐĞĞŵŝŶŐůLJƌŽƵƟŶĞƚĂƐŬ(calmly and clearly reading the ini-ƟĂůŝŶƐƚƌƵĐƟŽŶƐŽŶĐĞĐĂŶĚŝĚĂƚĞƐĂƌĞseated) can actually serve to mini-mize test anxiety and boost candi-dates’ sense of confidence. Both examples reinforce the importanceŽĨĞŶƐƵƌŝŶŐƚŚĂƚƚŚĞƚĞƐƚƌĞĐĞƉƟŽŶ process (the way that candidates
32
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 32
are physically greeted and treatedby examiners and invigilators bothprior to and during the test) is assmooth and professional as possi-ble.
5.2 General concerns
Candidate responses to both theƋƵĞƐƟŽŶŶĂŝƌĞĂŶĚƚŚĞƐĞŵŝ-structured interviews provid-ĞĚĂƌŝĐŚƌĞƉƌĞƐĞŶƚĂƟŽŶŽĨƚŚĞdiversity of opinions, feelings,ƉĞƌĐĞƉƟŽŶƐ ĂŶĚĂƫ ƚƵĚĞƐthat EXAVER candidates haveabout the tests. They also pro-vided EXAVER’s test develop-ers with important insightsŝŶƚŽƐŽŵĞŽĨƚŚĞƉŽƐŝƟǀ ĞĂŶĚŶĞŐĂƟǀ ĞĐŽŶƐĞƋƵĞŶĐĞƐĨŽƌtest candidates as a result ofpreparing for and taking a lan-guage test. With regard to theƋƵĞƐƟŽŶŶĂŝƌĞƚŚĞƋƵĂŶƟƚLJand variety of responses bringto mind Shohamy’s (2001) ob-ƐĞƌǀ ĂƟŽŶƚŚĂƚƚŚĞŽǀ Ğƌǁ ŚĞůŵŝŶŐmajority of test candidates not onlyhave a strong need and desire toexpress their feelings about the testthey took, but they also have theinherent right to do so, and it is theresponsibility of language teachersand testers to enable them to do so(p.156). By providing for this, shefeels that testers can help democra-ƟnjĞƚŚĞĂĐƚŽĨƚĂŬŝŶŐĂƚĞƐƚƐŽƚŚĂƚthe experience becomes more of aĐŽůůĂďŽƌĂƟǀ ĞŚŽƌŝnjŽŶƚĂůƉƌŽĐĞƐƐ rather than an authoritarian, top-down one (Shohamy, p. 136-137).
It can be argued, however, that themost important step that takesplace in the overall process of solic-ŝƟŶŐĐĂŶĚŝĚĂƚĞĨĞĞĚďĂĐŬŝƐǁ ŚĂƚtesters finally end up using it for.
For this reason, one might correctlyrefer to the ‘final consequences’ of ĐŽŶƐĞƋƵĞŶƟĂůǀ ĂůŝĚŝƚLJdŚĞĮ ŶĂůĂĐͲƟŽŶƐƚĂŬĞŶďLJƚĞƐƚĚĞǀ ĞůŽƉĞƌƐŵĂLJvery well serve to increase the oc-ĐƵƌƌĞŶĐĞŽĨƉŽƐŝƟǀ ĞĐŽŶƐĞƋƵĞŶĐĞƐfor future candidates and to de-
ĐƌĞĂƐĞƚŚĞŽĐĐƵƌƌĞŶĐĞŽĨŶĞŐĂƟǀ Ğconsequences.
dŚĞĨŽůůŽǁ ŝŶŐĂĐƟŽŶƐ ďĂƐĞĚŽŶĐĂŶͲdidate feedback from this and otherstudies, have already been taken (orare currently being undertaken) byEXAVER administrators and serve toillustrate how a language test boardcan convert candidate feedback intoƉŽƐŝƟǀ ĞŝŵƉĂĐƚĨŽƌĨƵƚƵƌĞĐĂŶĚŝͲdates:
^ƚƌĞĂŵůŝŶĞĚƌĞŐŝƐƚƌĂƟŽŶƉƌŽĐĞƐƐ making it much easier for currentand future candidates to registerfor the tests
EĞǁ ŽŶůŝŶĞŐƌĂĚĞĂůůŽĐĂƟŽŶƉƌŽͲĐĞƐƐƚŽƐƵďƐƚĂŶƟĂůůLJƌĞĚƵĐĞƚŚĞǁ ĂŝƟŶŐƟŵĞĨŽƌƌĞĐĞŝǀ ŝŶŐŐƌĂĚĞƐ
Sample Speaking Test for each ofEXAVER’s three levels uploadedto the EXAVER website so thatƉŽƚĞŶƟĂůĐĂŶĚŝĚĂƚĞƐŚĂǀ ĞĂŶidea of the format of the test aswell as the type of tasks they canexpect (These tests complementƚŚĞƐĂŵƉůĞƌĞĂĚŝŶŐǁ ƌŝƟŶŐĂŶĚlistening tests on the websiteƐŝŶĐĞys Z ƐŝŶĐĞƉƟŽŶ Ϳ
ƌĂŌŝŶŐŽĨĂůŝƐƚŽĨĂĚŵŝŶŝƐƚƌĂͲƟǀ ĞƉƌŽĐĞĚƵƌĞƐƚŚĂƚŵĂLJďĞƉƌŽďůĞŵĂƟĐĨŽƌĞdžĂŵŝŶĞƌƐĂŶĚinvigilators on the test day, alongǁ ŝƚŚĐŽƌƌĞƐƉŽŶĚŝŶŐƐŽůƵƟŽŶƐ
ŝƐƐĞŵŝŶĂƟŽŶŽĨĐĂŶĚŝĚĂƚĞĨĞĞĚďĂĐŬƋƵĞƐƟŽŶŶĂŝƌĞƐĂƐĂǁ ĂLJŽĨĐŽŶƟŶƵŝŶŐƚŽŵŽŶŝƚŽƌƚŚĞƉŽƐͲiƟve and negaƟve consequencesfor candidates taking the tests
Analysis and discussion of ap-ƉƌŽƉƌŝĂƚĞĂĐƟŽŶ;ƐͿďĂƐĞĚŽŶĐĂŶͲdidate responses
Follow-through to confirm that ĂƉƉƌŽƉƌŝĂƚĞĂĐƟŽŶǁ ĂƐƚĂŬĞŶ
6. Conclusion
By now it has perhaps become ap-parent to the reader that what can-ĚŝĚĂƚĞĨĞĞĚďĂĐŬŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐactually relates to is a type of as-sessment that is more inclusive andĚĞŵŽĐƌĂƟĐŝŶŶĂƚƵƌĞƚŚĂŶƚŚĞƚƌĂĚŝͲƟŽŶĂůĂƵƚŚŽƌŝƚĂƌŝĂŶƚLJƉĞƚŚĂƚǁ ĂƐprevalent in so many assessmentcontexts throughout the world dur-ing much of the 20th century and,indeed, prior to that. 7
When test developers refuse to so-licit candidate feedback, or do sowithout following through on it, theƵŶĚĞŵŽĐƌĂƟĐŶĂƚƵƌĞŽĨƚŚĞĂƐƐĞƐƐͲment can be seen in terms of the
“With regard to the question-
naire, the quantity and variety
of responses bring to mind
Shohamy’s observation that
the overwhelming majority of
test candidates not only have
a strong need and desire to
express their feelings about the
test they took, but they also
have the inherent right to do
so, and it is the responsibility of
language teachers and testers
to enable them to do so.”
33
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 33
power and control that testers ex-ert over candidates. Conversely,when test developers solicit candi-ĚĂƚĞĨĞĞĚďĂĐŬĂŶĚƚĂŬĞƉŽƐŝƟǀ ĞĂĐƟŽŶƐďĂƐĞĚŽŶŝƚ ƚŚĞĚĞŵŽĐƌĂƟĐnature of the assessment is evi-denced as a horizontal and collabo-ƌĂƟǀ ĞƉƌŽĐĞƐƐ D ŽƌĞŽǀ ĞƌƚŚŝƐƉƌŽͲĐĞƐƐĞŶĐŽƵƌĂŐĞƐƚŚĞƉĂƌƟĐŝƉĂƟŽŶof not merely a few, but a wide va-riety of stakeholders, therebystrengthening even further theĚĞŵŽĐƌĂƟĐŶĂƚƵƌĞŽĨƚŚĞĂƐƐĞƐƐͲment.
Another important point that lan-guage test developers should con-sider when judging the validity oftheir assessments is that languageƚĞƐƟŶŐůŝŬĞĂŶLJƚLJƉĞŽĨƚĞƐƟŶŐŝƐ at best, an inexact science. Thereare many things that can go wrongon the day of a test which can in-terfere in its validity. The air condi-ƟŽŶŝŶŐŝŶĂŚŽƚĂŶĚŚƵŵŝĚƌŽŽŵcould stop working, forcing candi-dates to finish the test in uncom-ĨŽƌƚĂďůĞƉŚLJƐŝĐĂůĐŽŶĚŝƟŽŶƐ Ŷoral examiner could ask a candidateǁ ŚĂƚƚŚĞLJĚŝĚŽŶƚŚĞŝƌůĂƐƚǀ ĂĐĂƟŽŶwithout knowing that someone inthe candidate’s family died at thatƟŵĞ/ŶďŽƚŚŽĨƚŚĞƐĞŶŽƚŽǀ ĞƌůLJextraordinary cases, the candi-ĚĂƚĞƐĐŽŶĐĞŶƚƌĂƟŽŶĐŽƵůĚƉŽƐƐŝďůLJbe affected, thereby modifying his/her performance. This couldĐŽŶƐĞƋƵĞŶƚůLJŶĞŐĂƟǀ ĞůLJŝŵƉĂĐƚŽŶthe candidate’s score and provide aĨĂůƐĞƌĞŇĞĐƟŽŶŽĨŚŝƐŽƌŚĞƌƚƌƵĞability.
The above examples represent realƐŝƚƵĂƟŽŶƐƚŚĂƚŚĂǀ ĞƚĂŬĞŶƉůĂĐĞduring real EXAVER test administra-ƟŽŶƐ ƐůĂŶŐƵĂŐĞƚĞƐƚĞƌƐǁ ŽƌŬ
with real people in the real world,ƌĞĂů;ĂŶĚŽŌĞŶƟŵĞƐƵŶĨŽƌĞƐĞĞĂďůĞor uncontrollable) problems areůŝŬĞůLJƚŽĐŽŶƟŶƵĞƚŽŽĐĐƵƌ, Žǁ Ğǀ Ͳer, when it comes to designing andadministering tests, there are manythings that testers can indeed con-trol, including:
Concern for the test’s most im-portant stakeholder: the candi-date
The psychometric quality of thetest, i.e. its validity and reliability
ŽůůĞĐƟǀ ĞĞůĞŵĞŶƚƐŽĨƚŚĞƚĞƐƚƐLJƐƚĞŵŽƌƚŚĞŽƉĞƌĂƟŽŶƐƚŚĂƚare external to the test per se,ƐƵĐŚĂƐƚŚĞƚĞƐƚƌĞŐŝƐƚƌĂƟŽŶ ŽƌŝͲĞŶƚĂƟŽŶ ĂŶĚƉƌĞƉĂƌĂƟŽŶƉƌŽͲcesses and the way that candi-dates are treated by examinersand invigilators on the day of thetest
ĞŝŶŐƌĞƐƉŽŶƐŝďůĞĂŶĚĞī ĞĐƟǀ Ğexaminers, e.g. giving fair andnon-ƉĂƌƟĂůƚƌĞĂƚŵĞŶƚƚŽĂůůĐĂŶͲdidates and following-up with a‘post-exam’ report with a list ofthings that went right and wrongduring the test
By concerning themselves withthese and other important
variables, language test-ers can help safeguardthe overall fairness andintegrity of the test andthe test system. In sodoing, they also help tounderscore the differ-ence between assess-ments that, on the onehand, are moving to-wards a more dynamic,responsible and demo-ĐƌĂƟĐŵŽĚĞůĂŶĚŽŶƚŚĞother hand, ones thatĐŽŶƟŶƵĞƚŽƌĞŵĂŝŶmore stagnant and con-ǀ ĞŶƟŽŶĂůŝŶŶĂƚƵƌĞ
References provided on request
Footnotes:
1. dƌĂŶƐůĂƟŽŶ /ŶŵLJŚƵŵďůĞŽƉŝŶͲŝŽŶ dŚĞƉƌĞƐĞŶƚĂƌƟĐůĞŝƐĂŶabridged version of “Consider thecandidate: using test-taker feed-back to enhance quality and validi-ƚLJŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐ ƉƵďůŝƐŚĞĚin e-TEALS: An e-journal of TeacherĚƵĐĂƟŽŶĂŶĚƉƉůŝĞĚ>ĂŶŐƵĂŐĞStudies 5 (2014): 1-23. ISSN 1647-ϳ ϭϮy ŚƩ Ɖ ůĞƌůĞƚƌĂƐ ƵƉ Ɖƚuploads/ficheiros/13086.pdf
“When test developers refuse to
solicit candidate feedback, or
do so without following through
on it, the undemocratic nature
of the assessment can be seen in
terms of the power and control
that testers exert over candi-
dates. Conversely, when test de-
velopers solicit candidate feed-
back and take positive actions
based on it, the democratic na-
ture of the assessment is evi-
denced as a horizontal and col-
laborative process.”
34
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 34
2. As of January 2017, the Mexicanminimum wage was approximately80 pesos per day.
3. For more details on the EXAVER ex-ĂŵŝŶĂƟŽŶƐĂŶĚƚŚĞys ZƚĞƐƚsystem, especially as they relate toůŽĐĂůŝnjĂƟŽŶ ƐĞĞďĂĚĞƚĂů“Developing affordable, ‘local’ tests: the EXAVER Project” in LanguagedĞƐƟŶŐdŚĞŽƌŝĞƐĂŶĚWƌĂĐƟĐĞƐ Ě Barry O’Sullivan (Palgrave Macmil-lan, 2011) pages 228-243.
4. ƵĞƚŽƐƉĂĐĞĐŽŶƐŝĚĞƌĂƟŽŶƐ Į ŶĚͲŝŶŐƐƌĞůĂƚĞĚƚŽƚŚĞƋƵĂůŝƚĂƟǀ ĞƉŚĂƐĞof the study could not be includedbut are available from the author [email protected]
5. The web-ďĂƐĞĚƋƵĞƐƟŽŶŶĂŝƌĞŽƌŝŐŝͲnally appeared in Spanish and wassubsequently translated into Eng-lish. Candidate responses to theƋƵĞƐƟŽŶŶĂŝƌĞĨŽůůŽǁ ĞĚƚŚŝƐƐĂŵĞƉĂƩ ĞƌŶ
6. While it could be argued that thispremise is based on common sense,it actually mirrored Bachman and
Palmer’s similar hypothesis (1996,p.32).
7. dŚĞƚƌĂĚŝƟŽŶĂůŽƌĂƵƚŚŽƌŝƚĂƌŝĂŶŵŽĚĞůŽĨĞĚƵĐĂƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚŝƐƐƟůůƉƌĞǀ ĂůĞŶƚŝŶŵĂŶLJƉĂƌƚƐŽĨƚŚĞworld today, including in many edu-ĐĂƟŽŶĂůĐŽŶƚĞdžƚƐŝŶD ĞdžŝĐŽ
35
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 35
You are responsible for developing an inter-
mediate level test for your school. There are
ten intermediate classes with ten different
teachers. They all use different course
books. If the students do well enough, they
can move to the next level. Teachers don’t
have a lot of time for marking. Tests will be
run monthly and there will be 150 students
taking the test.
Possible approaches/considerations:
- Collect course books used and analyse lan-
guage, functions, skills etc. used in each book.
- Identify common crossovers.
- Evaluate whether the test should include any
skills element, taking timing and resources into
account.
- Consider that you may need multiple versions.
- Pilot tests with the teachers (ask them to com-
plete within the time etc).
- For practicality, you could assess the function-
ing after the first tests (at a most basic level, ask
teachers to name the students in class that are
the strongest and weakest, and compare
against results).
- Changes can be made after testing to further
improve test functioning, possibly based on
placement of students and how well this is work-
ing.
You are designing a business English test
for telesales workers. You will be responsi-
ble for giving the test to new workers
(roughly 5 workers a month), and with the
results they will be given specific modules
to improve their working capability in Eng-
lish. All new employees already have a B2
level of English.
Possible approaches/considerations:
- Analyse the language, functions and skills that
are used in the role by studying existing work-
ers. Are they using a script? Do they go off
script?
- Consider the impact carefully; could employ-
ees lose their job due to this test? Do they know
or not know this? Do you know this?
- Practicality is less of an issue compared to
scenario 1, and you may be able to activate the
test language in authentic tasks that they will
carry out in their work.
- Ensure test design has clear links to the mod-
ules that you can recommend.
- Consider carefully when the tests will be done;
there could be some interactivity here between
their training for the job and their test perfor-
mance.
- Pilot the test on existing employees; they could
reap the benefits of free further training ideas!
IATEFL TEASIG WebinarDeveloping a test: Where do you start? When should you stop?
24 October 2017Jo Tomlinson
As we didn’t get time to discuss the scenarios at the end of the webinar, Jo has provided us witha small collection of thoughts and considera-tions. She is sure there are many more.
36
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 36
The unanswered questions asked during the
course of the webinar have been answered by
Jo in the following:
Is a test always a compromise governed by
rules?
I would say so. From my experience, test devel-
opment and the resulting test is always a series
of compromises. Test development often re-
quires two (possibly) distinct approaches. The
first is understanding and using good practice
standards. The second is pragmatism. Put simp-
ly, it’s the compromise between what should be
done and what can be done. I’m sure all of us
testers have an ideal way of testing in a specific
situation, but available resources often limit that.
We just have to do the best we can with what
we’ve got!
Isn't test development more cyclical than
linear? Or like a jigsaw puzzle where the
pieces never manage to fit together?
I think it definitely can be. Especially if the test is
to be repeated (perhaps with subsequent ver-
sions) over time. If there is a single test being
created, I personally like the stair metaphor as it
gives an idea of moving onwards (and up-
wards!), yet we must acknowledge and embrace
the fact that we’re bound to move up and down
quite a few stairs before getting to the top.
I also like the jigsaw puzzle idea. Perhaps our
aim should be to fit enough pieces together to
be able to ‘see the picture’!
How important is it that you speak to all
stakeholders in their own language – learn-
ers, school head, HR, bosses, parents?
When creating a test, I see no reason why we
shouldn’t do this if we can. The more we know
from everybody involved, the more we can un-
derstand the context. Almost nobody likes any-
thing that is forced upon them. The more we val-
ue the opinions of those involved, and express
this in a dialogue, the more likely they are to val-
ue the end product.
How much value do you place on ancillary
parts of test development such as item writ-
ing, rater training, statistical analysis, and
pilot feedback?
These areas are fundamental to test develop-
ment, but the part they play depends on the test
design and context. However, if your test design
includes these elements and they aren’t carried
out well enough, then you can’t actualise that
particular test design. Take, for example, criteria
that raters don’t particularly understand or are
self-conflicting; it doesn’t matter if you have the
best plans in the world, the results from that test
won’t be able to tell you much.
What type of test do you personally like con-
structing best?
This is a difficult question because I enjoy con-
structing all tests (honestly!). I would say I par-
ticularly enjoy analysing the construct. I work
with a colleague who comes from a linguistics
background in contrast to my testing back-
ground, so you can imagine the conversations. I
particularly enjoy work that gives us the creative
means to try something new. Integrated tests
are interesting to work on, simply because they
can be quite challenging from a development
angle.
Can you tell us something more about note-
taking as a task? Is note-taking a fair way of
testing these days, as we write so little? Or
could we do it on a tablet?
Let’s look at this from an EAP perspective. Note
-taking still exists, of course, and might be called
the most authentic (yet messy) way of testing
listening. There is a lot that can go wrong with
note-taking as a testing tool, especially in the
37
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 37
rating, yet there are ways in which these prob-
lems can be limited.
However, as we discussed in the webinar, note-
taking in general is moving on. More and more
students are recording their lectures and taking
photos of the slides. Now, I suppose an essen-
tial question for us as test developers is ‘What
are those students doing with that information
after the lecture?’
Good point regarding the way different ex-
aminers interpret particular assessment cri-
teria (e.g. spelling). How should we deal with
this and make sure that different assessors
interpret the assessment criteria they're sup-
posed to be using in the same way? Regular
monitoring and annual (re-)training sessions
don't seem to help a lot as the grades are
still different when we do random monitor-
ing.
Yes, I think it’s always a somewhat constant
struggle. If we take spelling into consideration,
then from my experience allowing degrees of
flexibility in spelling can be extremely problemat-
ic as what is an acceptable error for one rater
will be unacceptable for another. There are also
numerous problems with having a list of
‘acceptable spelling errors’. In this instance, I
would be tough and say no deviation from the
correct spelling is allowed. I also explain why
and ask the raters whether they think it is fair
that one student should ‘pass’ while another
should ‘fail’ due to a generous rater giving a few
more marks and a tougher rater giving a few
less. Most agree that it’s not acceptable.
In terms of wider interpretation of the marking
criteria, I think it is essential to discuss the fun-
damental terms for which we perhaps take for
granted that we all have a shared meaning (for
example, ‘task achievement’ or ‘cohesion’), as
well as analyse and discuss degrees of achieve-
ment and what they mean (e.g. ‘convincing’,
‘minimal’ etc.) and show examples of these.
Timing is also important; ensure the training/
standardisation is fresh in raters’ heads wherev-
er possible, and perhaps double mark and feed-
back (depending on whether this is feasible).
How much space do you think there is for
creativity in testing?
I hope there is a lot of space for creativity. The
more we can facilitate testing that is learner and
context specific, the more meaningful it can be
for the test-takers and also in terms of results
within the particular context. I think that one of
our aims as test developers (and perhaps this is
sometimes understated) is to assess in a way
that candidates find a test a joy to do, rather
than a hurdle to overcome.
38
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 38
Where are you based?Dubai, UAE
How long have you been there? / Which othercountries have you worked in?I’ve been in the Gulf for 25 years and have worked inthe US, France, Oman and the UAE.
How long have you been a member of IATEFL?As far back as I can remember.
Why did you choose TEASIG as your SIG?D LJƉƌŝŵĂƌLJĂƌĞĂŽĨƐƉĞĐŝĂůŝnjĂƟŽŶŝƐƚĞƐƟŶŐassessment.
Are you a member of any other SIGs?No not with IATEFL but I chair two SIGs with ind^K>ƌĂďŝĂ ƚŚĞdĞƐƟŶŐƐƐĞƐƐŵĞŶƚĂŶĚǀ ĂůƵĂͲƟŽŶ /' ĂƐǁ ĞůůĂƐƚŚĞ>ĞĂĚĞƌƐŚŝƉĂŶĚD ĂŶĂŐĞŵĞŶƚSIG.
What do you get out of TEASIG?TEASIG is a community within IATEFL where every-one has the same or similar interests; I enjoy thisĚLJŶĂŵŝĐĂŶĚĞƐƉĞĐŝĂůůLJŵĞĞƟŶŐƚĞƐƚĞƌƐĨƌŽŵĂƌŽƵŶĚthe world and learning about what they are doingŶŽƚŽŶůLJŝŶƚŚĞĮ ĞůĚŽĨ>dďƵƚŝŶůĂŶŐƵĂŐĞƚĞƐƟŶŐand assessment.
t ŚĂƚŽƚŚĞƌƐĞƌǀ ŝĐĞƐŽƌĂĐƟǀ ŝƟĞƐǁ ŽƵůĚLJŽƵǁ ĞůͲcome?I would welcome more peer-edited and reviewedƉƵďůŝĐĂƟŽŶƐĨƌŽŵŽƵƌĐŽŵŵƵŶŝƚLJ
What sort of work do you do?/ĂŵĂĨƵůůƟŵĞĨĂĐƵůƚLJŵĞŵďĞƌĂƚƵďĂŝD ĞŶ ƐŽůͲlege where I teach General Studies courses.
, Žǁ ŵƵĐŚŽĨLJŽƵƌǁ ŽƌŬƟŵĞŝƐƐƉĞŶƚŽŶƚĞƐƟŶŐĞǀ ĂůƵĂƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚ In my job about 15%ŽĨŵLJƟŵĞŝƐƐƉĞŶƚŽŶĂƐƐĞƐƐŵĞŶƚŽƌĂƐƐĞƐƐŵĞŶƚƌĞůĂƚĞĚĂĐƟǀ ŝƟĞƐ ŝŶŵLJƟŵĞŽƵƚƐŝĚĞƚŚĞǁ ŽƌŬƉůĂĐĞ/ƐƉĞŶĂĂŐƌĞĂƚĚĞĂůŽĨƟŵĞƚƌĂŝŶŝŶŐƚĞĂĐŚĞƌƐŚŽǁ ƚŽǁ ƌŝƚĞƚĞƐƚƐĂŶĚǁ ƌŝƟŶŐŬƐŽŶĂƐƐĞƐƐŵĞŶƚ
Do you see yourself asmore of a teacher oras more of a tester?I’d like to say ‘both’but in actual fact I amnow spending more ofŵLJƟŵĞĂƐĂƚĞĂĐŚĞƌ
Do you work in a teamor alone?I believe that team-work is crucially im-portant so I work inteams much more sothan alone.
Do you set tests?zĞƐ /ĚŽďƵƚŵŽƌĞŽĨŵLJƚĞƐƟŶŐƟŵĞŝƐƐƉĞŶƚĞĚͲŝƟŶŐƚĞƐƚƐƚŚĂƚŽƚŚĞƌƐŚĂǀ Ğǁ ƌŝƩ ĞŶĂŶĚĂĚǀ ŝƐŝŶŐŽŶƚĞƐƟŶŐĂŶĚĂƐƐĞƐƐŵĞŶƚŵĂƩ ĞƌƐ
What sort of tests are they? How high are thestakes for the test-takers?They are usually midterm or final assessments of ďŽƚŚƚŚĞĨŽƌŵĂƟǀ ĞĂŶĚƚŚĞƐƵŵŵĂƟǀ ĞŬŝŶĚ ƚŚĞůĂƩ ĞƌĂƌĞƉƌĞƩLJŚŝŐŚƐƚĂŬĞƐĂƐƚŚĞLJƵƐƵĂůůLJĐŽƵŶƚĨŽƌ30% of a student’s final grade.
Are the tests you set published in any form?Not any more although I worked for many years asthe Chief Examiner for the English B component ofƚŚĞ/ŶƚĞƌŶĂƟŽŶĂůĂĐĐĂůĂƵƌĞĂƚĞĂŶĚŵĂŶLJŽĨƚŚŽƐĞtests were published.
ŽLJŽƵĞŶũŽLJƐĞƫ ŶŐƚĞƐƚƐzƵƉ ĂŌĞƌƚƌĂǀ ĞůůŝŶŐǁ ĂƚĐŚŝŶŐ' ĂŵĞŽĨdŚƌŽŶĞƐĂŶĚscuba diving, it’s my favorite thing to do!
Can you use the same set of skills and principles inseveral different contexts? I believe so. I feel the way we package ourselvesand this includes our skills and principles is very im-portant.
TEASIG Member Spotlight: Dr Christine Coombe.
39
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 39
ŽLJŽƵĐĂƌƌLJŽƵƚƌĞƐĞĂƌĐŚŽŶƚĞƐƟŶŐĂŶĚƌĞůĂƚĞĚfields?Yes.
, Ăǀ ĞLJŽƵƉƵďůŝƐŚĞĚŽŶƚĞƐƟŶŐŝƐƐƵĞƐ Yes, I have a few research-ďĂƐĞĚĂƌƟĐůĞƐŝŶƚŽƉƟĞƌũŽƵƌŶĂůƐůŝŬĞ>ĂŶŐƵĂŐĞdĞƐƟŶŐd^K>Y ƵĂƌƚĞƌůLJĂŶĚLanguage Teaching. However, my favorite kind ofpublishing is on helping teachers increase their as-sessment literacy.
, Ăǀ ĞLJŽƵƐƉŽŬĞŶĂƚĐŽŶĨĞƌĞŶĐĞƐŽŶƚĞƐƟŶŐŝƐƐƵĞƐYes, lots.
ŽLJŽƵĚŝƐĐƵƐƐƚĞƐƟŶŐŝƐƐƵĞƐǁ ŝƚŚĐŽůůĞĂŐƵĞƐ D LJĐŽůůĞĂŐƵĞƐŽŌĞŶĐŽŶƐƵůƚŵĞŽŶƚĞƐƟŶŐŝƐƐƵĞƐ
Do you feel qualified to set tests? Yes very much so.
/ĨLJŽƵĚŽŶŽƚƐĞƚƐĞƚƐ ǁ ŚĂƚƚĞƐƟŶŐ-related work doyou do (advisory, marking, curriculum design …)?
/ƉƌĞƩLJŵƵĐŚĚŽŝƚĂůůƚĞƐƚƐĞƫ ŶŐĂĚǀ ŝƐŝŶŐŽŶƚĞƐƚƐ marking, examining etc.
ŚƌŝƐƟŶĞƚŚĂŶŬLJŽƵǀ ĞƌLJŵƵĐŚ
Email: [email protected]
40
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 40
Where are you based?I´m based in Brasília, the capital of Brazil, right in thecentre of the country.
How long have you been there?For almost five years.
Which other countries have you worked in?I´ve worked only in Brazil.
How long have you been a member of IATEFL?For about 4 years I think.
Why did you choose TEASIG as your SIG?ĞĐĂƵƐĞ/ƚŚŝŶŬƚĞƐƟŶŐĞǀ ĂůƵĂƟŽŶĂŶĚĂƐƐĞƐƐŵĞŶƚare crucial issues for any language teacher to knowabout.
Are you a member of any other SIGs?No, I´m not.
What do you get out of TEASIG?The most important thing, in my opinion, is thatTEASIG allows me to be in contact with people thatshare the same interests as mine – this allows me tolearn from other people, exchange ideas with peo-ple from different parts of the world and find solu-ƟŽŶƐƚŽĐŽŵŵŽŶƉƌŽďůĞŵƐǁ ĞŵŝŐŚƚŚĂǀ ĞŝŶŽƵƌdaily work.
t ŚĂƚŽƚŚĞƌƐĞƌǀ ŝĐĞƐŽƌĂĐƟǀ ŝƟĞƐǁ ŽƵůĚLJŽƵǁ ĞůͲcome?As I don´t live in Europe, I would very much welcomeŚĂǀ ŝŶŐŵŽƌĞŝŶĨŽƌŵĂƟŽŶŽŶǁ ŚĂƚǁ ĂƐƉƌĞƐĞŶƚĞĚand discussed in the events promoted by thed^/' ƉĂƌƟĐƵůĂƌůLJƚŚĞŽŶĞ-day ones!
What sort of work do you do?I am an English language teacher educator and Iteach both undergraduate (future English teachers)and MA students (most of them teachers working inpublic schools).
, Žǁ ŵƵĐŚŽĨLJŽƵƌǁ ŽƌŬƟŵĞŝƐƐƉĞŶƚŽŶƚĞƐƟŶŐ
Ğǀ ĂůƵĂƟŽŶĂŶĚĂƐͲsessment?A lot, not only as ateacher but also as aresearcher and as anMA tutor.
Do you see yourselfas more of a teacheror as more of a test-er?As a teacher, for sure.
Do you work in a team or alone?Mostly alone, unfortunately.
Do you set tests?Yes, I do.
What sort of tests are they?D ŽƐƚŽĨƚŚĞƟŵĞƚŚĞLJĂƌĞĐůĂƐƐƌŽŽŵƚĞƐƚƐ ďƵƚ/ĂŵĂůƐŽƉĂƌƚŽĨƚŚĞĐŽŵŵŝƩ ĞĞƚŚĂƚƐĞƚƐĞŶƚƌĂŶĐĞĞdžĂŵƐfor the post-ŐƌĂĚƵĂƟŽŶƉƌŽŐƌĂŵŵĞŝŶƉƉůŝĞĚ>ŝŶͲŐƵŝƐƟĐƐŽĨŵLJĚĞƉĂƌƚŵĞŶƚ
How high are the stakes for the test-takers?They are high states because they are related to thestudent´s progress at university – in the case of theĐůĂƐƐƌŽŽŵƚĞƐƚƐ ĂŶĚƚŚĞƐƵĐĐĞƐƐŽƌŶŽƚŝŶƐƚĂƌƟŶŐĂŶMA.
Are the tests you set published in any form?No, they´re for our own use.
ŽLJŽƵĞŶũŽLJƐĞƫ ŶŐƚĞƐƚƐ Yes, a lot!
Can you use the same set of skills and principles inseveral different contexts? Yes, sure, as long as you know the basic principles ofassessment.
ŽLJŽƵĐĂƌƌLJŽƵƚƌĞƐĞĂƌĐŚŽŶƚĞƐƟŶŐĂŶĚƌĞůĂƚĞĚfields?
TEASIG Member Spotlight: Gladys Quevedo Camargo.
41
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 41
Yes. At the moment I´m carrying out some researchon assessment literacy in Brazil for my post-docstudies.
, Ăǀ ĞLJŽƵƉƵďůŝƐŚĞĚŽŶƚĞƐƟŶŐŝƐƐƵĞƐ zĞƐ /Dzǀ ĞŐŽƚƐŽŵĞĂƌƟĐůĞƐƉƵďůŝƐŚĞĚŝŶƌĂnjŝůĂŶĚone abroad.
, Ăǀ ĞLJŽƵƐƉŽŬĞŶĂƚĐŽŶĨĞƌĞŶĐĞƐŽŶƚĞƐƟŶŐŝƐƐƵĞƐ Yes, I have.
ŽLJŽƵĚŝƐĐƵƐƐƚĞƐƟŶŐŝƐƐƵĞƐǁ ŝƚŚĐŽůůĞĂŐƵĞƐ Yes, a lot and whenever I have the opportunity!
Do you feel qualified to set tests? Yes, but assessment is a very broad and complexarea, so there´s always something to learn.
Gladys, thank you very much!
Email: [email protected]
.
42
TEASIG—EĞǁ ƐůĞƩ Ğƌ
Issue 63 42
ŽƉLJƌŝŐŚƚE ŽƟĐĞ
/d&>ƌĞƚĂŝŶƐƚŚĞƌŝŐŚƚƚŽƌĞƉƌŽĚƵĐĞƉĂƌƚŽƌĂůůŽĨƚŚŝƐƉƵďůŝĐĂƟŽŶŝŶŽƚŚĞƌƉƵďůŝĐĂƟŽŶƐ ŝŶĐůƵĚŝŶŐƌĞƚĂŝůĞĚŝƟŽŶƐŽŶƚƌŝďƵƟŽŶƐƚŽƚŚŝƐƉƵďůŝĐĂƟŽŶƌĞŵĂŝŶƚŚĞŝŶƚĞůůĞĐƚƵĂůƉƌŽƉĞƌƚLJŽĨƚŚĞĂƵƚŚŽƌƐ ŶLJrequests toƌĞƉƌŽĚƵĐĞĂƉĂƌƟĐƵůĂƌĂƌƟĐůĞƐŚŽƵůĚďĞƐĞŶƚƚŽƚŚĞƌĞůĞǀ ĂŶƚĐŽŶƚƌŝďƵƚŽƌĂŶĚŶŽƚ/d&>ƌƟĐůĞƐǁ ŚŝĐŚŚĂǀ ĞĮ ƌƐƚĂƉƉĞĂƌĞĚŝŶ/d&>ƉƵďůŝĐĂƟŽŶƐŵƵƐƚĂĐŬŶŽǁ ůĞĚŐĞƚŚĞ/d&>ƉƵďůŝĐĂƟŽŶĂƐƚŚĞŽƌŝŐŝŶĂůƐŽƵƌĐĞŽĨƚŚĞĂƌƟĐůĞŝĨƌĞƉƌŝŶƚĞĚĞůƐĞǁ ŚĞƌĞ++++++++++++++++++++++++++++++++++++++++++++++++++DisclaimerViews expressed in this IATEFL—those of the editor(s), of IATEFL TEASIG, of IATEFL or its staff or trustees.
Who is IATEFL for?
IATEFL is truly international with the majority of our members living in countries whereEnglish is not the first language. Members contribute greatly to the life of the Associationin a wide variety of ways: through participation in our 16 Special Interest Groups (SIGs);contributions to any of our magazines: IATEFL Voices and SIG publications; volunteeringon IATEFL committees; communicating developmental ideas to Head Office and giving pa-pers at, or simply attending our many conferences, workshops and seminars. The Associa-tion is managed and administered from an office in Faversham, UK. The nine remuneratedstaff are supported by volunteer committees, including the IATEFL Trustees and the Spe-cial Interest Group committees consisting of ELT professionals from a range of countries.
The aims of IATEFL are to:
· Benefit English language teachers all over the world providing opportunities forpersonal and professional development.
· Enable the international network of ELT professionals to grow, by encouraging andfostering the regional and local groupings, so that members can learn from eachother.
· Encourage grassroots professionalism where all categories of members at whateverstage of their career can make significant contributions and continue to learn.
Linking, developing & supporting ELT professionals worldwideIATEFL Patron: Professor David Crystal
Newsletter Issue 63 (December 2017) are not necessarily