what is 'eap speaking', exactly? - language assessment · for good practice in language...

LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACTLIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACTLIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACTCopyright University of Reading

WHAT IS 'EAP SPEAKING', EXACTLY?

Revisiting an EAP Speaking test, the ‘TEEP Speaking’

Bruce Howell, University of Reading

EALTA 2016

1

INTERNATIONAL STUDY AND LANGUAGE INSTITUTE

LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT

EAP SPEAKING MEANS…

•presentations

•seminar discussions

•tutorial discussions(maybe)

Conference theme

Assessment of what....? - Revisiting the issue of construct(s)2


TEEP SPEAKING IS …

•not ‘TEAP’!

3


TEEP SPEAKING

•TEEP = Test of English for Educational Purposes

• test of EAP (English for Academic Purposes)

• ‘exit test’ on pre-sessional EAP courses at University of Reading – i.e. ‘entrance test’ for degree study

•1000~1500 candidates per year (incl. some external)

• a new Speaking component introduced in 2010

• two-phase structure:

• monologue

• dialogue

4


TEEP SPEAKING - FORMAT

•25-minute timeslot (23 minutes + 2 mins’ turnaround)

•2 candidates, interlocutor, assessor

• scripted, timed

• recorded

5


6

PLANNING TIME: 1 MINUTE

phrased simply, to avoid construct irrelevance


7

PLANNING TIME: 4 MINUTES + MONOLOGUE: 3 MINUTES EACH

originally the instructions asked for a “presentation”


8

PLANNING TIME: 2 MINUTES + DISCUSSION: 4 MINUTES

representative of a ‘seminar’ (?)


9

DISCUSSION : 2 MINUTES

more of an open ‘intellectual discussion’we hope students normally have such discussions!!


TEEP SPEAKING – STAGES

10

stage of test what happens what EAP construct?

1focus question + roles (planning time)

1 minute to think about the general topic + 4 mins to plan own topic

study skills – preparing, making notes

2each candidate’s talk(monologue)

3 minutes each to introduce viewpoints, as a role play

simulated ‘presentation’

3scenario(planning time)

2 minutes to understand a specific situation

study skills – preparing, making notes, combining ideas

4discussion about scenario (dialogue)

4 minutes for an analysis and discussion

simulated ‘seminar’

5more general discussion (dialogue)

return to focus question: 2 minutes to discuss the ‘bigger picture’ question

simulated ‘intellectual discussion’


TEEP SPEAKING – VALUES• test created to operationalise values underpinning

existing (very established!) teaching and materials

• fundamental aim = replicate the experiences of speaking as a student in English

• test should be based on teaching materials/syllabus

• topic-based approach – suitably challenging and motivating 'academic' topics

•∴ planning time essential

• to feature both ‘explaining’ and ‘discussing’

• dyad pairing for genuine interaction; management of speaking considered vital (e.g. initiating, clarifying)11


TEEP SPEAKING – GRADING… also linked to values

Global criteria, given by both interlocutor and assessor:

•Explaining ideas and information

• Interaction

Analytical criteria, given by assessor only:

•Fluency

•Accuracy, range

• Intelligibility

Weighting = global 50% / analytical 50% 12

apparently tending towards representing ‘general’ proficiency

performance in specific tasks - more ‘EAP-like’


TEEP SPEAKING = AN ‘EAP SPEAKING TEST’•do you what ‘EAP Speaking’ is?

•does your local university have a definition?

• in European universities, more course using EMI (Colman 2012; Dimova et al 2015)

• so 'EAP Speaking' has increasing importance in Europe (and elsewhere)

(right?)

I have to be satisfied my EAP Speaking test really is ‘EAP’ 13


QUESTIONS TO ASK OURSELVESeasy ones:

•Are we happy that our test matches our syllabus?

•Are we happy that our test is ‘EAP’?

slightly more difficult questions (from ‘EALTA Guidelines

for Good Practice in Language testing and Assessment’):

•How clearly is/are the test purpose(s) specified?

• Is there a description of the test taker?

•Are the constructs intended to underlie the test/subtest(s) specified?

14


REVISITING THE CONSTRUCT DEFINITIONBachman & Palmer (2010:43):

“… we can define the construct from a number of perspectives, including everything from the content of a particular part of a language course to a needs analysis of the components of language ability that may be required to perform language use tasks in the target language domain, to a theoretical model of language ability.”

15


SO, WE’VE GOT SOME WORK TO DO•Frameworks such as Weir's (2005) define ‘Speaking’

•but where “… tasks … are designed to collect evidence in a small sub-set of contexts … test purpose should drive the definition of the construct” (Fulcher 2003)

•So, what aspects are relevant to ‘EAP Speaking’?? What does it include??

Douglas (1997:1): “ … international students … need to be able to speak proficiently to accomplish many tasks and purposes. First, they need basic interpersonal communication skills (BICS) to get around campus, buy books and materials in the bookstore, get meals in cafeterias and restaurants, ask a friend about a homework assignment … “

16


PART OF EAP SPEAKING?

17


ONE WAY OF LOOKING AT IT: BALEAP CAN-DO FRAMEWORK

18


BALEAP CAN-DO FRAMEWORK (SPEAKING)

19

• examples of the 50+ descriptors:

• S1.3.1 Demonstrate high level presentation skills

• S1.2.1 Take part in group discussion (and socializing)

• S1.2.4 Debate and communicate/share thoughts or feelings

• S1.2.3 Engage fully in discussion rather than providing superficial contribution

• S2.1.1 Ask for clarification

• S1.1.5 Take part in group work analysing and solving problems

• S1.1.2 Work independently as well as collaboratively

• S1.1.1 Demonstrate critical thinking

• S1.1.6 Tell other people when they are wrong

• S1.4.5 Engage in ‘Socration’ dialogue with supervisors

• S1.4.8 Challenge a lecturer

• S1.1.8 Introduce cultural insights

that’s interesting!

‘academic culture’ – part of EAP (?)


MORE QUESTIONS(NOT MANY ANSWERS YET)

• is our ‘ingrained’ definition of EAP speaking valid?

•have things changed?

•how many relevant sub-skills does TEEP Speaking already feature?

• are any omissions justifiable for practical reasons?

•we need to defining the ‘academic quality’ of topics

•how to express scores or profiles

• “the overall score is meaningless” (O’Sullivan 2012)

•have to extract the ‘EAP’ elements from CEFR, in order to make an equivalence claim 20


REFERENCES• Bachman, L. and Palmer, A. (2010) Language Assessment in Practice.

Oxford: Oxford University Press.

• Colman, J. (2012) in Doiz, A., Lasagabaster, D., Sierra, J. M. (Eds.) English-Medium Instruction at Universities: Global Challenges. Multilingual Matters

• Douglas, D. (1997) Testing Speaking Ability in Academic Contexts: Theoretical Considerations TOEFL Report MS-08 ETS available at https://www.ets.org/research/policy_research_reports/publications/report/1997/icia

• Dimova, S. , Hultgren, A. and Jensen, C. (Eds.) (2015) English-Medium Instruction in European Higher Education. English in Europe, Volume 3. Berlin, Boston: De Gruyter Mouton.

• EALTA (2006) EALTA Guidelines for Good Practice in Language testing and Assessment available at http://www.ealta.eu.org/guidelines.htm

• Fulcher, G. (2003). Testing Second Language Speaking London: Longman/Pearson Education.

• Weir, C. J. (2005) Language Testing and Validation: An evidence-based approach Houndgrave: Palgrave-Macmillan 21

https://www.ets.org/research/policy_research_reports/publications/report/1997/icia

http://www.ealta.eu.org/guidelines.htm

what is 'eap speaking', exactly? - language assessment · for good practice in language...

Documents