eaquals making cefr testsv3 - [(•)] square eye | easy...

The European Association for Quality Language Services

© EAQUALS: The European Association for Quality Language Services: Email: [email protected] Internet: www.eaquals.org

CEFR Tests

A simplified guide for EAQUALS Members on

selecting and developing CEFR progress tests


© EAQUALS: The European Association for Quality Language Services:

EAQUALS SECRETARIAT: P.O. Box 95, Budapest, H-1301 Hungary Email: [email protected] Internet: www.eaquals.org

Registered Office: Via Torrebianca 18 34132 Trieste, Italy

2

Contents Introduction Page 1 Rationale Page 3 Case Studies

Avo-Bell School of English, Sophia Page 9 Bell Krakow Page 17 Yüce School, Ankara 5 Tom and Emma School of English Page 34





1

Introduction This paper is a guide to help put together progress tests that support teacher continuous assessment in a language course. The EAQUALS assessment model is based upon teacher assessment in a CEFR-related curriculum. Naturally a proficiency assessment approach based on teacher assessment requires standardisation (training in what the levels mean, how to run the assessments) and moderation techniques (steps to limit subjectivity, adjust for excessive severity/lenience) in order to ensure an element of comparability. Such techniques are the subject of a parallel paper; they include the use of progress test results to confirm teacher impressions about the level of the group and individuals in the group. It is the selection or development of such tests for such a supportive role with which this paper is concerned. This paper is not a guide for developing examinations or for relating the results from existing tests or examinations to the CEFR by determining “cut-scores” that define a minimum performance for a particular level. A manual is provided by the Council of Europe for that

purpose1.

In order to give information about a learner’s level of proficiency the test needs to: • have a broad coverage (not the grammar and vocabulary done this month bur the grammar

and vocabulary most important at, for example, Level B1); • assess communicative ability (knowing how to use a language) as well as language

knowledge, or be balanced by a second assessment procedure that does that; • be appropriate to the context and type of learners concerned Assessing language level in relation to the CEFR does not usually mean throwing away all existing tests and starting from scratch. In addition, course books and examinations increasingly make explicit reference to CEFR “Can Do” descriptors in order to relate pedagogic aims to real-world objectives. Thus it may well be that suitable material for progress tests is at hand without the need for development from scratch. However, even in such a case, the material needs to be selected and possibly the test tasks associated with it may need to be adapted (e.g. True/False should never be used in a test as the learner has a 50% chance of a correct guess). What is necessary to set up a set of progress tests is a conscious review of available material and, if necessary, development of aspects that are lacking. The school needs to confirm that:

(a) the curriculum and syllabus reflect the CEFR level concerned - through the CEFR illustrative descriptors, reference works for the language concerned and objectives of examinations known to be related to the level concerned

1 Council of Europe (2009) “Relating Language Examinations to the Common European Framework of Reference for Languages:

Learning, Teaching, Assessment (CEFR): A Manual.” Available on www.coe.int/lang. North, B. and Jones J. (2009) “Relating Language Examinations to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR): Further material on standard setting through scaling and teacher judgement.” Available on www.coe.int/lang.





2

(b) the assessment tasks, texts and items are appropriate to the CEFR level concerned (c) the assessment tasks, texts and items reflect (i) the kinds of real-world communicative

tasks referred to or implied in the illustrative descriptors and/or (ii) the most important language points and the specific language skills required in order to be able to complete

such real-world tasks2.

Tips There are no golden rules to follow but the following axioms may be useful: - Start from your curriculum. - Exploit what is available in good course books that are well-targeted at the level. - Refer to CEFR descriptors for the skill(s) concerned for the level concerned:

o the 2008 edition edited by EAQUALS “Can-Do” group which are on the EAQUALS website – they define 11 levels (“plus levels” for all but C2);

o the summary of salient features for Reception (Listening and Reading) given in the next section.

- Don’t throw away previous tests. use the CEFR “Can Do” descriptors to review them and check whether they do test relevant communicative objectives.

- Look at what other people do (for example the case studies in this paper). - Be practical. Keep the time needed for the tests short: any time for testing is time taken

away from teaching and clients do not like too much of it. At least consider assessing speaking in class time through structured communicative tasks.

- Define and check implementation of procedures to ensure that everyone sets the same test conditions (time allowed, dictionaries allowed? play the tape once? twice?) and marks the tests same way.

- Ensure teachers involved have had CEFR standardisation training with the EAQUALS standardisation pack and the documented CEFR DVDs for English, French, German or Italian.

Brian North, Zürich, March 2009

2 At the C levels the emphasis switches from tasks to text:: appreciating the subtleties in comprehension

texts; being able to produce coherent, well-structured, appropriate text. To some extent this switch starts already at B2.





3

Rationale 1. Validity & Reliability:

Most people are familiar with the traditional expressions “validity” and “reliability.” First, however, there is a third, equally important principle: feasibility. Is the assessment practical (media, length, available expertise, teacher time)? There is no point having a wonderful test if it takes all day to administer and requires three people to moderate it.

In our context, the essential issues with validity and reliability can be summarised as:

Validity

> Are we actually assessing what we intend to assess? In this case are we testing language proficiency as described in the CEFR and its illustrative descriptors? (Or are we just testing something we find easier to test?)

> Does the test reflect curriculum content taught? (Or is the content arbitrary and designed to catch the learners out?)

> Are the effects of the test (a) good for learning, (b) ethical (Or does it damage classroom practice and corrupt relations between teacher and students into a conspiracy to get high scores, irrespective of practical language proficiency?)

> Do the results give “real world meaning” that is appropriate to the context concerned? (Or is it impossible to validly generalise to ability in the real world from results on this test?)

Reliability > Do the good people get the high scores? > Are results consistent (a) each time the test is used with different people (b) with

other tests – if those are taken as well by the same people? > Which taken together mean: are the results fair, stable and trustworthy? Twenty or thirty years ago one used to talk about a “tension” between validity and reliability. Nowadays, language testers are aware that a great part of any “reliability” can in fact be a “method effect.” A good, purely multiple choice, test will tend to be more reliable than a good test employing 5 items types (to sample skills and to avoid bias) since there are simply fewer things going on in the data; sampling is narrowed to something very controllable. However, apart from the fact that a lot of the important messiness of real life is excluded from a multiple choice test, in a purely multiple choice test, a good part of what is being assessed is the ability to do multiple choice questions (as opposed to “language proficiency”. One can produce a reliable test that is not valid (because it tests the wrong things); one cannot produce a valid test that is not reliable (because if you can’t rely on it how can it be valid? Consequently is now seen as an aspect of validity. If a test is reliable it will have “concurrent validity” meaning that results will correlate with those of another validated test.





4

2. Specification & Standardisation: Another way to look at validity and reliability is to focus on specification and standardization. The CEFR promotes educational standards in the sense of common reference points (Levels A1-C2). “Standards” with regard to assessment has two meanings – in addition to the concept of procedural standards: doing things properly. The common reference points involve “content standards” (input: what is to be assessed) and “performance standards” (output: what result is expected?). The former, content standards, is primarily concerned with specification; the latter, performance standards, is mainly concerned with standardisation.

2.1. Specification (Conceptual Validation)

> Are we actually assessing what we intend to assess? In this case are we testing language proficiency as described in the CEFR and its illustrative descriptors?

> Does the test reflect curriculum content taught? > Do the results give “real world meaning” that is appropriate to the context

concerned?

� Content standards: activities appropriate to and representative of an action-oriented curriculum = TASKS/ITEMS

> Select the communicative activities – e.g. types of listening activities > Design tasks – including sourcing texts > (Receptive): Write items – ensuring balance of item types > Define the marking scheme

� Performance standards: for productive skills: qualitative aspects of

language appropriate to task, context and learners = GENERIC CRITERIA > Use the EAQUALS CEFR/ELP descriptors produced by the “Can Do”

SIP and/or CEFR Table 3 (in EAQUALS Standardisation Pack) to select 3-5 qualitative categories appropriate to the context, for example:

> Spoken: Range, Accuracy, Fluency > Written: Range, Accuracy, Coherence

> Productive: Create a criteria grid focused on the relevant levels only, with a descriptor for each cell of the grid (each category at each level); Insert “mini-levels” in between the defined levels if desired, without defining these.

> Receptive: Define the marking scheme.

Specification cannot be separated from the issue of sampling. One needs a sample “appropriate to and representative of” an action-oriented curriculum. A test should sample different things, for example: > Speaking: Spoken production: a monologue, with follow up questions afterwards

Spoken Interaction: a spontaneous discussion, prompted in some way > Writing: Formal prose: description, essay, report, letter (= Written Production)

Written-spoken language: email, SMS, note (= Written Interaction) > Listening: 2-4 short pieces: identify the topic/main information / speaker attitude)

1-2 long pieces: detailed comprehension > Reading: 2-4 short artefacts: identify topic/main information / speaker attitude)

1-2 long texts: detailed comprehension





5

Another issue to consider is the length of the tests and the number of marks. The longer the test is the more reliable it will be – but the less practical it is. This is because a longer test has more points (marks) available to separate the stronger from the weaker:

> part of reliability is dependent on the ability of the marks to separate out the strong from the weak, therefore:

> standard error of measurement (SEM) is directly related to the number of marks; on a test with 36 items SEM will always be around 2 marks even if the test is “perfect.”

Try and ensure any test has 30-40 marks. Consider using a little scale (like 0-1-2-3) for each item if necessary to get more marks from fewer questions. The marking scale must be defined and you should provide an example. The scale could be something like: 0 wrong 1 an attempt - partly right 2 good, but something is still missing or wrong

3 model answer

Tips on Listening and Reading Tests

Do

> Test skills > Use different text types > Have several short texts

and one or two longer ones

> Write a specification:

text types, item types > Use only one or two

item types for one text > Ensure any test has 36-

40 marks

Don’t > Test memory (listening) > Use only one text > Choose texts according to your

own personal tastes

> Ask nasty questions to try and catch people out

> Use only one item type for the

whole test > Set open questions and take off

marks for spelling/morphology; test that separately

2.2. Standardisation (Optimising Reliability)

> Are the results fair, stable and trustworthy? > Is the assessment comparable?

� Procedures and instructions

Is the assessment implemented in the same way? > students clearly informed what to expect (incl. criteria) > teachers and students given unambiguous instructions > tests include an example item > teachers agree procedures to ensure comparable conditions





6

> play tape one? twice? > time allowed > preparation time? checking time? > help allowed (dictionaries?)

> teachers use same marking tool (not negative marking) > (some) double marking for speaking/writing

� Standardisation Training (See EAQUALS Standardisation pack)

Is performance interpreted in the same way? > illustration with documented illustrative samples (DVD, scripts): “This

is level B1, because ….” > small group discussion of other documented illustrative samples:

“What level do you think this is?” > individual rating of such samples, followed by small group, followed

by plenary > (optional) individual/group and then plenary discussion of some

samples collected from your students.

Brian North, Zürich, March 2009

CEFR Salient Elements: Reception




7

Setting Action What is understood Source Restrictions • Follow, maybe

with a little difficulty

• Films with a considerable degree of slang and idiomatic usage

• Poor quality, audially distorted public announcements

C1 • Abstract and complex topics encountered in social, academic and professional life, whether or not hey relate to own field/speciality

• Understand • Finer points of detail • Implied as well as stated

opinions • A wide range of idiomatic

expressions and colloquialisms • Register shifts • Implied attitudes and

relationships

• Lengthy, complex texts of various kinds • Extended speech – lectures, discussions, debates - even

when not clearly structured • Complex interactions between third parties in interaction and

debate • A wide range of recorded and broadcast texts, including

some non-standard • Any correspondence

May occasionally need to: • confirm details (with dictionary, from

speaker) if outside field • re-read difficult sections

• Follow, maybe with a little difficulty

• • Animated conversation between native-speakers B2+ • A wide range of familiar and unfamiliar topics encountered in social, academic and professional life

• Understand • • Spoken language, live of broadcast • Specialised texts (highly specialised if within field)

• Standard, non-idiomatic: • Adequate discourse structure • Low background noise • May occasionally need to confirm details

(with dictionary, from speaker) o if outside field o if above conditions not met

• Follow, maybe with a little difficulty

• Much of what is said • Discussion around him/her by native speakers

• Scan quickly • Relevance • Whether closer study is

worthwhile • Specific details

• Long and complex texts • News items, articles and reports

B2 • Reasonably familiar concrete and abstract topics related to field of interest/speciality

• Understand (with a large degree of independence)

• Main ideas • Essentials / essential meaning • Complex lines of argument • Speaker/writer mood, tone etc.

• Extended speech: lectures, talks, presentations, reports, discussions

• Propositionally and linguistically complex text • Technical discussions; lengthy, complex instructions; details

on conditions or warnings • Most TV and current affairs programmes • TV documentaries, interviews, talk shows, specialised sources • Announcements and messages • Most radio documentaries, recorded audio materials • Correspondence

• Standard • Clearly signposted/signalled with explicit

markers • If native speakers talking together modify

language • If can re-read difficult sections

• Follow, though not necessarily in detail

• Line of argument in treatment of the issue

• Argumentative text

• Scan • Desired information • Longer texts • Different texts, different parts of a text

B1+ • Common everyday or job-related topics

• Topics in his field of (personal) interest • Understand • Straightforward factual

Information content • General message • Man conclusions • Specific details

• Argumentative text • Lectures and talks within own field • Large part of many TV programmes: interviews, short

lectures, news reports • Majority of recorded and broadcast audio material

• Standard – (Familiar accent) • Straightforward • Clearly signposted/signalled with explicit

markers

CEFR Salient Elements: Reception




8

Setting Action What is understood Source Restrictions • Follow, though

not necessarily in detail

• Significant points • Extended discussion around him/her • Many films in which visuals and action carry much of the

story line • TV programmes: : interviews, short lectures, news reports • Straightforward newspaper articles

B1 • Familiar topics regularly encountered in a school, work or leisure context

• Topics in his field of (personal) interest

• Understand with satisfactory comprehension

• Main points • Relevant information

• Straightforward factual texts • Short narratives • Descriptions of events, feelings, wishes • Detailed directions • Short talks • Radio news bulletins and simpler recorded materials • Everyday written materials: letters, brochures, short official

documents • Simple technical information e.g. operating instructions

• Clear • Standard • Straightforward • Relatively slow

• Identify • Main point • TV news items reporting events, accidents etc in which visuals support the commentary

• Clearly and slowly articulated

A2+ • Familiar topics of a concrete type

• Understand enough to meet needs

• (Main point) • Basic types of standard letters, faxes (enquiries, orders, confirmations)

• Short texts with simpler, high frequency everyday and job-related language

• Regulations, e.g. safety

• Expressed in simple language

• Identify • Specific, predictable information

• Topic of discussion • Changes of topic • An idea of the content

• Simpler everyday material: advertisements, menus, reference lists, timetables, brochures, letters

• Discussion around him/her • Short newspaper articles describing events • Factual TV news items

A2 • Predictable everyday matters

• Areas of most immediate priority: basic personal, family, shopping, local area, employment • Understand • Main point

• Essential information • Short simple texts containing the highest frequency

vocabulary including a proportion of shared international vocabulary items

• Simple directions relating to how to get from A to B • Simple clear messages, announcements, recorded passages • Simple instructions on equipment encountered in everyday

life (e.g. telephone) • Short simple personal letters • Everyday signs and notices: directions, instructions, hazards

• Clearly and slowly articulated

• Identify • Familiar words, phrases, names

• An idea of the content

• Simple notices • Simpler informational material

A1 • The most common everyday situations

• Understand • (Main point) • Very short simple texts with visual support, a single phrase at a time: o messages on postcards o directions o descriptions

• Very slow, carefully articulated, with long pauses to allow assimilation of meaning

• Familiar names, words and basic phrases • A chance to re-read / get repetition





9

Case Study: AVO-Bell School of English, Sophia

1. The Context

The school offers extensive courses to Young Learners, teenagers and adults. There is a wide range of courses offered to our clients: General English, English for Communication, Business English, Exam preparation, as well as in-company tailor-made courses. Several years ago the school launched a Testing Project, which originally aimed at raising teachers’ awareness of the basic concepts about testing, and some key principles of test production and development. As a result, final tests for the General English courses were developed, which were based on a curriculum, which was not CEFR-bound yet. Still, the project value lay in the fact that teachers became competent about: item banking, reliability and validity, item writing, task types / format requirements, washback effect, test specifications and relation of tests to the course syllabus. All these set the basis for further improvement of tests’ contents and format. The other new trend, which influenced the development of the school tests, was the introduction of the CEFR and the development of course syllabi in line with these (See CEFR Curriculum Case Study). We also wrote Can-do-statements for each level (and sublevel), which became a point of reference for the progress and final tests’ development. All these led to the necessity of introducing skills tests alongside with the existing Language systems tests (which were mainly in multiple-choice format). At present, the underlying principle of the school curriculum is an eclectic approach to teaching, which extracts the essence of different teaching methods and approaches. Still, the core of it is the practice-oriented background of the CEFR. This is meant to be reflected in the course syllabuses, hence in the progress and final tests. This, subsequently, poses many other questions concerning: resources for testing materials, tests’ length and format, components and reliability / validity of our progress tests. 2. The Progress Tests I will briefly describe the progress test we are using at present for the General English Courses. We test the four skills: Reading, Writing, Speaking and Listening when the course is one third through, and the second progress test is a Language Systems one, which tests key syllabus items regarding grammar and vocabulary. The format of the tests is similar to the course book types of tasks in order to reduce students’ anxiety and not to impede their performance. The Reading / Listening tests contain between 6 and 10 items, preferably two task types per skill, and the Writing task is usually similar to what students would normally do in class. No particular Speaking tasks are produced for assessment purposes. What teachers do is conducting a routine speaking activity in class, and assessing it





10

in the end, using the marking criteria for the level. The Language Systems test contains between 40 and 50 items. Each of the above components carries equal weight, and scores are calculated in percentage for each skill. 3. Development of the Tests The resources we used for the Skills tests’ development were mainly course book tasks, or other relevant testing materials from different course books, e.g. New Opportunities, Longman Pearson, Face to Face series, FCE and CAE preparation course books, etc. We have also adapted some reading tasks from the past papers of Anglia Examinations, which relate topically to our General English syllabuses. We have also extracted material from the existing progress tests, which go along with the books we are currently using, as a basis for creation of the Language systems tests, but we have somewhat redrafted and reduced the items numbers for the sake of economy of time. Writing and Speaking tasks are more loosely related to the Can-do-descriptors for the level, but what’s more valuable is the fact that we have developed marking criteria for the productive skills, relating them to the descriptors for the six general levels of CEFR. These tests were piloted in a smaller number of groups first, and teachers were invited to provide feedback in terms of tests’ contents and reliability. They participate actively in the process of editing and improving them at end-of-course academic meetings, as well as at any time when this is pertinent. 4. Conclusions We have learned a lot of lessons from the tests development, and we have also asked ourselves a lot of questions:

- What makes a reliable reading / listening task and how can one guarantee that it matches the Can-do-statements (and description) for the level? The same applies, of course, to speaking and writing tasks.

- How do we know that the task in question measures the progress made until that particular point of the course, when the test is administered?

- What makes a reliable resource for testing material? - How can we ensure that test results are valid, if we administer a short task with one

task type? - What should be the relationship between the progress tests’ contents and the final test’s

one, i.e. should these overlap, or on the contrary, should they be testing different items of the syllabus? Hence, how can we claim for consistency of the progress and final test results?

Dos about progress test development:

- Involve teachers actively in the process of test evaluation and improvement, especially the more experienced ones, as they have an innate feeling for the students ‘ “level”;

- Pilot the tests with smaller groups of students to avoid negative incidents in a large way; - Review test results and try to analyse them in order to decide on the test reliability /

validity;





11

- Try to match topically (and structurally) the skills tasks to the course syllabus; - Try to stick to all ‘rules’ concerning test and item writing; - Conduct standardization sessions regularly, especially about assessment of speaking and

writing skills; - Observe the copyright law; make the appropriate acknowledgements on your tests;

Don’ts about progress test development:

- Do not use only one task type per skill, e.g. only True / False questions, gap filling exercises, etc.

- Do not digress into testing syllabus items which are of secondary importance to the level;

- Do not make tests too long / too short; - Do not rely on inexperienced teachers / teachers without proper training about test

writing – they are more likely to fall in the trap of inadequate testing, or testing for its own sake;

- Do not take your tests for granted – both language and learners undergo changes; - Do not try to write tests yourselves especially if you can adapt and select testing

materials from reliable resources; - Do not expect to produce a universal (or perfect) test;

Mila Angelova AVO-Bell School of English, Sofia March 2009





12

Progress test 1 – General English – A 2

AVO-Bell School of English

Name:……………………………………………………………………………….

LISTENING Task 1 Listen to the radio commercial and decide whether the statements below are true (T) or false (F). 1. Hatton Hotels are located in the city centre. T / F 2. Hatton restaurants serve only fish and vegetarian dishes. T / F 3. All bedrooms are equipped with cable television. T / F 4. A weekend tour of the surrounding area costs ₤5. T / F 5. On departure day every hotel guest gets a souvenir. T / F Task 2 Listen again. Read the questions and circle the correct answers: a, b or c.

1. How many Hatton Hotels are there all over the country? a) less than 40 hotels b) 40 hotels c) more than 40

hotels 2. In every room in Hatton Hotels is

a) a fridge b) a direct telephone line c) a teapot and tea cups

3. Every Hatton Cocktail Bar serves a) classic steaks and salads b) light meals and drinks c) fish and vegetarian

dishes 4. The special weekend tour of the area lasts for

a) 2 hours b) 3 hours c) 4 hours 5. The Hatton Hotel free phone number is

a) 0 800 768 563 b) 0 800 786 563 c) 0 800 768 536

Total ……/ 10 marks

WRITING

Task: This is part of an e-mail you receive from a classmate or English-speaking friend.

It’s Duncan’s birthday tomorrow, so we’re going for a walk tomorrow morning, and then we’re going to Café Bleu for lunch. Would you like to come?

Write an e-mail to your friend, explaining why you can’t come, and suggesting an arrangement for another day. Write about 50 - 70 words.





13

READING

I. Read the passage and choose the correct answer:

Example: People in China a) are enormous. b) eat a lot of rice. c) greеt each other with rice. d) need a lot of water.

1. Most of the world’s rice is grown

a) in Australia. b) in Asia. c) on hills. d) in Japan.

2. Asian countries a) always produce enough rice. b) don’t have any flat land. c) produce more potatoes than rice. d) sometimes have to import rice.

3. Rice is planted a) by machine in Asia. b) by hand and picked by machine. c) by hand in the United States. d) by hand in most Asian countries.

4. North Americans a) don’t like potatoes. b) eat more bread than rice. c) don’t eat rice. d) prefer rice from China.

5. Rice was grown in Egypt a) before it was grown in China. b) after it was grown in Italy. c) before it was grown in Europe. d) after it was grown in the USA.

RICE

Rice is one of the most important foods in the world. This is because people who live in China, Japan and India and other parts of Asia, live mainly on rice. In China the word “rice” is used in one of their greetings. People say, “Have you eaten your rice today?” This is because rice is so important to the Chinese. Although most of the world’s rice is produced in Asia, sometimes it has to be imported. This happens when rice doesn’t grow properly. If there is no rice harvest, people in Asia may die. Rice was first grown in China about five thousand years ago and it was then introduced into Egypt. Rice was first grown in Europe, in Italy, about six hundred years ago. It was not grown until three hundred years ago in the United States. North Americans eat some rice but they prefer bread and potatoes. Some rice is grown on hills but most rice is grown on flat land near lakes and rivers because rice needs a lot of water. In the rice-growing parts of the United States, rice is planted and picked by machine but in most Asian countries everything is done by hand.

……/5 marks





14

II. Read the text about Hollywood kids. Are these sentences True (T) or False (F).

HOLLYWOOD KIDS

In Hollywood, everybody wants to be rich, famous, and beautiful. Nobody wants to be old, unknown, and poor. For Hollywood kids, life can be difficult because they grow up in such an unreal atmosphere. Their parents are ambitious, and the children are part of the parents’ ambitions. Parents pay for extravagant parties, expensive cars, and designer clothes. When every dream can come true, kids learn the value of nothing because they have everything. A 13-year-old boy, Trent Maguire, has a driver, credit cards, and unlimited cash to do what he wants when he wants. “One day, I will earn more than my Dad,” he boasts. Parents buy care and attention for their children because they have no time to give it themselves. Amanda’s mother employs a personal trainer, a nutritionist, a bodyguard/chauffeur, a singing coach, and a counsellor to look after all her 15-year-old daughter’s needs. Often there is no parent at home most days, so children decide whether to make their own meals or go out to restaurants, when to watch television or do homework. They organize their own social lives. They play no childhood games. They become adults before they are ready. Hollywood has always been the city of dreams. The kids in L.A. live unreal lives where money, beauty, and pleasure are the only gods. Will children around the world soon start to think the same? Or do they already? 1. Everybody in Hollywood is rich and famous………………………………………………… 2. Hollywood kids don’t lead ordinary lives……………………………………………………. 3. They understand the value of what they have………………………………………………… 4. Trent Maguire is spoilt and ambitious………………………………………………………… 5. The adults try hard to be good parents………………………………………………………… 6. Amanda’s mother listens to all her daughter’s problems……………………………………… 7. The kids are often home alone…………………………………………………………………

……../7 marks

Total : ……../12 marks

Based on Language to Go, Pearson Longman Listening based on NEW Opportunities, Education for Life, Pre-Intermediate, Test book, Pearson Longman, 2008





15

Progress test 2– General English - A 2

AVO-Bell School of English

Name:……………………………………………………………………………….

1. Expand the notes to make new sentences using the correct tense.

1. We / rearrange / the office / yesterday.

2. John / just / move / the desk. Now / it / nearer / the window.

3. Tomorrow / we / buy / some plants.

4. We / not hang / the / new / pictures / yet.

5. Mary / already / do / all the filing.

________ / 5 points

2. Tick (√√√√) the correct sentences.

1 a) I’m so boring – there’s nothing to do! ________ b) I’m so bored – there’s nothing to do! ________ 2 a) I was amazed when I saw Pete. He looked so old. ________ b) I was amazing when I saw Pete. He looked so old. ________ 3 a) ‘I’ve got terrible toothache.’ ________ ‘Have you? I’m go to get you an aspirin.’ ________ b) ‘I’ve got terrible toothache.’ ________ ‘Have you? I’m going to get you an aspirin.’ ________ 4 a) I would marry you tomorrow if you asked me to. ________ b) I will marry you tomorrow if you asked me to. ________ 5 a) It’s shocking how many homeless people there are. ________ b) It’s shocked how many homeless people there are. ________ 6 a) Do you have to wear a uniform for your job? ________ b) Do you must wear a uniform for your job? ________ 7 a) I have given up smoking last week and I feel great. _______ b) I gave up smoking last week and I feel great. ________ 8 a) I’d like to be a waiter – you meet all sorts of interesting people. ________ b) I like be a waiter – you meet all sorts of interesting people. _______ 9 a) Hey! You don’t have to park here. It’s not allowed. ________ b) Hey! You can’t park here. It’s not allowed. ________ 10 a) I was asking a question when she interrupted me. ________ b) I am asking a question when she interrupted me. ________

________ / 10 points





16

3. Complete the second sentences so that they mean the same as the first sentences. Use the words in brackets.

1. They make these jackets in Turkey. These jackets _____________________________________________________ . (made)

2. It isn’t necessary for you to do the washing-up. You don’t _________________________________________________________ . (have)

3. ‘I’ve decided to go to Mexico on holiday.’ I ___________________________________________________________ . (planning)

4. I can’t play the drums. It would be nice to learn. I’d __________________________________________________________ . (like)

5. The car is clean. Peter has _____________________________________________________ . (wash)

________ / 5 points 4. Match the sentences to make conversations.

1. The phone’s ringing. a) OK, have a nice time. 2. I’ve just washed the car. b) Why? Are they poisonous? 3. Is it OK if I call Paul? c) No, I’m afraid it isn’t allowed. 4. What would you do? d) Yes, I’ve just sent it. 5. You mustn’t eat those mushrooms. e) Did you have a good journey? 6. I’m going to go for a swim. f) That’s fine. I’ll give you a lift. 7. Have you already written your e-mail? g) Don’t worry, I’ll answer it. 8. Can I smoke here? h) Yes, of course it is. 9. I went to New Jersey yesterday. i) I think I’d apologise. 10. I have to be home by midnight. j) Thanks, it looks very clean. 1________ 2________ 3_________ 4__________ 5__________ 6________ 7________ 8_________ 9__________ 10__________

________ / 10 points 5. Underline the correct options.

1 A: So, what’s your idea of the perfect / good job? B: One which gives me activity / responsibility. 2 A: Do you want a career / task in sales? B: Yes, I do. And I want to earn a good payment / salary. 3 A: Would you like to work for / with animals? B: No. I rather / would like to work in a big office. 4 A: What’ wrong? You look sick / sickening? B: I’ve got a very aching / sore throat. 5 A: Do you enjoy working lonely / alone? B: Yes, it’s quite interested / interesting in a way. 6 A: Have you turned / changed your mind about getting married? B: No, not at all. I just want you to tell the honesty / truth. 7 A: This play is the longest-running / longest-walking play on Broadway. B: Is it? I’m not surprised / surprising. It’s very good. 8 A: Have you got a cold? You’re puffing / blowing your nose a lot. B: No, I’ve got an allergic / allergy to flowers. They make me sneeze. 9 A: I hear that Chicago is full of street gangs / teams. B: No, it’s not! Who said / told you that? 10 A: It’s our wedding birthday / anniversary today. B: I know it is. I’ve bought you a real / true gold bracelet. ________ / 20 points Total:__________ / 50 points

Based on Language to Go, Pearson Longman





17

Case Study: Bell Krakow

1. The Context

Bell Krakow offers general English and examination courses, as well as courses tailored for individual needs of corporate or one-to-one clients. The courses are offered to both adult and young learner students starting at the age of six as well as to infants and very young children who are exposed to rather than taught English through music, art, and drama. The school promotes an eclectic approach to methodology where the main communicative approach is supplemented by an array of other methods and techniques. Practical examples, variety and context-oriented activities as well as grammar explanations and grammatical exercises are all used in the lessons. The syllabi for particular levels consist of communicative objectives specified in terms of “can-do” statements adapted from the European Language Portfolio within four categories: 1) listening and speaking, 2) reading, 3) writing, and 4) grammar and vocabulary, and the linguistic content specified through the lists of linguistic items divided into: grammar, vocabulary, writing and functions (See CEFR Curriculum Case Study). Schemes of work for each term are prepared for all the groups at a given level on the basis of the content of the syllabi by directors of studies in consultation with the teachers. Detailed versions of schemes of work for particular groups are prepared by individual teachers. 2. The Progress Tests The testing and assessment system forms a natural part of each course undertaken in a group or individually. Learners are assessed directly by teachers on the basis of their “classroom” work as well as by formal testing on the basis of tasks carried out under examination conditions.

Small progress tests are offered at least three times in a term and are prepared by the teachers. Final end-of-term tests on the basis of which learners are promoted to higher levels are prepared by directors of studies. End-of-term tests are always offered in a written form; tests run during the term can have both a written and oral form. Feedback on performance is offered immediately after the test or during the next meeting.

Written progress tests offered during a term test progress: i.e. have learners understood, and can they use the grammar and lexis taught in lessons. Such tests are book-based but also reflect what has been taught in class, thus include extra lexis, grammar etc. taught outside of the coursebook. A test run during a term includes grammar and lexis taught, or a skill – listening and/or reading – based on the material/topics taught. Question types for grammar and lexis, as well as formats for listening and reading, are familiar to learners. The following general assessment criteria are applied to writing tasks:

• the extent to which the student fulfils the task,





18

• the extent to which the student organises writing coherently, • the extent to which the student uses a range of appropriate grammar accurately, • the extent to which the student uses a range of appropriate vocabulary, • the extent to which the student uses spelling and punctuation accurately.

Detailed assessment criteria for written tests are provided for each level by the directors of studies. Speaking is assessed on a continuous basis throughout the term using a simple matrix of 5 marks each for Fluency, Accuracy, Range and Interaction (pronunciation is incorporated into accuracy). Learners are graded by two teachers teaching each group during lessons, or particularly during set activities when they always know they are being graded. Since learners should be able to see how they are doing during the term, marks are altered regularly according to their effort and ability. Homework assignments are substantial and marked by teachers since they are part of the end of term mark. Homework assignments are split between two teachers teaching each group with 5% each, 5 homework assignments, at 1% each designated for the final result. Except for exam-preparation groups where these are writing assignments, a minimum of 2/3 are writing assignments per teacher, depending on the level. The time limit on completing such assignments is two weeks. Classwork grade is based on the overall effort and ability of learners. Consequently, a learner who is very able but not making an effort is graded lower than one who is weak but tries hard to improve. Learners have the chance to discuss this mark at least twice during the course. End-of-Term Test is a combination of progress and level test prepared and marked by the respective director of studies. End-of-Term Test includes Use of English, Listening and Reading, split 40:30:30 in mark weighting and approximate time taken to complete. Learners’ final results and ability to move up a level are assessed in the following way: Speaking 20% Written tests 10% Classwork 10% Homework 10% End term (level) test 50% The pass mark is 65% “Can-do” checklists The system of formal tests is supplemented with the “can-do” checklists of the Portfolio-type descriptors prepared for each level. Twice each term, both learners and teachers are asked to tick the abilities attained in order to focus their attention on practical abilities that the learners are supposed to gain at each level. When a learner can do more than 70% of the statements, they will have achieved the corresponding CEF level. The checklists of descriptors for teachers were adopted from the ALTE/EAQUALS Portfolio; the checklists of descriptors for students were adopted from the Polish translation of the Portfolio (Europejskie Portfolio Językowe, CODN Warszawa 2005: accredited model 6.2000).





19

Teachers’ resource packs with activities that teach/test ‘can do’ statements are being developed. Regular training in assessment exercises and techniques, standardization sessions, and an agreed system helps ensure fairness and consistency throughout the school. 3. Conclusion Bell Krakow teachers understand the need for creating a uniform system of assessment and certification in Europe. They also appreciate the focus on practical abilities emphasised by “can-do” statements and agree that the use of “can do” descriptors to measure progress/levels is an appropriate and appealing new way to assess learner ability. In the context of the existing requirements set by the Polish Ministry of Education however, they are greatly relieved to see that the intention of Bell Krakow is not to get rid of the “old” syllabi system and replace it with CEFR levels and descriptors but to supplement and support the existing syllabi and assessment procedures with the new approach which facilitates the learning process. Ela Jarosz Academic Director Bell Krakow





20

SPEAKING AND WRITING TASKS LEVEL 3 – CEFR: A2

FOOD

Student A: _____________________________________________________

Student B: _____________________________________________________

1. Role-plays 1. Two housewives talking about what their families like to have for breakfast/lunch/dinner. 2. Two chefs on a TV cooking programme explaining how to make a simple dish. 3. Two friends ordering a meal in a restaurant. Fluency Accuracy Range Interaction Overall 2. Writing 1. Write a short letter to a friend (50 – 100 words) telling him/her about a great restaurant that you have been to. Task fulfilment Task organisation Range of appropriate grammar Range of appropriate vocabulary Spelling and punctuation





21

MID-TERM TEST LEVEL 3 – CEFR: A2

(70 points) Name: _____________________________ __ Total:

%_____________ I. Choose the correct word and underline it. Example: He is a / the doctor.

1. How ever / often / never do you go to the cinema?

2. Has she / you / they got any brothers?

3. ‘What are they?’ ‘They’re a / an / --- apples.’

4. I need a / some /any milk to make this cake.

5. When / How many / How much days a week do you work?

6. Sorry, I don’t / ‘m not / can’t help you. I don’t know the answer to your question.

7. I don’t like watch / watching / watched TV.

8. I didn’t eat / ate / eating dinner last night.

9. He wasn’t / weren’t / were not at work yesterday.

10. What do you usually did / doing / do at the weekend?

11. We went / go / fly to Italy last summer.

12. When did you be / did you was / were you in Paris?

13. We don’t go to / aren’t going to / aren’t go to buy a new car.

14. Be quiet ! The baby sleeps / is sleep / is sleeping .

14

II. Add apostrophes. (‘ ) Example: I dont live in London. > I don’t live in London.

1. I havent got any brothers.

2. Its Marks house.

3. I didnt go swimming yesterday.

4. Shes always hungry.

5. Dont go now. Lets wait for Tom.

6. Theyre waiting for us.

7. Whos got a car?

9

70





22

III. Write negations. Example: They live in Kraków. They don’t live in Kraków.

1.She bought some shampoo at the supermarket.

.........................................................................................................................

2. She goes to the swimming pool every Sunday.

.........................................................................................................................

3. They are listening to the radio.

..........................................................................................................................

4. They went home before the end of the film.

..........................................................................................................................

5. We like Italian food.

..........................................................................................................................

6. They were very tired when they came home last night.

...........................................................................................................................

7. We met Susan in Spain last year.

..........................................................................................................................

8. It’s going to be hot and sunny next weekend.

..........................................................................................................................

9. She can sing beautifully.

..........................................................................................................................

9

IV. Complete with a preposition. Example: I’m Egyptian. I’m from Cairo.

1. The picture’s ................... the wall.

2. I work from 8.30 ................ 5.15.

3. She’s arguing …………… her husband.

4. They never work ............... Sunday.

5. He goes to work ................ bus.

6. The TV is in front …………. the cupboard.

7. Are you interested ............... fashion?

8. Did you go ............. holiday last summer?

9. Ann is sitting ................ to Mark.

10. I was born ………….. 1987.

10





23

V. Write a , an, some or any.

1. I haven’t got ............... children.

2. There is ……………….milk in the fridge.

3. We’ve got .............................. very good friend in London.

4. Her husband is …………….. actor.

5. Have you got ............. ice?

5

VI. Underline the word that doesn’t fit. ( look at the meaning, not form) Example: banana apple table orange melon 1. kitchen bathroom shelf hall

2. breakfast lunch dessert dinner

3. vegetables fruit snack coffee

4. pineapple orange lettuce banana

5. sugar milk juice water

6. between opposite square behind

6

VII. Match the verbs and phrases. Example: have a shower make have take go get 1. ………….. a good time 2. ………………. a taxi 3. …………… a noise

4. ………….. a party 5. …………….. away for the weekend 6……………photos.

6

VIII. Complete the following sentences with one word from the box. last many much one

1. How ……………… cigarettes do you smoke?

2. I passed my exam .................... year.

3. I don’t drink ……………… coffee in the evening.

4. How …………….. money do you spend on books?

5. ‘Which pen would you like?’ ‘ The green ..............’

5





24

IX. Make sentences from the words given. The first word is underlined.

Example: a doctor is he He is a doctor.

1. on doesn’t he Sunday work

.............................................................................................. .

2. turn off please mobile all phones

.............................................................................................. .

3. What she do next is going to summer ?

.............................................................................................. .

4. going wear what you are to ?

.............................................................................................. .

5. are glasses some there the in cupboard.

.............................................................................................. .

6. she what now is reading ? .............................................................................................. .

6





25

Case Study: Yüce School, Ankara

1. The School

Under Yüce four schools operate: Yüce Primary ( medium of instruction-Turkish), Yüce High School (medium of instruction –Turkish), Yüce College (IBDP- medium of instruction-English) and Yüce Science High School (medium of instruction- Turkish). The school teaches both English and German starting from the kindergarten. 2. The Context and the Progress Tests: The school is using a curriculum of which has the learning objectives derived from the European Language Portfolio. The approach of the curriculum is communicative. Teachers write the syllabi under the guidance of the curriculum. The scheme of work is written weekly and shared with the students. Students have chance to write notes on the schemes of work telling the teacher where they need support. Lesson plans are written by looking at the scheme of work (weekly plans). At the beginning of the school year, starting from the sixth grades, students are given level assessment test which includes four skills. Students are streamed into two groups: Lower and upper levels. Lower group students are supported with weekend courses free of charge. At the beginning of the school year students are introduced with the exit level objectives so that they know what they will be able to do and what will be covered. Students also do self assessment twice a year using the ELP descriptors aligned with the curriculum. We sent parents information letters every other week so the parents know what is going on in and out of the classroom ( in school activities, field trips etc.) . 3. The Development of the Tests:

Teachers are grouped according to their levels and there is one guide teacher (who is more experienced with the system) for each level. The assessments are prepared by the teachers, checked first by the guide teachers and then by the head of departments. At the end of each week a progress test is given and three times a semester written tests (covering listening, reading and writing skills and use of English) are given. Writing is evaluated and signed by two teachers. Speaking is evaluated during the level assessment (before the school starts) and during the lessons (e.g. oral presentation and group work). Listening and reading are assessed both in the written exams and in addition there are progress tests (quizzes).





26

Levels and descriptors of levels are written on both the written exams and progress tests (quizzes) so that both the teachers and students can follow what is being covered from the curriculum. The same is done for the scheme of work. Conclusions

1. In our city foreign language teachers have a very professional attitude towards their occupation. When the school experiences a high turnover in staff, sometimes the guide teachers have to be appointed from experienced teachers who are new to the system. Although in-service training is given before the school starts teachers seem lost without clear guidance. However, regular observation and feedback help.

2. Writing and speaking skills assessments have to be monitored very closely by the heads

of departments as they may be neglected.

3. The parents’ approach to communicative lessons and testing: “My Johnny doesn’t write enough in his notebook”. “My daughter said they always play games”. The school needs to explain the rationale of a communicative approach.

4. Lower school grades (=years) have self assessment in addition to tests. Teachers have a

say in the self-assessment too. Nevertheless, teachers and parents may well not agree on the results.

5. Higher grades also have self assessment in which teachers have a say. In this case

teachers and students may not agree on the results. As self assessment is for the student and doesn’t affect the report card grades, we are planning to remove the teacher’s column from the self-assessment forms for the higher grades.

6. The school doesn’t give exit certificates as it is not a procedure under local educational

regulations. However, we are looking forward to being able to award EAQUALS certification.

Saadet Roach Director Yüce Schools, Ankara, Turkey March 2009





27





28





29





30





31





32





33





34

Case Study: Tom and Emma School of English, Belgrade

1. The Context

The majority of our students are young learners (about 80%), who tend to enrol in September and study with us for at least a year. A large number of our students start in the 1st grade or at preschool level and study with us until they have completed either the Cambridge PET or Cambridge First Certificate in English. In general with each group we work through the CEF levels using progressively more advanced course books. We tend to work with one course book for a year as we have 72 classes of between an hour and an half an a half depending on the level. We have young adult groups who are predominantly preparing for a Cambridge exam: FCE, CAE or CPE. Our adult students also tend to enrol for at least a year and predominantly want to improve their spoken English. They come twice a week for an hour and a half and also use one course book a year. Until now, we have only had adults up to B1 level. 2. Curriculum

Our curriculum for young learners has been devised using the YALS (Serbian Association of Language Schools) curriculum which was developed in 2002 and course books from a variety of publishers which have been developed for specific CEFR levels. We have determined what topics, grammar and functions we think should be covered at each CEFR level and sub-level and devised our own can do statements for each level. Our end of year tests for young learners have been designed to test the core topics, grammar and functions and to ensure that we have tested that they have met the can do statements for that level. Our curriculum for young adults and adults was based on the YALS curriculum from 2002 and has been revised with reference to the CEFR descriptors. Our end of year tests for adults have been designed to test the core topics, grammar and functions for each CEFR level and sub-level and writing and speaking tests have been designed to ensure that students fit within the CEFR descriptors for that level. 3. Progress Tests Progress testing is done approximately once a month for both young learners and adults. We use the progress tests which accompany the course books. Where they are not available or we have not been able to get hold of them, we create the tests in-house using materials taken from course books of the same level. We also add to these tests if for example a skill is not tested e.g. writing - in this case we would test whichever form of writing they have learnt in that unit. However, we rarely ‘test’ their spoken English – this is done by continuous assessment through speaking activities done in class. We don’t, however, use any specific framework for this assessment.





35

4. Development of End of Year Tests Young Learners We created our own tests for levels A1.1, A1.2, A1.3, which use matching pictures with words, colouring in a parts of the body, gap-fill sentences with prepositions, writing simple words next to pictures, drawing the hands on a clock, writing basic sentences about abilities, listening exercises which require students to draw happy and sad faces and answer basic comprehension questions. Oral tests are done in class with only the class teacher present and are focused on describing visual stimuli designed in-house using course book materials and practice tests from The Euro Exam. From A2 level onwards all four skills are tested using materials taken from course books or test books. The lowest two levels do the tests in class and they take no longer than one class to complete i.e. the whole test is not longer than 50 minutes. From A2.3 level the students come to school at a designated time and complete the written part of the test which includes grammar, reading and writing in 1 hour. They have an oral test with a partner from their own group which lasts 10 minutes. The oral test is performed by two teachers who don’t teach them. The listening test which is also not longer than 10 minutes is done in class.

• Grammar - We use a variety of types of task to test grammar; multiple choice, open

cloze, gap fill.

• Reading - One text with multiple choice questions, true false questions or ordering

pictures according to the text.

• Listening - Two extracts using multiple choice questions with pictures or words or

matching sentences with speakers.

• Writing - A2 - Writing is simple sentences about a picture with word prompts for the

lowest level and then a piece of writing with questions to think about or topics to include

e.g. a story, or an informal email for the higher levels. B1 onwards – Students are given

a choice of 3 tasks and have to write between 80-120 words depending on the level.

• Speaking - Speaking for the lowest level is to describe the differences between two

pictures of the same scene, for the higher levels they have some questions about

themselves and one/two of the following; picture comparison, information exchange

exercise, telling a story following a series of picture prompts.

Adults We have created tests for all levels from A1 to B1 using in-house materials and other materials from course books of the relevant level. The written tests are done on one day outside of class time and take no longer than an hour. They have an oral test with a partner from their own group which lasts 10 minutes. The oral test is performed by two teachers who don’t teach them. The listening test which is also not longer than 10 minutes is done in class.

• Grammar - We use multiple choice questions and gap fills with the words given to

choose from to test grammar.

• Reading - One text taken from a course book with true false questions.

• Listening - One extract using true/ false and multiple choice questions.

• Writing - A1 – an informal letter. Students are given an informal letter and asked to

write the response in about 60 words for the lower level and 80 words for the higher

level. A2 – students are given 3 options to choose from; an informal letter, a short story





36

or a description of a place in about 100 words. B1 – students are given 3 options to

choose from; an informal letter, a review, a description of a person in 100-120 words.

• Speaking - All levels start with some general questions – at the higher level these

questions are designed to elicit specific tenses. Other tasks within the test are

describing pictures, information exchange, telling a story following a series of picture

prompts and problem solving activity.

5. Moderation All writing tests are double-marked according to a marking framework out of 20 points with marks for content, target reader, organisation and cohesion, range, register and format and accuracy. All speaking tests are conducted by two teachers according to a marking framework out of 10 points with marks for fluency, accuracy, pronunciation. 6. Conclusions

Over the last couple of years we have been developing, modifying and hopefully improving our end of year tests. The following points are the most important things that we have learnt from the experience:

• Instructions must be crystal clear.

• Standardisation training in oral and written assessment for all teachers is essential.

• Activities for the oral assessment should be as ‘free’ as possible and allow students the

opportunity to show what they know.

• The questions for the teacher-controlled phase of the oral assessment need to be very

carefully thought out and should be worded so as to elicit specific grammar

constructions.

• Reading texts should not be too long with 5-8 questions per text.

• Listening texts should be of good quality and test familiar topics.

• Writing tests should be double-marked.

• Grammar is best tested with multiple choice questions. (Not everyone would agree with

this. What is clear is that students perform differently on “recognition items” like

multiple choice than they do on gap-fill items. However, good multiple-choice grammar

items are very, very difficult to construct. ed)

Justine Spasojevic

Director of Studies

Tom and Emma School of English, Belgrade

eaquals making cefr testsv3 - [(•)] square eye | easy...

Documents