SOCIO-COGNITIVE APPROACHES TO TESTING AND ASSESSMENT

PROSET - TEMPUS 1

by Marina Gvozdeva, Natalya Milyavskaya, Tatiana Sadovskaya, Violetta YurkevichBased on material by Cyril Weir

“How do examination boards make distinctions between the tests they offer at different levels on the proficiency continuum?”(Prof. Cyril Weir)What does it mean to say that one language learner is more proficient than another?What makes some language tasks easier to perform than others?What makes a test suitable for higher or lower ability learners?

PROSET - TEMPUS 2

Proficiency levels

The CEFR describes language ability on a scale of levels from A1 (Breakthrough) for beginners up to C2 (Mastery) for those who have reached a high level of proficiency in a language.

Common European Framework of Reference

(CEFR)

From http://www.cambridgeenglish.org/exams-and-qualifications/cefr

A SOCIO-COGNITIVE FRAMEWORK

Centre for Research in English Language Learning and Assessment

Construct Validity

CONTEXT VALIDITY

COGNITIVEVALIDITY

TEST TASK

PERFORMANCE

SCORING VALIDITY

Cognitive Validity

The extent to which the tasks we employ elicit the cognitive processing involved in task solving.

8

Receptive language processing

Productive language processing

9

Context Validity

Context validity relates to the appropriateness of both the linguistic and content demands of the text to be processed, and the features of the task setting that impact on task completion.

COGNITIVE VALIDITY: the example of reading

11

Cognitive demand at different proficiency levels

In many ways the CEFR specifications are limited in their characterisation of task-solving ability at the different levels and we need to be more explicit for testing purposes about:• the types of tasks demanded at each of the stages;• how well calibrated the cognitive processing demands made

upon candidates are in the design of the tasks;• the cognitive load imposed by relative task complexity at each

stage.

12

Word Recognition

Word recognition is concerned with matching the form of a word in a written text with a mental representation of the orthographic forms of the language.

13

Lexical access

Accessing the lexical entry containing stored information about a word’s form and its meaning from the lexicon.

The form includes orthographic and phonological mental representations of an item and possibly information on its morphology.

The lemma includes information on word class and the syntactic structures in which the item can appear and on the range of possible senses for the word.

14

Syntactic parsing

Once the meaning of words is accessed, the reader has to group words into phrases, and into larger units at the clause and sentence level to understand the text message.

15

Establishing propositional meaning at the clause or sentence level

An abstract representation of a single unit of meaning: a mental record of the core meaning of the sentence without any of the interpretative and associative factors which the reader might bring to bear upon it.

16

Inferencing

Inferencing is necessary so the reader can go beyond explicitly stated ideas as the links between ideas in a passage are often left implicit. Inferencing in this sense is a creative process whereby the brain adds information which is not stated in a text in order to impose coherence. If there were no such thing as inferencing, writing a text which includes every piece of information would be extremely cumbersome and time consuming.

17

Establishing a mental representation across texts

In the real world, the reader sometimes has to combine and collate macro-propositional information from more than one text. The need to combine rhetorical and contextual information across texts would seem to place the greatest demands on processing.

Cognitive processing at A2 to C2

KET A2 PET B1 FCE B2 CAE C1 CPE C2 Word recognition * * * * * Lexical access * * * * * Parsing * * * * * Establishing propositional meaning * * * * * Inferencing (*) * * * * Building a mental model (*) * * * * Creating a text level structure * * Creating an organised representation of several texts

*

18

CONTEXT VALIDITY

20

Contextual Parameters in Reading

Task Setting:• text length.Linguistic Demands:• discourse mode• lexical resources• structural resources• functional resources• nature of information.

21

The cognitive demands imposed by relative text complexity at each stage

Overall Number of words

Lexis Structure

KET

Approximately 740-800words

Restricted to common items which normally occur in the everyday vocabulary of native speakers.

Mainly simple sentences

PET

Approximately 1460-1590 words

General vocabulary sufficient for most topics in everyday life.

Mostly simple sentences but some use of relative and other subordinate clauses.

FCE

Approximately 2000 words

Good range of vocabulary. Topics are addressed in detail and with precision.

A range of sentence patterns– from the simple to the complex.

CAE

Approximately 3000 words

Broad range of vocabulary including idiomatic expressions and colloquialisms as well as language relating to opinion, persuasion and ideas.

This level is typified by: many complex sentences Frequent use of modals Some use of ellipsis Complex approaches to referencing – range of pronouns and adverbials, as well as use of synonymy.

CPE

Approximately 3000 words

Very wide range of vocabulary including idiomatic expressions and colloquialisms as well as language relating to opinion, persuasion and abstract ideas.

Most sentences are long and complex. No restriction on the types of structure employed by the text. Many examples of structures typically used for effect in writing – sentences with several subordinate clauses, for example.

22

Text length

Examination Overall Number of words Number of texts

Maximum for any single text

KET (A2) Approximately740-800 words 4 250 PET (B1) Approximately 1450-1600 words 5 550 FCE (B2) Approximately 2000 words 3 700 CAE (C1) Approximately 3000 words 6 1100 CPE (C2) Approximately 3000 words 9 1100

23

Examination Discourse Mode KET (A2) Genre : public signs & notices,(such as those found on roads, railway stations, airport);

newspapers & magazines (e.g., reviews, letters, consumer information, advertisements); informational sources (e.g., encyclopaedias, leaflets, brochures) Rhetorical Task : Descriptive, narrative, instructive

PET (B1) Genre : public signs & notices,(such as those found in shops, banks, restaurants); personal messages (text messages, notes, postcards, emails); newspapers & magazines (e.g., reviews, letters, consumer information, advertisements); informational sources (e.g., web pages, simple encyclopaedias, leaflets, brochures) Rhetorical Task: Descriptive, narrative, expository, instructive.

FCE (B2)

Genre: newspapers & magazines (e.g., articles, reports), fiction books (extracts), informational sources (e.g., guides, manuals) Rhetorical Task: Descriptive, narrative, expository, argumentative, instructive

CAE (C1) Genre : newspapers, magazines & journals (e.g., articles, reports), fiction & non-fiction books (extracts), promotional and informational sources (e.g., guides, manuals) Rhetorical Task: Descriptive, narrative, expository, argumentative. Instructive

CPE (C2) Genre : newspapers, magazines & journals (e.g., articles, reports, editorials), fiction & non-fiction books (extracts), promotional and informational sources (e.g., guides, manuals) Rhetorical Task: Descriptive, narrative, expository, argumentative, instructive