asking the right questions assessing language skills 2008 presentation to atesl central local sheri...
TRANSCRIPT
Asking the Right Questions
Assessing Language Skills
2008 Presentation to ATESL Central Local Sheri Rhodes,
Mount Royal College
What is the first thing you think of when you hear the word
“evaluation”?“assessment”?
“test”?
Example of a Speaking Test
• One item• Single-word repetition• Testing a discrete point • Stimulus is the prompt, “Say now Shibboleth.”• Scoring criteria – right / wrong• Consequences – potentially dire!!
Purpose of Assessment
Why?
Formative vs. Summative
• How will we use the test?
Reasons for Testing
• Placement Tests• Diagnostic Tests• Progress or Achievement Tests• Proficiency Tests• Portfolio Assessments
What makes a good test?
• Validity• Reliability
Test Design
Discrete
•A verb tense•A word
Integrative
•Use a variety of language and skills for a task
Indirect
•Knowledge•Study
Direct •Activation•Integrative
Test Design
Question Types - Indirect
•Multiple Choice
Question Types - Indirect
•Transformation•Sentence re-ordering•Edit•Match
Question Types - Direct
• May use items that look like indirect items • Others are specific tasks to the skills• Replicate real-life interaction
– Mirror an authentic purpose
Question Types - Direct
• Remember– Items should look like the kind of tasks students
have been practicing in their lessons.– Create a “level playing field”.– Marking direct test items will be more
difficult than indirect items.
Constructed-response assessments
• Fill-in assessment
Performance-based assessments
• Simulates real-life language use• Offers open-ended tasks• Allows for creative and divergent responses• Often employs strategies• Requires students to synthesize information• Promotes recursive learning • Evaluates performance based on
well-defined critera
Testing Reading and Listening
• Choose the best summary• Put pictures in order• Complete a phone message form (listening)• Fill out a summary form (reading)
Testing Reading and ListeningGlobal understanding
Testing Reading
• Many skills at different levels of reading from word recognition, reading speed, skimming techniques, etc.
• Primary evaluation is looking at reading comprehension at every level.
Testing Listening
• Use a recorded audio track to ensure test relability
• Ensure the recording is clearly and easily audible
Testing Writing
• Produce written texts – Leaflets based on information in a text– Compositions (narrative or discursive essays)– Transactional letters (reply to advertisement)
Testing Speaking
• Seen as the most challenging of exams to prepare, administer and score.
Testing Speaking
• Interview students• Pair tasks
– Compare two pictures– Solve a problem– Discuss a topic– Role-play a situation
Grammar and Vocabulary
• Knowledge versus Ability• Isolated (discrete points) or integrated
Marking
• Backwash– Positive– Negative
Marking Writing
A + ?
Are you ever tempted to do this?
Marking Writing
• Holistic Scoring– Rubrics assigning a single score
• Analytic Scoring– 5 aspects of writing weighted and combined
(Jacob)– 7 scales (TEEP by Weir)– 3 scales (Michigan Writing Assessment)
Other Applications of Rubrics and Scales
• Speaking• Performance of Tasks• Portfolio evaluation
1. List the things you want to test.2. Decide how important each item is.
• Give them more time or space on the test
• Weight the marks to reflect the importance
3. Consider the test situation.4. Write the test. 5. Consult and share.6. Test the test. (the trial)7. Reflect and refine.
Steps in Test Design
Another list (Brown and Hudson)• Require students to perform, create, produce, or do something• Use real-world contexts or simulations• Allow students to be assessed on what they normally do in class every day• Use tasks that present meaningful instructional activities• Focus on processes as well as products• Tap into higher-level thinking and problem-solving skills• Provide information about both the strength and weaknesses of students• Are multi-culturally sensitive when properly administered• Ensure that people, not machines, do the scoring using human judgment• Encourage open disclosure of standards and rating criteria• Call upon teachers to perform new instructional and assessment roles