gregory light, phd brazil science, technology, humanities, engineering and mathematics (sthem)...
TRANSCRIPT
Gregory Light, PhD
Brazil Science, Technology, Humanities, Engineering and Mathematics (STHEM) Consortium
1ST WORKSHOP: LORENA, BRAZIL, MAY 26 30, 2014‐
Changing Teaching, Transforming Learning
Session II: Assessing Learning
OverviewSession I: Design Principles & Learning Outcomes
• Global & National Challenges (Interactive Presentation)
• Teaching for Learning: a Framework (Interactive Presentation)
• Constructing Learning Outcomes (Activity)
Session II: Assessing Learning
• Assessing Student Learning (Discussion)• Dimensions of Assessment (Interactive
Presentation/Activity)• Aligning Assessment (Activity)
“The quickest way to
change student
learning is to change
the assessment system.”
-- Elton & Laurillard, 1979, p. 10
ASSESSMENT
Group Question:
Why do we assess students?
At TablesShare
Purposes of AssessmentSummative
• To pass or fail a student• To grade or rank a student• To license to proceed• To select for future courses• To license to practice• To predict success in future courses• To predict success in employment• To select for future employment
Purposes of AssessmentFormative
• To provide feedback to students to improve their learning
• To motivate students• To diagnose a student’s strengths and
weaknesses• To help students develop their skills of
self-assessment• To provide a profile of what a student
has learnt
Purposes of AssessmentEvaluative
• To provide feedback to teachers• To improve teaching• To evaluate a course’s strengths and
weaknesses• To make the course appear
‘respectable’ and credit worthy to other institutions and employers
Dimensions of Assessment
Dimensions of Assessment
FormativeSummative
Reliability The extent to which the results of the assessment method can be trusted
ValidityThe extent to which the
assessment methods reflect student learning and the learning
goals of the course
Essentially designed to sum
up someone’s achievement
Essentially designed for use
in helping the learning process
Practicality
Criterion Referenced
Norm-Referenced
Based on self-reflection of
learning achieved on the course
Self-ReferencedBased on
knowledge and skills learned on
course
Based on comparisons with others in
the group
Peer-ReferencedBased on peer
appraisal of learning achieved
on the course
Dimensions of Assessment
FormativeSummative
Reliability The extent to which the results of the assessment method can be trusted
ValidityThe extent to which the
assessment methods reflect student learning and the learning
goals of the course
Essentially designed to sum
up someone’s achievement
Essentially designed for use
in helping the learning process
Practicality
Criterion Referenced
Norm-Referenced
Based on self-reflection of
learning achieved on the course
Self-ReferencedBased on
knowledge and skills learned on
course
Based on comparisons with others in
the group
Peer-ReferencedBased on peer
appraisal of learning achieved
on the course
Dimensions of Assessment
FormativeSummative
Reliability The extent to which the results of the assessment method can be trusted
ValidityThe extent to which the
assessment methods reflect student learning and the learning
goals of the course
Essentially designed to sum
up someone’s achievement
Essentially designed for use
in helping the learning process
Practicality
Criterion Referenced
Norm-Referenced
Based on self-reflection of
learning achieved on the course
Self-ReferencedBased on
knowledge and skills learned on
course
Based on comparisons with others in
the group
Peer-ReferencedBased on peer
appraisal of learning achieved
on the course
Dimensions of Assessment
FormativeSummative
Reliability The extent to which the results of the assessment method can be trusted
ValidityThe extent to which the
assessment methods reflect student learning and the learning
goals of the course
Essentially designed to sum
up someone’s achievement
Essentially designed for use
in helping the learning process
Practicality
Criterion Referenced
Norm-Referenced
Based on self-reflection of
learning achieved on the course
Self-ReferencedBased on
knowledge and skills learned on
course
Based on comparisons with others in
the group
Peer-ReferencedBased on peer
appraisal of learning achieved
on the course
Dimensions of Assessment
FormativeSummative
Reliability The extent to which the results of the assessment method can be trusted
ValidityThe extent to which the
assessment methods reflect student learning and the learning
goals of the course
Essentially designed to sum
up someone’s achievement
Essentially designed for use
in helping the learning process
Practicality
Criterion Referenced
Norm-Referenced
Based on self-reflection of
learning achieved on the course
Self-ReferencedBased on
knowledge and skills learned on
course
Based on comparisons with others in
the group
Peer-ReferencedBased on peer
appraisal of learning achieved
on the course
Dimensions of Assessment
FormativeSummative
Reliability The extent to which the results of the assessment method can be trusted
ValidityThe extent to which the
assessment methods reflect student learning and the learning
goals of the course
Essentially designed to sum
up someone’s achievement
Essentially designed for use
in helping the learning process
Practicality
Criterion Referenced
Norm-Referenced
Based on self-reflection of
learning achieved on the course
Self-ReferencedBased on
knowledge and skills learned on
course
Based on comparisons with others in
the group
Peer-ReferencedBased on peer
appraisal of learning achieved
on the course
TEACHING-CENTERED
LEARNING-CENTERED
Activity: Assessment Methods
Choose an assessment method(s) you currently use and “map” it on to 1) a learning outcome2) these dimensions.
Do individually, then share at tableThen with whole group
Assessment Methods(Some Examples)
• final exam (written)
• Homework• projects• reports• Presentations• observed
discussion groups
• office hours
• final exam (multiple choice)
• quizzes• cold calling• career
performance• standardized
tests• personal
response systemsHigher validity – less reliable Higher reliability – less validity
Aligning Assessment
Using Rubrics
(Making valid assessments more
reliable)QuickShare
Aligning Assessment with a Learning Outcome (Biology Example)
Course Goal
Learning Outcome
FormativeAssessment
Summative Assessment
What will students learn?
If they have learned it, what will students know and be able to do?
What will students do to learn it?
How will students demonstrate they know it or are able to do it?
Students will understand the transfer of information from DNA to proteins.
Students will be able to predict changes in amino acid sequences caused by mutations.
In groups students are given sequence of DNA corresponding to amino acid sequence. Students identify reading frame and predict amino acid changes due to mutations in that sequence.
On exam students will predict the new amino acid sequence that results from a mutation in a given gene sequence.
Knight, (2011)
A Typical Rubric Format STUDENT EVIDENCE
(1) WeakLittle or
no evidence
of outcome
(2) BasicSome
evidence of outcome
(3) ProficientDetailed
evidence of outcome
(4) StrongHighly
creative; outcome
TEACHERCRITERIA
Criterion 1Argument
Criterion 2 Integration of literature
Criterion 3Writing quality
Adapted from Beauchamp et al 1996
Assessing Writing: Example Rubric
(1) WeakLittle or
no evidence
of outcome
(2) BasicSome
evidence of outcome
(3) ProficientDetailed
evidence of outcome
(4) StrongHighly
creative; outcome
Criterion 1Argument
Rambling; poor use of logic; personal opinion
Basic structure with some use of evidence
Strong structure & logic, evidence used throughout
Strong structure & evidence; sophisticated discussion
Criterion 2 Integration of literature
No or minimal use of sources
Sources used, but no integration of writer’s ideas
Sources well integrated into author’s ideas
Sources well integrated; critique made
Criterion 3Writing quality
Poor organization, grammar, syntax
Some effort to organize ideas; grammar/syntax problems
Good organization; very few grammar/ syntax problems
Well-developed flow; error-free; elegant style
Adapted from Beauchamp et al 1996
Benefits of Rubrics
Used for grading (Summative) or feedback (Formative)
Clear criteria (Criteria Ref.) Ensure grading aligns with learning
outcomes (Validity) Shared with multiple teacher
graders/situations (Reliability) Can be used with students (Self Ref.)
and peers (Peer Ref.)
Activity
In small group, identify
and design a rubric for a
specific Learning outcome
Large Group Discussion
Aligning Assessment
Using Multiple Choice Questions (MSQ)
(Making reliable assessments more valid)
QuickShare
MCQs as formative instruction
Answer and provide your rationale:
1. Which of the following has/have intrinsic pacemaker characteristics?
a) Medulla c) Sinoatrial node
b) Pons d) Atrioventricular node
Use rationales (short open-ended explanation for choice) to let students demonstrate their learning - can be graded or not..
Stanford Learning Lab: Nash & Shaeffer, 1999; Schaeffer et al., 1999
Ideal rationale: SA node is the normal pacemaker for the entire heart. AV node also has pacemaker potential, but is overshadowed by SA node. Medulla has pacemaker potential for breathing rhythm as well. Pons helps refine rhythm, but does not have pacemaker potential.
Less-than-ideal rationales: Offering an incomplete answer: Normally the SA node is responsible for generating heart rate, and it is able to do this because of its intrinsic rhythm. The AV node also has an intrinsic rhythm, but it is “overshadowed” by that of the SA node.
Providing a book definition: The sinoatrial node is the pacemaker of the mammalian heart.
Providing irrelevant information: Stretch receptors are located in the aortic arch and the carotid sinus. They have the ability to respond to changes in pressure.
Restating the question: The SA node, AV node, and medulla all possess intrinsic pacemaker characteristics as they all serve as intrinsic pacemakers.
Blind appeal to authority: This answer is right because Professor Heller said that it was, and Professor Heller is cool.
Activity
In small group, share other
ways in which the validity
of MCQs might be
enhanced.
Share with Large Group
Testing low-level knowledge
Purely economic loss is recoverable in a product liability action.a) Trueb) False
Purely economic recovery will be barred in which of the following causes of action?
c) Negligenced) Fraud e) Defamationf) Product liability
“What’s the rule?”
No context, not allowing for interpretation/analysis
from Case & Donohue, 2008
Revision: Higher-level understanding
A restaurant hired an exterminator to eliminate cockroaches from the basement under the restaurant. Around midnight, the exterminator applied to the basement floor and walls an effective pesticide that he had purchased from the manufacturer. A toxic gas released by the pesticide penetrated into the restaurant kitchen and did not disperse by the next day. As a result, the restaurant was required to close that day.
The restaurant brought a tort action based on product liability against the pesticide manufacturer for lost profits.
Will the restaurant prevail?
a) No, because in this action purely economic loss in not recoverable. *b) No, because the exterminator was the proximate cause of the restaurant's
damages. c) Yes, because the manufacture of pesticides is an abnormally dangerous
activity. d) Yes, because the pesticide was being used as intended.
from Case & Donohue, 2008
3 ideas to take with you
Assessment is teaching
Students need to be involved
Assessment gives you no more than what you ask for
Final
QUESTIONS?
References• Entwistle, N. & Tait, H. (1990) ‘Approaches to learning,
evaluations of teaching and preferences for contrasting academic environments’, Higher Education, 19 (2): 169–94.
• Knight, J. (2011) University of Colorado. NAS/HHMI Summer Institute on Undergraduate Biology Education, Madison, WI.
• Light, G, & Micari, M. (In press) Making Scientists: Six Principles for Effective College Teaching, Harvard University Press.
• Beauchamp, McConaghy, Parsons & Sanford. (1996) Teaching From the Outside In. Duval: 1996, 37.