revisiting obtl with assessment...
TRANSCRIPT
Revisiting OBTL with Assessment Rubrics
Eva Wong
Director, Centre for Holistic Teaching and Learning
Hong Kong Baptist University
Email: [email protected]
Abstract
The chief aim of Outcomes-based Teaching and Learning (OBTL) is the creation of a
student-centered learning environment with two important characteristics:
• effectively encourages students’ active engagement;
• provides timely feedback to motivate students to work towards a high standard.
To achieve this, we are required to set clear and specific course outcomes so that our
students know what they are expected to learn. We then have to make use of engaging
pedagogies to facilitate learning of the course outcomes. Finally, we must help students
gauge their progress in achieving the outcomes with assessments and feedback. The
systematic and constructive alignment of engaging pedagogies and assessments with
outcomes is crucial in the OBTL approach and it is also the means of enhancing learning
and teaching. The use of assessment rubrics both formalized and simplified the alignment
process because through the use of rubrics, both teachers and students know what to
expect and the level of performance required.
Timely feedback provided to students can help them improve and maintain their
motivation in learning. With some examples of engaging students in learning activities
through the adoption of OBTL, this seminar will discuss how to make use of assessments,
both formative and summative, coupled together with appropriate assessment rubrics, to
achieve the aim of OBTL in creating a learner-centred environment.
Bio of Dr Eva Wong:
Dr Eva Wong is the Director of the Centre for Holistic Teaching and Learning (CHTL),
at the Hong Kong Baptist University (HKBU). With education and student learning as the
main focus of her work, she joined HKBU in February 2010 to take up major
responsibilities for the professional development of academic staff, assisting the
implementation of the outcome-based approach to teaching and learning, and supporting
the University’s e-learning endeavours, with the main focus of enhancing student
learning with a holistic approach.
Dr Wong worked in a number of tertiary institutions in the United Kingdom during the
early part of her career, returning to Hong Kong with her family in 1991. She then joined
the City University of Hong Kong (CityU) as a faculty member in the Department of
Information System, and received the University’s Teaching Excellence Award in 1996-
1997. Since receiving the Award, Dr Wong had worked in various units within CityU,
bringing her experience and expertise in IT systems implementation to good use for the
successful implementation of various academic systems.
Dr Eva Wong
Director
Centre for Holistic Teaching and Learning
Hong Kong Baptist University
Hang Seng Management College
Revisiting OBTL with Assessment Rubrics
Rubrics Examples
Seminar by Dr Eva Wong
19 January 2012
Centre for Holistic Teaching and Learning
Hong Kong Baptist University
Topics Page
Rubric Use and Development 1
Rubric for Written Assignments 7
Rubric for Oral Presentation 9
A Guide to Grading Your Class Participation 10
33
RUBRIC USE AND DEVELOPMENT
Using rubrics is a way of ensuring that students, teachers, and parents alike know
the purpose of the work that students are being asked to do. The use of rubrics as a
tool for scoring work has the potential for giving students the power and
responsibility that goes with knowing what is being asked of them and how to
achieve it.
What is a rubric?
A rubric is an evaluation tool that describes the criteria for performance at various
levels using demonstrative verbs. It is a performance-based assessment process that
accurately reflects content skills, process skills, work habits, and learning results.
There are generally two types of rubrics: holistic and analytic. It is important to
analyze the task, activity or project being assessed and determine which type of
rubric is most appropriate to apply. A holistic rubric describes a student’s work as
a single score--the report or project as a whole is assigned a score. Therefore,
holistic rubrics are best suited to tasks that can be performed or evaluated as a
whole and/or those that may not require extensive feedback.
Analytic rubrics specify criteria to be assessed at each performance level, provide a
separate score for each criterion, and may include a composite score for overall
performance. In some cases, the composite score is weighted based on the
importance of each dimension.
34
Why should rubrics be used?
Using rubrics focuses both students and teachers on two essential questions:
• What do we want students to know and do?
• What would exemplary demonstration of this learning look like?
Rubrics serve several purposes in the assessment process. These purposes include:
• Creating a common framework and language for evaluation.
• Providing students with clear expectations about what will be assessed, as
well as standards that should be met. Send messages about what is most
meaningful.
• Increasing the consistency and objectivity of evaluating (especially scoring
or rating) performances, products, and understanding.
• Providing students with information about where they are in relation to
where they need to be for success.
• Identifying what’s most important to focus on in instruction.
• Giving students guidance in evaluating and improving their work. Students
can learn how to think about evaluation.
How do you develop a rubric?
At first developing rubrics is very difficult. The greatest challenge is for teachers
and, ultimately, students to translate the performance of various assignments to the
rubric fairly and reliably. For this they need support, time, and practice.
Rubrics can be developed using the following 8-step process.
Step 1: Determine what the assessment will encompass
35
Step 2: Review previous student work and/or other rubrics to identify any
additional assessment criteria
Step 3: Define each dimension
Step 4: Adopt a scale for describing the range of products/performances and
write a description for each dimension for each point on the scale
Step 5: Develop a draft rubric
Step 6: Evaluate the rubric
Step 7: Pilot test, revise, and try the rubric again
Step 8: Share the rubric with students and their parents
Tool #11 in the Aiming High ToolKit (Resource Section Tab 15) explains the
process for the development of rubrics and provides examples. Additional tools
for designing rubrics are provided on the following pages. They include the
“Rubric to Evaluate the Quality of a Rubric,” an examples of a rubric; “Words and
Phrases for Prompt and Rubric Design”; and “Descriptors for Weaker and Stronger
Performance Levels.”
36
RUBRIC TO EVALUATE THE QUALITY OF A RUBRIC
Criteria
Needs To Be Reworked
Acceptable But Needs More Clarity If Used For High
Stakes Testing
Clearly Written
Performance Levels Addressed
Scoring guide is open-ended
The scoring guide provides for different performance levels
The scoring guide is descriptive of each level of performance
Description of Performance Levels
There are no specific descriptions of the different performance levels
Differences between the levels rely on looking for a number of examples or responses
The descriptions define clear and significant differences between the performance levels
Language Specificity
Vague words are used to discriminate between levels: some, many, few, good, excellent
Subjective words (good, excellent, some) are used to discriminate between levels but are further defined
The critical attributes between each level of performance are included
Usefulness
The ratings do not provide useful instructional information
Ratings provide instructional information that needs further task analysis
Ratings provide useful instructional information
Developed by the SBE Design Team Northern Colorado BOCES
37
WORDS AND PHRASES FOR PROMPT AND RUBRIC DESIGN Developed by the SBE Design Team, Northern Colorado BOCES
Instruction Verbs for Five Levels of Thinking
KNOWLEDGE/ COMPREHENSION
APPLICATION ANALYSIS SYNTHESIS EVALUATION
list
repeat
record
relate
locate
review
restate
describe
discuss
explain
recognize
identify
define
report
name
recall
tell
use
show
apply
employ
interpret
operate
sketch
schedule
illustrate
translate
demonstrate
dramatize
inspect
inventory
examine
diagram
analyze
compare
contrast
relate
question
test
measure
differentiate
distinguish
calculate
experiment
plan
create
design
program
manage
arrange
compose
propose
set up
collect
assemble
prepare
construct
formulate
organize
rate
score
choose
value
select
assess
estimate
appraise
evaluate
revise
judge
debate
oppose
defend
criticize
FOUR LEVELS OF DIFFERENCE IN DEGREE DEGREES OF UNDERSTANDING
DEGREES OF FREQUENCY DEGREES OF EFFECTIVENESS
thorough/complete substantial/extensive minimal/general partial/some
misunderstanding
nearly always/always often/frequently sometimes/occasionally rarely/almost never/
never
highly effective effective moderately effective minimally effective/
ineffective
38
Descriptors for Weaker Performance Levels
recognizes and describes briefly
incomplete attempt
with some errors
without complete understanding
generally explains
general, fundamental understanding
uses a single method
represents a single perspective
identifies few connections
without drawing accurate conclusions
without explaining the reason
presents confusing statements and facts
without demonstrating complete
understanding of the characteristics
with limited details
demonstrates beginning understanding
has a general sense
with inaccuracies
takes a common, conventional approach
overlooks critical details
relies on single source
vague or incomplete description
unable to apply information in problem
solving
does not perceive a pattern
presents concepts in isolation
omits important details, facts, and/or concepts
no evidence of future projections
Descriptors for Stronger Performance Levels
thoroughly understands and explains efficient, thorough solution without errors thorough, extensive understanding provides new insight thorough mastery of extensive knowledge uses multiple methods represents a variety of perspectives draws complex connections draws logical conclusions which are not
immediately obvious clearly explains the reasoning provides clear, thorough support demonstrates complete understanding of all
the characteristics in elaborate detail sophisticated synthesis of complex body of
information shows an impressive level of depth with precision and accuracy takes an original, unique, imaginative
approach provides comprehensive analysis uses multiple sources thorough explanation of critical analysis solves problem by effective application of
information identifies an abstract pattern relates concepts using a variety of factors thorough presentation of important details,
facts, and concepts predicts future changes
Rubric for Written Assignments
Category/Criteria
Lacking or
Missing 1 pts
Poor 2 pts
Fair 3 pts
Good 4 pts
Very Good 5 pts
Conceptual
Writer has not
understood lectures,
readings, discussion,
or assignment.
Shows inadequate
command of course
materials or has
significant factual and
conceptual errors;
does not respond
directly to the
demands of the
assignment; confuses
some significant ideas.
Shows an
understanding of the
basic ideas and
information involved
in the assignment;
may have some
factual, interpretive,
or conceptual errors.
Shows a good
understanding of the
texts, ideas and
methods of the
assignment; goes
beyond the obvious;
may have one minor
factual or conceptual
inconsistency.
Has cogent analysis,
shows command of
interpretive and
conceptual tasks
required by
assignment and
course materials:
ideas original, often
insightful, going
beyond ideas
discussed in lecture
and class.
Thesis
No discernible thesis. Thesis vague or not
central to argument;
central terms not
defined.
General thesis or
controlling idea; may
not define several
central terms.
Clear, specific,
argumentative thesis
central to the essay;
may have left minor
terms undefined.
Essay controlled by
clear, precise, well-
defined thesis: is
sophisticated in both
statement and insight.
Development & Support
Little or no
development; may list
facts or
misinformation; uses
no quotations or fails
to cite sources or
plagiarizes.
Frequently only
narrates; digresses
from one topic to
another without
developing ideas or
terms; makes
insufficient or
awkward use of
textual evidence.
Only partially develops
the argument; shallow
analysis; some ideas
and generalizations
undeveloped or
unsupported; makes
limited use of textual
evidence; fails to
integrate quotations
appropriately.
Pursues thesis
consistently: develops
a main argument with
clear major points and
appropriate textual
evidence and
supporting detail;
makes an effort to
organize paragraphs
topically.
Well-chosen
examples; persuasive
reasoning used to
develop and support
thesis consistently:
uses quotations and
citations effectively;
causal connections
between ideas are
evident.
Structure
No transitions;
incoherent
paragraphs; suggests
poor planning or no
serious revision
Simplistic, tends to
narrate or merely
summarize; wanders
from one topic to
another; illogical
arrangement of ideas.
Some awkward
transitions; some
brief, weakly unified
or undeveloped
paragraphs;
arrangement may not
appear entirely
natural; contains
extraneous
information.
Distinct units of
thought in paragraphs
controlled by specific
and detailed topic
sentences; clear
transitions between
developed, cohering,
and logically arranged
paragraphs that are
internally cohesive.
Appropriate, clear and
smooth transitions;
arrangement of
paragraphs seems
particularly apt.
Build free rubrics at www.iRubric.com
Mechanics
Usually contains so
many mechanical
errors that it is
impossible for the
reader to follow the
thinking from
sentence to sentence.
Example of a Grading
Rubric For a Term
Paper in Any
Discipline.
Usually contains either
many mechanical
errors or a few
important errors that
block the reader's
understanding and
ability to see
connections between
thoughts.
Usually contains
several mechanical
errors, which may
temporarily confuse
the reader but not
impede the overall
understanding.
May contain a few
errors, which may
annoy the reader but
not impede
understanding.
Almost entirely free of
spelling, punctuation,
and grammatical
errors.
Language
Numerous
grammatical errors
and stylistic problems
seriously distract from
the argument.
Some major
grammatical or
proofreading errors
(subject-verb
agreement; sentence
fragments); language
marred by clichés,
colloquialisms,
repeated inexact word
choices; inappropriate
quotations or citations
format.
More frequent
wordiness; several
unclear or awkward
sentences; imprecise
use of words or over-
reliance on passive
voice; one or two
major grammatical
errors (subject-verb
agreement, comma
splice, etc.); effort to
present quotations
accurately.
Some mechanical
difficulties or stylistic
problems; may make
occasional problematic
word choices or
awkward syntax
errors; a few spelling
or punctuation errors
or cliché; usually
presents quotations
effectively.
Uses sophisticated
sentences effectively;
usually chooses words
aptly; observes
conventions of written
English and
manuscript format;
makes few minor or
technical errors.
Sources
What sources have
been used is not clear.
Some information has
been collected.
Information is
gathered from a few
electronic and non-
electronic sources.
Information is
gathered from
multiple electronic and
non-electronic
sources.
Information is
gathered from a
minimum of 3-each
electronic and non-
electronic sources.
Amount of
Information
No reference is cited
to support statements.
References are seldom
cited to support
statements.
Although attributions
are
occasionally given,
many
statements seem
unsubstantiated.
The reader is confused
about the
source of information
and ideas.
Professionally
legitimate sources
that support claims
are generally
present and
attribution is, for the
most part, clear and
fairly
represented.
Compelling evidence
from
professionally
legitimate
sources is given to
support
claims. Attribution is
clear and
fairly represented.
Quality of
Information
Information has little
or nothing to do with
the main topic.
There are virtually no
sources
that are professionally
reliable.
The reader seriously
doubts
Most of the references
are from
sources that are not
peer-reviewed
and have uncertain
reliability. The reader
Although most of the
references
are professionally
legitimate, a few
are questionable (e.g.,
trade books,
References are
primarily peer-
reviewed
professional journals
or other approved
sources (e.g.,
the value of the
material and
stops reading.
doubts
the accuracy of much
of the
material presented.
internet sources,
popular
magazines, …). The
reader is uncertain of
the
reliability of some of
the sources.
government
documents, agency
manuals, …). The
reader is
confident that the
information
and ideas can be
trusted.
Citations & Bibliography
Sources are not
accurately
documented in APA
format in either the
body of the paper or
the bibliography.
Sources are not
accurately
documented in APA
format.
All sources
(information and
graphics) are
accurately
documented, but
many are not in APA
format.
All sources
(information and
graphics) are
accurately
documented, but a
few are not in APA
format.
All sources
(information and
graphics) are
accurately
documented in APA
format.
Oral Presentation
Exceeds Standard Meets Standard Nearly Meets Standards Does Not Meet Standard Language Use and Delivery The student communicates ideas effectively
Effectively uses eye contact. Speaks clearly, effectively and confidently using suitable volume and pace. Fully engages the audience.
Dresses appropriately, Selects rich and varied words
for context and uses correct grammar.
Maintains eye contact. Speaks clearly and uses suitable volume and pace. Takes steps to engage the audience. Dresses appropriately. Selects words appropriate for context and uses correct grammar.
Some eye contact, but not maintained. Speaks clearly and unclearly in different portions. Occasionally engages audience. Dresses inappropriately. Selects words inappropriate for context; uses incorrect grammar.
Uses eye contact ineffectively. Fails to speak clearly and audibly and uses unsuitable pace. Does not engage audience. Dresses inappropriately. Selects words inappropriate for context; uses incorrect grammar.
Organization and Preparation The student exhibits logical organization.
Introduces the topic clearly and creatively. Maintains clear focus on the topic. Effectively includes smooth transitions to connect key points. Ends with logical, effective and relevant conclusion.
Introduces the topic clearly.
Maintains focus on the topic. Include transitions to connect key points. Ends with coherent conclusion based on evidence.
Introduces the topic. Somewhat maintains focus on the topic. Includes some transitions to connect key points. Ends with a conclusion based on evidence.
Does not clearly introduce the topic. Does not establish or maintain focus on the topic. Uses ineffective transitions that rarely connect points.
Ends without a conclusion.
Content The student explains the process and findings of the project and the resulting learning.
Clearly defines the topic or thesis and its significance. Supports the thesis and key findings with an analysis of relevant and accurate evidence Provides evidence of extensive and valid research with multiple and varied sources Provides evidence of complex problem solving and learning stretch. Combines and evaluates existing ideas to form newinsights.
Clearly defines the topic or thesis. Supports the thesis and key findings with evidence. Presents evidence of valid research with multiple sources.
Provides evidence of problem solving and learning stretch.
Combines existing ideas to form new insights.
Defines the topic or thesis. Supports the thesis with evidence. Presents evidence of research with sources. Provides some evidence of problem solving and learning stretch. Combines existing ideas.
Does not clearly define the topic or thesis. Does not support the thesis withevidence. Presents little or no evidence of valid research.
Shows little evidence of problem solving and learning stretch.
Shows little evidence of the combination of ideas.
Questions and Answers
Demonstrates extensive knowledge of the topic byresponding confidently,precisely and appropriately to all audience questions and feedback.
Adapted from: http://course1.winona.edu/shatfield/air/Science_Rubrics.pdf
Demonstrates knowledge of the topic by responding accurately and appropriately to questions and feedback.
Demonstrates some knowledgeof the topic by respondingaccurately and appropriately to questions and feedback.
Demonstrates incomplete knowledge of the topic by responding inaccurately and inappropriately to questions and feedback.
