Designing Assessment:

How much is enough?

Assessment practice is surrounded by uncertainty (Allen 1998)

Academics design tasks, award grades and provide feedback that,

They feel comfortable with,They believe the student will feel is fair,Can with stand robust scrutiny by

colleagues.They make decisions that lead to preferred

and expected outcomes using their experience to predict the best and worst outcomes!

“Good practice” in assessment underpins fairnessClear descriptions of grading criteria &

grade related performance standardsTimely feedback that linked to the

assessment criteria Appropriate tasks that encourage

learningDecision-making guided by posted

criteria and standards when grading (adapted from Carroll 2004)

Scholarship on Learning & Assessment

Students require risk and ambiguity to learn Students require challenge to engage in their

learning Students need support in their learning Students who are meta-cognitive are more

likely to succeed Students learn for the longer term when they

have to DO SOMETHING with the knowledge and concepts they learn

Much assessment is taken up with surface learning and trying to ‘spin out a grade’ for the institution.

Ultimately assessment is a fallible and human activity.

Meta-cognitiveis knowing what you know and don’t

know and how you learn and how to go about finding out what you don’t know.

is awareness of your own approach to learning and the current status of your learning.

Assessment DesignInforms & Shapes Learning

Tells students what you, the teacher/assessor, think is important

Relevance Relatedness

Dictates students’ learning activities Formative (Assessment as Learning)

Defines students to themselves as learners Effort Ability Persistence

What Informs Assessment Design?Traditions (essays and exams)Conventions & common practice

(Britzman 2003)Wisdom of experience (Shulman 1979

& beyond)Contemporary trends (e.g. portfolio

assessment)Expedience and practicalitiesScholarship?

About assessment design! It is the least understood aspect of curriculum

and teaching. It has educational, technical and ethical

dimensions that encompass:

1. Purposes: Why assess?2. Goals: What to assess?3. Processes How to assess?4. Interpretation How to grade?5. Response How to feedback & report?

(Rowntree, 1977)

Focus : Assessment Design1. Purposes: Why assess?2. Goals: What to assess?3. Processes How to assess?4. Interpretation How to grade?5. Response How to feedback & report?

Impacts on, should consider, and be influenced by the four other dimensions.

Key Issues in designing assessment?

What is the most important for learning goals for students in this unit?

How to engage students from the start? What will be appropriate for the unit level? How to build on, & develop existing capabilities? How to encourage Higher Order Thinking (HOT)? How to support students with special needs? How to provide feedback opportunities? How to design tasks that will generate evidence

of students learning achievements?

Important for learning goals

Assessment designs have 3 functions: Define for students what is important to learn Shape students approach to learning Provide a means of assurance of learning for

the university and society.

Clear learning goals Tasks that align with them

What attributes would you like to engender in your graduates?

Knowledgeable disciplinary specialists? Ethical, efficient & trust worthy professionals? Collaborative, reliable colleagues and team

players? Community & corporate citizens? Effective problem solvers? Creative & critical reasoning thinkers? Effective verbal & written communicators? Culturally literate? Technologically literate?

Important for learning goals

Identify a Unit you will teach next session.Unit……….. Level:…………Learning goals:1.2.3.4.5.6.

Engaging students from the startWhat capabilities do you assume students will bring to the Unit?

Terminology Conceptual understanding Capabilities Dispositions

How will you confirm this? Pre-tests/Online tests/ reflective logs/ Hurdle assessment, work required

What will signal the learning priorities and approaches to students?

Engaging students from the start

Appropriate for the unit levelAssessment design should alter in terms of complexity across the years.First year: Acquire discipline language, terminology Develop research analysis, communication &

psychomotor skills Identify relevance to the big picture How experts proceed & facts become factsLater years Less tasks, less testing, greater complexity. Application of the information to novel contexts Increased research led learning Learn the values of the discipline

Appropriate for the unit level

Build on existing capabilitiesOften units are taught and assessed in isolationwhich increases the burden on each unit to ‘cover’ and ‘assess’.Assessment designs need to: account for their own contribution to the overall

degree program capitalize on and develop what has gone before and

is currently being taughtProgrammatic approaches to assessment assist students to: synthesize, effectively utilize multi-source feedback become more metacogntively aware become responsible for their own learning

Build on existing capabilities

What are the concurrent and previous units that can contribute to student performance in this unit?

What would be the essential learning that this unit contribute to a programmatic approach?

Develop Higher Order Thinking The essence of a university education is higher order thinking.

Blooms Taxonomy

CreatingEvaluatingAnalyzingApplyingUnderstanding

Remembering

Higher Order thinking

Biggs (1992)Pre-structural

Uni-structural

Multi-structural

Relational

Extended Abstract

Biggs (1992)

Pre-structural

Uni-structural

Multi-structural

Relational

Extended Abstract

Biggs (1992)Pre-structural

Uni-structural

Multi-structural

Relational

Extended Abstract

Scholarship to Inform Standards

ReasoningPerry’ Ethico Moral Reasoning in College Students

Black & white thinking

Multiplicity + search for truth.

Relativism but no commitment

CommitmentConditional Commitment

Develop Higher Order Thinking Consider a good performance by a student in your unit.

What was the student able to demonstrate that lead you to assess it as higher order performance?

How might the assessment tasks encourage that kind of performance?

Supporting students with special needsStudents have diverse needs across their course of study.

It is important to anticipate challenges for students

It is important that academics don’t take on issues as their sole responsibility.

Supporting students with special needsWhat flexibility is in your unit to accommodate unpredictable circumstances & needs?

What resources can academics call upon at UWA?

Provide feedback opportunities Students NEVER feel they get enough feedback Students rarely go back and respond to the

feedback with which they they have been provided Student ought to be active participants in the

feedback process Self and peer feedback processes increase student

metacognition Feedback and the response to feedback needs to

be designed into the tasks with successful performance and grades contingent on students engagement with feedback.

Provide feedback opportunities How might students be engaged in the

feedback process in your unit?

Generating evidence of learning achievementThe most valuable assets students can graduate with are: a sound understanding of what they have

learned and achieved in the degree program validated evidence to support their new

capabilities E.g. Portfolios, projects, research

Evidence of achievementWhat processes will assessment in this unit assist students to gain an accurate awareness of their own emerging capabilities?

What tasks can provide students with validated evidence of achievement and capability?

What designs best produce desired learning and graduate capabilities?

Authentic Tasks: Assessment as Learning E.g.. Library search (critical selection & team

activity) Forensic role play (investigation/problem solving) Observation (noticing)

Case presentations (theory/practice application)

Debates and position statement (evidence-based, opinion formation)

Mastery learning (Skill development)

How much assessment is enough? Enough to engage them from the start and

keep them engaged for 12 weeks. Enough to extend their existing abilities and

develop higher order thinking. Nothing that is there because its university and

we always do it at university. Enough to produce sound evidence of

achievement.

Some Design examples First year technical (accounting, economics)

Regular (weekly/) work assessed in class, presented in class, collected, worked examples and a sample of 3-4 assessed at the end of semester.

Plus an end of semester exam. First year, expressive areas

Comparison of popular media treatment(web?) of a topic with academic literature, (Individual or group)

Presentation of findings Website design to illustrate findings

Final year: Proposal, plan or contract of work to be don Work in progress self peer and expert formative evaluation Completed project e