using learning outcomes and assessment criteria peter noakes department of electronic systems...

52
Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

Upload: ryan-haley

Post on 28-Mar-2015

223 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

Using Learning Outcomes and Assessment Criteria

Peter Noakes

Department of Electronic Systems Engineering

University of Essex

Page 2: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

2

Purpose of this Talk

To provide an introduction to topics and related background information that will help us to provide more concise module specifications

To improve your efficiency in generating appropriate Learning Outcomes, Assessment Methods and Assessment Criteria for the modules you teach

At the same time provide better transparency for students by making module specifications clearer and unambiguous

and improve students commitment to self learning by clarifying what is expected of a student

Consequently this should improve student progression and encourage better student retention

Page 3: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

3

Structure of Talk

Background

Learning Outcomes

Level Descriptors (QAA, SEEC, EPC)

Assessment Methods

Assessment Criteria

An Example for You

Report Back and Discussion

Page 4: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

4

Reference Sources

This presentation is based on the content of the following publications:

“How to use Level Descriptors” by Jenny Moon, SEEC*, 2002

“How to Use Learning Outcomes and Assessment Criteria” by David Gosling and Jenny Moon, SEEC*, 3rd Edition 2002

“Assessment: A Guide for Lecturers” by George Brown, LTSN Generic Centre, 2001

“Guide for Busy Academics”, LTSN Generic Centre

*Southern England Consortium for Credit Accumulation and Transfer

(http://www.seec-office.org.uk )

Page 5: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

5

Background

A Programme defines study or learning required to achieve an award or qualification

A Programme Specification is required by the QAA for each award or qualification and defines the threshold learning outcomes for the programme

A Programme comprises a number of Modules each of which is separately assessed and earns credit when successfully completed

Using the outcomes model each Module Description defines the intended (threshold?) learning outcomes, the syllabus coverage and the assessment methods and criteria for the module.

Achievement of Module Learning Outcome should contribute to a student’s satisfaction of the programme learning outcomes

Page 6: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

6

Learning Outcome-based Model

Traditionally an academic would first define the syllabus coverage, then develop how its taught and finally determine the method of assessing the student’s absorption of the material.

The outcome-based model has three interconnected components:1. An explicit statement of learning intent (intended learning outcome)

which focuses on what the student is expected to know and be able to do by the end of the module, expressed in a form that permits their achievement to be demonstrated and measured

2. The processes and resources to enable the outcomes to be achieved and demonstrated (curriculum, teaching, learning methods and materials, assessment and support and guidance methods)

3. The criteria for assessing whether the intended learning outcomes have been achieved and for differentiating the performance of students.

They are dependent on the “level” at which the module is targeted

Page 7: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

7

Level / Qualification Descriptors?

A Level is an indicator of the relative demand, complexity, depth of study and learner autonomy

A Level (Qualification) Descriptor is a generic statement describing the characteristics and context of learning expected at each specific level.

Module Learning Outcomes and Assessment Criteria are reviewed with respect to a level descriptor when developing a module and assigning credit at the appropriate level.

QAA defines Qualification Descriptors in the Qualification Framework for Higher Education as Level C, Level I, Level H and Level M. These broadly correspond to Years 1, 2, 3 and first year postgraduate level respectively

SEEC defines Level Descriptors as Level Zero, Level 1, Level 2, Level 3 and Master’s Level. These broadly correspond to preliminary Year, Years 1, 2, 3 and first year postgraduate level respectively

Page 8: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

8

Level Descriptors? Think of Level Descriptors as a means of communication about

expectations of student’s study They are not rigid but developmental

Student Learning is commonly described in terms of : complexity of knowledge and understanding standard of cognitive skills key or transferable skills achieved the expected responsibility of the learner the autonomy or independence of the learner amount of guidance required by the learner

Be careful of differences in the implied standard of learning! Is it defined for a threshold student, an average (or typical!)

student or the best student?

Page 9: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

9

Hierarchy of the Cognitive Domain

6 EvaluationAbility to make a judgement of the worth of something

7 Synthesis Ability to combine separate elements into a whole

8 Analysis Ability to break a problem into its constituent parts and establish the relationships between each one

9 Application Ability to apply rephrased knowledge in a novel situation

10 Manipulation Ability to rephrase knowledge

1 Knowledge That which can be recalled

Based on Bloom’s Taxonomy of Educational Objectives

Page 10: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

10

Programme Learning Outcomes

Guidance is provided by:

QAA Level H Descriptors and Benchmark Statements for

Engineering

Computing

Biosciences

SEEC’s generic HE Level 3 Definition

Engineering Professors Conference’s 26 “Ability to” Statements for Engineering Programmes

Also see the accreditation requirements used by Professional Bodies (e.g. Engineering Council’s SARTOR 97 or latest UK SPEC)

Page 11: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

11

QAA General Honours Descriptors Level H - A Students successfully completing programme requirements at this

level will have demonstrated:

1. a systematic understanding of key aspects of their field of study, including acquisition of coherent and detailed knowledge, at least some of which is at or informed by, the forefront of defined aspects of a discipline;

2. an ability to deploy accurately established techniques of analysis and enquiry within a discipline;

3. conceptual understanding that enables the student: to devise and sustain arguments, and/or to solve problems, using ideas

and techniques, some of which are at the forefront of a discipline; and to describe and comment upon particular aspects of current research, or

equivalent advanced scholarship, in the discipline;

4. an appreciation of the uncertainty, ambiguity and limits of knowledge;

5. the ability to manage their own learning, and to make use of scholarly reviews and primary sources (e.g. refereed research articles and/or original materials appropriate to the discipline).

Page 12: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

12

QAA General Honours Descriptors Level H - B Typically, successful students at this level will be able to: a) apply the methods and techniques that they have learned to review,

consolidate, extend and apply their knowledge and understanding, and to initiate and carry out projects;

b) critically evaluate arguments, assumptions, abstract concepts and data (that may be incomplete), to make judgements, and to frame appropriate questions to achieve a solution - or identify a range of solutions - to a problem;

c) communicate information, ideas, problems, and solutions to both specialist and non-specialist audiences;

and will have: d) qualities and transferable skills necessary for employment requiring:

the exercise of initiative and personal responsibility; decision-making in complex and unpredictable contexts; and the learning ability needed to undertake appropriate further training of a

professional or equivalent nature.

Page 13: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

13

SEEC Generic Level Descriptors: Development of Knowledge and Understanding

HE Level 1 HE Level 2 HE Level 3

The Learner has a given factual and /or conceptual knowledge base with emphasis on the field of study and appropriate technology

The Learner has a detailed knowledge of major theories of the discipline and an awareness of a variety of ideas, contexts and frameworks

The Learner has a comprehensive / detailed knowledge of a major discipline with areas of specialisation in depth and awareness of the provisional nature of knowledge

The Learner can demonstrate awareness of ethical issues in current areas of study and is able to discuss these in relation to personal beliefs and values

The Learner is aware of the wider social and environmental implications of area of study and is able to debate issues in relation to more general perspectives

The Learner is aware of personal responsibility and professional codes of conduct and can incorporate a critical ethical dimension into a major piece of work

Kn

ow

led

ge

B

as

eE

thic

al

Iss

ue

s

Page 14: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

14

SEEC Generic Level Descriptors : Cognitive / Intellectual Skills - A

HE Level 1 HE Level 2 HE Level 3

The Learner can analyse with guidance using given classifications / principles

The Learner can analyse a range of information with minimum guidance using given classifications / principles and can compare alternative methods and techniques for obtaining data

The Learner can analyse new and / or abstract data and situations without guidance, using a range of techniques appropriate to the subject

The Learner can collect and categorise ideas and information in a predictable and standard format

The Learner can reformat a range of ideas and information towards a given purpose

The Learner with minimum guidance can transform abstract data and concepts towards a given purpose and can design novel solutions

An

aly

sis

Sy

nth

es

is

Page 15: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

15

SEEC Generic Level Descriptors : Cognitive / Intellectual Skills - B

HE Level 1 HE Level 2 HE Level 3

The Learner can evaluate the reliability of data using defined techniques and / or tutor guidance

The Learner can select appropriate techniques of evaluation and can evaluate the relevance and significance of the data collected

The Learner can critically evaluate evidence to support conclusions / recommendations, reviewing its reliability validity and significance. Can investigate contradictory information / identify reasons for contradictions

The Learner can apply given tools / methods accurately and carefully to a well defined problem and begin to appreciate the complexity of the issues

The Learner can identify key elements of problems and choose appropriate methods for their resolution in a considered manner

The Learner is confident and flexible in identifying and defining complex problems and can apply appropriate knowledge and skills to their solution

Ev

alu

ati

on

Ap

plic

ati

on

Page 16: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

16

SEEC Generic Level Descriptors : Key / Transferable Skills - A

HE Level 1 HE Level 2 HE Level 3

The Learner can work effectively with others as a member of a group and meet obligations to others (e.g. tutors, peers and colleagues).

The Learner can interact effectively with a team / learning group, giving and receiving information and ideas and modifying response where appropriate.

The Learner can interact effectively with a team / learning group / professional group, recognise, support or be proactive in leadership, negotiate in a professional context and manage conflict.

The Learner can work within an appropriate ethos and can use and access a range of learning resources.

The Learner can manage learning using resources for the discipline. Can develop working relationships of a professional nature within the discipline.

The Learner with minimum guidance can manage own learning using a full range of resources for the discipline. Can work professionally within the discipline.

Gro

up

W

ork

ing

Le

arn

ing

R

es

ou

rce

s

Page 17: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

17

SEEC Generic Level Descriptors : Key / Transferable Skills - B

HE Level 1 HE Level 2 HE Level 3

The Learner can evaluate own strengths and weakness within criteria largely set by others.

The Learner can evaluate own strengths and weakness, challenge received opinion and develop own criteria and judgement.

The Learner is confident in application of own criteria of judgement and can challenge received opinion and reflect on action. Can seek and make use of feedback.

The Learner can manage information, collect appropriate data from a range of sources and undertake simple research tasks with external guidance.

The Learner can manage information. Can select appropriate data from a range of sources and develop appropriate research strategies.

The Learner can select and manage information, competently undertake reasonably straight-forward research tasks with minimum guidance

Se

lf

Ev

alu

ati

on

Info

rma

tio

n

Ma

ng

eme

nt

Page 18: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

18

SEEC Generic Level Descriptors : Key / Transferable Skills - C

HE Level 1 HE Level 2 HE Level 3

The Learner can take responsibility for own learning with appropriate support.

The Learner can take responsibility for own learning with minimum direction.

The Learner can take responsibility for own work and can criticise it.

The Learner can communicate effectively in a format appropriate to the discipline and report practical procedures in a clear and concise manner.

The Learner can communicate effectively in a manner appropriate to the discipline and report practical procedures in a clear and concise manner in a variety of formats.

The Learner can engage effectively in debate in a professional manner and produce detailed and coherent project reports

Au

ton

om

yC

om

mu

nic

ati

on

s

Page 19: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

19

SEEC Generic Level Descriptors : Key / Transferable Skills - D

HE Level 1 HE Level 2 HE Level 3

The Learner can apply given tools / methods accurately and carefully to a well defined problem and begin to appreciate the complexity of the issues in the discipline.

The Learner can identify key areas of problems and choose appropriate tools / methods for their resolution in a considered manner.

The Learner is confident and flexible in identifying and defining complex problems and the application of appropriate knowledge, tools / methods to their solution.

Pro

ble

m

So

lvin

g

Page 20: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

20

SEEC Generic Level Descriptors : Practical Skills

HE Level 1 HE Level 2 HE Level 3

The Learner can operate in predictable, defined contexts that require use of a specified range of standard techniques.

The Learner can operate in situations of varying complexity and predictability requiring application of a wide range of techniques.

The Learner can operate in complex and unpredictable contexts, requiring selection and application from a wide range of innovative or standard techniques.

The Learner is able to act with limited autonomy, under direction or supervision, within defined guidelines.

The Learner is able to act with increasing autonomy, with reduced need for supervision and direction, within defined guidelines.

The Learner is able to act autonomously, with minimal supervision or direction, within agreed guidelines.

Ap

plic

ati

on

o

f S

kills

Au

ton

om

y in

S

kill

us

e

Page 21: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

21

Engineering Professors Conference EPC Generic “Ability to” statements - A

1. Ability to exercise Key Skills in the completion of engineering-related tasks at a level implied by the benchmarks associated with the following statements.

Key Skills for engineering are Communication, IT, Application of Number, Working with Others, Problem Solving, Improving Own Learning and Performance.

2. Ability to transform existing systems into conceptual models. This means the ability to: a) Elicit and clarify client's true needs. b) Identify, classify and describe engineering systems. c) Define real target systems in terms of objective functions, performance

specifications and other constraints (i.e. define the problem). d) Take account of risk assessment, and social and environmental impacts, in the

setting of constraints (including legal, and health and safety issues). e) Select, review and experiment with existing engineering systems in order to

obtain a database of knowledge and understanding that will contribute to the creation of specific real target systems.

f) Resolve difficulties created by imperfect and incomplete information. g) Derive conceptual models of real target systems, identifying the key

parameters.

Page 22: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

22

Engineering Professors Conference EPC Generic “Ability to” statements - B

3. Ability to transform conceptual models into determinable models. This means the ability to: a) Construct determinable models over a range of complexity to suit a range of

conceptual models. b) Use mathematics and computing skills to create determinable models by

deriving appropriate constitutive equations and specifying appropriate boundary conditions.

c) Use industry standard software tools and platforms to set up determinable models.

d) Recognise the value of Determinable Models of different complexity and the limitations of their application.

4. Ability to use determinable models to obtain system specifications in terms of parametric values.

This means the ability to: a) Use mathematics and computing skills to manipulate and solve determinable

models; and use data sheets in an appropriate way to supplement solutions. b) Use industry standard software platforms and tools to solve determinable

models. c) Carry out a parametric sensitive analysis. d) Critically assess results and, if inadequate or invalid, improve knowledge

database by further reference to existing systems, and/or improve performance of determinable models.

Page 23: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

23

5. Ability to select optimum specifications and create physical models. This means the ability to: a) Use objective functions and constraints to identify optimum specifications. b) Plan physical modelling studies, based on determinable modelling, in order to

produce critical information. c) Test and collate results, feeding these back into determinable models.

6. Ability to apply the results from physical models to create real target systems.

This means the ability to: a) Write sufficiently detailed specifications of real target systems, including risk

assessments and impact statements. b) Select production methods and write method statements. c) Implement production and deliver products fit for purpose, in a timely and

efficient manner. d) Operate within relevant legislative frameworks.

7. Ability to critically review real target systems and personal performance. This means the ability to: a) Test and evaluate real systems in service against specification and client needs. b) Recognise and make critical judgements about related environmental, social,

ethical and professional issues. c) Identify professional, technical and personal development needs and undertake

appropriate training and independent research.

Engineering Professors Conference EPC Generic “Ability to” statements - C

Page 24: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

24

Working with Level Descriptors

They should be seen as helpful guides rather than dictates

They are generic and may contain sections not appropriate to a particular programme

They may not cover all possible learning that is relevant to the programme.

The words become more meaningful if you look at descriptors at the previous and next level

Look at the relationship between descriptors at the same level as they do not function independently of each other

Use them to provide an appropriate vocabulary to describe learning

Page 25: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

25

A Module Specification from a Clean Sheet!

Generic Level DescriptorsGeneric Level Descriptors

Translate Level Descriptors into Subject Descriptors

Translate Level Descriptors into Subject Descriptors

Identify Aim of Module

Identify Aim of Module

Write Learning OutcomesWrite Learning Outcomes

Write Threshold and Grading Assessment

Criteria

Write Threshold and Grading Assessment

Criteria

Develop Assessment Method to test achievement of assessment

criteria

Develop Assessment Method to test achievement of assessment

criteria

Develop a teaching strategy to enable learners to reach the learning outcomes / assessment criteria

Develop a teaching strategy to enable learners to reach the learning outcomes / assessment criteria

Develop the module and rethink it including the

initial learning outcomes

Develop the module and rethink it including the

initial learning outcomes

Page 26: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

26

Writing Module Specifications

Clearly identify the intended level of the module Formulate clear and unambiguous intended threshold and possibly

desirable learning outcomes for the module The threshold learning outcomes identify the essential learning to

merit the award of the credits for this module Desirable learning outcomes can be included to provide guidance of

learning above threshold which will be assessed to provide grading Identify assessment criteria that encourage learning at the

appropriate level Threshold assessment criteria should specify how satisfactory

performance of the threshold module learning outcomes can be demonstrated

Grading-related assessment criteria are used to provide incentive for higher achievement above threshold performance

Page 27: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

27

Writing a Module Description in Reality!

Generic Level DescriptorsGeneric Level Descriptors

Translate Level Descriptors into Subject Descriptors

Translate Level Descriptors into Subject Descriptors

Existing Aims, Objectives and Syllabus Content of Module

Existing Aims, Objectives and Syllabus Content of Module

Rewrite Objectives as Learning Outcomes

Rewrite Objectives as Learning Outcomes

Write Threshold and Grading Assessment

Criteria

Write Threshold and Grading Assessment

Criteria

Modify Existing Assessment Methods to test achievement

of assessment criteria

Modify Existing Assessment Methods to test achievement

of assessment criteria

Modify current teaching strategy to enable learners to reach the learning outcomes / assessment criteria

Modify current teaching strategy to enable learners to reach the learning outcomes / assessment criteria

Develop the module and

rethink it including the initial learning

outcomes

Develop the module and

rethink it including the initial learning

outcomes

Existing Assignments and Laboratories

Existing Assignments and Laboratories

Review Aim of Module

Review Aim of Module

Page 28: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

28

Writing Learning Outcomes

A well written learning outcome is likely to contain :

A verb that indicates what the learner is expected to be able to do at the end of the period of learning

Word(s) that indicate on what or with what the learner is acting. If the outcome is about a skill then the word may describe the way the skill is performed

Word(s) that indicate the nature (in context or terms of standard) of the performance required as evidence that the learning was achieved

Page 29: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

29

Examples of Learning Outcomes

The learner is expected to be able to:

demonstrate understanding of the purpose and operation of the hardware and software components present in personal computers, microprocessors and embedded processor applications. (Level 1 Computer Systems)

explain the physical basis of the operation of Metal Oxide Semiconductor Transistors (MOSFETs) and obtain the small signal model of a MOSFET (Level 2 Electronic Devices)

Page 30: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

30

Features of Good Learning Outcomes

Must be achievable by students within the time available and at the level of learning at which the students are. Be realistic!

Written in terms of the “learner being expected to be able” or as “intended learning outcomes”

Should specify areas of learning rather than specific curriculum There should be in the range of 4 to 10 learning outcomes per

module – too many makes statements of the assessment criteria unmanageable

Should be written in a language that is understood by all and is unambiguous

Each intended learning outcome should represent a major achievement expected by students at the end of the module

Learning outcomes must be assessable by a reasonable and manageable form of assessment within the time allocated to the module

Achievement of each threshold learning outcome is essential in order to pass the module

Page 31: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

31

Vocabulary for Writing Learning Outcomes and Assessment Criteria - A Verbs which require evidence of knowing:

Be aware of, define, describe, extract, identify, know, label, list, match, measure, name, organise, outline, present, recall, recognise, recount, relate, repeat, select, state, underline, write.

Verbs which require evidence of comprehension: Clarify, classify, compare, comprehend, contrast, convert, defend,

describe, discuss, distinguish, estimate, exemplify, explain, express, extend, find, formulate, generalise, give examples of, identify, illustrate, indicate, infer, interpret, judge, justify, name, paraphrase, perform, predict, present, report, represent, restate, rewrite, select, summarise, translate, understand.

Verbs which require evidence of knowledge / understanding: Apply, arrange, assess, change, choose, compute, construct,

demonstrate, discover, draw (up), exemplify, explain how, find, give examples, illustrate, manipulate, modify, operate, order, practice, predict, prepare, produce, relate, select, show, solve, use, verify

Page 32: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

32

Verbs which require evidence of analysis: analyse, break down, calculate, categorise, compare, conclude, contrast,

criticise, devote, diagnose, differentiate, distinguish between, divide, elucidate, evaluate, examine, identify, illustrate how, infer, justify, outline, point out, precis, question, recognise, relate, resolve, select, separate, subdivide.

Verbs which require evidence of synthesis: account for, alter, argue, build up, combine, compile, compose, conclude,

create, derive, design, develop, devise, engender, enlarge, explain, formulate, generalise, generate, integrate, manage, modify, order, organise, plan, prepare, present, produce, propose, put together, rearrange, reconstruct, relate, reorganise, report, restate, revise, select, structure, suggest, summarise, synthesise, teach, tell, write.

Verbs which require evidence of evaluation appraise, assess, choose, compare, conclude, contrast, criticise, defend,

describe how, determine, discriminate, estimate, evaluate, judge, justify, measure, question, rate, value.

Vocabulary for Writing Learning Outcomes and Assessment Criteria - B

Page 33: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

33

Examples of Learning Outcomes

Discuss and comment on the following threshold learning outcomes:

Describe the structure of telecommunications networks (Level 1)

Understand the ideas of differentiation and integration (Level 1)

Understand the principles of human cognitive systems and motor performance when operating interactive computer systems (Level 2)

Demonstrate oral and written communication skills (Level 2)

Appreciate the use of the z-transform in digital signal processing (Level 3)

Improve at working in software engineering teams (Level 3)

Describe the principal characteristics of human vision, hearing and speech relevant to audiovisual communication and their exploitation in image, video, and audio compression (Level 3)

Page 34: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

34

Purposes of Assessment

To provide a licence to proceed to the next stage of the programme or to graduation

To classify the performance of the student in rank order

To improve student learning by providing feedback on his or her strengths and weaknesses

To motivate the student to learn

To provide information for future selection or as a licence to practice

To evaluate the course and improve teaching

These may overlap or conflict!

A common error is to assume that the results of an assessment task used for one purpose are appropriate for another purpose

Page 35: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

35

Some Principles of Assessment

Assessment shapes learning therefore change assessment to change learning

Match the assessment tasks to the learning outcomesMatch the assessment criteria to the task and the

learning outcomesKeep the assessment criteria simpleBe fair, reliable and valid in your markingProvide meaningful and timely feedback

Do we apply these principles?

Page 36: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

36

Some Assessment Methods Formal Examinations

Knowledge and Understanding Analytical Ability Problem Solving Communication Skills

Progress tests Knowledge and Understanding Simple Problem Solving

Formal Practical Experiments Following Instructions and Recording Results Practical Ability Written Reports or Software Documentation

Projects Requirement Analysis, Research, Problem Solving / Synthesis Practical Development Oral and Written Communication Possibly Group Working

Other Coursework Assignments

Page 37: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

37

Common Weaknesses in Assessment

Tasks do not match the stated outcomes Criteria do not match the tasks or outcomes Criteria not known to and/or not understood by the students Overuse of one method of assessment Overload of students and/or staff

Insufficient time for students to do the assignments To many assignments with the same deadline Insufficient time for staff to mark examination or assignment

Absence of well defined criteria so consistency is difficult to achieve Unduly specific criteria which create a straightjacket for students Inadequate or superficial feedback provided to students Wide variation in marking between modules and within assessors Variation in assessment demands of different modules

Page 38: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

38

Designing Assessments

Some questions to be considered

What are the learning outcomes to be assessed? What are the capabilities / skills either implicit of explicit, within the

learning outcomes? Is the method of assessment chosen appropriate to the outcomes

and skills? Is the method relatively efficient in terms of student time and staff

time? What alternatives are there and what are their advantages and

disadvantages? Does the specific assessment task match the outcomes and skills? Are the marking schemes or criteria appropriate?

Page 39: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

39

Assessment Criteria – a definition

Not to be confused with assessment methods or tasksAssessment Criteria provide a clear indication of how

achievement may be demonstrated Often specified with respect to each learning outcome,

they describe what a learner is expected to do in order to demonstrate that the learning outcome has been achieved.

Assessment Criteria may be used in three ways: To confirm achievement of threshold standards To define what is expected in order to achieve each of the

grades being awarded To specify a template of characteristics or qualities against which

the students’ performance of the assessment task will be judged

Page 40: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

40

Writing Assessment Criteria

1. Consider the learning outcome being tested

2. Consider the assessment task set

3. Brainstorm requirements for, or attributes of, successful performance of the assessment task

4. If necessary specify the range to clarify contextual factors and the level

5. Focus on what is essential and categorise the requirements or attributes into clearly worded criteria

6. Check that the criteria are measurable or assessable in valid and reliable ways and that the criteria are clear and unambiguous

7. Repeat steps 3, 4 and 5 until you are fully satisfied

8. Publish the assessment criteria with the assessment task and identify the intended learning outcome or outcomes that the task is assessing

Page 41: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

41

An Example

The learner is expected to be able to: demonstrate understanding of the purpose and operation of the

hardware and software components present in personal computers, microprocessors and embedded processor applications. (Level 1 Computer Systems)

Assessment Task? Hardware Laboratory: Test and evaluate the operation of

computer hardware components by constructing a serial adder on a logic patch-board using available TTL devices.

Assessment Criteria: Connect correctly the following functional elements on the patch-

board provided: EX-OR, Half Adder, Full Adder, 4 to1 Multiplexer, D-type bistable carry store, 4-bit register, 4-bit counter, and links modules to form a 4-bit serial adder.

Test each module and record results Complete the multi-choice test and score better than 70%

Page 42: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

42

Assessment Criteria – Exercise 4 U

Final Year Individual Project 30 credits: Aim: The project provides the student with the opportunity to

apply knowledge and practical skills gained during the degree programme to the solution of a problem agreed with the supervisor.

Form Groups of 3 or 4

1. Write a Threshold Learning Outcome for this module

2. Develop an Assessment Task to address this learning outcome

3. Write Assessment Criteria for this task

Repeat 1, 2 and 3 for a desirable graded learning outcome!

Page 43: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

43

Report Back and Discussion

Each group to report their learning outcomes assessment tasks assessment criteria

Comments

Discussion

Page 44: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex
Page 45: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

Achievement of Module Threshold Learning Outcomes

Peter NoakesDepartment of Electronic Systems Engineering

University of Essex

Page 46: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

46

Current Developments at Essex

Internally Funded Project

The objective is to develop module specifications and structures where the threshold and extended learning outcomes, and associated assessment methods and criteria are clearly defined for students.

As a result a student’s satisfaction of threshold learning outcomes should be readily determined

Initially concentrating of Year 1 Modules offered by ESE

Working with colleagues in Departments of Computer Science and Biological Sciences

Page 47: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

47

The Module Description

For each module Staff and Students will have a clear view of : the threshold learning outcomes their method of assessment and the assessment criteria used. the extended (desirable) learning outcomes the assessment activities used for grading the assessment criteria used for grading above

threshold the expected time commitment for the activities

This should ensure that a graduating student attains his or her maximum potential

Page 48: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

48

Project Proposals

In future a module description will identify separately the threshold and extended (desirable) learning outcomes and syllabus

The threshold syllabus defines the topics and associated skills, their method of assessment and associated assessment criteria, that will be used to demonstrate threshold level achievement for the module

The extended syllabus defines the topics, their method of assessment and associated assessment criteria, that will be used to demonstrate understanding and application of their knowledge above the threshold level for the module allowing grading above Third class.

The threshold aspects of the module will be taught conventionally with clear guidance to required reading, and supported by regular formative on-line MCQ testing

The extended aspects of the module will be taught by a combination of special topic lectures and directed self study with associated supporting problem classes

Page 49: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

49

Module Specifications

Module specifications to be captured by filling form to provide entry to a central database

Data to be extracted for various audiences by running different reports

Linked to marks database to provide clear abstraction of satisfactory threshold achievement

Page 50: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

50

Module Assessment Current Approach

Largely based on End of Year Examinations

All have an in-term Multi-choice Progress Test

Many modules have a Practical Laboratory (Hardware / Software / CAD / Database / Networks / Microprocessor / Matlab / Web) – Assessment in various ways including log book, demonstration, presentation, report, oral, OMR test

Some have Assignments – electronic submission, paper submission, demonstration, oral

Some modules include Projects – demonstration, presentation, report, oral

Module mark generated by forming a weighted aggregate of the marks awarded to individual elements

A 40% overall aggregate does not ensure that all learning threshold outcomes have been satisfied!

Page 51: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

51

Proposed Module Assessment

Each module has 100 marks available

Progress Test – threshold material 10 marks

Examination – 2 hour paper – 2 parts

Part A – say 8 to 20 questions on threshold material to produce say 20 marks

Part B – 3 to 5 on extended material answer 2 or 3 for full marks – applying knowledge, problem solving – producing say 40 marks

Laboratory or Assignment Coursework – contribute say 20 marks on threshold material and say 10 marks on extended material.

Page 52: Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex

LTSN Workshop at University of Essex 26th June 2003

52

Graphical Representation

10

20

30

40

50

60

70

80

90

100

Progress Test

Practical Activity

Part A examination

Practical Activity

Part B examination

Threshold Assessments

Degree Graded Assessments

Module Marks