Using REAP Principles and a Student Timeline to help Ulster staff reflect on

their Assessment & Feedback Strategy

Enhancing the Student Experience: Project findings

Fiona Doherty, Instructional Technologist fp.doherty@ulster.ac.uk

Viewpoints Project, JISC Curriculum Design Programme

1st May 2009

Background

Viewpoints – JISC Curriculum Design Programme Aim: Create a series of simple, user-friendly reflective tools

for staff, promoting and enhancing good curriculum design.

Student Profiler – Student focussed cohort review tool

5 reflective tools/strands: Assessment and Feedback Information Skills and Resources Community Teaching Styles Course Pathways / module selection

Importance of Assessment & Feedback

“Assessment is the curriculum.”“Students will study what they think will be assessed.”

(Ramsden, 1992)

“Assessment is at the heart of the undergraduate experience. Assessment defines what students regard as important, how they

spend their time and how they come to see themselves as students and then as graduates. It follows, then that it is not the curriculum

which shapes assessment, but assessment which shapes the curriculum and embodies the purpose of higher education.”

(Brown & Knight, 1994)

Ramsden, Biggs, Race, Rowntree, Nicol, Macfarlane-Dick, Gardner, Snyder, Brown, Gibbs, Simpson, Knight, Yorke, Tinto, Bowden, Calder, Boud, Sadler, Rust, etc. JISC, HE Academy, QAA

Ulster Assessment & Feedback

Assessment handbook (Academic Office website) Assessment resources (Staff Dev website) T&L Strategy, STAR & 1st Year Teaching Guidelines

Constructive alignment (Biggs, 1999) Assessment for learning rather than of learning Formative v summative assessment Feedback v feed-forward Assessment & feedback methods Who assess – self, peer, tutor? Discipline specific, level specific

Defining the Student Timeline

Break down and define the student lifecycle based on a typical 3 year undergraduate course at the University of Ulster.

Looking at the Ulster’s Academic Calendar for 09/10 – 11/12. Identifying important points in the student lifecycle, e.g. the first few weeks with regard

to retention. Use generic phases that should map to a majority of courses from a variety of faculties. Semester by semester approach.

0 Wk 1 Wks 2-3 Wks 4-7 Wks 8-12

Exam PeriodInduction Orientation First few weeks Mid-semester Final phase

How to prompt A&F Reflection?

Use of principles/guidelines, such as:

National University of Ulster

Re-Engineering Assessment Practices (REAP) Principles of good formative assessment and feedback.

12 formative assessment principles developed by the University of Strateclyde by the Assessment Working Group. These principles are based on recent research on assessment (Yorke, 1987: Nicol & Macfarlane-Dick 2004, 2006, in press: Boud, 2000: Knight, 2002: Knight and Yorke, 2003), the QAA guidelines on assessment of student learning (2006) and published studies of University policies and practices that are associated with high levels of student success (Kuh, Kinzie, Schuh and Whitt, 2003: Tinto, 1991).

Ulster’s Assessment Handbook (May 2007)

160 page doc – practical guide – intended to promote best practice. Covers many aspects of assessment practice – offers an intro to topics and acts as a key reference doc for policies relating to assessment.

Ulster’s Guidelines for First Year Undergraduate Teaching

Developed alongside the T&L strategy. Section 4 has 4 guidelines relating to Assessment & Feedback.

Chickering & Gamson’s Good Practice in Undergraduate Education

7 principles proposed in the US by Chickering and Gamson (1991) based on their review of good undergraduate education. These principles are a starting point in trying to understand the relationship between the theory and practice of assessment.

Ulster’s STAR Guidelines for the Management of Student Transition

Guidelines for the Management of Student Transition – The STAR Project (Student Transition and Retention), Cook, Rushton, McCormick, Southall (2005).

About the REAP Project & Principles

Re-engineering Assessment Practices (REAP)

REAP Partners include University of Strathclyde, University of Glasgow and Glasgow Caledonian University

Draws on current educational research (Gibb’s & Simpson’s 11 Conditions, Knight & Yorke, Boud)

Emphasis on student responsibility with goal to develop in students the ability to monitor, manage and self-direct their own learning.

11 Principles of good assessment design (7 empower, 4 engage students) 12 Principles for effective assessment and feedback with questions &

implementation techniques (David Nicols, published by QAA)

Principles of Good Assessment Design

"empower" 1. Engage students actively in identifying or formulating criteria 2. Facilitate opportunities for self-assessment and reflection 3. Deliver feedback that helps students self-correct 4. Provide opportunities for feedback dialogue (peer and tutor-

student) 5. Encourage positive motivational beliefs and self-esteem 6. Provide opportunities to apply what is learned in new tasks 7. Yield information that teachers can use to help shape teaching

Principles of Good Assessment Design (cont.)

"engage" 8. Capture sufficient study time and effort in and out of class 9. Distribute students’ effort evenly across topics and weeks. 10. Engage students in deep not just shallow learning activity 11. Communicates clear and high expectations to students.

Adapted from Nicol and Macfarlane-Dick (2006) and Gibbs and Simpson (2004)

Other Principles

Chickering& Gamson

Good Practice in Undergraduate Education4. Gives prompt feedback

Ulster’sFirst yearGuidelines

Assessment and feedback Section The assessment strategy needs to be co-ordinated across modules. There should be greater emphasis on formative feedback early in the semester. Marking criteria should be made explicit to students. Feedback should be timely and designed to enhance student progress; feedback

should be forward looking rather than dwelling on past mistakes.

STARGuidelines

Curriculum Development Section Students should receive regular, formative evaluations of their work early in their

course or course component.

How to reuse the REAP Principles?

1. Clarify good performance 7. Develop self-assessment and reflection

2. Encourage time and effort on tasks 8. Provide students with choice

3. Deliver high quality feedback 9. Involve students when developing criteria

4. Provide opportunities to act on feedback 10. Develop learning communities

5. Have a positive impact on learning 11. Motivate students

6. Encourage interaction and dialogue 12. Inform and shape your teaching

A&F Mapping to a Student Timeline

Wks 0 1 2-3 4-7 8-12 13-16

Induction Orientation First few weeks Coursework Exams

RP 11- Motivate 8 – Choice11- Motivate1 - Clarify

expectations

3- Quality feedback to self-correct

4- feed forward1 - Clarify expectations5 – Constructive alignment7 – reflection & self-

assessment10- learning communities11. motivate

7 – reflection & self-assessment

2 – deep learning6 – discussion10 – learning communities12 – inform teaching4 – feed forward

1 - Clarify expectations5 – Constructive

alignment8- choice4 – feed forward3 – quality feedback7 – reflection9 – involve in process

1 - Clarify expectations

5 – Constructive alignment

9 – involve in process

8- choice

CG Gives prompt feedback Emphasises time on tasks

Emphasises time on tasks

UG There should be greater emphasis on formative feedback early in the semester.

Feedback should be timely and designed to enhance student progress; feedback should be forward looking rather than dwelling on past mistakes.

SG Students should receive regular, formative evaluations of their work early in the course or course components.

Semester Timeline

Does your assessment

and feedback…

Clarify expectations

Encourage deep learning

Provide quality feedback

Allow feedback to feed-forward

Constructively align to the

teaching

Allow for feedback

discussion

Facilitate reflection and

self-assessment

Provide students with

choice

Involve students in

process

Support learning

communities

Activate motivation in

students

Processes inform teaching

0 Wk 1 Wks 2-3 Wks 4-7 Wks 8-12Examination

PeriodInduction Orientation First few weeks Mid-semester Final phase

Clarify expectations

Clarify expectations

Activate motivation in

students

Provide quality feedback to self-

correct Allow feedback to

feed-forwardClarify expectationsActivate motivation

in studentsConstructively align

to the teaching

Encourage Deep Learning

Allow feedback to feed-forward

Facilitate reflection and self-assessment

Provide quality feedback to self-

correctConstructively align

to the teachingProcesses inform

teaching

Clarify expectations

Constructively align to the teachingAllow for feedback

discussionFacilitate

reflection and self-assessment

Activate motivation in

students

Clarify expectations

Constructively align to the teaching

Activate motivation in

students

Provide students with choice

Support learning communities

Allow for feedback discussion

Provide students with choice

Involve students in process

Allow for feedback discussion

Support learning communities

Provide students with choice

Involve students in process

Activate motivation in students

Provide students with choice

Involve students in processAllow for feedback

discussion

Provide students with

choiceAllow for feedback

discussionAllow feedback to feed-forward

Criteria

Essential

Desired

Initial concept of a reflective A&F Tool

Initial Feedback on Tool Concept – Ulster Staff

A&F Reflective Task & Questionnaire

Sent to 21 academic staff from a variety of disciplines on 23 April 09

Feedback so far (based on 5 responses) – Found task quite easy and useful Found the principles easy to

understand and comprehensive Helped them reflect on their A&F

strategy

Initial Feedback on Tool Concept – Ulster staff (cont.)

“Allowed me time to think through the assessment processes and my expectations at each stage.”

“It allowed me to consider the student experience of assessment and what assessment should be achieved in the development of the

student.”

“It was very useful reflecting on what the student will be exposed to from initiation to the module right through to assessment.”

“It helped me consider how reflection on assessment could be configured in various stages throughout the module.”

Development Partners/ Stakeholders

Ulster T&L PVC CHEP/Staff Development Quality Management and Audit Unit Co-ordinators of Student Learning Access & Distributed Learning

External REAP Project – David Nicols Other JISC Curriculum projects JISC Cluster – Open University, University of Strathclyde Project Critical Friend – Peter Bullen HE Academy

Development of the A&F Tool

Deliver focus groups & workshops with Ulster staff

Develop from concept to working tool (based on feedback)

Consider – Usability, Outputs, Purpose & Audience

Identify benefits for: Users Learners Institution

Resources

Viewpoints project blog - viewpointsproject.blogspot.com/JISC Curriculum Design Programme

jisc.ac.uk/whatwedo/programmes/elearningcapital/curriculumdesign/fundedprojects.aspx

REAP Project & Principles - reap.ac.uk/QAA Enhancement Themes – Curriculum design for the first year -

enhancementthemes.ac.uk/documents/firstyear/Curriculum_Design_final_report.pdfChickering & Gamson’s Good Practice in Undergraduate Education -

honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/7princip.htm

Ulster Assessment Handbook - ulster.ac.uk/academicoffice/download/Handbooks/Assessment%20Handbook%20-%202007.pdf

Ulster Assessment Resources - staffdev.ulster.ac.uk/index.php?page=assessmentSTAR Guidelines - ulster.ac.uk/star/resources/star_guidelines.pdfGuidelines for 1st Year Undergraduate Teaching - ulster.ac.uk/star/STARfellows/s0743guidelines.pdfUlster Academic Calendar - ulster.ac.uk/academicservices/student/common/attendance.pdf

Biggs, J. (1999) Teaching for Quality Learning at University. Buckingham: Open University Press/SRHE.

Brown, S. & Knight P. (1994) Assessing Learners in Higher Education. Kogan Page. Ramsden, P. (1992) Learning to teach in higher education London Routledge.