welcome. first year assessment redesign – a programme approach galvin, Á, o’neill, g, noonan,...
TRANSCRIPT
First Year Assessment ReDesign – A Programme Approach
Galvin, Á, O’Neill, G, Noonan, E, Jennings, D.
University College Dublin
Introduction
• Evaluation of a programme approach to the review and re-design of first year assessment to achieve effective learning and students engagement, while still being efficient to staff.
• Central strand of a wider strategic initiative – Focus on First Year –… “to foster early and lasting student engagement”. (UCD Education Strategy, 2009-2014)
Literature Review Curriculum Design & Programme Mapping• …importance of addressing assessment from a strategic
perspective (Knight, 2000)• …efficiency of the programme approach & importance of
alignment of assessment with programme philosophy, outcomes & GAs, i.e. curriculum mapping (Fink, 2003; Ornstein & Hunkins, 2009; Wiggins & McTighe, 2005)
Curriculum change processes• …clear sense of purpose, strong leadership & local ownership –
all important factors in successful curriculum change initiatives (Blackmore & Kandiko, 2012; Dempster et al, 2012)
• …“discipline-based department teams rather than individuals can be strategic targets for effective change” (Healy et al, 2013: p.42)
Proj
ect P
erso
nnel
:D
ean,
Pro
gram
me
Lead
, Sta
ge
Coor
dina
tors
Proj
ect
Pers
onne
l: Pr
og. L
eade
rs,
Stag
e &
M
odul
e Co
ordi
nato
rs
Workshop A:Articulation of
Programme & Stage Outcomes
Workshop B: Mapping Programme
Outcomes to assessment activities and identifying opportunities and issues
Workshop C: 1st Assessment Redesign against UCD Assessment
Principles
Equivalence document
Specific Assessment
ReDesign workshops
Project Methodology
7
Develop a coherent approach to use of assessment – mapping assessment to programme outcomes at each Stage
1= to indicate students are INTRODUCED to the programme outcome 1 A= ASSESSED (Summative)
2= to indicate the outcome is REINFORCED and the students afforded opportunity to practice 2 Blank= NOT ASSESSED3= indicates that students have had sufficient practice and can now demonstrate MASTERY 3
1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A
Programme Outcome 1 1 A 1 A 1 1 1 1 1 1 1 1 10.00 0.00 0.00 10 2
Programme Outcome 2 2 0.00 1.00 0.00 2 0
Programme Outcome 3 3 0.00 0.00 1.00 3 0
Programme Outcome 4 0.00 0.00 0.00 0 0
Programme Outcome 5 0.00 0.00 0.00 0 0
Programme Outcome 6 0.00 0.00 0.00 0 0
Programme Outcome 7 0.00 0.00 0.00 0 0
Programme Outcome 8 0.00 0.00 0.00 0 0
Programme Outcome 9 0.00 0.00 0.00 0 0
Programme Outcome 10 0.00 0.00 0.00 0 0
Programme Outcome 11 0.00 0.00 0.00 0 0
Programme Outcome 12 0.00 0.00 0.00 0 0
Sem
este
r One
Frequencyof occurance
Sem
este
r Two
Total Introduce
d
Total Reinforce
d
Total Master
y
Wieghted Total
How Often
Assessed?
Module name Module nameModule nameModule nameModule name Module nameModule nameModule nameModule nameModule name
Research Methodology (I)
• Face-to-face interviews with individual Programme Leaders (n=7) – a representative from each of five participating programmes was interviewed
• Focus: – Reflections on the project process– Extent to which project objectives were achieved– Analysis of changes to assessment
Research Methodology (II)
• Online survey circulated to First Year Module Coordinators (n=35)
• 22 MCs completed survey (62% response rate), representing 31 (76%) of modules in the project
• Focus: Extent & nature of planned changes to assessment, aligned with first year design principles
Research Results: Programme Leaders’ PerspectiveMotivation for getting involved• Existing interest/concern/need
• Programme Dean was the main driver
Strengths of the project process• Flexible & adaptable approach – customised to local needs
• The workshops – well structured/outcomes orientated/efficient • The role of T&L facilitators – ‘outside’ perspective & new ideas
Measuring the success of the project• Examples of changes to 1st Year assessment cited, e.g.
Assessment FOR learning; some revision to weighting, timing & quantity BUT query around the extent of changes implemented –
radical revision of Stage 1 had not occurred (yet)• Articulation of programme & stage outcomes – highly valued
• New stage 1 ‘concept’ module introduced
I think the fact that they were adaptable was very important, a real
strength
The mapping exercise was very useful…it highlighted the kinds of demands on
students & ‘bunching’ of assessments
So if there is someone watching from outside it’s always interesting, because first of all they tidy up
what you’re doing and make sense of it, and second of all they point out to you things that are
invisible to you because you are ‘native’.
We knew already there were a couple of modules that we weren’t completely happy with the mode of
assessment, and we have restructured those as a result of
going through assessment redesign.
Change takes time – at least two-three years.
The more we zoned in on the first year modules and what we would do differently it felt like the law of diminishing returns. The
‘big picture’ programme stuff was great, but when it came to ‘just change that bit in that
module’, it was like ‘so what’.
Research Results: Programme Leaders’ Perspective
Research Results: Module Coordinators’ Perspective Assessment Design Principles W* N ** %
Consider the demands of other parallel modules in the stage when planning my assessments
34 14 64%
Reduce assessment work-load for staff, i.e. attention to word-count, reducing number of submissions
33 11 50%
Design a more efficient & effective sequencing of the learning & assessment activities
30 10 45%
Reduce assessment workload for students, i.e. reduce number & size of assessment tasks
30 8 36%
Allow more assessment FOR learning opportunities, i.e. in/out of class activities where the primary focus is to allow students get feedback on their progress
30 12 56%
Introduce more authentic assessment 28 9 41%
Develop multiple opportunities for collaborative learning 25 7 32%
Create more time for introducing learners to key challenging discipline or subject concepts
24 7 32%
W* = Weighting to include the amount (a little, a lot); N** = Number of MCs who applied this principle to their module(s)
Research Results: Module Coordinators’ Perspective
• Strong focus on ‘efficiency’ of assessment– 14 MCs (64%) planned to ‘considering the demands
of other parallel modules when planning their assessments’.
• Assessment FOR learning opportunities planned by 12 MCs (56%)– In-class quizzes, in-class discussion groups, in-class
group work (despite large class sizes)– Online formative MCQs & problem-solving
activities
Discussion & Conclusions• Some areas of consistency, as well different
emphasis, emerged from the two data sets• Positive feedback on curriculum mapping tool• Key success factors of project: – Macro approach to curriculum change involving
wider community of colleagues in collaborative process
– Flexible & adaptable approach– Workshop format was both effective & efficient
Key Recommendations• Programme review process is transferrable to
other strategic projects • Striking the balance between the long-term
impacts of a more strategic approach versus the achievement of immediate module-level changes - local priorities will be key driver
• Further research on the curriculum mapping tool needs to be carried-out & disseminated
…..“learner centred, evidence informed design, developed in the peer-supported environment, is fundamental to successfully embedding new modes of delivery to pedagogic innovations.”
(Dempster et al, 2012, p.135)