designing work integrated assessment- tools & techniques for creating 'authentic'...

Designing work-integrated assessment: tools & techniques for creating 'authentic' assessments

Running order

The Collaborate project (20 mins)Designing a Work-integrated Assessment (Group activity - 15 mins)

Tea break (15 mins)

Affordances: What are they? (20 mins)Aligning digital technologies (Group activity - 10 mins)

Evaluating a redesigned assessment (10 mins)What can we do? (Group activity - 10 mins)

Where next / final questions (5 mins)

"Bringing together staff, students and employers to create

employability focused assessments enhanced by technology“

July 2011 - July 2014

JISC Institutional Change, Assessment & Feedback Strand

The Collaborate Project

Key threads, key questions

Assessment

Can current forms of assessment close the gap?

Technology

What role can technology play to support this?

Employability

Is there a gap in current provision?

Where to find answers?

Literature Interviews Analysis

Employability is potentially the biggest force driving people to

University ...

Amongst the foremost reasons for those 22+ considering a

university education (Pollard et al, 2008)

Almost 80% of students stated improving job opportunities

was the most important reason to go to university (CBI/NUS,

2011)

Driven by Employability

Wilson Review (2012) suggested a gap between business

and higher education.

One recommendation of many:

“Universities should reflect on the opportunities that are

provided for students to develop employability skills through

the formal learning methodologies used within the university”

(Wilson, 2012, p.10)

Driven by Employability

Career Zone

Many Employability Initiatives

Enterprise and self-employment

CVs, applications, interviews

Graduate jobs

Part-time jobs

Skills training and events

Volunteering

Placements

The Exeter Award

Global employability

Internships

Career research

Embedded Curriculum Approach

Curriculum EmploymentEmployability

Internship

Key skills training

Work experience

module

New forms of assessment

Assessment Drives Behaviour

What students attend to, how much work they do and how they go

about their studying is completely dominated by assessment

(Gibbs, 2004)

But can exams really prepare

students for the office?

Other forms of assessment

exist which may be better

suited ... e.g. ‘authentic’ assessment

Why Authentic Assessment?

“The types of assessment we currently use do not promote

conceptual understanding and do not encourage a deep

approach to learning” [encourage] “strategic or mechanical

approach”

Newstead (2002, p. 72)

Newstead, S. (2002). Examining the examiners: Why are we so bad at assessing students. Psychology Learning and

Teaching, 2(2):70-75.

Authentic Assessment

Originated in US K-12 system in the early 1990’s, focused on real

world problems in real world contexts.

“an assessment requiring students to use the same competencies,

or combinations of knowledge, skills, and attitudes, that they need

to apply … in professional life”

J. T. M. Gulikers, et al. (2004). `A five-dimensional framework for authentic assessment'. Educational

Technology Research and Development 52(3).

Authentic Assessment

Can be challenging in higher education - 'authentic' has multiple

meanings:

“the term inappropriately implies that some assessment

approaches are superior to others because they employ tasks that

are more ‘genuine’ or ‘real’”

J. Terwilliger (1997). `Semantics, Psychometrics, and Assessment Reform: A Close Look at "Authentic"

Assessments'. Educational Researcher 26(8):24+.

Work-Integrated Assessment

Combines concepts of authentic assessment with local research.

Defined by six 'dimensions':

‘Real World’ Problem / DataSet an overall real world

problem, supported by real world data

Collaborative WorkingCreate teams of students from

the outset, encourage collaboration

Varied AudiencesAim to set explicit audiences for each assessment point

Multiple Assessment PointsMove to a more distributed

pattern of assessment; consider introducing ‘surprise’

points

Light StructureLightly structure the overall assessment; reward student

approaches

Peer / Self ReviewInclude peer and/or self review

explicitly in the assessment process

Visualising dimensions

1 Analyse

2 Design

3 Evaluate

Why use Work-Integrated Assessment?

PSY1301: Professional Issues and Development

Original Assessment

75% written examination (1½

hours)

25% oral presentation

(undertaken in pairs, 10

minutes each)

Redesigned Assessment

40% written project: 750 word

patient information leaflet in

groups of up to 6 students

20% oral presentation

(undertaken in groups)

40% MCQ

Assessment

Problem / Data: Creation of a patient information leaflet

Time: Across the module, depending in part on the lectures

Audience: Two externals as new audiences / markers

Collaboration: Students in assigned teams of six

Review: Team working for informal review

Structure: Signposts to relevant guidance

The Leaflets

On the assessment

1 Analyse

2 Design

3 Evaluate

Small groups of 3-4Dummy/own assessment

Analyse & redesign

Discussion

How was the model useful in supporting thinking about the assessment?

Could you see yourself using this in your own context?

What would you change about the model?

Affordance Theory

What do these mean to you?

Affordance: Gibson’s definition

“The affordances of the environment are what it offers the animal,

what it provides or furnishes, either for good or ill ... It implies the

complementarity of the animal and the environment.”

Gibson (1979, p. 127)

Gibson, J.J. (1979) The Ecological Approach to Visual Perception, Houghton Mifflin Harcourt (HMH).

Gibson vs Norman: Wikipedia’s version

“An affordance is a quality of an object, or an environment, which

allows an individual to perform an action. For example, a knob

affords twisting, and perhaps pushing, while a cord affords pulling.

...

Psychologist James J. Gibson originally introduced the term ...

defined affordances as all ‘action possibilities’ latent in the

environment.”

Integrating Technologies

We align our learning outcomes, teaching methods and assessments - but what about technologies?

Can Affordances theory help?

“The affordances of the environment are ... what it provides or furnishes” Gibson (1979, p. 127)

What does Blogger provide?

What does LinkedIn provide?

What does Skype provide?

Rating technologies against dimensions

e.g. Time Question: Does the technology provide the learner with a

framework for constructing and managing multiple contributions

extended over time?

e.g. Collaboration Question: Does the technology provide social

presence in ways which allow multiple learners to work together

digitally, sharing ideas and content creation?

“Tech Trumps”

How do we use technology?

"There is no shortage of technology, only a shortage of the educational vision necessary to use the technology to create new educational environments"

Berge and Collins (1995, p. 5)

“I just need to know what programmes do what .... they could be very useful to me.”

Politics undergraduate studentBerge, Z. L., & Collins, M. P. (Eds.) (1995). Computer Mediated Communication and the Online Classroom Vol. III. Distance learning. Cresskill, NJ.: Hampton Press.

PSY1301 "Tech Trumps"

Technology demonstrations

Normal scheduled lecture

Brief description of project background

10 minute overview of potentially useful technologies

Finding your niche

The iTest

What is the iTest?

Thirty categorised

questions/statements

I make use of digital technology that few

other colleagues are using

I tend to avoid digital technology as I do not

trust it to work as it should

I can apply technology in novel ways to

meet specific needs

Students have the hardware, but can they use it in an

academic context?

High expectations for students coming to university to be

digitally literate ... not always the case

Is there a digital divide that needs to be managed?

Kennedy, G., Judd, T., Dalgarno, B., and Waycott, J. (2010). Beyond natives and immigrants: exploring types of net

generation students. Journal of Computer Assisted Learning, 26(5):332-343.

Where does the iTest fit?

Back in small groupsRedesigned assessmentNow align technologies

http://goo.gl/HA0Y05

Discussion

Could you see technologies that might suit your design?

What kind of thinking did the tech trumps provoke?

What's missing?

Evaluation package: student centered evaluations

"Mark on the Line" activity

"Mark on the Line" activity

Open answer postcards

Open answer postcards

Drag and drop skills evaluation

Drag and drop skills evaluation

Drag and drop skills evaluation

Group activity

http://goo.gl/vRQo8M

Back in small groupsSkill cards drag and drop

What can you do?

Contact:

Richard Osborne

Education Advisor (Technology Enhanced Learning)

University of Exeter

r.m.osborne@exeter.ac.uk

01392 72 4951

Questions?

Collaborate websitehttp://www.exeter.ac.uk/projects/collaborate/