assessment 2009
DESCRIPTION
Assessment 2009. IMPACT ASSESSMENT – ACTION RESEARCH – PLEs – OLEs – CGOs – Kirkpatrick – ACCESS – Bertelsmann – Schön – Reflective Practice – Societal Impact – Personal Impact – Operative Attention. Scared or Inspired??. - PowerPoint PPT PresentationTRANSCRIPT
ASSESSMENT 2009
IMPACT ASSESSMENT – ACTION RESEARCH –
PLEs – OLEs – CGOs – Kirkpatrick – ACCESS –
Bertelsmann – Schön – Reflective Practice – Societal Impact – Personal Impact – Operative Attention
SCARED OR INSPIRED??
TIME TO REALLY ASSESS VALUE…
THE OBLIGATORY JOKE How many Academic Administrators does it
take to change a light bulb?
CHANGE !!??
GENERATION PONG – TIME TO EVOLVE
THESE NEW-FANGLED IDEAS… 1998 Bertelsmann report:
• Cognitive tools empower learners to design their own representations of knowledge rather than absorbing representations preconceived by others.
• Cognitive tools can be used to support the deep reflective thinking that is necessary for meaningful learning.
• Ideally, tasks or problems for the application of cognitive tools will be situated in realistic contexts with results that are personally meaningful for learners.
• Using multimedia programs as cognitive tools engages many skills in learners such as: project management skills, research skills, organization and representation skills, presentation skills, and reflection skills.
LEARNERS AS DESIGNERS
Spoehr (1993) reports that students who build and use hypermedia develop a proficiency in organizing knowledge about a subject in a more expert-like fashion. They are able to represent multiple linkages between ideas and organize concepts into meaningful clusters.
Salomon et al. (1991) - Educators should empower learners with cognitive tools and assess their abilities in conjunction with the use of these tools. Such a development will entail a new conception of ability as an intellectual partnership between learners and the tools they use.
ASKING THE RIGHT QUESTIONS.
Tools –
Questions – tbd
START with Action research : a reflective process of progressive problem solving led by individuals working with others in teams or as part of a "community of practice" to improve the way they address issues and solve problems.
Dr. Joan McMahon [email protected] John Sener – [email protected]
SO WHAT?
Impact Assessment basically asks the question “So what?” After anything you do or try, simply asking that question gets you closer to the “impact” issue. –
Dr Joan McMahon
KIRKPATRICK’S EVALUATION OF T+L
Kirkpatrick's 1975 book Evaluating Training Programs defined his originally published ideas of 1959, increasing awareness of them, so that his theory has now become arguably the most widely used and popular model for the evaluation of training and learning.
Kirkpatrick's four-level model is considered an industry standard across the HR and training communities.
The four levels of Kirkpatrick's evaluation model essentially measure:
KIRKPATRICK’S FOUR LEVELS L1 – Teaching or reaction of student - what they
thought and felt about the training L2 - Learning - the resulting increase in
knowledge or capability L3 – Results - extent of behavior and capability
improvement and implementation/application L4 – Impact - the effects on the business or
environment resulting from the trainee's performance
All these measures are recommended for full and meaningful evaluation of learning in organizations, although their application broadly increases in complexity, and usually cost, through the levels from level L1-4.
WHAT QUESTIONS CAN YOU ASK TO
GET TO “IMPACT”?
T = I taught it.
TL = I taught it but did they learn it?
TLR = I taught it but did they learn it AND RETAIN IT?
TLRI = I taught it but did they learn it and retain it AND SO WHAT?
EXAMPLES – BETTER QUESTIONS!!
The teacher encouraged interaction – Yes / No 1 2 3 4 5
The teacher was tasked to encourage interaction among students – what would have motivated you personally to interact more with your peers?
Did you meet the Learning Goals or the course as in the syllabus?
What has been the impact of this course in your work or personal life? What new skills have you used to affect change?
THE EVOLUTION OF ASSESSMENT
This slide attributed to Bobby Elliot - Scottish Qualifications [email protected]
ASSESSMENT 1.0
Paper-based Classroom based Formalised Synchronised Controlled Industrialised
Changed little since early 20th Century
This slide attributed to Bobby Elliot - Scottish Qualifications [email protected]
ASSESSMENT 1.5
Computer-based assessment Characteristics
E-testing E-portfolios Simulations
Embedded in most VLEs Stand-alone systems Familiar to students and teachers
This slide attributed to Bobby Elliot - Scottish Qualifications [email protected]
STUDENT PERCEPTIONS (OF 1.0/1.5)
Artificial and contrived
Something that is done to them
Doesn’t measure anything important
(It’s a) Hurdle to be jumpedNot part of their learningSole purpose of their learning
This slide attributed to Bobby Elliot - Scottish Qualifications [email protected]
WEB 2.0
User-generated content Architecture of participation Network effects Openness Data on an epic scale Power of the crowd
ASSESSMENT 2.0 NEEDS TO BE…
Authentic Personalised Negotiated Collaborative Recognising existing skills
WEB 2.0 SERVICES
Web Service Example Functions
Personal Portal Netvibes Evidence Organization
E-mail Gmail Evidence Storage
Blog Wordpress E-portfolio, diary, log-book
RSS Bloglines Evidence Discovery
Social Bookmarking Del.icio.us Evidence Capture
Instant Messaging MSN Discussion Groups / Collaboration
VOIP Skype Audio Evidence /
Wiki Wikispaces Collaboration / Groupwork
Video Upload YouTube Create / Store Video Evidence
Social Networking Facebook Evidence discovery / Collaboration
This slide attributed to Bobby Elliot - Scottish Qualifications [email protected]
REASONS TO OPEN THINGS UP
“Because you’re pouring money into a black hole that students don’t like, which is unnatural to them, which can’t possibly keep up with developments on the Web, and which is little more than a comfort blanket to teachers who can’t or won’t embrace the 21st Century.”
Bobby Elliot
A LITTLE MORE CONSERVATIVELY
It’s crude but it’s an important evolutionary step
Not every student is a digital native Not every teacher can use Web 2.0
“I can’t get my staff to use the quiz in Moodle so what chance is there that they’ll embrace Web 2.0?”
MELD THE RIGHT QUESTIONS TO THE APPROPRIATE (STUDENT-SELECTED MEDIA)
Ask the right questions – Open vs Closed / Fat vs. Skinny / IMPACT Assessment = SO WHAT?!
Review objectives to encourage multiple means of achieving success.
Even when you are in a Blackboard/Moodle (confined) world, be open to a variety of deliverables, and look to nurture that creativity.
Accept OLEs (Online Learning Environment) / PLEs (Personal Learning Environment) – Voicethread / Skitch …
(Impactful) QUESTIONS??