investigating the pedagogical push and technological pull of computer assisted formative assessment

Download Investigating the Pedagogical Push and Technological Pull of Computer Assisted Formative Assessment

Post on 21-Oct-2014

1.810 views

Category:

Technology

0 download

Embed Size (px)

DESCRIPTION

Discussion of computer assisted formative assessment projects at The Open University.

TRANSCRIPT

OU Conference 2007 - Open Comment

DMW 2008 The Open University's Institute for Educational Technology Investigating the Pedagogical Push and Technological Pull of Computer Assisted Formative AssessmentDenise WhitelockThe Open University, Milton Keynes

d.m.whitelock@open.ac.uk

DMW 2008 The Open University's Institute for Educational Technology

DMW Learning for Life Workshop March 2008Behaviourist prescriptions for assessment: wolves in sheep's clothing

DMW Learning for Life Workshop March 2008

DMW Learning for Life Workshop March 2008Dont forget the human reaction

DMW Learning for Life Workshop March 2008

DMW Learning for Life Workshop March 2008www.storiesabout.comwww.storiesabout.com/creativepdpc.mckillop@rgu.ac.uk

DMW Learning for Life Workshop March 2008

DMW 2007 The Open University's Institute for Educational Technology

Formative AssessmentBlack & Wiliam (1998)improvement 0.4 to 0.7 on standardised tests

MacFarlane-Dick (2006)empower students as self regulated learners

Feedback and assessment moving forward

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Collaborative Formative Assessment

CSCL . on task talkPredict Look and Explain Collaborative problem solving find a complex problemTools to support distance students to communicate, find a partnerAppropriate simulation

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

BuddySpaceAutomatic map construction to show locationsAutomatic roster (buddy list) construction using OU login authentication so it knows which workgroup or tutorial group Im inMany views: map, timelines, etc.Plug-ins for future enhancements

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

BuddySpace

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

State of mindgoals, plans, intentionsdevicesuser profiling

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

AutomaticMapGeneration

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

AutomaticMapGenerationRight-click to drill downinto cluster nodes (provides scaleability)

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

BuddyFinderWho else is facing a similar problem?Where can I find complementary approaches?Has anyone found contrary evidence?Generally: Find me the person who can really help me now.More specifically:

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Arbitrary keywordsat users discretionBuddyFinder

DMW 2008 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Easysearch

DMW 2008 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Matcheslight up

DMW 2008 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

InvokesGoogles Web Servicebehind the scenes,if requested,and auto-highlightswithin users own map!BuddyFinder

DMW 2008 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Finding an appropriate science application

Complex problem solving with Global Warming7 variables for manipulationSolar constant, CO2, H2O, Aerosol content, albedo, ice & snow, cloud coverImmediate effects of manipulation displayed as graphical output

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Global Warming

DMW 2007 The Open University's Institute for Educational Technology

DMW, Institute of Educational Technology, September 1997

DMW 2007 The Open University's Institute for Educational Technology

Global Warming

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Next: Yoked apps via BuddySpaceStudent AStudent B(yoked, but without full screen sharingrequired!)

DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Global Warming: Simlink Presentation

DMW 2007 The Open University's Institute for Educational Technology

DMW 2008 The Open University's Institute for Educational Technology

DMW 2008 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Debate with SIMLINKMore questioningExplanation became more importantSocio emotive communication more evidentSome couples separated for a while and then came back together

DMW 2007 The Open University's Institute for Educational Technology

DMW 2008 The Open University's Institute for Educational Technology What is Open Comment?

Automated formative assessment toolFree text entry for studentsAutomated feedback and guidanceOpen questions, divergent assessmentFor use by Arts Faculty

DMW 2008 The Open University's Institute for Educational Technology

DMW 2008 The Open University's Institute for Educational Technology Causal models of explanationFirst step:Identification of question types where students exhibit causal reasoningLooked for questions with:Salient variablesDescription of these variablesIdentification of trendsIdentification of relationship between the variables i.e. causality

DMW 2008 The Open University's Institute for Educational Technology

DMW 2008 The Open University's Institute for Educational Technology Three common types of question types were selected for Open Comment in History

Analysis of statistics, usually presented to the student as a tableComprehension of a set textIdentifying similarities and differences for a given event

DMW 2008 The Open University's Institute for Educational Technology

Knowledge elicitation/ethnography

DW and SW acted as studentsAnswered all three question typesC gradesImproved only when an answer model elicitedDMW 2008 The Open University's Institute for Educational Technology

DMW 2008 The Open University's Institute for Educational Technology

DMW 2008 The Open University's Institute for Educational Technology Open Comment components

A Java-based feedback systemA web service shellA graphical interface for testingA Moodle-based question typeA forms-based editing tool

DMW 2008 The Open University's Institute for Educational Technology

Pedagogical models: checking them out with you now!

List the activities that tutors engage with when they are marking a script. What things are they looking for e.g. error detection. Categorise the activities into a scheme or stages from which you could build instructions for a robot to mark the scripts.

DMW, IET, Managing Assessment SIG, June 2008

DMW, IET, Managing Assessment SIG, June 2008

DMW 2008 The Open University's Institute for Educational Technology Stages of analysis by computer of students free text entry for Open Comment: advice with respect to content (socio-emotional support stylised example)STAGE 1a: DETECT ERRORS E.g. Incorrect dates, facts. (Incorrect inferences and causality is dealt with below)Instead of concentrating on X, think about Y in order to answer this question Recognise effort (Dweck) and encourage to have another goYou have done well to start answering this question but perhaps you misunderstood it. Instead of thinking about X which did not.. Consider Y

DMW 2008 The Open University's Institute for Educational Technology

DMW 2008 The Open University's Institute for Educational Technology Computer analysis continuedSTAGE 2a: REVEAL FIRST OMISSIONConsider the role of Z in your answer Praise what is correct and point out what is missing Good but now consider the role X plays in your answerSTAGE 2b: REVEAL SECOND OMISSIONConsider the role of P in your answer Praise what is correct and point out what is missing Yes but also consider P. Would it have produced the same result if P is neglected?

DMW 2008 The Open University's Institute for Educational Technology

DMW 2008 The Open University's Institute for Educational Technology Final stages of analysisSTAGE 3:REQUEST CLARIFICATION OF KEY POINT 1STAGE 4:REQUEST FURTHER ANALYSIS OF KEY POINT 1(Stages 3 and 4 repeated with all the key points)STAGE 5:REQUEST THE INFERENCE FROM THE ANALYSIS OF KEY POINT 1 IF IT IS MISSINGSTAGE 6:REQUEST THE INFERENCE FROM THE ANALYSIS OF KEY POINT 1 IF IT IS NOT COMPLETESTAGE 7:CHECK THE CAUSALITYSTAGE 8:REQUEST ALL THE CAUSAL FACTORS ARE WEIGHTED

DMW 2008 The Open University's Institute for Educational Technology

Feedback5 main categoriesPraiseDeveloping/expandingPresentation and structure essayMisunderstandingAdvice concept linking, analysisDMW 2008 The Open University's