monitoring soft outcomes d2 egk
DESCRIPTION
This presentation is used in a webinar run by the University for Industry and their support to UK Online centres, ICT centres for community learners. It was created by Elizabeth Gray-Kinf of GKG.TRANSCRIPT
Elizabeth Gray-KingMonitoring Soft Outcomes
1. Introductions2. Expectations of this session, using the Whiteboard3. Knowing the differences – Outputs and Outcomes4. Why Monitor?5. Finding outcomes and creating indicators6. Monitoring Tools7. Resources8. Mopping up Expectations
Monitoring Outcomes Webinar
There are two sorts of outcomes…
› Hard outcomes– The things which can be counted, almost completely
objective – the statistical kind of stuff
› Soft outcomes– The things which are difficult to count, but which
demonstrate growth – the touchy/feely kind of stuff
Hard outcomes
› courses booked
› courses completed
› qualifications earned
› progression made
› Any statistical kind of evidence able to be demonstrated by clear records
Soft Outcomes
› Interpersonal skills – social skills and coping with authority
› Organisational skills – personal organisation and the ability to order and prioritise
› Analytical skills– the ability to exercise judgement, managing time or problem
solving
› Personal skills – insight, motivation, confidence, reliability and health awareness
» (Dewson et. al. 2000a).
The fundamental question:How do you know you’ve made a difference?
Outcome
confident people
Output
3 Drop in Sessions
Outcomes and Outputs example….
Services Outcome Output
Dads n’kids Saturday 10-12
More confident fathers Stronger family relationships Less stressed mothers
10 fathers and 15 children at each session from a range of 30 families each Saturday in term.
Thursday Late Thursday nights theatre workshop
More confident young people
Increased courage to seek specific support on a range of issues
Maximum 25 young people at each session from a wider group of 50; 13 sessions each term
XYZ Information Service Provision of information on welfare benefits and advocacy
More healthy older people Reduced fear in accessing
information Increased confidence in self
management Increased visibility of
appropriate information
1000 leaflets distributed 2 Your Rights weeks in each
year 30 home visits each quarter 40 volunteers trained Advocacy information produced
for 100 leaflets and 2 web sites
Why monitor at all?
› Evidence – that activities were designed effectively– that learners needs were met– for fundraising– for regulatory bodies– for funders
› Ideas and evidence to – target new learners– market effectively– design future learning– design future monitoring tools
Where will we find our outcomes to measure?
› Business Plan, proposals and tenders (the difference intended to be made)
› Vision or Mission statement of organisation or project
› Publicity about the project/course/services – what are the ‘selling points?’ and therefore the outcomes on offer?
How do we translate outcomes to measures?
› Consider the features of a successful outcome…
› For instance, Confident people can be spotted because they:– Talk with other people easily– Listen actively– Ask for help clearly– Informally offer help and guidance to others– Etc (quickthink with others what success LOOKS like)
› Chart those descriptions to create– success indicators [what success looks like]– success measures [how you will count/observe/collect information]
› Often these are called performance indicators
success indicators
What can be seen to indicate – to show – that positive changes have been made?
› Run a Quick think or a participation exercise (or a simple mind map)
› What would success LOOK like? (more relaxed expressions, engagement with others, clear preparation)
› What would success SOUND like? (clear answers, effective questions, effective listening, compliments rather than complaints)
› Each kind of course or intervention will have similar indicators and a range of different ones. Each one has a comparative and/or short accurate description (more, less, clear, fewer, more frequent, etc)
success measures
how you will count/observe/collect information?
› Agree the tools to collect the success indicators
› Agree what could be set in place to see the changes– observations at course start and at intervals– reports from tutors/volunteers at start and at intervals– self perception scales at course start and at intervals– Web based self perception scales at course start and at intervals
› Ensure whatever is created is understood by all who will use it and asks information which is relevant (not merely interesting)
Tools for monitoring soft outcomes 1
› Learning Records– Records of learner self-, group- and peer- assessment– Tutor records of assessment activities and individual/group progress– and achievement– Learners’ files, journals, diaries, portfolios, artwork, videos, audiotapes, – performances, exhibitions and displays– Individual or group learner testimonies: artefacts, photographs and other
forms of evidence are also acceptable (LSDA/NIACE, 2003).
Tools for monitoring soft outcomes 2
Beyond recorded learning: Participatory approaches
http://www.peopleandparticipation.net/display/Methods/Participatory+Appraisal
“The term Participatory Appraisal describes a family of approaches that enable local people to identify their own priorities and make their own decisions about the future, with the organising agency facilitating, listening and learning.
Participatory Appraisal uses visual and flexible tools to ensure that everyone can join in regardless of background. It can be carried out in a place where people already meet in their everyday lives.”
› User friendly and user driven
Tools for monitoring soft outcomes 3
› Existing programmes and web based tools› Bespoke tools
– target charts
– scaled lines
– "race courses"
– visuals
Target charts for Distance measure and travel
T Chart to gain impressions and future direction
Pairwise chart to choose from a range of options
Sticky dot voting to agree a choice
Resources
› www.pms.ac.uk/mymop/ - health based personal measures and tools
› http://www.dwp.gov.uk/asd/asd5/WP8_annexe.pdf - examples of different tools used to record personal soft skills and outcomes
› http://communityplanning.net› http://www.peopleandparticipation.net/display/Methods/Home
[example - http://www.peopleandparticipation.net/display/Methods/Open+Space]
Mopping Up
› Unmet expectations?› Examples from UK online centres?› Other issues?