impact and evaluation
TRANSCRIPT
Welcome – Impact and Evaluation
Sam Linton – Inspiring Scotland
Maria Williamson – Action for Children
Programme level
• Logic Model devised and introduced across portfolio in
Nov 2012 with 27 Outcomes
• No means of aggregating impact of programme
• Inspiring Scotland external evaluation completed Nov
2013
• CashBack programme evaluation completed and
launched in debate by Justice Secretary in June 2014
Partner level
• All partners in phase 2 had evaluations completed
• Part of phase 3 process involved completion of evaluation
plan with strong link to Logic Model
• Schedule for external and internal evaluation processes now
much clearer across programme
• Each project using stronger self evaluation/case study
approaches
External Evaluation recommendations – Impact and Evaluation
• SG and DT to review Logic Model and map outcomes with more focus
on positive destinations
• SG to ensure Grant Offer Letters set out outcomes to be delivered and
indicators to be used for tracking
• SG and DT to develop consistent approach for using indicators across
programme
• Partners to focus on relatively small number of key outcomes
Ouct
omesNumber of Indictors Indicators Organisations Projects
Culture YouthworkYouthwork
SportSport Employability
1 Increased participation in positive activity 29 9 16 1 2 2 3 1
2 Increased participation by difficult to engage and equalities groups 32 9 14 2 1 3 3
3 Increased opportunities for new experiences or activities for participants 24 6 11 2 2 2
4 Increased opportunities to develop interests and skills 33 8 12 1 2 2 3
5 Increased opportunities for continued participation by linking up and signposting to other provision 26 4 10 2 2
6 Increased involvement in structured pro-social and healthy activities 10 4 5 1 2 1
7 Participants have places to go where they feel safe and comfortable 14 4 7 2 1 1
8 Greater confidence and self-esteem among participants 18 8 12 2 1 2 3
9 Participants demonstrate new skills and positive changes in behaviour 29 7 14 1 1 1 2 2
10 More participants have achieved accreditation for their learning 15 6 10 1 1 2 2
11 Participants develop aspirations for further learning and development 17 7 9 1 1 2 3
12 Participants are more involved in community-based activities 4 2 4 2
13 Participants develop positive peer networks and relationships 12 4 7 2 2
14 There is increased community-based interaction 8 2 3 2
15 Sustained participation in positive activities 26 5 12 1 1 3
16 Sustained improvements in health and well-being 2 1 1 2
17 Sustained improvements in self-esteem and confidence 6 2 4 2
18 Sustained improvements in the awareness of the benefits of play, interactive, physical and social activities 15 3 5 1 2
19 Increased supportive social networks and feelings of belonging 10 4 5 1 1 2
20 More participants progress onto positive destinations (employment, further learning, training) 31 9 14 1 3 3 2
21 Increased horizons and improved outlook amongst participants 7 2 4 1 1
22 Participants have influenced the opportunities available to them in their community 11 3 6 1 2
23 Sustained participation in community-based activity 10 2 5 2
24 Reduced levels of crime and anti-social behaviour 6 3 3 1 2
25 Increased community self-esteem and capacity to shape local development 0 0 0
26 Better community integration and cohesion 0 0 0
27 Better community integration and cohesion 10 5 5 2 3
CashBack for
Creativity
YouthLInk SSS SRU AfC
Youth Scotland OYT SFA GCC
SSF BbS CFCF
PTDA
KEY
those indicators with in excess of 20 options chosen against it
to remove as no partner has chosen them
to consider removing as indicators partners specified to SG relate purely to case study or external evaluation and not a quantifiable figure to be noted on a quarterly basis
to review - as hard to measure and prove impact of Cashback work alone on the achievement of the outcome
CashBack Phase 3 – aggregated view
• Grouping partners according to culture, youthwork, youthwork/sport, sport or
employability
Most common outcomes
� increased participation in positive activity: >610,000 individuals
� increased opportunities to develop skills & interests: 0% to 100%
� greater confidence & self-esteem amongst participants: 75%to 100%
� more participants have achieved accreditation for learning: up to 90%
� more participants progress into positive destination: up to 90%
Maria WilliamsonEmployability Children’s Services Manager
Our approach to evaluation
Employability Pipeline
Young Person’s Journey Through Services
Stage 1 Referral &
Engagement
Action for Children Education/schools
SDS Social Work
Needs identification
Initial Paperwork Referral form
LS/CMI Risk/Needs Ass. Solution Focused
Service Plan developed Benefits confirmed
Community and Local Authority Partners identified
Family Support
Individual Keyworking & transition support
Relationship built Clear action plan
Stage 5 In-Work/ Aftercare
Local Employers
AfC Employability Services
Education/Employment Sustainability
Aftercare In Work Support plan
Mentor training Ongoing career development
Conflict resolution Re-access service if job ends
Continued skills development Sustained employment
Stage 4 Employer
Engagement
Action for Children
Education/Employment Vocational training Work experience
Youthbuild Employer Engagement Employer led training
Skills portfolio
Wage Subsidy Tailored placement
12 week paid work (ERI)
Increased confidence Vocational qualifications
Work Experience Employer/industry
Certification Work/MA
Stage 3 Vocational
Activity
Action for Children
Education/Employment Vocational training Work experience
Get Set for… Community Challenge
Forestry Sector Specific training &
accreditation
Labour market understanding
Targeted job search
Work Placement Employability skills SCQF 4 attainment
Work prep Transition & in work support
Improved communication Improved employability Certificates achieved
Work experience SCQF 4 Accreditation
Progress to Stage 4, Work or education
Stage 2 Barrier
Removal
Action for Children Other relevant partners
Antisocial Pattern/unhealthy
lifestyle Poor Motivation/Attitude
Poor self-esteem/care skills Interpersonal Skills
Chaotic family
YouthStart Community participation
Goal Setting Barrier focussed work
Core Skills Development SCQF3 attainment
Practical tasters Work Tasters
Interpersonal skills Resilience Building Time management
Increased self esteem Sense of achievement
Stabilised personal circumstances
Reduced offending Positive relationships
Healthy & active SCQF Accreditation
Progress to Stage 3, work or education
Provider
Needs Addressed
Outputs
Outcomes
Transitional Support Continuum Generic
Plan
DeliverEvaluate
Learning/Delivery Cycle
Our Approach
Young Person
1 2 1 meetings
Database tracking
questionnaire
Outcome star
Risk management
Action planning
Outcomes and Indicators (1)
Activity Performance Indicators
• Engaging young people within the eligibility groups across 5 target LA areas
Quality performance Indicators
• Young people completing an outcomes-based action plan with defined long-term personal job goal
• Young people achieving a positive destination
• 180 (as a minimum)
• 100%
• 75% (135 young people)
Outcomes and Indicators (2)
Outcomes
• 96 young people
participate in PX2 personal
development training
• 84 young people
participate in a targeted
Progression Pathway
Indicators
• 86 complete PX2 and make positive behaviour changes
• 86 increase their ambition for the future
• 86 progress into PDs
• 60 secure additional training, 6 enter FE, 20 gain jobs
• 80 gain accredited training
• 72 complete a targeted pathway
• 67 complete a paid work placement
• 50 secure a job or apprenticeship
Mentoring Through Sports
SQA level 1 award Active Healthy Living, Health and
Activity focus, SFA coaching award,
Badminton Scotland Award, CP certificate, First
Aid, Football refereeing
2 x 12 = 24 YPs
Clyde College - Glasgow
Entry to Construction
CSCS, Safety Awareness, Work at Height, Small plant and tools, PAMA, UKATA
4 x 12 = 48 YPs
Glasgow, Edinburgh, WD, South and North
Lanarkshire
Steps to the Future
Personal Development, PX2 accredited
programme, effective thinking skills,
CV/employability workshops
8 x 12 = 96 YPs
Glasgow, Edinburgh, WD, South and North
Lanarkshire
Entry to oil/gas and engineering
Entry level bespoke course in partnership with
employers, industry related certificates, folk lift, IT,
packaging,
1 x 12 (pilot)
Glasgow
Positive Choices
Build on previous experience
Test new models of delivery
Young people’s input
Employer feedback
External evaluation commissioned
Building on Success
Process of External Evaluation
• Commission Evaluator
• Early Planning meetings
• Agree Timeline
• Identify Baseline Data Collation/Methods
• 8 x Focus Groups
• Partner views and feedback
• Exit Survey (after each programme)
• Interim report
• Final Evaluation report
Lessons Learned
• Work in collaboration
• Remain open and transparent
• Make provision available
• Planning
• Gain meaningful feedback
• Be clear on expected outcomes
• Build on tools already in use
• Allocate adequate time
• Be prepared to change/modify your model
• Track and measure from the onset
• Remain objective
Welcome – Equalities
Arran Dewar– Inspiring Scotland
Neil Carrie – Scottish Rugby
Pete Young – Scottish Rugby
Programme level
• CashBack core values of “open to all” and inclusion
• EQIA introduced as part of annual reporting across
portfolio
• Identification of areas for improvement, and promoting
cycle of continual review
• CashBack programme evaluation
Partner level
• All partners in phase 2 had completed EQIAs
• Part of phase 3 process focussed on inclusion of protected
characteristics groups within Equalities Action Plan for each
partner
• Building on EQIA findings for improvement, or adoption within
own CashBack programme to address
External Evaluation recommendations
• Partners to act on the findings of their EQIA, setting this out in an Equality
Action Plan
• Equality Action Plan to set out how partners ensure their projects are open
to all
• Partner and DT to review EQIAs regularly
• SG to review their 2010 EQIA action points and develop a new EQIA to
adopt changes under Equalities Act 2010
• Equality gaps to inform SG of future balance / direction of CashBack
25 November 2014
Scottish Rugby,
Equalities Action Plan
Welcome
Neil Carrie
Head of Domestic Rugby Support
Pete Young
Equalities & Outreach Manager
Overview
• CashBack Investment – Integrated
programmes
• Equality Action Plan
Overview
• Equity Standard – Intermediate
• EQIA
• Policy Initiatives
• Reorganisation
Other influences…
1st September 2014
• Restructure
• 1 mile deep vs 1 mile wide
• Capacity to do more
• Specialists vs generalists
• Equalities and Outreach Manager
Age
Cashback focused on young people, but there is a lot of work also done to
encourage senior players to return to the game as coaches, officials and
volunteers
• Golden Oldies rugby- affiliated to SRU who provide insurance for players
up to 55 years old.
• Players up to 86 still competing in festivals
• Broad delivery, primary schools;
• Secondary schools;
• Club development for adults
• Game On – NEETS 16-25 yo.
Disability
• Working with Disability Sport to develop programmes and opportunities
for participants with physical and learning disabilities
• Disability training for all Development Staff and MA’s
• Unified Rugby “ The Clan”: rugby for players with learning disabilities.
Established group in Kilmarnock and new group in Edinburgh
• Wheelchair rugby- Caledonian Crushers in Glasgow.
- New wheelchair 7’s group linked to Glasgow East Rugby club.
• Work in Special Schools- planned sessions in Kaimes Feb2015
Race
• Currently working with BEMIS (Black and Ethnic Minorities in Scotland) to
look at developing opportunities for under-represented groups
Examples of projects:
• School of Rugby in Shawlands popular amongst Asian students
• Referral group in Angus- helped with integration of young Polish students
Sex and Gender
• New head of Women's Rugby appointed- Sheila Begbie.
• Women and Girls strategy to 2025 being developed
• Equality Champion on Scottish Rugby Board- Lesley Thomson
• Successful girls events at Lochinch in September and November with over
300 U15 and U18 girls and 12 new teams- supported by Cashback
• New women's teams in Shetland, Greenock, Howe of Fife
Religion & Belief
� 366 (1462) Catholic schools receiving CDO input
� Primary – 1924 sessions with 45,000 attendance (15,132 sessions, 357,000
attendance)
� Secondary – 867 sessions with 16,000 attendance (14,464 sessions,
229,000 attendance)
However…
� Unsure as to make-up of club membership (stakeholder data project)
� Challenges in converting taster sessions into teams at some schools
� Other religious groups
Sexual Orientation
Caledonian Thebans- Scotland's only open sexuality rugby club.
• Affiliated member club of SRU
• Supported coaching from Murrayfield Wanderers
• Annual coaching clinic with 9 teams from UK and Europe attending-
coaches provided from Edinburgh Rugby/ DO’s and hosted at Murrayfield
Thanks
Questions?
Celebrating Success
External Evaluation recommendations:
• Increase shared learning
� Put to use the intense monitoring, reporting and evaluating
• Local Authority Activity Packs
� work with local Partners to identify where there may be gaps in the
detail or how the information is provided as well as how to maximise
the opportunities for promoting these more effectively.
Celebrating success
CashBack for Creativity invests in 14 organisations Scotland wide
Scalloway Youth Club opens
Youth Work Fund Open for Applications
Gateway to Employability Programme launched 8 October 2014
Basketball Scotland launches Schools Hub
School of football yields results for the East Region
£4million ploughed into youth projects in
Edinburgh
Opportunities to share
• Summary of LA activity
• Quarterly forward planner of activity (highlighting significant events)
• Press releases
Bespoke LA pack
SCSN Monthly Newsletter
Website
e-newsletter
Blog
Scottish Government meetings
Ministerial visit opportunities – New
Cabinet Sec
- Holiday time
- Cabinet tours
www.cashbackforcommunities.org.uk
@CashBackScot• Encourage
• Share
• Embed in practice