yosec - assessing how much of 'what works' programmes do - kaye mclaren

Overview of YOSEC

Development and use of the Youth Offending Services Effectiveness Checklist

All intellectual property rights and copyright for this presentation held by Kaye McLaren. To be used only with her permission.

Kaye.mclaren@paradise.net.nz

YOSEC – what is it?! Youth Offending Services Effectiveness

Checklist

Checks how much of ‘what works’ to reduce youth offending a programme is actually doing

Main YOSEC documents

Y o u t h O f f e n d i n g S e r v i c e s E f f e c t i v e n e s s C h e c k l i s t

Y O S E C c h e c k l i s t3 7 i t e m v e r s i o n ' l o n g '

2 0 i t e m v e r s i o n ' s h o r t 'T h i s i s w h a t y o u f i l l i n

S c o r i n g G u i d e l i n eu s e d t o s c o r e c h e c k l i s tI n c l u d e s s c o r i n g p r o f i l eG i v e s p i c t u r e o f s c o r e s

P r o g r a m m e D e v e l o p m e n t G u i d e l i n eE x p l a i n s e a c h i t e m s c o r e

G i v e s r e s e a r c h b a c k g r o u n dG i v e s i d e a s f o r p r a c t i c a l a c t i o n s

Y O S E C d o c u m e n t s

History of YOSEC Long-held dream of mine

Funded by multiple youth justice agencies in NZ who could see the potential

No changes without my agreement – protects YOSEC quality

YOSEC – aims

2. Help funders makes decisions on who to fund, taking into account how much of ‘what works’ a provider is doing

4. Help providers identify their strengths to protect, and where it’s worth putting time and money into development

Informing funding

Funding systems lack way of assessing fit with evidence-base

Funders can quickly and easily assess this using YOSEC

Two halves of the apple

Programme capability AND likely impact on crime

Capability only can mean noless crime or even more crime

‘What works’ only can meaneffective programmes fall over

The impact of doing ‘what works’1. 1/Provide more hours of services to high risk - 10%

less crime

3. 2/ Target factors known to be causally linked to crime - 19% less crime

5. 3/ Use behavioural techniques - 23% less crime

Do all three = 32% reduction in reoffending

Do none = 7% increase in reoffending

Aiding provider development

Works like a mirror – hold up to providerto see areas that are looking good

and those that are not so flash!

Aiding provider development 2 Means you know what

to work on and what to spend money on

Helps funders identify what development to fund most urgently

Development process I systematically reviewed international

research on ‘what works’

Converted findings into a draft checklist

Wrote programme development guide

Development process Held 6 national consultation meetings

with youth justice and cultural experts

Revised checklist and did scoring guide – marathon task!

Cultural consultation process Included Maori and Pacific cultural and

programme experts in initial six consultation meetings nationwide

Reviewed cultural research

My research reviewed by a cultural expert

Cultural consultation process

3 days of hui with Maori cultural experts

3 days fono with Pacific cultural experts

Largely happy but had some suggestions for improving it

Cultural sections Only one section is specifically about

‘culture’

Cultural issues built into most questions

Reflects idea that cultural practice is normal good practice

Flow chart of using YOSEC

Who fills it out Designed to be self-administered by

provider – ‘fudge factor’

Independent assessment is more reliable and builds partnership

Surprised by funder willingness to train and resource assessment staff

How long it takes to do

Both long and short versions take 1.5 to 3 hours to fill in

Depends on how much people talk!

Then more time for looking at documentation or talk to stakeholders – worth it for extra accuracy

YOSEC meetings are not like this!

How long it takes to do Doing development plan

takes a few more hours

Less than half the checklist is questions - less pages wouldn’t mean less time

To get information any other way takes months or even years

Involves process or outcome evaluation

Eg of detailed action plan

Date: 1 February 2007

Item no YOSEC score

Action plan to build or protect strengths Do by date

    Level 1 items  

B1 Engaging with clients  

1rain and encourage staff to be supportive in practical ways, e.g. helping with benefit hassles, food or transport

        In staff meeting identify main areas of practical help we can provide by brain storming then choosing the most viable ones (Staff responsible: NAME)

        Develop brief training session of information and skills needed to provide this help (Staff responsible: NAME)

        Develop information kit on relevant services e.g. benefits, policy on transporting clients etc (Staff responsible: NAME)

1 Feb 08  1 April 07  1 June 07  1 June 07

Using YOSEC over time Can do YOSEC more than once – cheap

and quick (relatively speaking)

Shows you where programme got closer to ‘what works’ over time

Obvious where strengths drifted

Can track what actions still need doing

Considerations re funding

Suggest don’t cut or stop programme funding after first YOSEC assessment

Not fair to providers – historically been little development and training in ‘what works’ in New Zealand

‘Throw money at it and hope for the best’

Considerations re funding Recommend doing development for at least

one year before re-assessing

YOSEC score not only reason for cutting funding – how well staff relate to clients, how willing they are to change

Recommend funding for development

Main provider concernsVery busy – YOSEC another thing to fit

into a hectic schedule

Already have a lot of compliancepaperwork whichthey are not paid to do

Main provider concerns

Anxious about impact on funding, scared of losing it

May not interested in changing programme – think it’s good enough already

Addressing provider concerns Important to find out what concerns are

and reassure providers

Cover how to do this in training

Using a powerpoint presentation like this at the start can help the provider

Incentives to do YOSEC Pay provider for time doing YOSEC

Guarantee not to stop funding for specific development period

Tag YOSEC reports to funding applications

Provide a significant development fund