a rct evaluation of the letterbox club in northern ireland dr karen winter and jennifer mooney...
TRANSCRIPT
![Page 1: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/1.jpg)
A RCT evaluation of The Letterbox Club in Northern Ireland
Dr Karen Winter and Jennifer [email protected]
![Page 2: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/2.jpg)
• Poor educational outcomes
• Measures: attendance rates; suspensions; exclusions; attainment in tests, exams and formal qualifications
• Contributory factors: interplay of individual child; familial and care related characteristics
• Disability; abuse; trauma; physical/emotional ill health; poor familial relationships; multiple transitions; poor attachment; stigma; labelling; lack of support for carers
Context
![Page 3: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/3.jpg)
• Legislation
• Policy and practice initiatives
• Macro level - structures and processes
• Micro level – direct to child interventions
• Concern with evidence, what works, cost effectiveness
Initiatives
![Page 4: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/4.jpg)
The Letterbox Club
• Once monthly parcel for 6 months • May to October• Books, stationery, number games• Addressed to the child• Personalised letter• Interest level, not ability level• Aims • www.letterboxclub.org.uk
![Page 5: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/5.jpg)
Contents
![Page 6: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/6.jpg)
• Letterbox Club began 2002• 2003-2006 pilot work• 2007 – national pilot funded by government• 2008; 2010 evaluations by programme designers
in England and Northern Ireland• Gains in reading and number skills • 2011 independent evaluation data in Northern
Ireland – gains reported• www.qub.ac.uk/cee
Available research
![Page 7: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/7.jpg)
Can the gains in reading and number skills be attributed directly to the Letterbox Club
intervention?
The gap
![Page 8: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/8.jpg)
Comparison using standardised scores - limits
Importance of control group – identical, random allocation, evenly balanced, variations accounted for
Essential if we are serious about addressing inequality Next slides consider 3 elements to study design: the RCT, logic model and the process evaluation.
The gap
![Page 9: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/9.jpg)
• Does not account for previous research around reading
• Assumes a linear movement • Hinges on feelings of ownership• Assumes children do not have access to
materials• Middle class deficit view of children in care
Logic modelMismatch between inputs and anticipated outcomes?
![Page 10: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/10.jpg)
Current studyThe RCT study
![Page 11: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/11.jpg)
Flow chartRCT study participants
![Page 12: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/12.jpg)
Measures• Neale Analysis of reading ability• Elementary Reading Attitude Survey (Garfield)
Other data• Age• Gender• Type of placement• Trust area
Logic modelData collected
• Placement moves• Carer information• Length of time in care• Siblings/ foster siblings• Letterbox fun days
![Page 13: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/13.jpg)
![Page 14: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/14.jpg)
• Attitudes• Ethics• Collaborators• Fieldworkers• Results • Responses to results
RCT challenges
![Page 15: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/15.jpg)
• Interviews foster children and their carers• Exploring:• what actually happens at time of receipt of
the parcel; • engagement with parcels;• views on parcel content;• and ongoing support with the materials.
Logic modelProcess evaluation
![Page 16: A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney k.winter@qub.ac.uk jmooney06@qub.ac.uk Dr Karen Winter and](https://reader035.vdocuments.net/reader035/viewer/2022070401/56649f1c5503460f94c31dd6/html5/thumbnails/16.jpg)
• Further Studies- nurture groups and school based interventions
• Collaborations- with colleagues in Canada and REES centre (Oxford)
• Development of the programme
Many Thanks!
Logic modelThe Future