randomised trials in uk social work: a discussion of possible approaches
DESCRIPTION
Randomised Trials in UK Social Work: A Discussion of Possible Approaches. Donald Forrester Professor of Social Work Research Director of the Tilda Goldberg Centre. Proposed RCT . Early stages of development (2012-14) Want to openly discuss issues and challenges - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/1.jpg)
Randomised Trials in UK Social Work: A Discussion of Possible Approaches
Donald ForresterProfessor of Social Work Research
Director of the Tilda Goldberg Centre
![Page 2: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/2.jpg)
Proposed RCT
• Early stages of development (2012-14)
• Want to openly discuss issues and challenges– Not got all answers and would rather have errors pointed
out now...
• Focus: whether Motivational Interviewing improves engagement of parents who misuse drugs or alcohol
• But first a bit of history, theory and background
![Page 3: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/3.jpg)
Social Work’s Evidence Base: Emperor’s New Clothes?
![Page 4: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/4.jpg)
RCTs are one of the greatest intellectual
achievements of the 20th century
![Page 5: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/5.jpg)
What is an “RCT”?
![Page 6: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/6.jpg)
What are advantages?
• The best way of ruling out bias• Bias defined as explanations other than the
intervention that might explain results
• Key types of bias:– Self selection bias– People are active and resolve problems– Social work is about maintenance not cure...– Self-report not good indicator of impact – people usually
appreciate services
![Page 7: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/7.jpg)
History of RCTs
• First comparative trial – in the Bible. Daniel compares vegetarian diet with meat and drink.
• RCTs in education and social work 1930s
• Medicine from the late 1940s
• Now dominant approach in medicine
![Page 8: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/8.jpg)
Selective history of RCTs
Cambridge and Sommerville Study• USA 1930s• Intensive, long-term befriending from social
worker for boys considered at risk of offending• Much appreciated by young men• Increased offendingFrom Oakley, 2002
![Page 9: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/9.jpg)
Social work and RCTs
• Dominant social work approach in 1960s, psycho-dynamic casework
• Reid and Shyne tested and found:– Long-term work LESS effective than short term– Developed task-centred approaches based on
approach
![Page 10: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/10.jpg)
Social work and RCTs
• Homebuilders Intensive Family Preservation Service
• Aimed at preventing children entering care• Success rates from 70 – 100%• Large-scale RCT – no impact
![Page 11: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/11.jpg)
Field social work RCTs in the UK 1970-2010
![Page 12: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/12.jpg)
Problems with RCTs
Bias
ContextGeneralisability
![Page 13: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/13.jpg)
ContextPawson and Tilley – Realist critique – context crucial for
understanding nature of intervention and its impact
Orford (2008) from within the RCT field – reflecting on the “Dodo Bird” effect
• Whatever is happening across interventions more important than differences between interventions
• Context of service, client or research
• Requires emphasis on understanding context, process as well as outcomes
![Page 14: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/14.jpg)
Generalisability• RCTs on drugs establish general knowledge• This isn’t true for psychosocial interventions:– Implementation fidelity and innovation effects– What the control group receives eg “nothing works
in Sweden…”– Or Dodo effects – everything works equally…
• This is a fair criticism of naïve positivism• But ALL research makes claims to generalisability – and
all should be considered critically
![Page 15: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/15.jpg)
Outcomes and social work: the politics of research
Client’s View
Expert View
Society’s View
![Page 16: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/16.jpg)
Reimagining RCTs• Starting point to develop a critical form of
RCTs that addresses these concerns – a characteristically and perhaps distinctively social work approach to RCTs
• Many of problems with RCTs are of RCTs as done not as should or could be
• So how should we do RCTs...
![Page 17: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/17.jpg)
Early social work RCTs
• Reid and Shyne (1969) – brief vs open-ended social work with middle class couples with relationship problems– Randomization purely on length of intervention– Taped interviews and found this led to more focussed
approach– Extensive outline of context of intervention– Both women and (particularly) men preferred this– Social work, service user and standardised measures of
outcome all agreed ST work produced better outcomes at 12 months
![Page 18: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/18.jpg)
Goldberg (1970): Helping the Aged
• Randomised older people to “normal service” (with unqualified workers) or service from qualified social workers
• Detailed description of needs, services and outcomes and combination of qualitative and quantitative methods
• High levels of need, improvements in both groups
• Major problem: the “intervention” differed in multiple ways
![Page 19: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/19.jpg)
Evaluating Helping the Aged
• The descriptive work was fantastic• The research is transparent and self-critical
• Contributed to the programme of work that culminated in “Community Care”
• Should have been the start for RCTs in social work...
![Page 20: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/20.jpg)
Field social work RCTs in the UK 1970-2010
![Page 21: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/21.jpg)
Not the research questions
• Can we show that any of the approaches taught to social workers makes a positive difference – compared to “normal practice”?
• Can we do this in a “normal” local authority setting? (chosen child and family work)
• Can we make an RCT “work” in social work?
![Page 22: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/22.jpg)
Key Considerations: Outcomes
• What outcome/s to measure?
• Legal basis (Children Act 1989)– Protection from harm– Meeting needs
• Every Child Matters outcomes (2004)– Health– Safety (eg from abuse)– Education– Making positive contribution– Achieving economic wellbeing
![Page 23: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/23.jpg)
Key Considerations: Outcomes
• Complex factors influencing outcome/s– Social situation (eg poverty, racism etc)– Personal problems of parents (eg addiction)– Difficulties in child (eg challenging behaviour)– Patterns of family functioning and parenting
• Complexity of social work “intervention”– Direct work– Interagency linkage– Risk assessment and decisions– Not voluntary clients seeking help
![Page 24: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/24.jpg)
Key Considerations: Outcomes
Single biggest problem likely to be family participation rate in research
Commonly as low as 5% - we need 60-70%Or a good proxy...
![Page 25: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/25.jpg)
Key Considerations: Intervention
• Chose Motivational Interviewing as:– Strong evidence base in substance misuse field– Very flexible style of working– General style of communication aimed at engaging
resistant clients– Consistent with social work values– Initial piloting study undertaken – which
highlighted challenges
![Page 26: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/26.jpg)
Simple Overview of Intervention Process
Improved Communicatio
n Skills
Increased Engagement
Positive Impact on Parent and
Family
Child Protected and
Thriving
![Page 27: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/27.jpg)
Piloting approach to outcomesImprove
Engagement of Parents
•Tapes– parent response (MISC and bespoke) •Parent views of SW (simple ratings)•Number and length of visits (worker and parent)•Engagement with other services (worker and parent)
Improve Parent and Family Wellbeing
•Reduced family stress (PSI)•Reduced alcohol and drug use (MAP)•Reduced parental stress (GHQ)
Protect child and improve welfare
•Whether abused/neglected (problematic) – using parent and social worker measures and one SI•Child’s emotional and behavioural welfare (SDQ)•NOT expecting to show an impact...?
![Page 28: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/28.jpg)
Current Pilot
• Combining small-scale RCT of MI with quasi-experimental study of “Hackney Model”
• 40 workers in 10 teams/units in 2 contrasting LAs receive intensive MI input
• 40 workers do not
• Piloting MI input, types of data collection and success of different approaches to families
![Page 29: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/29.jpg)
Elements of Data Collection Piloted• Direct tapes of practice
– Measured for MI-ness – Social work skill ratings being developed with group of experts by experience through
iterative process
• Brief questionnaires to all families allocated– With follow-up from administrators
• Research interviews with families using various standardised instruments– Key problem likely to be low response rate
• Social worker questionnaires on families – Key problem validity – To be tested by comparing with research intvws
• Exploring using LA data measures
![Page 30: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/30.jpg)
Concluding Thoughts
An RCT is the most robust test of effectiveness, because it minimises bias
Social work needs to engage with the complexity of RCTs in order to evaluate whether what we do works
A critical social work perspective is also likely to create better RCTs
![Page 31: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/31.jpg)
Concluding Thoughts
Developing RCTs fit for use in social work settings is not straightforward or easy
Doing so will not provide all of the answers
However, hopefully it will provide some evidence about what skilful social work is and the difference it can make...
![Page 32: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/32.jpg)
“If we knew what it was we were doing, it would not be called research, would it?”
Einstein
![Page 33: Randomised Trials in UK Social Work: A Discussion of Possible Approaches](https://reader036.vdocuments.net/reader036/viewer/2022081520/56816764550346895ddc4224/html5/thumbnails/33.jpg)
ReferencesForrester, D. (2010) Playing with fire or rediscovering fire? The perils and Potential for Evidence
Based Practice in Child and Family Social Work”, in Ayre, P. and Preston-Shoot, M. (Eds) Children’s Services at the Crossroads. A Critical Evaluation of Contemporary Policy for Practice, Russell House Publishing; Dorset
Goldberg, E.M., with Mortimer, A. and Williams, B.T. (1970) Helping The Aged—A Field Experiment In Social Work National Institute for Social Work Training, No. 19, George Allen & Unwin: London. 1970.
Holosko, M. J. (2010) “What Types of Designs are We Using in Social Work Research and Evaluation?”Research on Social Work Practice; Nov2010, Vol. 20 Issue 6, p665-673
Oakley, A. (2000) Experiments in Knowing: Gender and Method in the Social Sciences, Polity PressOrford, J. (2008) “Asking the right questions in the right way: the need for a shift in research on
psychological treatments for addiction”, Addiction, Volume 103, Issue 6, pages 875–885, June 2008
Reid, W.J. and Shyne, A.W. (1969) Brief and Extended Casework, Columbia University PressTorgerson, D.J. and Torgerson, C.J. (2008) Designing Randomised Trials in Health, Education and
the Social Sciences, An Introduction, Palgrave Macmillan; Basingstoke