using research to inform professional decision making in a context of risk brian j taylor department...
TRANSCRIPT
Using Research to Inform Using Research to Inform Professional Decision Professional Decision Making in a Context of RiskMaking in a Context of Risk
Brian J Taylor
Department of Social Work
SSRG 09 April 2008 [email protected] 2
Social Work takes place in a context of uncertainty or ‘risk’ Assessing & managing ‘risk’ is a
central feature of practice, involving decision making in situations of uncertainty*
*Skills for Care and Development (2005) “National Occupational Standards for Social Work” London: SCD (paraphrased)
SSRG 09 April 2008 [email protected] 3
Assessing and Managing Risk What do we do when we “assess risk”? We assess ‘Risk’, BUT what do we do next? What do we mean by “manage risk”? What are ‘reasonable steps to minimise risk’? Is it all about Health & Safety at Work
legislation? What are the issues for organisations,
professions, policy makers, managers, trainers and practitioners?
SSRG 09 April 2008 [email protected] 4
Organisational assessment of risk
Department of Health (2003) Controls Assurance Standards January 2003 London: Department of Health
Department of Health, Social Services & Public Safety (2003) Risk Management: Core Standard Belfast: DHSSPS
LIKELIHOOD
CONSEQUENCE
1
Insignificant
2Minor
3Moderate
4Major
5Catastro
phic
5Almost certain
Low5
Medium10
Medium15
High20
High25
4Likely
Low4
Medium8
Medium12
High16
High20
3Possible
Very Low3
Low6
Medium9
Medium12
Medium15
2Unlikely
Very low2
Low4
Low6
Medium8
Medium10
1Rare
Very low1
Very low2
Very Low3
Low4
Low5
SSRG 09 April 2008 [email protected] 5
Organisational management of risk Publicly accountable members of the Board
must have in place systems so that they know what the major risks are in their health and social care organisation Financial Legal Professional care and treatment (statutory duty
of quality)Department of Health, Social Services And Public Safety (2001) Best
Practice – Best Care: A Framework for Setting Standards, Delivering Services and Improving Monitoring and Regulation in the HPSS: A Consultation Paper Belfast: DHSSPS
SSRG 09 April 2008 [email protected] 6
Professional management of risk Accountability of professions for managing
risk as part of corporate risk management “… focus on safety, accountability and
improvement of services”* “… culture of self-evaluation and
continuous improvement.”*Department of Health, Social Services and Public Safety (2002)
Departmental Guidance HSS(PPM) 10/2002 Governance in the HPSS – Clinical and Social Care Governance: Guidelines for Implementation Belfast: DHSSPS
*Simmons L (2007) Social Care Governance Workbook Belfast: DHSSPS and London: SCIE for Clinical and Social Care Governance Support Team
SSRG 09 April 2008 [email protected] 7
By now we are all familiar with undertaking a ‘risk assessment’ on activities – even Dilbert is gradually getting used to the idea!
copyright©1997 United Feature Syndicate, Inc. Reproduced by permission
SSRG 09 April 2008 [email protected] 8
How meaningful is this risk assessment?
What value does it have?
How does risk assessment fit with professional values, client values, & the professional task?
copyright©1997 United Feature Syndicate, Inc. Reproduced by permission
SSRG 09 April 2008 [email protected] 9
Do we all see “risks” the same way ?!
copyright©1997
United F
eature S
yndicate, Inc. R
eproduced by perm
ission
SSRG 09 April 2008 [email protected] 10
Three examples of using research in relation to risk and decision making Qualitative research
in work with Older People Experimental research
in Mental Health ‘Theoretical research’
in Family and Child Care
SSRG 09 April 2008 [email protected] 11
EXAMPLE 1: Professional Perspectives on Risk and Decision Making To make organisational &
professional approaches to assessing & managing risk effective, we need to understand how practitioners ‘do the risk business’
What tools and concepts do professionals use to ‘do risk’?
SSRG 09 April 2008 [email protected] 12
Perceptions of health & social care staff on decision-making on long-term care of older people Grounded theory approach Sample: 4 of 11 HSS Trusts in NI that
deliver community health & social services one in each Board area (policy effects) at least one providing only community
health & social services at least one ‘integrated Trust’ (with acute
services)(also a proxy for rural and urban)
SSRG 09 April 2008 [email protected] 13
Data gathering method & sample Focus groups with social workers,
community nurses, occupational therapists, hospital discharge support teams, home care managers
Interviews with consultant geriatricians, general medical practitioners, older people (with their carers, where appropriate)
99 staff in total; 9 older people
SSRG 09 April 2008 [email protected] 14
Identified Risk Factors for Admission to Institutional Care (fits the evidence!) mental impairment
including dementia falls & fractures ADL limitations managing medication incontinence health-related needs sleep problems nutrition
SSRG 09 April 2008 [email protected] 15
What evidence do you use in assessment & decision making? We only predict from our experience and
knowledge (Care Manager) We know the number of people who end up
in institutional care after a fracture … most of that [predictive data on the probability of harm] would be helpful (Geriatrician)
BUT generally little explicit use of research or synthesised data in relation to individuals
SSRG 09 April 2008 [email protected] 16
Use of theory (based on evidence?) (1) I don’t think we have put <risk> on the
training agenda explicitly (Geriatrician) There is no tick chart or formula that I
work to. It’s just what is staring me in the face when I go out there (GP)
I’m used with the Brearley model to make my mind focus on issues … You know when people are shouting at you “they are at risk out there” (SW)
SSRG 09 April 2008 [email protected] 17
Building A Model of Admission Decisions Some findings led to a new conceptualisation of decision making about admission of an older person to a residential or nursing home
We illustrate using one factor >
SSRG 09 April 2008 [email protected] 18
Rated very seriously
And then there is so much robbery now and beating old people up. If she <elderly aunt> falls she would always get better again but if somebody came in and beat her up or robbed her, she would never get over it. It would kill her. (Family carer interviewed with her aunt, an older woman living at home against professional advice)
SSRG 09 April 2008 [email protected] 19
Client fears – even refusing services Everybody knows who you are. I’ve got my pension and I suppose they know how much you are worth every week and are liable to break in. Sure it’s happening all the time. (Older Man)
She needed a handrail at the front door but <refused it> because there were so many break-ins recently she thought that it might make her look more vulnerable if she had a rail. (Occupational Therapist)
SSRG 09 April 2008 [email protected] 20
But not regarded as a legitimate or effective reason for admission Researcher: Is this a factor in admission to
institutional care, this vulnerability to intruders or burglary that you mentioned?
Social Worker: It would not be of course officially in an assessment
This and other elements led to a new model:
Taylor BJ & Donnelly M (2006) ‘Professional perspectives on decision making about the long-term care of older people’ British Journal of Social Work 36 (5): 807-26.
SSRG 09 April 2008 [email protected] 21
Elements of Admission Process
Person living alone loses confidence
Person living with family carer who is no longer able or
willing to meetneeds
Professionals assessing risk considering law, policy, theory, etc
Availabilityof servicesincl, funding & people
APPLYING THIS EVIDENCE IN:Development of a Single Assessment Tool for the Health and Social Care Needs of Older People – Jan 2006 to Jun 2008 (DHSSPS)http://www.dhsspsni.gov.uk/ec-single-assessment-tool
PROJECT TEAM – University of UlsterProfessor Brendan McCormack, Institute of Nursing ResearchDr Brian Taylor, Social WorkMrs Bridget Murray, Rehabilitation SciencesMrs Joanne McConville, Project OfficerDr Paul Slater, Research Fellow
http://www.science.ulster.ac.uk/sat/
SSRG 09 April 2008 [email protected] 23
Elements of the Tool – developed with 350+ stakeholders Contact Screening Core Assessment >1 GP and Medical Practitioner Report Specialist Referral Specialist Summary & Recommendations Carers’ Assessment Complex Assessment >2
SSRG 09 April 2008 [email protected] 24
Core Assessment – informed by a range of health & social care research evidence1. Physical Health2. Mental Health & Emotional Well-being3. Awareness & Decision Making4. Medication Management5. Communication & Sensory Functioning6. Walking & Movement7. Personal Care & Daily Tasks8. Living Arrangements & Accommodation9. Relationships10. Work, Finance & Leisure
SSRG 09 April 2008 [email protected] 25
Complex Assessment Focus on risks to independent living in
relation to each domain in Core Component of the Tool Recent events, crises and their impact on the older
person, including their confidence to live at home Disruption or distress due to identified needs
(concerns, risks) ability to cope with needs rehabilitation and learning potential health promotion possibilities additional care or support required
How do identified difficulties impact on ability to make choices and decisions about daily living?
SSRG 09 April 2008 [email protected] 26
Example 2: Applying experimental evidence (knowledge) to manage uncertainty
Basic ethical principle ‘primum non nocere’ = first, do no harm ““Counselling and psychotherapy* is probably pretty Counselling and psychotherapy* is probably pretty
useless and therefore mostly harmless” useless and therefore mostly harmless”
*‘psychotherapy’ = “A generic term for the treatment of mental illness or emotional disturbances primarily by verbal or nonverbal communication” [Medline database] ie includes art therapy, drama therapy as well as most varieties of ‘counselling’
BUT let’s look at an example of emerging evidence:
SSRG 09 April 2008 [email protected] 27
5
10
15
20
25
30
35
40
Baseline 4 months 3 years
Imp
ac
t o
f E
ve
nt
Sc
ale
Debrief (Hi) Control (HI)
Critical Incident Stress Debriefing Study
Richard Mayou et al, 2000
SSRG 09 April 2008 [email protected] 28
Risks in inappropriate counselling? MAY make people worse than without treatment In depression – behavioural activation is more
effective than just talking about problems After the Omagh bomb in Aug 1998 the local Health
and Social Care Trust activated its emergency plan & set up a Community Trauma and Recovery Team (CTRT)
Of people coming because of Post Traumatic Stress Disorder (PTSD) 50% had previous unsuccessful psychological therapy or counselling which generally added to their sense of hopelessness
WHY MIGHT THIS BE? (theory)
SSRG 09 April 2008 [email protected] 29
Omagh bombing study - factors most highly associated with Chronic PTSD Injured .38 Beliefs: ‘Unsafe World’ .38 In Market Street .40 Thought would die .42 Thought /Emotion Suppression .43 Rumination .54 Negative View of Symptoms/Self .72
Duffy M, Gillespie K & Clark DM (2007) ‘Posttraumatic Stress Disorder in the Context of Terrorism and Other Civil Conflict in Northern Ireland: A Randomized Controlled Treatment Trial’ British Medical Journal 334(7604):1121-2.
SSRG 09 April 2008 [email protected] 30
What will help? How do we know? The Omagh CTRTeam (included Social Work,
Psychiatric Nursing, Psychology, Psychiatry, Occupational Therapy) had to face the question:
What helping approach shall we use? How will we know if it is effective? The team leader Michael Duffy (SW) and colleagues
developed a randomised controlled trial of the effectiveness of Cognitive Behavioural Therapy (the main treatment used) for the treatment of Post Traumatic Stress Disorder in the context of civil conflict – a previously untested application of CBT
SSRG 09 April 2008 [email protected] 31
CBT for common problems recovery
rateMajor depressive disorder 50%Panic disorder 75%Posttraumatic stress disorder 80%Social phobia 75%Generalised anxiety disorder 50%Obsessive compulsive disorder 45%Specific phobias 75%
Recovery rate is the approximate proportion of individuals who do not have the problem at the end of intervention
SSRG 09 April 2008 [email protected] 32
The end of psychosocial treatments? ABSOLUTELY NOT!! BUT We need to note & appraise the evidence which is much stronger for STRUCTURED counselling models E.g. “…problem-solving therapy, interpersonal
psychotherapy, and pharmacotherapy would be considered efficacious interventions for major depression, with cognitive-behavioral and cognitive therapy considered possibly efficacious.”
Wolf NJ & Honko DR (2008) ‘Psychosocial and pharmacological interventions for depressed adults in primary care: A critical review’ Clinical Psychology Review, 28(1):131-61
SSRG 09 April 2008 [email protected] 33
Report by Lord Richard Layard
Mental illness is the UK’s greatest social problem - £3 million to be set aside for training in evidence based psychotherapies 2008-2010 (in England)
Layard R (2004) ‘Mental Health: Britain’s Biggest Social Problem?’ Paper presented to Government Strategy Unit, London
SSRG 09 April 2008 [email protected] 34
Social Work & Psychosocial Interventions If we in Social Work lose our role in
relation to psychosocial interventions it will be because of our entrenched
attitudes and systems not because such treatments do not work
or could not be shown to work or because of a lack of public funding
cf Layard R (2006) ‘The case for psychological treatment centres’ British Medical Journal 332 (7548) 1030-2
SSRG 09 April 2008 [email protected] 35
Using Evidence:
in commissioning new services
in commissioning training
in decisions about
individuals
SSRG 09 April 2008 [email protected] 36
Applying evidence to decisions about care of individuals Going back 5 slides to the one on recovery
using CBT CBT was generally shown to be effective for
a range of conditions, BUT Between 20% and 55% of people were NOT
helped sufficiently to no longer ‘have that problem’
What approach for THIS individual? Can we refine our decision making?
SSRG 09 April 2008 [email protected] 37
Example 3: Slated by (ab)use of research?!
“Now tell me Miss Social Worker, in relation to my client Mrs Whitehouse and her beloved child Amy, have you read this piece of research by Sniff and Jones about successful treatment of addictions …?”
SSRG 09 April 2008 [email protected] 38
Research Evidence for Court Decisions What value is one research study? How will we judge the quality of a study? What value is a theory?
E.g. attachment and bonding; stigma; loss; rehabilitation; nurturing environment; discipline
Can we put more weight on a synthesis of studies than a single study?
What criteria do we set for a good synthesis? How does this study apply to Mrs Whitehouse? How will we train Social Workers for this work?
SSRG 09 April 2008 [email protected] 39
Research-based reasoning in court?
“Effective clinical reasoning requires skill in developing arguments, establishing the relevance of information to an argument, and evaluating the plausibility of assertions or claims…” Osmo R & Landau R (2001) ‘The need for explicit argumentation in ethical decision–making in social work’ Social Work Education 20(4) 483-492 (page 489)
“Expressions of opinion must be supported by detailed evidence and articulated reasoning” (Munby J in Re M (Care Proceedings: Judicial Review) (2003) 2FLR 171 p.183)
“Mrs R’s Article 8 rights required that her child should not be taken from her unless every feasible alternative was thoroughly explored and rejected for good reason.” (AR v Homefirst Community Health and Social Services Trust (2005) p.18 para 101)
SSRG 09 April 2008 [email protected] 40
One model of explicit argumentation1. CLAIM Make a claim or conclusion
2. WARRANT Explain relationship between claim & data
3. DATA Provide grounds, data & evidence
4. JUSTIFY relationship between evidence and claim
5. QUALIFY the claim (degree of confidence)
6. Highlight LIMITATIONS & reservations to the claim (including conditions where it might be weakened)
7. JUSTIFY (2) with additional evidence in relation to qualifiers and limitations
Stephen Edelston Toulmin (1958) ‘The Uses of Argument’ Cambridge: Cambridge University Press (adapted)
SSRG 09 April 2008 [email protected] 41
A Child Care example Ms G has three children who are in care –
problems of alcohol abuse & neglect Child A (12) & Child B (14) are with different
foster families, Child C (15) is in residential care
Ms G has been erratic regarding contact and there have been problems of aggression
Decision required about contact – considered detrimental to the children
SSRG 09 April 2008 [email protected] 42
Evidence in child protection (1)?1. CLAIM - Rights under Article 8 HRA should
be over-ridden in the interests of the children2. WARRANT – outline issues & relationship of
these To whose benefit is continued contact? Emotional well-being of children: identity, distress,
future prospects, stability Impact of Ms G’s addiction on children
3. DATA on visits, views of children etc, & research evidence related to these issues to inform professional judgment
4. JUSTIFY connection: evidence and concerns
SSRG 09 April 2008 [email protected] 43
Evidence in child protection (2)?5. QUALIFY – research relating to severity &
likelihood of harm and benefit for continuing contact and for ceasing it
6. LIMITATIONS – different situation of each child, possible change in problematic maternal behaviour
7. JUSTIFY (2) – research on resilience, capacity to change once addicted etc
SSRG 09 April 2008 [email protected] 44
What is the research process to test & bring this model to practice, if it proves suitable at each stage?1. Consider application of theoretical model
2. Apply to historical or hypothetical cases?
3. Seek professional views?
4. Pilot studies?
5. Controlled trial of effectiveness, usability & acceptability?!
SSRG 09 April 2008 [email protected] 45
Key Issue: Prediction of Harm Is it ‘safe’ to return this child home?
Has this parent recovered sufficiently from mental illness or addiction?
Is it ‘safe’ to discharge this person from psychiatric hospital in terms of possible violence or suicide?
Is it ‘safe’ for elderly Mrs Brown to return home from hospital after her stroke?
Is it ‘safe’ for ‘us’ to support this step towards independent living of young Mr O’Kane with a disability?
SSRG 09 April 2008 [email protected] 46
Predicting (Screening for) ‘problems’ Suicide rate for Northern Ireland 2006
227 male; 64 female Treat as approximately 300 in a population
of about 1.5 million i.e. 2 in 10,000
Could we use a screening tool to identify these people so as to target services?
SSRG 09 April 2008 [email protected] 47
Predicting rare events If we had a test that was 90% accurate and we are trying to predict a rare event such
as abuse, violence or suicide that occurs say 2 in 10, 000 people per year (as in this example)
amongst e.g. a population of 100,000 people EXERCISE: IDENTIFY ISSUES IN USING
THE TOOL BY CALCULATING HOW MANY ARE CORRECTLY AND INCORRECTLY IDENTIFIED
SSRG 09 April 2008 [email protected] 48
Effectiveness of a tool to predict harmTOOL: yes no
REALITY
yes <1> <2>
TP FN
No <4> <3>
FP TN
SSRG 09 April 2008 [email protected] 49
Predicting rare events
The real incidence of abuse or suicide or other risk would be 20 cases per year in the population of 100,000 [2 in 10,000], and so
there would be 99,980 cases that year where this harm did not occur.
[total = 100,000]
SSRG 09 April 2008 [email protected] 50
False Negatives
With a 90% accurate tool, we would detect
18 of the 20 cases of harm correctly (item 1, TP),
but would miss the other 2 “false negatives” (2, FN) because the test wrongly said there was no risk
SSRG 09 April 2008 [email protected] 51
False Positives
OF the 99,980 cases that were not a risk, the test would correctly show 89,982
[90%] as being no risk situations (3, TN) but would incorrectly pick up 9,998 (10%)
of the no-risk situations as being risky “false positives” (4, FP) because the test incorrectly labels them as risky when they are not.
SSRG 09 April 2008 [email protected] 52
Issues in using a tool to predict harmTOOL: yes no total
REALITY
yes 1818 22 20
No 9,9989,998 89,98289,982 99,980
Total 10,016 89,984100,000
SSRG 09 April 2008 [email protected] 53
Using a 90% accurate tool to predict NI suicides (pop 1.5M)TOOL: yes no total
REALITY
yes 270 (TP) 30 (FN) 300
No 149,970 1,349,730 1,499,700
(FP) (TN)
Total 150,240 1,349,760 1,500,000
SSRG 09 April 2008 [email protected] 54
Problem of False Positives
Trying to predict rare events such as abuse, violence or suicide in a much larger population presents particular problems
Many non-risky situations will be identified as being “risky” with consequences for individuals and for workloads of professions, teams & organisations
How can we know for certain whether or not this individual is the one where the harm will occur?
IMPOSSIBLE!
SSRG 09 April 2008 [email protected] 55
The Best Prediction Tools The very best prediction tools in social work
and criminal justice are achieving rather lower accuracy than the example above (perhaps 70% - 80%)
e.g. predicting the likelihood of a convicted offender being re-convicted by committing a similar type of serious crime
False positives & false negatives are an even greater problem with a less accurate tool
SSRG 09 April 2008 [email protected] 56
“A Continuum of Risk” “Research therefore cautions us that [in
mental health] as in other fields such as medicine and child protection there is no such thing as a ‘risk free’ assessment….
… There are no criteria which enable us to place individuals into sharply-defined, once-and-for all categories of ‘dangerous’ or ‘not dangerous’. Rather there is a continuum of statistical risk with uncomfortably limited predictive capacity.”
Perry, J and Sheldon, B (1995) Richard Phillips Inquiry Report, London: City of Westminster, and Kensington & Chelsea & Westminster District Health Authority (p.18)
SSRG 09 April 2008 [email protected] 57
Prospects for Actuarial Risk Assessment Tools continually improving for predicting probability
of harm esp. criminal justice & mental health Actuarial prediction often discriminates better
between risk levels than clinical judgment Might inform priority for services Recognise limitations regarding predicting harm Take a broader view of ‘assessment’ Accept more diffuse approaches to targeting
preventive measures E.g. postcode vs identifying individual ‘children in need’
Study how we make professional decisions
SSRG 09 April 2008 [email protected] 58
Conclusions (1) To make organisational & professional
approaches to assessing & managing risk effective, we need to understand how practitioners ‘do the risk business’
Assessment tools to support decision making can be valuably informed by research and can support professional clinical judgment but we cannot predict rare events (harm) accurately
Be confident in our SW skills in psycho-social interventions – but we need to focus on treatments based on robust evidence
SSRG 09 April 2008 [email protected] 59
Conclusions (2) Evidence can be applied more readily to
commissioning services and training than to decisions about care of individuals
We need a ‘network’: to energise research on risk, evidence and
decision-making in Social Work to build theoretical models of decision making To develop models for bringing research from
theoretical concepts through empirical studies and into practice
Using Research to Inform Using Research to Inform Professional Decision Professional Decision Making in a Context of RiskMaking in a Context of Risk
THE END