promoting child well-being by using machine learning ... welfare...analytics purpose: predict an...
TRANSCRIPT
![Page 1: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/1.jpg)
Community Science
Promoting Child Well-Being by Using Machine Learning Algorithms
Gaithersburg, MD 7/21/2017
![Page 2: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/2.jpg)
THE STATE OF THE CHILD WELFARE SYSTEM
o 3.6 million child abuse & neglect referrals per year, 6.6 million children
o 5-7 child deaths per day
o A child abuse and/or neglect report every 10 seconds
o $124 billion in economic costs, annually
o NO significant change over the last 10 years
o One of the worst records among all industrialized countries
![Page 3: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/3.jpg)
TOTALS AND AVERAGES DON’T HELP
THOSE ON THE FRONTLINES
![Page 4: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/4.jpg)
QUICK OVERVIEW OF TERMS
o Big data
o Social science
o Data science
o Analytics
o Machine learning
![Page 5: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/5.jpg)
BIG DATA
![Page 6: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/6.jpg)
WHAT ARE “BIG” DATA?
Big Large Small
![Page 7: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/7.jpg)
SOCIAL SCIENCE
&
DATA SCIENCE
![Page 8: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/8.jpg)
The Population
Average Moved
(p<.05)
SOCIAL SCIENCE
Evidence-Based
Decision-MakingIf we repeat the intervention as it
was implemented during the
experiment, the population average
will improve 95 times out of 100.
![Page 9: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/9.jpg)
You are like me!
Can you tell me
how you did it?
DATA SCIENCEProbability-Based
Decision-MakingIf a specific type of intervention
experience worked for 90% of the
cases that were very similar to the
current case, then there’s a 90%
chance it will work.
![Page 10: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/10.jpg)
ANALYTICS
![Page 11: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/11.jpg)
DESCRIPTIVE ANALYTICS
![Page 12: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/12.jpg)
PREDICTIVE ANALYTICS
“Predictive
analytics tells
you what will
[likely]
happen”
![Page 13: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/13.jpg)
PRESCRIPTIVE ANALYTICS
“…prescriptive
analytics tells you what
to do about it.”
Jeff Bertolucci, Information Week
Find the
Positive
Deviants!
![Page 14: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/14.jpg)
MACHINE LEARNING
![Page 15: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/15.jpg)
MACHINE LEARNING
Machine learning is “like the
scientific method on steroids,
making observations, forming
hypotheses, testing hypotheses, and
refining hypotheses, millions of times
faster than any scientists could do.”
Pedro Domingos
Professor
Dept. of Computer Science & Engineering
University of Washington
![Page 16: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/16.jpg)
HOW DO WE LEARN ABOUT
WHAT WORKS?
![Page 17: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/17.jpg)
EXPERIMENTSPurpose: Understand causation for a population
Method: Conduct small to large scale experiments, collecting new data on
subjects, ideally using the scientific method
Metric of Success: Significant [average] difference between experimental and control group
Pro: Determines what actually causes improvement for a population (tries to
remove bias)
Con: Can’t guide decision-making on case-by-case basis
Implication: Good for policymaking and funding decisions, not for individual decision-
making
![Page 18: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/18.jpg)
WHEN EVIDENCE-BASED RESEARCH MEETS THE REAL WORLD
![Page 19: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/19.jpg)
ANALYTICS
Purpose: Predict an outcome for an individual
Method: Conduct algorithmic modeling (statistical, machine learning/AI) on existing
datasets to learn patterns of associations
Metric of Success: Predictive accuracy across every member of the population
Pro: Accurate decision-making for individual cases
Con: Not causation (includes bias in the models)
Implication: Good for case prediction, not for unbiased prescription or evaluation
![Page 20: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/20.jpg)
THE BIG IDEABRIDGING SOCIAL SCIENCE AND DATA SCIENCE, USING MACHINE LEARNING
![Page 21: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/21.jpg)
SIMPLE INSIGHTS FOR ACTION
SIX STEPS
![Page 22: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/22.jpg)
STEP 1: ACCESS & PREPARE ADMINISTRATIVE DATAMACHINE LEARNING CAN ADDRESS THE ‘GARBAGE IN, GARBAGE OUT’ DATA QUALITY PROBLEM
1. More than a few
hundred cases
2. Clean up missing data
issues
3. Don’t sweat statistical
assumptions
Intake
History
Background
Situation
Interventions
Dosages
Methods
Experiences
Outcomes
Results
Milestones
Goals
![Page 23: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/23.jpg)
STEP 2: DISCOVER WHAT WORKSMACHINE LEARNING ALGORITHMS FIND EVERY PATHWAY TO SUCCESS
Services
Programs
Environments
Conditions
Situations
Placement + Parenting Program + >365 Days of Case Management
Placement + >180 Days of Residential Treatment
Placement + Permanency w/Other Family Member w/in 180 Days
No Placement + 180 Days of Residential Treatment + 6 or More Family Counseling Sessions
No Placement + Completion of Parenting
Program + Job Placement w/in 90 days
Any One of
these Solutions
Reduces the
Odds of Return
by at Least 50%
![Page 24: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/24.jpg)
3rd incident, physical
abuse & neglect
Single parent, bipolar
& addicted
Homeless X 3
Child cognitive
impaired
---------------
20% likely to engage
3rd incident, physical abuse & neglect
Single parent, bipolar & addicted
Homeless X 3Child cognitive impaired
---------------20% likely to engage
3rd incident, physical
abuse & neglect
Single parent, bipolar &
addicted
Homeless X 3
Child cognitive impaired
---------------
20% likely to engage
3rd incident, neglect Single
parent, workingAbused by partner
Three siblings effectted---------------
20% likely to engage
Repeat incident, physical abuse & neglect
Single parent, mental health
Unstable HousingChild cognitive impaired
---------------20% likely to engage
70% will return
STEP 3: FIND MATCHED COMPARISON GROUPS BACKGROUND & HISTORY THAT PREDICT LIKELIHOOD TO ENGAGE IN AND/OR RECEIVE WHAT WORKS
1st incident, physical
abuse
Single parent,
unemployed
Child does well in
school
---------------
70% likely to
get/engage
15% will return
![Page 25: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/25.jpg)
STEP 4: DETERMINE WHAT WORKS FOR EACH GROUPMACHINE LEARNING FINDS NATURALLY OCCURRING EXPERIMENTS WITHIN THE DATA
No
Placement +
Parenting
Program +
Job
Placement
N=70
All Other
Treatment
Options Tried
N=30
P<.001
5% Return
30%
Return
1st incident, physical
abuse
Single parent,
unemployed
Child does well in
school
--------------------
N=100
---------------
70% likely to
get/engage
15% will return
![Page 26: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/26.jpg)
STEP 5: EVALUATE SUCCESSMODELS EVALUATE HOW MANY CASES GOT WHAT WAS NEEDED, NOT POPULATI ON AVERAGES
Group
#1
N=100
Group
#2
N=100
Group
#3
N=100
Group
#4
N=100
Group
#5
N=100
Got What
Worked &
Didn’t
Return
70 90 85 75 55+ + + + = 375
75% Didn’t Return Due to Receiving the Right Solutions
![Page 27: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/27.jpg)
STEP 6: USE THE MODELS FOR DECISION MAKINGBUILD APPLICATIONS & LEARNING COMMUNITIES
NO NEW DATA
SYSTEM REQUIRED!
EXISTING DATABASE
SCORING ENGINE
DECISION MAKING
MODELS
CASE-SPECIFIC
INSIGHTS
Likelihood to engage: 70%
Likelihood to return: 15%
What Will Improve Success:
• Drug treatment for parent
• Parenting program
COMMUNITY OF LEARNING
![Page 28: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/28.jpg)
BROWARD SHERIFF’S OFFICE
![Page 29: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/29.jpg)
BROWARD SHERIFF’S OFFICE: 24% OF ALL CASES RETURN
43%
14%
23%
11%
5%
3%
0%
<6mos
6 to 12 mos
12 to 24 mos
24-36 mos
36-48 mos
48-60 mos
60+ mos
% of All Returns
![Page 30: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/30.jpg)
BROWARD SHERIFF’S OFFICE
o Accurately predicts next incident in 83 cases out of 100
o A prescription model that can accurately recommend which cases should go
into either intensive out-of-home services or in-home community-based
programs, and if these recommendations were applied, the result would be a
30% reduction in return cases
o A rigorous impact evaluation model, using machine-guided matching, that
proves that in 60% of the cases, intensive out-of-home case management
services from a specific provider significantly reduces their likelihood for a
subsequent incident.
o A rigorous impact evaluation model that also proves that 40% of the cases are
misplaced in these intensive services, and as a direct result, are 175% more
likely to return with a subsequent incident.
o Specific investigators make a difference.
![Page 31: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/31.jpg)
CASEY FAMILY PROGRAMS
![Page 33: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/33.jpg)
SYSTEM IMPLICATIONS
![Page 34: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/34.jpg)
IMPLICATIONS- A CAUTIONARY TALE
o Data analytics do not have to be the next "shiny new object;"
o Changing child welfare practice means developing the capacity of
organizations, systems and communities;
o Outside knowledge is needed;
o Algorithms inherit system values and bias;
o Don’t forget social or contextual factors; and
o The purpose is to accelerate learning how to protect and improve
the well being of children and youth.
![Page 35: Promoting Child Well-Being by Using Machine Learning ... Welfare...ANALYTICS Purpose: Predict an outcome for an individual Method: Conduct algorithmic modeling (statistical, machine](https://reader034.vdocuments.net/reader034/viewer/2022051605/60129058afb0586202457c38/html5/thumbnails/35.jpg)
QUESTIONS