cricos code 00025b western public...the referral data used to build the model covers the period from...
TRANSCRIPT
CRICOS code 00025B
CRICOS code 00025B
2019 John Western Public Lecture
Data analytics in the public sector – the tortoise or the hare?
Professor Rhema Vaithianathan
@IssrUq
@uqsocsci
#johnwestern2019
A video recording of the John Western Public Lecture is available here:https://issr.uq.edu.au/article/2019/09/2019-john-western-public-lecture
the privatesector
the private sectorearly adopter
1998 2007
Amazon Recommender Google AdWords Walmart
2000 2004
Facebook Analytics
the private sectordata drives profitability
Consent in exchange for customer value
Data is a strategic asset
AI/ML seamlessly incorporated into the business process
End-to-end data businesses
the private sectorinnovative advantage
the publicsector
the public sectorslow adoption
2005
NHS Hospital Admission Tool
(UK)
NEET Prediction Tool (NZ)
Hospitals use IBM Watson (GER & US)
2012 2016
COMPAS tool (US)
2017
CentrelinkRobo-debt
(AUS)
1998
the public sectormultiple hurdles
High risk with low customer value
Data use is often unconsented
Expectation of transparency
Need for a human-in-the-loop
the public sector should be more like the private
“
Aleksandr Kogan, (about his app that collected personal details of 80 million
Facebook users for Cambridge Analytica)
Honestly we thought we were acting perfectly appropriately. We thought we were doing something that was really normal.
And Cambridge Analytica is not alone in these reversals…
the private sectorwhat went wrong?
Consent in exchange for customer value
No social licence
Data is a strategic asset
Data centred vs. human centred
AI seamlessly incorporated into the business processOpaque
End-to-end data business
Weak data-rights
The data science conundrum
enjoy!
We can do amazing things with data.
Should we?
public sector case study: child abuse prediction
The Allegheny Family Screening Tool (AFST)
annual child abuse referrals US
3.6 million
AFSTthe problem
AFSTthe problem
One third of American children experience a CPS investigation (for child abuse or neglect) before they turn 18
AFSTthe problem
Half of all children who are critically or fatally abused were never the subject of a CPS investigation beforehand
Child & Family
History
Behavioral Health
County Prison
Alcohol & Drug
Juvenile Justice
ProbationPublic Welfare
Parental CPS
Family’s Welfare
Community indicators
*Only if an MCI_ID is successfully established. The referral data used to build the model covers the period from 25 Aug 2008 to 13 March 2015, and there are 58,801 referrals (calls) and 50,076 (unique) victims in total. 2,236 victims (4.5%) did not have established MCI_ID
Call comes in
Score is produced for each child named on the call
Harvests data from the data warehouse
AFSThow it works
AFSTwhat it predicts
A score of 1 to 20 gives the likelihood that a child on the referral will have an out-of-home placement in 2 years
0%
10%
20%
30%
40%
50%
60%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Score
Observed Two Year Placement Rates Following Risk Scoring
AFSThow well it predicts
California Prototype
California prototype can we use less data?
AFSTmultiple databases
California Prototypesingle child welfare
database
California prototype how well it predicts
010
2030
4050
6070
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
Per
cent
California prototype maltreatment deaths
010
2030
4050
6070
Per
cent
1 2 3 4 5 6 7 8 9 10
California prototype cancer deaths
The data science conundrum
enjoy!
We can do amazing things with data.
Should we?
human centered data scienceelements of a winning strategy
Agency leadership
Transparency and fairness
Community voice
Multi-disciplinary team
Ethical review
Independent Evaluation
Agency leadership
Emily Putnam-Hornstein (USC)
Diana Benavides-Prado, (CSDA, AUT).
Emily Kulick (CSDA, AUT), Kyle Jennison (Allegheny County)
Child Welfare
Data Science
Technical Implementation
Katie Arvay (Allegheny County)
Tim Dare (UoA), Eileen Gambrill(UC Berkeley)
Business Process Implementation
Ethics
Fairness/Disparities
Aimee Wilkins (CSDA, AUT)
Comms/MediaJeremy Goldhaber-Fiebert(Stanford Medical)
Evaluation
Leadership
Marc Cherna, Erin Dalton (Allegheny County)
Alex Chouldechova (CMU)
AFSTRhema Vaithianathan
(PI)
Multi-disciplinary team
Transparency and fairness
“
Dare and Gambrill, UoA and UC Berkely
Ethical Analysis: Predictive Risk Models at Call Screening for Allegheny County
In our assessment, subject to the recommendations in this report, the implementation of the AFST is ethically appropriate.
Indeed, we believe that there are significant ethical issues in not using the most accurate risk prediction measure.
35
Ethical review
Community voice
“
Goldhaber-Fiebert, Stanford University, Independent Evaluation of AFST
Implementation of the AFST … increased the accurate identification of children who needed further intervention services, without increasing the workload on investigators
37
Independent Evaluation
The data science conundrum
enjoy!
We can do amazing things with data.
How should we?
human centered data scienceelements of a winning strategy
Agency leadership
Transparency and fairness
Community voice
Multi-disciplinary team
Ethical review
Independent Evaluation
lessons
The way forward
The public sector can lead the way by adopting human centred data science
what next?
what next?
Emily Putnam-Hornstein, PhDDirector, Children’s Data NetworkAssociate Professor, University of Southern CaliforniaResearch Associate, UC Berkeley
Acknowledgements
Erin Dalton, Director of Research and AnalyticsAllegheny County , PA
csda.aut.ac.nz
