searching for effective dui interventions— a comparison of four san diego dui programs
TRANSCRIPT
Mary M. Skorka, DUI/PSN SupervisorCalifornia Department of Health Care ServicesSubstance Use Disorder Compliance Division
Sacramento, CA 95811
Searching For Effective DUI Interventions—A Comparison of Four San Diego DUI
Programs
Some Background on CA DUI Situation
According to the latest DMV report, more than 180,000 arrests were made in 2011, suggesting a common and widespread public safety problem.
The overall situation of DUI offenses is actually improving in California.
The latest DMV-MIS report showed that alcohol-involved traffic fatalities decreased for four consecutive years (after an 8-year increase) before a slight increase in 2011.
DUI related arrests also decreased, so did the number of people injured in alcohol-involved crashes.
But the crash fatalities in 2010 still represent a 63% increase over a decade ago (2000).
Among convicted DUI offenders arrested in 2009, 73.0% were first offenders and 27.0% were repeat offenders, a considerable decrease since 1989 (37%).
Factors Contributing to the Overall Decline in DUI Offenses
Social and cultural shifts in attitudes towards drunk driving,
Public awareness campaigns, Increasing numbers of advocacy groups, Driver licensing restrictions, School-based education programs Aggressive law enforcement activities One important contributor—the large network of
DUI programs across the state that are providing probably the largest education and counseling based intervention services anywhere in the country.
Diverse Range of DUI Program Providers
There are 487 different DUI programs, under 267 program providers.
Providers range from large corporations to small stand-alone operators.
While all program operations are governed by Title 9, specific curricular and intervention activities vary tremendously across the state.
More than half of all DUI program providers have been in continuous operations for more than two decades.
Program Outcomes Are Not Easy to Assess Due to Diversity of Providers
With a few exceptions, most DUI programs are not engaged in regular program outcome assessment.
CA state oversight agency in recent years began to establish benchmarks and explore ways where common data can be gathered for outcome assessment.
DMV Annual Data Report Tracks Two Outcome Measures
(1) alcohol-involved crashes; and (2) DUI incidents, which include alcohol-
involved crashes, DUI convictions, Administrative Per Se suspensions and DUI failure-to-appear notices (FTA).
DMV reports do not seek effective programs or intervention models.
DMV reports in general find limited evidence in support of DUI programs on recidivism.
Findings on Subsequent Crashes from DMV Studies (2012)
For 8 years in a row, assignment to a DUI program was not significantly associated with 1-year subsequent crash rates;
But in 2009, for those referred to DUI programs, the crash rates are slightly lower this year (3.57 per 100 drivers) than in the previous year’s evaluation (4.08 per 100 drivers).
However, the drop in the crash rates may reflect the overall statewide decline in crashes that occurred in 2009.
3-6-9 Month Programs Make No Difference in Outcomes
DMV Studies repeatedly found no association between recidivism and length of programs.
Findings are consistent for many years that the 9-month do not produce better results than the 3-month programs; or the 6-month programs were any better than the 3-months.
David J. DeYoung, Chief of Research at DMV, previously examined the effectiveness of lengthening SB 38 alcohol treatment programs from 12 to 18 months for second offenders and found no evidence that the additional 6 months contributed to reducing DUI recidivism.
Need for Evidence-Based Practices
Two likely explanations for the overall lack of favorable outcomes from California’s DUI programs: 1) the design limitations of the DMV study produced
flawed estimates, or 2) ineffective programs produced a washed-out effect,
bringing down the performance of outstanding providers.
Goals and Challenges in Current State Oversight Functions
To identify and promote promising practices To establish greater accountability; Seek to establish consistent mechanism to
collect common data elements for outcome assessment;
Four data components are under consideration for measuring program outcomes: Demographics (what you are clients) Treatment activities (what have been provided) Program operation (to what extent proposed services have
been provided—the dosage measure) Impact of treatment (e.g., knowledge gain, recidivism)
Four San Diego Programs
San Diego County is one of few counties in CA that gather common data elements.
A common data collection instrument (e.g., socio-demographic variables of all program participants, their substance abuse histories,
offense locations, residential zip codes, and DUI program activities.) Consistent data submission protocol Established programs with stable operations Stable service catchment areas Distinct Program Characteristics Invariant pricing structure
Distinct Program Models Model A: a non-profit social service agency that has its own operational
manuals for many years and in recent years adopted a curriculum from another large non-profit DUI program provider in Northern California that has a wide presence in the state.
Model B: a faith-based community service provider that until very recently did not have a unified or well-articulated curriculum. The program was highly flexible, depending largely on the personal experiences and skills of individual counselors. Much latitude was exercised in curriculum activities and delivery styles.
Model C: a University-based non-profit DUI program provider that has an in-house curriculum that is heavy on clinical social work principles that focus on personal growth. The curriculum has been published and available to the public, and it represents a large single-site operation.
Model D: a large private entity with a proprietary curriculum and procedure, and has a wide presence in the state with operations in six counties.
San Diego Data Problems
Until this study, San Diego County did not examine its DUI program data that have been collected for close to 15 years.
No data cleaning, matching, analysis or reporting ever took place.
Lack of resources at the County level prevented County AOD administrator to seek technical support or provide feedback and quality control to DUI program providers.
Separate data are collected at two points in time: Intake and Discharge.
Mis-matched IDs and duplications were a problem. Extreme values were never checked nor cleaned. Two immediate recommendations: (1) maintain only one data
(preferably exit data); and (2) perform regular data cleaning tasks, at least on an annual basis.
Common Data Elements Revealed:
San Diego performed above CA state average. But not all DUI programs performed at the
same level. Program completion rates appeared to reflect a
combination of management styles and curricular activities.
Differences in managerial styles seemed apparent in reasons of program termination.
Table 1. Enrollment by Providers in San Diego County
DUI Program Provider
Model A(N=33,080)
Model B(N=32,663)
Model C(N=46,919)
Model D(N=43,560)
3 Month 64.4 64.2 64.9 60.9
6 Month 4.0 3.7 6.3 5.3
9 Month 4.6 5.7 5.1 5.3
12 Month 0.2 0.2 0.1 0.1
18 Month 21.6 24.9 19.8 19.312 Hour (Wet Reckless) 3.8 1.0 2.7 9.0
12 Hour (Age 18-20) 1.3 0.2 1.1 0.0
3 Month (2nd Offense) 0.0 0.0 0.0 0.0
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-20100.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
69.7% 69.1% 68.0% 68.9% 70.5% 70.8% 70.3% 68.8% 68.6%71.7% 71.5%
64.0%
72.0%75.0% 75.6% 75.5% 75.6% 74.2% 75.5% 76.8% 77.0%
80.6%
Figure 1. San Diego DUI Programs Out-performed Statewide Average (All Programs): CA = 69.9%; San Diego = 75.4%
California San Diego
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-2010
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
75.6% 75.1% 74.9% 75.7% 77.2% 77.2% 76.4% 76.2% 76.3% 78.0% 79.3%
70.9%
78.2% 79.8% 80.6% 80.9% 81.2% 80.0% 81.7% 83.8% 83.2%86.7%
Figure 2 Completion Rate of 3-Month Programs OVerall: CA=76.5%; San Diego=81.2%
California San Diego
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-2010-5.0%
5.0%
15.0%
25.0%
35.0%
45.0%
55.0%
65.0%
54.8% 53.9%51.2% 52.2% 52.6% 52.5% 52.7%
50.9% 50.5%52.4%
54.0%
12.0%15.2%
53.2%55.7%
50.5%54.0% 52.8% 54.1% 54.7%
59.4% 59.7%
Figure 3. Completion Rate of 18-Month ProgramsOverall: CA=52.6%; San Diego=52.5%
California San Diego
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-20100.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
26.8% 27.7%28.8%
27.7%26.3% 25.8% 26.2%
27.3% 27.2%
24.8% 25.1%
33.5%
26.6%
23.0% 22.7% 22.3% 22.6%23.9%
22.5%21.3% 21.1%
17.9%
Figure 4. Termination Rate (All Programs) Overall: CA = 26.6%; San Diego = 22.7%
California San Diego
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-2010
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
21.5%22.3% 22.5%
21.7%20.2% 20.1% 20.8% 20.7% 20.5%
19.1%18.0%
27.3%
20.9%
18.7%18.1%17.3%17.3%18.5%
16.9%15.2%15.6%
12.3%
Figure 5. Termination Rate of 3-Month Programs Overall: CA=20.7%; San Diego=17.5%
California
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-2010
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
40.4% 41.5%43.8% 42.8% 42.7% 42.4% 42.2% 43.7% 43.2% 42.1% 41.1%
81.4%78.4%
43.0% 41.1%45.1%
43.0% 44.1%42.0% 41.1%
37.0% 36.8%
Figure 6. Termination Rate of 18-Month ProgramsOverall: CA=42.3%; San Diego=43.7%
California
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-20100.0%
0.5%
1.0%
1.5%
2.0%
2.5%
3.0%
3.5%
4.0%
4.5%
3.5%
3.2%3.3% 3.4%
3.2%3.4% 3.5%
3.9%4.1%
3.6%3.4%
2.5%
1.5%
1.9%1.7%
2.2%
1.8% 1.9%2.0%
1.9% 2.0%
1.6%
Figure 7. Transfer Rate (All Programs) Overall: CA = 3.5%; San Diego = 1.9%
California San Diego
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-2010
0.0%
0.5%
1.0%
1.5%
2.0%
2.5%
3.0%
3.5%
2.9%
2.6% 2.6% 2.6% 2.6%2.7%
2.8%3.1%
3.2%
2.8%2.7%
1.8%
.8%
1.5%1.3%
1.8%
1.5% 1.5%1.4%
1.1%1.2%
.9%
Figure 8. Transfer Rate of 3-Month Programs Overall: CA=2.8%; San Diego=1.3%
California
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-2010
0.0%
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
4.8%4.6%
4.9% 5.0%4.7%
5.1% 5.1%5.5%
6.2%
5.6%
4.9%
6.6%6.3%
3.9%
3.2%
4.4%
3.0% 3.1%
3.9%4.2%
3.6% 3.5%
Figure 9. Transfer Rate in 18-Month ProgramsOverall: CA=5.1%; San Diego=3.8%
California
Provider
Model A(N=19,61
7)
Model B(N=18,5
42)
Model C(N=26,10
2)
Model D(N=25,3
90)
Attempts to Program Completion
1 84.20 84.83 97.63 98.832 12.24 11.20 2.21 1.083 2.74 2.94 0.15 0.074 0.64 0.77 0.01 0.025 0.13 0.19 0.00 0.006 0.05 0.05 0.00 0.007 0.01 0.01 0.00 0.008 0.00 0.02 0.00 0.00
Table 2. Program Completion by Numbers of Attempts*
*These percentages were based on all individuals who had successfully completed the program.
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-201050%
55%
60%
65%
70%
75%
80%
85%
90%
95%
Figure 10. Program Completion Rate by DUI Program Models
Model A Model B Model C Model D California
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-2010
60%
65%
70%
75%
80%
85%
90%
95%Figure 11. 3-Month Program Completion by Providers
Model A Model B Model C Model D California
1999-
2000
2000-
2001
2001-
2002
2002-
2003
2003-
2004
2004-
2005
2005-
2006
2006-
2007
2007-
2008
2008-
2009
2009-
2010
15%
25%
35%
45%
55%
65%
75%
85%Figure 12. 18-Month Program Completion by Models
Model A Model B Model C Model D California
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-20105%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Figure 13. Termination Rate (All Programs) by Program Models
Model A Model B Model C Model D California
1999-
2000
2000-
2001
2001-
2002
2002-
2003
2003-
2004
2004-
2005
2005-
2006
2006-
2007
2007-
2008
2008-
2009
2009-
2010
5%
10%
15%
20%
25%
30%
35%
40%Figure 14. Termination Rate (3-Month Programs) by
Models
Model A Model B Model C Model D California
1999-
2000
2000-
2001
2001-
2002
2002-
2003
2003-
2004
2004-
2005
2005-
2006
2006-
2007
2007-
2008
2008-
2009
2009-
2010
10%
20%
30%
40%
50%
60%
70%
80%
90%Figure 15. Termination Rate (18-Month Programs) by
Models
Model A Model B Model C Model D California
1999-2000
2000-2001
2001-2002
2002-2003
2003-2004
2004-2005
2005-2006
2006-2007
2007-2008
2008-2009
2009-20100.50%
1.00%
1.50%
2.00%
2.50%
3.00%
3.50%
4.00%
4.50%
5.00%Figure 16. Transfer Rate (All Programs) by Models
Model A Model B Model C Model D California
1999-
2000
2000-
2001
2001-
2002
2002-
2003
2003-
2004
2004-
2005
2005-
2006
2006-
2007
2007-
2008
2008-
2009
2009-
2010
0.00%
0.50%
1.00%
1.50%
2.00%
2.50%
3.00%
3.50%
4.00%
Figure 17. Transfer Rate (3-Month Programs) by Models
Model A Model B Model C Model D California
1999-
2000
2000-
2001
2001-
2002
2002-
2003
2003-
2004
2004-
2005
2005-
2006
2006-
2007
2007-
2008
2008-
2009
2009-
2010
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%Figure 3.9. Transfer Rate (18-Month
Programs) by Models
Model A Model B Model C Model D California
Model A Model B Model C Model D Freq. % Freq. % Freq. % Freq. %Exceeded Absences 1,198 10.1 2,521 23.6 2,547 20.3 3,623 26.0Non-Payment 2,211 18.6 1,998 18.7 1,836 14.6 1,142 8.2Illness-Death 17 0.1 43 0.4 104 0.8 23 0.2No Contact 6,968 58.5 2,637 24.7 5,529 44.1 7,135 51.2County Standards 462 3.9 2,064 19.3 100 0.8 378 2.7Program Sobriety 179 1.5 501 4.7 1,162 9.3 526 3.8Subsequent DUI 124 1.0 277 2.6 88 0.7 102 0.7Court Termination 91 0.8 55 0.5 408 3.3 1 0.0Other 0 0.0 15 0.1 0 0.0 0 0.0Transfer to Model A 325 2.7 38 0.4 35 0.3 50 0.4Transfer to Model B 46 0.4 154 1.4 45 0.4 97 0.7Transfer to Model C 43 0.4 48 0.4 35 0.3 79 0.6Transfer to Model D 69 0.6 72 0.7 144 1.1 0 0.0Out-of-County Transfer 184 1.5 263 2.5 516 4.1 778 5.6
Total 11,917 100.0 10,686 100.0 12,549 100.0 13,934 100.0
Table 2. Reasons for Termination by Provider (N=49,086)
Take-Home Points
DUI programs in CA represent a large-scale social-behavioral intervention; therefore its performance and impact need to be assessed.
Common data elements are first step to seek evidence-based practices.
San Diego common data elements showcased the possibility of such data gathering across the state.
Information about differences in program performance and operations is critical for county and state oversight agencies.