DATA-DRIVEN APPOINTMENT SCHEDULING IN THE PRESENCE OF NO-SHOWSMichele Samorani Linda LaGangaOctober 16

2012

The No-Show Problem0Mental Health Center of Denver (MHCD)0 Large nonprofit organization0 36% of the appointments were no-shows!

0 MHCD can’t charge for not showing0 MHCD already uses reminder calls0Progress in reducing no-shows for psychiatrist

appointments0About 25% on average, and varies between doctors

0 What can they do?

Solution 1: Open Access

0 Show rate decreases if “lead time” increases

0 Give only same-day or next-day appointments 0 If too many patients call in for same-day appointments, defer

them to tomorrow (Robinson and Chen 2010)

Same-day Next-day 2 days 3 days 4 days

Show rate .87 .74 .75 .72 .71

Solution 2: Overbooking0 Compress slots (LaGanga and Lawrence 2007)

p p p p

12:00

11:30

11:00

10:30

10:00

9:30

9:00

Regular Scheduling

Overbookingp p p pp

9:00

9:20

9:40

10:00

10:20

10:40

11:00

11:20

11:40

p p p p

12:00

p p p pp9:00

9:20

9:40

10:00

10:20

10:40

11:00

11:20

11:40

p p p p

12:00

Lucky CaseLow waiting time

Low overtime

Unlucky CaseHigh waiting time

High overtime

Overbooking

Data-Driven Appointment Scheduling

Solve Scheduling

ProblemAppointmentRequest

ClassificationRule

Day & Slot

Day 2:00 PM 2:20 PM 2:40 PM

0

1

2

Current Schedule

Current day

SSN

SS

S S

N

Use Analytics to Schedule Appointments

Scheduling Horizon

(h)

Show in day 0Show in day 1

No-Show in day 2

Day-dependent show outcomes!• Lead time• Personal schedule• Day of week• Weather

Minimize overtime and waiting time

Goals0 Understand the causes of no-shows (descriptive analytics)0 Accurately predict show outcomes (predictive analytics)0 Optimally schedule appointments (prescriptive analytics)0 The scheduling policy must be practical (descr. + prescr. anlyt.)0 Provide guidelines on clinic design (prescriptive analytics)

Classification Rule

Solve Scheduling

ProblemAppointmentRequest

Day & Slot

ClassificationRule

Understand the causes of no-shows (descriptive analytics) Accurately predict show outcomes (predictive analytics) Optimally schedule appointments (prescriptive analytics) The scheduling algorithm must be interpretable (descr. + prescr. anlyt.) Provide guidelines on clinic design (prescriptive analytics)

0 Any Classification algorithm requires a mining table

0 Typically, the mining table is built manually0 We build it automatically0 Through Propositionalization (Samorani et al. 2011)

Classification Rule

Appointment Attribute 1 … Attribute n Show?

1 D12 … 12.3 Yes

… … … … …

56,000 Q21 … 0.0 No

Appointment

Service Type

Day of week

Location

Lead time

Show outcome

Client

Gender

Diagnosis

Age

Recovery Markers

Employment

Housing

…

Staff

Discipline Code

0..N1

0..N10..N

1

Propositionalization

1. Pick a path starting from Appointment2. “Roll-up” attributes3. Add a new attribute to the table Appointment

More than 3,000 attributes built in 3 hours!

Age of the client

Average Age of the clients seen in location

Average Age of the clients seen by the staff in location

0 What attributes are most discriminant?0Lead time0Location0Previous no-show rate0Service type0Staff characteristics:0 Number of times they

performed group therapy0 Number of times they

performed case management0 Number of times at a

certain location

New Knowledge!

Expected

Unexpected

Sensitivity

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Spec

ific

ity

0.0

0.1 0.2 0.3 0.4 0.5 0.6

0.7 0.8 0.9

1.0

Light grey: Random prediction Orange: Prediction quality at MHCD with Bayesian Network LB = lower bound; UB = upper bound

Performance frontier

Prediction Quality0 Sensitivity = accuracy among the non-showing appointment requests0 Specificity = accuracy among the showing appointment requests0 Cost-sensitive classification to shift quality towards sensitivity or specificity

UB

LBLB

LBLB

LBLB

LBLB

LBLB

LB

ClassificationRule

Solve the Scheduling Problem

AppointmentRequest

Day & Slot

Solve Scheduling

Problem

Understand the causes of no-shows (descriptive analytics) Accurately predict show outcomes (predictive analytics) Optimally schedule appointments (prescriptive analytics) The scheduling algorithm must be interpretable (descr. + prescr. anlyt.) Provide guidelines on clinic design (prescriptive analytics)

s.t.

Mathematical Model-Patient categories for

waiting time

-Day- and patient-dependent revenues

-Solved via column generation

Interpret the output of the scheduling algorithm

Solve Scheduling

ProblemAppointmentRequest

Day & Slot

ClassificationRule

Understand the causes of no-shows (descriptive analytics) Accurately predict show outcomes (predictive analytics) Optimally schedule appointments (prescriptive analytics) The scheduling algorithm must be interpretable (descr. + prescr. anlyt.) Provide guidelines on clinic design (prescriptive analytics)

Develop a Heuristic

0 Target sequence: S S N S S N

0 A further analysis reveals that0 No-shows tend to be scheduled far in advance 0 Shows tend to be scheduled in the near future

0 Heuristic: Schedule predicted shows soon in S-slots Schedule predicted no-shows far in the future in N-slots

-2.1% profit compared to optimal procedure

0 Let’s analyze the output of the scheduling algorithm:

Guidelines on clinic design

Solve Scheduling

ProblemAppointmentRequest

Day & Slot

ClassificationRule

Understand the causes of no-shows (descriptive analytics) Accurately predict show outcomes (predictive analytics) Optimally schedule appointments (prescriptive analytics) The scheduling algorithm must be interpretable (descr. + prescr. anlyt.) Provide guidelines on clinic design (prescriptive analytics)

Sensitivity and Specificiy

*Bars and labels = profit

Same-day scheduling is worst if prediction quality is high

Sensitivity and Specificity regulate the trade-off between Patients seen

and Wait. / Over times

Comparison to Open Access

Policy Profit Wait time (minutes) Overtime (minutes)

Open Access 6.05 0.00 48.79

HP (.9, .5) 7.14 14.74 15.22

HP (.7, .7) 6.92 15.58 20.47

HP (.6, .8) 6.82 15.42 22.97

Lower Bound 6.44 18.14 31.11

Upper Bound 7.89 5.18 0.37

0 Open Access: same- or next-day scheduling without overbooking (Robinson and Chen 2010)

0 HP (sn, sp): heuristic procedure with sensitivity sn and specificity sp. Scheduling horizon = 5 days

0 10,000 day simulations

+18.0%

+10.9%(benefit of analytics)

0 It can be shown that analytics is less beneficial for shorter scheduling horizons

At MHCD

0 Low show rate Shift prediction quality to high sensitivity0 High show rate Shift prediction quality to high specificity

*In selected MHCD clinics with low show rate, assuming capacity = 8, slots = 12

Benefits of Analytics

Heuristic

L. Bound

Benefits of Analytics

*In selected MHCD clinics with low show rate, assuming capacity = 8, slots = 12

Heuristic

L. Bound

Conclusions

Contributions and Managerial Insights

0 Find causes of no-shows

0Develop a dynamic scheduling algorithm that uses individual day-dependent no-show predictions

0 Develop an effective heuristic procedure that is interpretable and easy to implement

0Find that same-day appointment is the worst policy if predictive analytics is used

0 Outperform open access by 18% at MHCD

0 Reduce system variability

Innovation in Analytics

0 Descriptive Analytics:0 Propositionalization to find new knowledge

0 Predictive Analytics:0 Cost-sensitive classification to favor one of the two

conflicting objectives

0 Prescriptive Analytics:0 Suggest when to lean towards sensitivity or specificity0 Study the output of optimization through analytics

Implementation at MHCD0 The implementation of the scheduling system at a

MHCD clinic is currently in progress0 First phase (DONE): implement an “observer”0 Second phase: implement it in a real clinic

Thank you for your attention