scott davidoffdissertation defense familycontrolsmarthome the project on routine as resource for the...

Scott DavidoffDissertation Defense

THE PROJECT ON

Routine as Resource for the Design of Learning SystemsScott Davidoff Dissertation Defense

A pattern of behavior that is followed repeatedly, but is subject to change if conditions change

Winter 1964

“”2

Koestler 1967

Routines reduce our attention needs

Zerubavel 1981

Reduce attention needed for tasks

Wakkary & Maestri 2007

Free attention for bigger challenges

Wolin + Bennett 1984

The cyclic nature of routine makes it a natural target for machine learning

Machine learning can use this order

Suchman 1983, Tolmie et al. 2002

Idiosyncrasies are hard to model

Certain human routines can be modeled, increasing the scope of activity recognition

Machine learning

Opportunities

The ability to use learned routines in end-user applications would solve a variety of human problems

Opportunities

Dual-income family logistics

Sequence of place and transportation (rides) that occur on daily, weekly, and seasonal cycles

“”13

Darrah et al. 2000

Managing details can be difficult

Frissen 2000

Routines give a feeling of control

Beech et al. 2004

Life does not always follow routines

Breakdowns lead to loss of control

Perry et al. 2001, Ling + Campbell 2003

A constant need to follow updates

Gneezy + Rustichini 1998, Darrah 2009

A constant source of anxiety

We can learn a model of family logistical routines, and present that information to families to help them feel more in control of their lives

Scott DavidoffDissertation Defense 21

2We can use sensing and modeling to synthesize missing information resources

3Show how to use the model and evaluate the impact of the information

1We can use fieldwork to identify missing but needed information resources

Fieldwork Modeling Validation

Create an ordered list of places and rides

Attend to the details of a plan as it unfolds

Planning and coordinationLogistics

Coordinate

Fieldwork1

22 Davidoff et al Ubicomp 2006Months

Months

Months0606

12Families

Families

Davidoff et al Ubicomp 2007

Davidoff, Dey + Zimmerman CHI 2009

45 Months24Families

Semi-Structured Interviews

Davidoff et al. Ubicomp 2006

Scott DavidoffDissertation Defense scott davidoff, min kyung lee, anind dey + john zimmerman

Needs Validation

User Enactments

Phone Calls

CalendarsDavidoff, Dey + Zimmerman CHI 2009

528 phone interviews

109 activity Interviews

108 calendar months

Less than 20% of days go exactly as planned1Routines are not documented

People have incomplete knowledge of other people’s routines

People make plans that depend on incorrect information

Family E

Routine

Deviation, Scheduled

Deviation, Unscheduled

Family EAll Families

Routine

Deviation, Scheduled

Deviation, Unscheduled

Less than 20% exactly as planned

Routineness of Family E Activities

Routines are not documentedFieldwork Finding

August September October November

People have incomplete knowledge of other people’s routines

Fieldwork Finding

People make plans that depend on incorrect informationFieldwork Finding

Orthodontist

Scouts

School

Dad Work

4:00 5:00 6:00 7:003:002:00 8:00

S16Dad

View of Plan: Dad

Check-up

Track Practice

Orthodontist

Scouts

School

Dad Work

4:00 5:00 6:00 7:003:002:00 8:00

S16Dad

View of Plan: S16

Paper Route

Track Practice

Orthodontist

Scouts

School

Dad Work

4:00 5:00 6:00 7:003:002:00 8:00

S16Dad

Who Will do the Paper Route?

Check-up

Track Practice

Information gaps can break down coordination

No resources exist to find needed information

Modeling2

2 31 Ride Detection

Driver Prediction

Predict Lateness

In Situ Observation

Maintain Current Behaviors

Ubiquitous Sensing

Parent drives kid to an activity

Parent drives kid from activity

Pick-ups and drop-offsRide

Pick-up

Drop-off

Ride Detection

Day Care

Using GPS to Sense a Drop-Off

Parent Child 62

t3 t4 t5 t6t2t1 t7

Day Care

Parent Child 63

t3 t4 t5 t6t2t1 t7

Day Care

Parent Child 64

t3 t4 t5 t6t2t1 t7

Day Care

t4 t5 t6t2

Parent Child 65

Day Care

Using GPS to Sense a Pick-Up

Parent Child 66

t3 t4 t5 t6t2t1 t7

Day Care

Parent Child

t3 t4 t5 t6t2t1 t7

Day Care

Parent Child

t3 t4 t5 t6t2t1 t7

GPS Ride Detection EvaluationFamily Precision Recall

How many of the sensed ridesare right

How many of the total ridesare sensed?

Pick-up Drop-off Pick-up Drop-off

A .991 .987 .912 .910

B .966 .962 .979 .981

C .913 .824 .971 .921

D .878 .873 .980 .944

E .931 .684 .959 .985

Mean .936 .866 .960 .950

.901 .955

Smarter power management

Wi-Fi, Bluetooth, cell tower ID

Apply heuristics to modelingCultural norms, Individual Behavior

Single Location Sensor

GPS Sampling Rate

Driver Prediction

Name Meaning Values

Ln Location of pick-up or drop-off Place ID

RType Ride type Pick-up, Drop-off

DoW Day of week 0,1,2,3,4,5,6

ToD Discretized time of day (15 min) 1,2,3…96

drivert-j Driver for the last 5 rides to Ln Mom, Dad

Driver distribution model [0,1]

Driver Prediction Model Features

Driver Prediction Performance x Number of Training Weeks of Data

Lateness Prediction

TnowTideal

Lateness in Minutes

Number of Examples

Distribution of Parent Late Arrivals

N = 83

Name Meaning Values

R Whether the parent remembers

True, False

Lchild Location of the child Place ID

Lcurr Current location of a parent

Place ID

J Driver prediction model Mom, Dad

Name Meaning Values

True, False

Place ID

T If the parent is traveling True, False

Lstart Starting location of a parent

Place ID

Name Meaning Values

True, False

Place ID

Empirical cumulative distribution (ecdf) of

on-time arrivals to Lchild

at time TnowTideal

Name Meaning Values

True, False

Place ID

at time TnowTideal

D Destination of a parent Place ID

Name Meaning Values

True, False

Place ID

at time TnowTideal

D Destination of a parent Place ID

tideal - 30

A’ = 0.659

tideal

A’ = 0.801

tideal +10

A’ = 0.826

Validation3

Do parents understand the view?

Do parents perceive that the information is valuable?

Do parents feel more in control having this view?

45 Davidoff et al Ubicomp 2006Davidoff et al Ubicomp 2007Davidoff, Dey + Zimmerman CHI 2009Davidoff et al. CHI 2010Months

Months24

24Families

Families

Davidoff et al. Ubicomp 2011(in preparation)

Davidoff et al. Ubicomp 2011 in preparation

Experience prototype: Doctor

Davidoff et al. Ubicomp 2011 in preparation

Experience prototype: Kitchen

It's nice to have a map of the day. I don't literally write all this stuff down, so sometimes it's just hard to keep in my head.

Visual Distills Details

−P2“”

Because the calendar [is] all words and numbers, so you have to really think about …how long everything takes…

Visual Simplifies Calculation

−P10

…With the visual you can see who's doing what at what time. You can make a decision about who's activity to change.

Visual Simplifies Calculation

−P10“”

I would be able to know where everyone was gonna be, instead of having to ask around.

Visual Clarifies Intentions

“”98

−P11

Venkatesh + Davis 2003Technology Acceptance Model-3

…helps me do my job “as a parent.”

…would want my family to use it.

…more in control of details, more ready for changesPerception of Control

Behavioral Intention

Perceived Usefulness

Perception of Control

Supports Planning

Supports Awareness

Contributions

Suchman 1983, Tolmie et al. 2002

Certain routines can be modeled

2 31 Ride Detection

Driver Prediction

Predict Lateness

Commodity GPS with real power constraints

Breakdowns lead to loss of control

Perception of Control

Supports Planning

Supports Awareness

Gneezy + Rustichini 1998, Darrah 2009

A constant source of anxiety

Empower Parents

Future Work

Improve algorithms

Use algorithms to enable other family coordination systems

Add additional sensor inputs

Extend to other forms of routine

Scott DavidoffDissertation Defense Neustaedter + Brush CHI 2006 114

6pmOrthodontist6pmOrthodontist

Neustaedter + Brush CHI 2006 115

6pmOrtho6pmOrthoPaper RoutePaper Route

Neustaedter + Brush CHI 2006 116

Scott DavidoffDissertation Defense Brown et al. Ubicomp 2007

Baseball

Brown et al. Ubicomp 2007 118

The project on

Thanks for your time, brain power, and support

Thanks to our sponsors

NSF IIS 1017429

Google Research Award

Thanks to our supporters

Thanks to my collaborators

Thanks to my committee

Thanks to my advisors

Questions

Image Credits

QUESTIONSLIDES

ROUTINE MODELINGRELATED WORK

FORMULAS

Driver Distribution

131€

driver = P1Ln ,Rtype,DoW

∑ + driver = P2Ln ,Rtype,DoW

132€

d(y)LWq =

y ie− j

i xij −q j( )

i∈labeled data,i≠q

e− j

i xij −q j( )

∑i=q∑

Bayesian Network

P(Rd | Lstartd ,Lstart

m ,Lcurrd ,Lcurr

m ,Lchild ,Td ,Tm ,φ)

RIDE SENSING TECHNICAL VERSION

Davidoff, Dey & Zimmerman CHI 2011

Two Kinds of Drop-Offs

Day Care

5:30 5:45 6:005:00

4:45 6:15

t3t2 ,t1 ,

Day Care

5:30 5:45 6:005:00

4:45 6:15

t3t2 ,t1 ,

Two Kinds of Pick-Ups

Day Care

5:30 5:45 6:005:00

4:45 6:15

t3t2 ,t1 ,

Day Care

5:30 5:45 6:005:00

4:45 6:15

t3t2 ,t1 ,

Day Care

5:30 5:45 6:005:00

4:45 6:15

Day Care

5:30 5:45 6:005:00

4:45 6:15

States = Ln,T |CoT,else{ }Using GPS to Sense a Drop-Off

Day Care

5:30 5:45 6:005:00

4:45 6:15

States = Ln,T |CoT,else{ }

People = P,C{ }

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

(t1,P,CoT)∧(t1,C,CoT)∧

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

(t2,P,Ln )∧(t2,C,Ln )∧

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

(t2,P,Ln )∧(t2,C,Ln )∧

(t3,P,¬CoT)∧(t3,C,Ln )

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

(t2,P,Ln )∧(t2,C,Ln )∧

(t3,P,¬CoT)∧(t3,C,Ln )

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

(t1,P,¬CoT)∧(t1,C,Ln )∧

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

(t1,P,¬CoT)∧(t1,C,Ln )∧

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

(t1,P,¬CoT)∧(t1,C,Ln )∧

(t2,P,Ln )∧(t2,C,Ln )∧

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

(t1,P,¬CoT)∧(t1,C,Ln )∧

(t2,P,Ln )∧(t2,C,Ln )∧

(t3,P,CoT)∧(t3,C,CoT)

Day Care

5:30 5:45 6:005:00

4:45 6:15

People = P,C{ }

(t1,P,¬CoT)∧(t1,C,Ln )∧

(t2,P,Ln )∧(t2,C,Ln )∧

(t3,P,CoT)∧(t3,C,CoT)

SCHOOL HALF DAY BREAKDOWN

Baseball

School

Dad Work

4:00 5:00 6:00 7:003:002:00 8:00

S16Dad

View of Plan: S16

Track Practice

Baseball

School

Dad Work

4:00 5:00 6:00 7:003:002:00 8:00

S16Dad

View of Plan: S16

Track Practice

Baseball

School

Dad Work

4:00 5:00 6:00 7:003:002:00 8:00

S16Dad

Dad Started Baseball League

Track Practice

OTHER APPLICATIONS

Orthodontist

Scouts

School

Dad Work

4:00 5:00 6:00 7:003:002:00 8:00

S16Dad

Detect conflict with unlabeled routine

Track Practice

Check-up

Paper Route

Baseball

School

Dad Work

4:00 5:00 6:00 7:003:002:00 8:00

S16Dad

Dad Started Baseball League

Track Practice

ACTIVITY SCATTERPLOT

RIDE SENSING ERRORS

Smarter power managementGPS Sampling Rate

Wi-Fi, cell tower, bluetoothSingle Location Sensor

GPS Sampling Rate

5:30 5:45 6:005:00

4:45 6:15

Simultaneous Departure Errors

School

Parent Child 166

Wi-Fi, cell tower, bluetooth

Apply heuristics to modelingCultural norms, Individual Behavior

Single Location Sensor

GPS Sampling Rate

1:00pm

2:00 3:00 4:0012:00

11:00 5:00

Lost Signal During Daytime

School

Parent Child 168

DRIVER PREDICTION ERRORS

USER ENACTMENT PICS

User Enactments

scott davidoffdissertation defense familycontrolsmarthome the project on routine as resource for the...

machine learning slide

zimmerman chi

families davidoff et

calendar months davidoff

learned routines

peoples routines fieldwork

peoples routines people

anind dey

Documents

articles design davidoff

towards a consistent machine learning …...

investor day 3 davidoff feb 2013 slides - imperial...

davidoff catalogo 2012

celebrating c oour milestones · kirk-davidoff, pamela a....

cv , of john g. davidoff current copy 23.07.2015

chivas.com | enjoy chivas responsibly listings...camel king...

davidoff 05a

assessing the performance of routine immunization …. scott...

introduccion a la psicologia_linda davidoff

ippt20011120, ecj, davidoff · ippt20011120, ecj, davidoff...

catalogo sigari davidoff

davidoff linda l - introduccion a la psicologia

introduccion a la psicologia linda davidoff

76105813 introduccion a la psicologia linda davidoff

capitulo 5 memoria (introduccion a la psicologia linda...

eighth annual davidoff education day

bat davidoff strategic brand review 1992-1998

advocacy and pluralism in planning* - paul davidoff€¦ ·...

davidoff бриф доп