1

Assisted Cognition

Henry Kautz, Oren Etzioni, Dieter Fox,

Gaetano Borriello, Larry ArnsteinUniversity of Washington

Department of Computer Science & Engineering

2

An Epidemic of Alzheimer’s Disease

3

4

5

Statistics for United States4.6 million people with Alzheimer’s16 million people by 2050Today costs $100 billion @ year for care

Additional $61 billion in lost productivity from family members

$ ½ Trillion total cost by 2050!

6

Lost CompetenciesShort-term memoryAbility to carry out complex tasks (driving, paying bills, cooking, house-hold tasks)Ability to orient self in time and space Memory of eventsDressing, bathing, cooking, eating Memory of conceptsSelf-initiativeRecognize friends, relatives

7

Cognition in ContextCan often compensate for physical disabilities by change in environment

Wheelchairs

Cognitive competence also depends on environment

PhysicalSocial

8

Social Context

9

Social Context

cooking

dressing

gardening

self-medicating

personal grooming

shoppingexercise

10

ProblemCaregiver burnout

½ of all family caregivers suffer depression“The 36 Hour Day”

Far too few professional caregivers to provide constant 1-on-1 help in institutional settings

Already a nationwide shortage of good staff

11

Assisted Cognition Systems

Learn to interpret human behavior from low-level sensory data

General commonsense knowledgePatterns of behavior idiosyncratic to the particular userExternal data sources

Actively offer prompts and other forms of help as neededAlert human caregivers when necessary

13

Architecture

14

Applications

The Activity CompassThe Adaptive Prompter

15

The Activity CompassGoal: help person safely and independently move about the community (including use of public transit)

User carries GPS/wireless equipped PDAAC system tracks user’s position, predicts where user is going based on past experienceSystem offers help when

Inferred plan is likely to fail otherwise (e.g. miss bus)User is likely to be lost or disoriented (wandering, on wrong bus)User explicitly consults PDA

16

Gathering Data

17

green – GPS readings (10 sec), yellow – location estimation (probability distribution)

18

Creating the User ModelTraining Data:

20,000 GPS readings3 weeks of occasional useReduce noise using Kalman filter

Hand labeled by mode of transportationWalking, In Car, On Bus, Riding Bike, Inside

Predicting current modeInput: current location/time/velocityDecision tree learning: 98.9% accuracy (10 FCV)

Predicting next mode transition(s)Input: current mode/location/time/velocityDecision tree learning: 98.8% accuracy (10 FCV)

19

Crisis Prediction (I)When might user need help?

“Cutting it too close”Associated with each leaf of decision tree is a spatio-temporal windowCompute expected position of next transition within that windowIf position is at or near upper temporal boundary, increased probability that expected transition will fail to hold (e.g., user will miss the bus!)

20

Crisis Prediction (II)When might user need help?

External changes in the worldSome kinds of transitions (e.g. board bus) are enabled by external forces (the bus!)Real-time Seattle transit information available onlineUse information to more finely label training dataCrisis = prediction that is inconsistent with external information

21

Crisis Prediction (III)When might user need help?

Novel eventsAfter-the-fact discovery that predicted behavior did not occurAsk user to confirm actions are intended

Explicit error modelsDangerous locationsWandering trajectory

22

Intervention StrategiesMust balance

disutility of crisis cost of annoying userprobability of crisisdo not want to over-rely on negative reinforcement

Qualitative preference language“Never let me miss a bus late at night”

23

User Interface

24

The Adaptive PrompterGoal: help a person carry out a multi-step task

Smart home tracks residents and objectsHierarchical recognition model

Simple behavior (sleeping, walking)Simple actions (get into bed)Meaningful patterns of actions – plans

Model of possible failures and interventions

25

Example1. Joe enters bathroom at 9:00 am.2. He turns on water, and picks up toothbrush.3. Nothing happens for 15 seconds. AC

system recognizes “tooth brushing” activity has stalled.

4. Prompts Joe to pick up toothpaste. Joe does so and completes task.

5. Joe leaves bathroom with water still running. AC system gently encourages Joe to go back and turn it off.

26

Towards a Behavior Description Language

Requirementsprobabilities on fluents and eventscontinuous (or finely discretized) timeprobability distributions on temporal relations between eventshierarchical eventsplans – intended complex eventsdefective planssystem interventionsutilities of defects and interventions

27

ApproachDevelop scenarios for AP in consultation with experts on Alzheimer’s carePrototype specification languageSemantics via translation into Dynamic Bayesian Networks

Interventions: Dynamic Decision Networks

A terrific KR challenge!See work by Martha Pollack, Hung Bui, Geib & Goldman, Daphne Koller

28

Data Source: Elite CareOakfield Estates

                                           

29

PeopleUniversity of Washington Computer Science & EngineeringUW Medical CenterAlzheimer’s Disease Research Center (ADRC)UW Institution on AgingIntel Research

People & Practices – User studies of future technology needsIntel Research Seattle – Ubiquitous computing

Elite CareOakfield Estates assisted living

http://assistcog.cs.washington.edu/

30

Advertisement

UbiCog 2002 – Workshop on Ubiquitous Computing for Cognitive Aids

September 29, 2002Gothenberg, SwedenPart of UBICOMP-2002, the major ubiquitous computing conferenceSlots for speakers still available, email Henry Kautz <kautz@cs.washington.edu>