1

Context Aware Computing

2

First Paper I used mostly “A Survey of Context-Aware Mobile Computing

Research”, Chen and Kotz, Dartmouth (Tech Report) instead of Schilit’s paper (which is the original paper coining the term)

Context-aware computing “Mobile computing application that can discover and take

advantage of contextual information” A first step towards ubiquitous computing?

What is “contextual information”? Example: user profile, location, time, temperature, nearby people or

facilities, …

“Instead of adapting systems and applications to mobility so that mobility is hidden, explore and support new mobile aware applications”

3

Category Clause

Location in-room 35-2-2-00Co-Location

with {{Name User:adams}}with {{Name Host:*} {Features color}}

Path path 35-2-2-00 35-2-2-10

Distance distance 35-2-2-10 35-2-1-01

Contextual Reminders

Select nearby stuff Browse info by location

Context Aware Computing

4

Part I: Definitions/Taxonomy

5

Defining Context

Dictionary definition: “the interrelated conditions in which something exists or occurs”

One definition [Schilit]: Computing context: connectivity, communication cost,

bandwidth, nearby resources (printers, displays, PCs)… User context: user profile, location, nearby people, social

situation, activity, mood … Physical context: temperature, lighting, noise, traffic

conditions …

Author’s add: Time context (time of day, week, month, year…) Context history can also be useful

6

Context (cont’d)

Is all this information necessary?

“Context is the set of environmental states and settings that either determines an application’s behavior or in which an application event occurs and is interesting to the user” Active context: influences the behavior of the application

• Location in a call forwarding application Passive context: context that is relevant but not critical

• Active map application: display location name and other people in the room

Is all this information measurable? Temperature? Location? People around? Social situation?

Mood?

7

Context-Aware computing

How to take advantage of this context information? Schilit’s classification of CA applications:

1. Proximate selection: user interface where nearby objects are emphasized/made easier to choose

2. Automatic contextual reconfiguration: a process of adding/removing components or changing relationships between components based on context change

3. Contextual information and commands: produce different results according to the context in which they are issued

4. Context-triggered actions: rules to specify how the system should adapt

Are these fundamental/inclusive?

8

Alternative Taxonomy [Pascoe],[Dey]

Pascoe: Taxonomy of context-aware features Contextual sensing: how to determine the context Contextual resource discovery Contextual augmentation: how to associate data with the

context

Dey maps to “services” for CA apps: Presentation of information/services to users Execution of services Tagging of context to information for later retrieval

9

Author’s classification

Follows from the definition of context

1. Active Context Awareness: “an application automatically adapts to discovered context by changing the application’s behavior”

1. Passive context awareness: “an application presents the new or updated context to an interested user or makes the context persistent for the user to retrieve later”

10

Part 2: Survey of Context-aware Applications

11

App. Survey

10+ applications surveyed, chronologically ordered We will look at a few Try to notice common threads

Our second paper is a detailed look at one of these applications

12

PARCTab System

Characteristic From Xerox Palo Alto Research Center 12 applications

Applications Active Map

• Location information (Room number)

The others

• Some helps the user find the most convenient local resources– e.g. nearest printer

• An application makes ParcTab as a remote control

Figure : ParcTab Terminal

13

Active Badge System

Characteristic From Olivetti Research Lab in 90’s One of the first context-aware systems Office personnel wear badges (IR signal)

Applications Call Forwarding

Figure : Locations of the office personnel with badges

14

Call Forwarding

Location of employees is shown to receptionist Receptionist forwards calls to the nearest phone Eventually, receptionist was removed from the loop; calls

forwarded automatically

Is this a good idea? What if you were in an important meeting? (or taking a nap!) What if a private call is forwarded to a non private room? “Context-aware communication”

What is the context? What about an active map application?

15

Mobisaic Web Browser

Extend browsers to refer to dynamic contextual information

Dynamic URLs with contextual/environment variables Variables get bound to current context when the URL is

referenced

Also support active documents As the context changes, the document is automatically

updated

What types of context can be useful here? Active or passive?

16

Shopping Assistant

Scenario: you are at home depot trying to figure out what you need to finish your basement

Shopping assistant can Tell you what parts you need Where to find them relative to your location in the store What is on sale Do comparative pricing Use your previous profile information to customize shopping

and delivery

What context? Active/passive?

17

CyberGuide

Characteristic From Georgia Tech Provide information services to a

tourist

• Direction

• Background information

• Travel diary (automatically recorded)

Indoor and Outdoor Version

• Indoor – Using IR (infrared)

• Outdoor – Using GPS Figure : Outdoor

Cyberguide Screenshot, etc.

18

Augmented Reality

User’s view of the real world is augmented with additional information

Scenario 1: you wear sun-glasses with a display and headphones and walk in downtown Rome As you move around, the glasses can tell you your location As you look at the coliseum you get information about it

Scenario: special ops squad infiltrating an enemy complex wearing the same gizmos In addition to information as above, the glasses change to

night vision if lighting is not sufficient

Context?

19

GUIDE

Developed in Lancaster University For Lancaster City visitors

Using WaveLAN as communication infrastructure A tourist comes to a region(cell), then

he receives information of the region.

Information provided using Fujitsu TeamPad 7600 portable PC Java based http browser

Screenshot of GUIDE

20

Adaptive GSM phone/PDA

PDA: notepad application changes its characteristics depending on user activity Large font when walking, small font when stationary Change the intensity level depending on the lighting

conditions

Phone: decide ring volume or vibration depending on situation In hand, in a suitcase, on a table, in a

classroom/conference?

What is the context?

21

Conference Assistant

Assistant checks out conference schedule and user’s interest to highlight which presentations should be attended

Automatically present speaker names, title of paper/other relevant info

May record slides, audio, questions, take notes and tag them with the appropriate place in the talk for later retrieval

22

Location Aware Information Delivery

Generate reminders to users as they enter locations Voice synthesis used to generate audio reminders “Sorry, dear, I forgot” is going to become obsolete?

Several projects (MIT, Stanford, GaTech)

CyberMinder (GaTech) uses a complex context including nearby people, weather conditions

But what is the basic context needed here?

23

Observations

Human-Computer interaction can be improved with context info Intelligent applications? AI?

Focus on simple context(s): e.g., location, identity, time

Why aren’t other contexts used more heavily? Not that useful? Difficult to sense/model? Need more imagination? A natural consequence of these being mobile applications?

No killer app… little gizmos of limited use Still to come?

24

StartleCam

From MIT Media Lab Composed with

Wearable video camera Computer sensing system

Save Image when the

wearer is interested By sensing skin conductivity

signal

Figure : StartleCam System

25

Sensing Context

26

How to sense the context Sensing Location – Use Global Positioning System

(GPS)

Some info: http://www.cmtinc.com/gpsbook/index.htm GPS determines the distance by measuring the time it takes

a signal to propagate from Satellite to receiver Need to have very good synchronization of clocks Receive signal from three satellites to determine location Need a fourth satellite to estimate elevation Satellite GPS accuracy is getting reasonable (10-20 meters) Sub-meter accuracy possible with differential GPS

But what to do indoors?

27

Indoors

GPS doesn’t work because the satellite signal is weak or reflected (lowers accuracy)

Need to build our own location tracking system difficult problem

Use ideas similar to GPS: compute distances from known locations that send beacons

28

Location Tracking

We will look at the Bat system from AT&T labs (as part of our second paper)

Cricket (MIT, Mobicom 2001): RF and ultrasonic signals sent from each basestation Mobile estimates distance by time interval between the two

signal arrivals Accuracy within a few feet

RADAR (Microsoft) and other efforts Measure signal strength (or SNR) to estimate distance A set of static receivers track positions of transmitters They are queried for location

29

Guide System Location Tracking

Cellular system with base stations at areas of interest Non-overlapping cells

Coarse Granularity Location You only know what area you are from by knowing the

basestation that you are in range of

Cells not overlapping; can lose track of location completely

30

Sensing other low level contexts

Time? Physical sensors possible

Light, acceleration, tilt, sound, temperature, pressure, proximity of humans

Nearby objects? If system keeps track of location of objects, query to a

database Bandwidth? Odyssey Orientation?

Orientation sensor based on two mercury switches (Newton MessagePad)

Directional antennas/directional transmitters?

31

High Level contexts

This is by far the more difficult problem Not just an engineering challenge Psychology, sociology, …??

How does a node know whether this is a party or a wake? Check your calendar or detect the cake/coffin?

32

Sensing Context changes

Publish subscribe model? Monitor polls the current context Notifies subscribers to the context Centralized or on a per-node?

Polling rate is a function of rate of change of context

33

Modeling Context Information

34

Location Model

How do we answer questions such as: Given an object what is its location Given a location, return the set of objects there Determining paths between locations How does a vehicle navigation system do it?

Geographical Information Systems/Spatial Databases Already pretty mature (part of Oracle, mysql, etc..) Model geometry and topology Typically locations are organized hierarchically Sonal Dedhia’s project

35

Security and Privacy Issues

I personally hate the idea that someone can know my location all the time I still refuse to get a cell phone, but it looks like I will finally

cave in! Par for the course, many of these systems ignore privacy

and security issues

36

Second Paper

The Anatomy of a Context-Aware Application – Andy Harter, Andy Hopper,

Pete Steggles, Andy Ward and Paul Webster. AT&T Labs, Cambridge, UK (cont)

37

Overview Paper describes the system components necessary to support

a context-aware application1. Location system2. Data model3. Distributed object model4. Resource Monitors5. Spatial monitors

Working system, real details

So what is the application? Teleporting/follow-me Application follows you as you move around a building Hmm!! Virtual dog?

• Definitely some gaming applications

• But what else? Virtual nanny?

• Framework not specific to this application

38

Location System

Would like: Fine grained (accurate in space) High update rate (accurate in time) Inexpensive Works indoors

Argue for ultrasonic Optical: expensive detectors; line of sight limitations GPS: dies indoors RF: multipath

39

Bat Unit

Radio transceiver, ultrasonic transducer and control logic

Each bat has a GUID Base station transmits a periodic message with the GUID;

corresponding bat responds with ultrasound

Use speed of sound in air (estimated from ambient temperature) to estimate location Ultrasonic receivers detect the delay and map it to distance

Use multiple receivers to get 3D location using multilateration

Reflections of ultrasonic waves – statistical outlier elimination (can same techniques be applied to RF multipath?)

40

Bat (cont’d)

41

Bat Unit (cont.)

It takes 20 ms between bat readings = 50 timeslots per base station per second Allow echos to die

Location can be used to measure orientation Attach many bats to the same object. Use the measurements to

infer the orientation If too cumbersome, can check “shadow” of a single bat

Base station can provide Location Quality-of-service to allocate time slots to bats based on the expected update frequency Bats carried by people – few times a second Bats attached to workstation – once every few minutes

42

Bat Unit (cont.)

Bats perform handover when moving from one base station to another (similar to the cellular networks) Hand off decisions can also be made based on the Bat location

Battery consumption is low, power consumed depends on the update frequency and power state Several updates a second = several month lifetime

95% of samples within 9cm’s of actual location 95% of samples within 25 degrees with multiple bats; within 70

degrees for a single bat (orientation)

Bat is good enough to be used as a 3D mouse; it has buttons and can communicate with basestation

43

How well does it work?

44

Modeling the environment

Detailed model describing entities in the real world and their possible interactions Environment consists of real objects: should use OO

modeling (agree?)

Modeling language based on entity relation diagrams and multiple inheritance Modeled people, computers, keyboards, monitors, networks,

telephones and furniture

Use CORBA and databases to implement persistent distributed objects

45

3 Tier Architecture

46

Populating and Updating the Model

Some elements are static (e.g., furniture) …but some are dynamic (e.g., is a keyboard in use)

• need to be updated automatically

Centralized data repository (to optimize information access)

Three classes of resource monitors: Machine activity e.g. keyboard activity Machine resource e.g. CPU usage, memory usage Network point-to-point bandwidth and latency

47

Client level event filters

Update Frequency The frequency at which items are monitored is based on

how quickly the item tends to change

Relevancy If a value has not changed significantly, it is not sent. This

value depends on the data being monitored

Caching Caching improves performance at the cost of consistency

48

Location Update

Each Bat location is tracked Translated into object location in a type specific way (e.g.,

person different than workstation)

49

API Absolute and relative spatial facts

“Person is at (x,y,z) facing in direction ” .vs. “Person is standing in front of the monitor”

Geometric containment is used for relative spatial facts

monitor person

Contained(person, screenspace)

50

Reasoning about Space

51

Scalability

With many devices, containment is complex

They use containment tree indexing system (a quad-tree based approach)

52

Implementation

BAT teleportation system With their earlier active badge based teleportation system,

they only knew that a user was in the room and so they had to cycle between multiple displays in a room

If a particular display was being use, they would still cycle that display because they did not monitor machines

If a machine is dead, their system would still wait because they did not monitor machines

With the BAT system, they have more accurate location information

Event driven programming style

53

Zones and Buttons

Action zone triggers teleportation Within maintain zone, teleported desktops are

maintained

Action

monitor

Maintain