note: some of the material in this …moncef/sgn-5857-pcm/p2.pdf · vtt technical research centre...

NOTE: SOME OF THE MATERIAL IN THIS

PRESENTATION IS COPYRIGHTED

BUT FREE FOR RESEARCH PURPOSES.

CHECK WITH

pertti.huuskonen@nokia.com

BEFORE USING

OUTSIDE THE

SGN-5858 SEMINAR.

Wednesday, March 11, 2009

Wednesday, March 11, 2009

Wednesday, March 11, 2009

What is context?S O M E E X A M P L E S

Wednesday, March 11, 2009

Wednesday, March 11, 2009

Could I have some

Camels, please.

2 packs, please.

Cigarettes! Cigarettes,

you idiot!

CONTEXT IN DIALOGUE

Yes, up here in my room.

What? Animals not allowed?

Wednesday, March 11, 2009

Wednesday, March 11, 2009

June 25, 2008 UbiComm Slide

Ubiquitous Computing

“Ubiquitous, a. Existing or being

everywhere; omnipresent” – The New Webster’s Dictionary

“Computer access will be

everywhere: in the walls,

on wrists, and in ‘scrap

computers’ (like scrap

paper) lying about to be

grabbed as needed. “

This is called ‘Ubiquitous Computing’, or ‘ubicomp’.-- Mark Weiser, IEEE Computer, October 1993

http://www.ubiq.com/

Wednesday, March 11, 2009

NOKIA Point & Find

paf.nokia.comWednesday, March 11, 2009

Wednesday, March 11, 2009

Wednesday, March 11, 2009

© NOKIA Pertti Huuskonen

How are contexts recognised?

Higher level context

Context recognition(time series sensor fusion)

Indoors

WalkingFrequency

Car

Trip to work

Tim

eArrival

Elevator

WalkingFrequency

Indoors

WalkingFrequency

0.9

0.8

1.0

0.93

0.7

1.0

0.6

1.0

1.0Context atoms in a time sequence vector

Context atoms Probabilities

Wednesday, March 11, 2009

Multichannel sensor interpretation

Source: VTT Electronics/Panu Korpipää

Phase of the scenario Correct set of contexts

1. Drive car (Car, Activity)

2. Run to the door (Other sound, Running, Outside, Activity)

3. Walk to the elevator (Other sound, Walking, Inside, Activity)

4. Wait for the elevator (Other sound, Inside)

5. Elevator (Elevator, Inside)

6. Walk to the door (Other sound, Walking, Inside, Activity)

7. Open the door, sit down (Other sound, Walking, Inside, Activity)

8. Make a phone call (Speech, Inside)

9. End call, put on music (Other sound, Walking, Inside, Activity)

10. Classical music (Classical music, Inside, Still)

11. Stop, change disc, play (Other sound, Walking, Inside, Activity)

12. Rock music (Rock music, Inside, Still)

13. Stop music, walk to tap (Other sound, Walking, Inside, Activity)

14. Take tap water (Tap water sound, Inside)

15. Walk to the table, sit (Other sound, Walking, Inside, Activity)

16. Eat (Other sound, Inside, Activity)

17. Conversation (Speech, Inside)

9 channels: 3x acceleration, 2x light intensity, temperature, humidity, skin conductivity, audio

Wednesday, March 11, 2009

Wednesday, March 11, 2009

Semantic Data

15

Wednesday, March 11, 2009

Modeling people / the world / life

... is hard.

Wednesday, March 11, 2009

Wednesday, March 11, 2009

and / or common sense :-)

Google ad upon searching “Context Awareness”

Google needs some context awareness

Wednesday, March 11, 2009

Wednesday, March 11, 2009

Why Middleware?

Context Application0.784V,

0.348 mA,

-0.84i+0.23j

Context Application0.784V,

0.348 mA,

-0.84i+0.23j

Outdoors,

In Hand,

Running

Middleware

Wednesday, March 11, 2009

Context Framework

Series 60

Wednesday, March 11, 2009

Nokia N97 contains the context framework in its

Symbian operating system.

Wednesday, March 11, 2009

Wednesday, March 11, 2009

© 2007 Nokia

File: - E:\PICS\IMG_8699.JPG. . Make - Canon. Model - Canon EOS 300D DIGITAL. Orientation - Top left. XResolution - 180. YResolution - 180. ResolutionUnit - Inch. DateTime - 2006:11:05 18:45:52. YCbCrPositioning - Centered. ExifOffset - 196. ExposureTime - 2 seconds. FNumber - 5.60. ISOSpeedRatings - 400. ExifVersion - 0221. DateTimeOriginal - 2006:11:05 18:45:52. DateTimeDigitized - 2006:11:05 18:45:52. ComponentsConfiguration - YCbCr. CompressedBitsPerPixel - 3 (bits/pixel). ShutterSpeedValue - 2 seconds. ApertureValue - F 5.60. ExposureBiasValue - 0.00. MaxApertureValue - F 3.50. MeteringMode - Multi-segment. Flash - Not fired, compulsory flash mode. FocalLength - 22 mm. UserComment - . FlashPixVersion - 0100. ColorSpace - sRGB. ExifImageWidth - 3072. ExifImageHeight - 2048. I n t e r o p e ra b i l i t y O f f s e t - 2 3 6 6 . Fo c a l P l a n e X R e s o l u t i o n - 3 4 4 3 . 9 5 . FocalPlaneYResolution - 3442.02. FocalPlaneResolutionUnit - Inch. SensingMethod - One-chip color area sensor. FileSource - DSC - Digital still camera. CustomRendered - Normal process. ExposureMode - Auto. WhiteBalance - Auto. SceneCaptureType - Landscape. . Maker Note (Vendor): - . Macro mode - Off. Self timer - Off. Quality - Fine. Flash mode - Auto + red-eye reduction. Sequence mode - Single or Timer. Focus mode - One-Shot. Image size - Large. Easy shooting mode - Landscape. Digital zoom - None. Contrast - High , +1. Saturation - High , +1. Sharpness - High , +1. ISO Value - 400. Metering mode - Evaluative. Focus type - Auto. AF point selected - . Exposure mode - Easy shooting. Focal length - 18 - 55 mm (1 mm). Flash activity - Not fired. Flash details - . Focus mode 2 - 65535. White Balance - Auto. Sequence number - 0. Flash bias - 0.00 EV. Subject Distance - 65535. File number - 000 - 0000. Image Type - IMG:EOS 300D DIGITAL JPEG. Firmware Version - Firmware Version 1.1.1. Camera Serial Number - 1570502381. Image Number - 2868699. Owner Name - MarjaN.

Wednesday, March 11, 2009

© NOKIA

uk.weather.com

GPS+Place server

GSM clock

Message log

Bluetooth

WLAN

Presence &Calendar

Metadata

Wednesday, March 11, 2009

Wednesday, March 11, 2009

Wednesday, March 11, 2009

images, video, audio, messages, contacts, playlists, bookmarks,…

personal settings

game content

macros & scripts

social network data

superdata

landmarks

remixes

collections & combinations (benchmark: Apple iLife suite)

…

Content history and relation

“What was the MP3 I was listening to when I took this photo?”

UI skins, voice commands, profiles, T9

dictionaries, …

items, movies, characters,

saved games,…

New types coming up all the time; slow adoption

Emerging content types

Wednesday, March 11, 2009

Wednesday, March 11, 2009

3 © 2007 Nokia

Wednesday, March 11, 2009

Wednesday, March 11, 2009

Wednesday, March 11, 2009

Wednesday, March 11, 2009

Forget-me-not by Xerox Research (around 1993)

•“Intimate computing”: carry a computer

(ParcTab) at all times

•Collect contextual data into a biography

• Personal location (by tabs)

• Encounters with people (by tabs)

• Workstation activities: programs, files, email

• Phone calls

•Context as retrieval key

•Access with icon-based hypertext system

on the tabs

http://www.lamming.com/mik/Papers/fmn.pdf

Wednesday, March 11, 2009

Context Movies

! Presents collected context through fun and entertaining video diaries

! Minimal labor – yet graphically spectacular movies!

! Save to gallery or send via MMS

Calendar

Set Time span, determine movie length and fetch background music from media gallery

Then preview, and save to gallery …

…. or send as MMS

Wednesday, March 11, 2009

© 2007 Nokia

Wednesday, March 11, 2009

Wednesday, March 11, 2009

Photo: Gardiner Pearson (http://www.state.gov/r/pa/ei/pix/events/secretary/2003/16993.htm)

WatchingListening

Snoring

Texting

Conversing

Sleeping

Wednesday, March 11, 2009

IEEE Wireless Communications,

October 2002, pp. 39-45

Wednesday, March 11, 2009

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

40

Case study: Sports data sensing

• Based on the Palantir dataset

• Several people are doing the same activity

• 9 Context classes

• All recognizers have slightly different recognition accuracy

From: Jani Mäntyjärvi & Ville Könönen, VTT

Improves recognitionfrom 72% to 92%

Wednesday, March 11, 2009

Wednesday, March 11, 2009

C.S.P. (Context Service Provider)

KNOWITALL.COM

W E K N O W Y O U R C O N T E X T .

(Picture depicting the agents from the movie Matrix removed)

Wednesday, March 11, 2009

> 38°C> Quite humid> Not moving> Touch detected> Inferred location:

by the pool _

Wednesday, March 11, 2009

Burp.

> 38°C> Quite humid> Not moving> Touch detected> Inferred location:

by the pool _

Wednesday, March 11, 2009

Wednesday, March 11, 2009

© NOKIA Pertti Huuskonen

Why is context awareness needed?

!"Wednesday, March 11, 2009

Wednesday, March 11, 2009