CAPNET ProgramContext-aware Pervasive Networking

Mobile Forum’s Research Focus Area

Contents

Introduction to the CAPNET Program

Further information on application scenarios illustrated in videos

CAPNET ProgramAim

the essential foundation for new information and communications technologies in the

context-aware mobile ubiquitous computing domain.

Focus

the mobile user perspective, where other terminals and services are utilised via a personal

mobile device in a gateway-like fashion.

Goal

iterative and progressive development of technologies enabling truly pervasive networking

environments for applications, using existing and emerging computing platforms and

environments as a technological basis.

Motivation for CAPNET Program• Pervasive networking is a promising area for creation of environment aware personalised mobile services.

• The large penetration, capability increase and miniaturising of mobile networks and devices enable the creation of cheap pervasive services. => huge business potential exists!

• Next generation value nets will be built on physical sub-nets & smart personal spaces.

(E.g. local mobile commerce, smart communication, p2p interactions & consumer-initiated content creation, sharing of services/content, virtual value communities)

Support infrastructure for pervasive services:• Flexibility in network programmability• Multi-channel, multi-zoned service offering• Advanced resource and profile management• Easily adaptable, self-configurable services

Main Aim of the CAPNET Program

To create ubiquitous technology enablers – map lucrative discontinuities

“Grow” necessary workforce/competence to launch ubi business:spin-offs, new product businesses & services, fill white-spaces

Take lead in new exiting “4G” area in a concrete fashion

CAPNET Partner Network 2002

MediaTeamUniv. of Oulu

www.mediateam.oulu.fi

ISGUniv. of Oulu

www.ee.oulu.fi/mvis/research/isg/

HCIUniv. of Oulu

www.tol.oulu.fi/tutkimus/index.html

Umiacs/LAMPUniv. of Maryland

www.umiacs.umd.edu

Research

CAPNET Areas

Ubicomp via mobile device

Advanced UI

Context solving

Content Processing

Intelligent SW

Adaptive content

Environmentmanagement

ProximityTransactions

Reliability

Zoned networking

Ubi concept design

Business

Nokia

Ventures Organization

Mobile Phones

Nokia

Hantro

IBM

CCC

Sonera

University of Linköping

Tekes Nets Programme

CAPNET Research Process

User experience

An application scenario driven research approach

•to ensure that real human needs are the driving force for the technology development

•research problems are derived from actual usage needs and scenarios

•requirements derived from the application scenarios are used as a basis for architecture and technology development.

•prioritised list of scenarios is used for defining the order of implementation of CAPNET demonstrations

•selected application scenarios are illustrated in the visualizations at the end of this presentation and on the videos provided here

User experience evaluation

•methods and frameworks for evaluating and measuring the factors affecting to the user experience in ubiquitous environments

•evaluation of CAPNET demonstrations and implementations from the user experience point of view

CAPNET Architecture

Each device contains a CAPNET Runtime Environment - which in turn contains the engines

Engines are the basic entities of a distributed system requiring very little resources as a basic setup

Engines consist of components

Core component offers basic functionality for an engine type

Dynamic component can be tailored according to services needs

CAPNET RUNTIME ENVIRONMENT (CRE)CAPNET RUNTIME ENVIRONMENT (CRE)

ENGINECORE

COMPONENT

DYNAMICCOMPONENT

DYNAMICCOMPONENT

ENGINECORE

COMPONENT

DYNAMICCOMPONENT

DYNAMICCOMPONENT

CRE

CRE

CRE

CAPNET UniverseCAPNET Universe

CRE

Engines Are the Building Blocks of the Architecture

Each engine is targeted to a domain area– Component Management Engine controls all components and partitions application optimally

– Connectivity Management Engine controls all connections

– Service Discovery Engine finds components, engines, and external services

– Context Engine provides contextual information for other engines, etc

UBI enabler engines have to be included in every CRE

Light engines in mobile terminals, more capable engines in the network

Application level

Value adding service enablers

UBI enablersComponent

Management

Connectivity

Management

Service

Discovery

Media

Processing

Context

Management

Virtual

Storage

UIApplication

Logic

……

……

……

Architecture Taxonomy1. Direct Engine-Application Connection (direct value add)

2. Local Domain of Networked Engines (proximity value add)

3. Hybrid Domains of Networked Engines (richest value add)

E

Browser Application

Local/Mobile Environment

E

Browser Application

Local/Mobile Environment

E E

E

E

Local Hosted Environment

E

Browser Application

Local/Mobile Environment

E EE

E

Local Hosted Environment(Local domain)

E E

E

Remote Hosted Environment(Global domain)

or

or

or

• Engine can be used directly from Web browser or application• Engine can be installed in mobile device or network server in local/global domain• Engine implements a micro-server functionality (calm: waits to be used, proactive: functions under scripts)

• Engines can establish temporary hierarchy• Application can use multiple engines• Engine can use engine(s)• Engines can be located in different local domains

• Nature of connection may vary• Engine can move to optimize service• Engine functionality can be upgraded

• Engines can establish a fixed hierarchy• Engines can seek services offered by other engines• Engine can function as a transparent gateway for other engines.• Engines can form functionality islands (larger servers) coping with load, errors, QoS demands

Main properties (cumulative 1-3):

Prototyping

•Implementing the architecture and selected application scenarios

•Aim is to

–demonstrate service concepts

–test developed architecture

–test individual technologies within a complex system

•e.g. new service discovery and routine learning algorithms

•First prototype

–Scenario: Know your where abouts

–Utilized existing technology: PersonalJava, XUL, XML-RPC, Jini

–Implemented in Java but uses language independent communications

CAPNET Test environmentProvide a realistic platform for the deployment of software prototypes that are produced within Prototypes program area.

–Basic architecture and common components can be tested and evaluated in the environment

–Prototype software packet as a new service scenario can be verified in the environment

–Provide feedback to refine both architecture and fundamental methodology creations

New invented techniques as well as applications scenarios are to be evaluated in the test environment in terms of usability, from the viewpoint of perceived user experience

–How the users experience the ubiquitous environment and its applications,

–What parameters affect the user experience.

–Evaluation results will provide feedback for the new application scenarios creation

The test environment can be used for the demonstration of the concrete deliverables of the CAPNET program.

1st Test Environment focuses on indoor Wireless Local Area Network, based on IT-NET –With wireless positioning system; PDA as the main user device; simulating main building & offices

Next Test Environment will be extended to outdoor WLAN & Wireless Wide Area Network–Outdoor WLAN based on SmartRotuaari; wireless positioning; urban downtown & business centre;

–WWAN based on Octopus environments; GPRS connection; suburban, rural, general coverage;

–Sensor technology for human activity analysis will provide context-information from the test environment.

Application Scenarios

Lunch Assistant

Simo, Raimo, and other employees of CAPNET program are having a meeting in Tietotalo’s meeting room. While the chairman is speaking the display of Simo’s mobile device starts to blink. The device is in a meeting mode.

–The device has recorded its locations from the beginning. The profile changes and the use of the device (times/places where/when he does not answer the phone etc.) have been saved into a database as well. From this data, the device has identified and analyzed Simo’s routines.

• Enabling phenomena: routine learning.

The device displays a message informing that Simo has another meeting at 13:00, and he should order a meal in advance so that he has time to eat, and be back in time for the next meeting. The device suggests different restaurants based on the travel time and available time, displays the menu of the restaurant Simo chooses, and orders the meal.

–The device knows how much time it will take to travel between locations. The device uses Simo’s calendar information, and combines this information with his context-information. Also, the device has access to restaurant databases, and it has built a profile for Simo according to his preferences.

• Enabling phenomena: routine learning, profiling, easy downloading of client applications from the restaurant, WLAN connection between Simo’s mobile device and restaurant servers, advanced human-computer interface for browsing and ordering.

Lunch AssistantWhile Simo is in the elevator, the mobile device informs that Simo is going to play badminton in the evening, reminds him of his wife’s nameday, and asks wheter he wants the sauna to be warm after badminton game as usual.

–The device notices that Simo has time to receive notifications about his evening program. It has learned that Simo wants the sauna to be warm every time after badminton, and offers to do it automatically.

• Enabling phenomena: routine learning, location awareness.

Simo goes to the restaurant, his meal is delivered to him, and he eats while Raimo is having his lunch outside at Piccolo fast food. After the meal, Simo pays for the food with his mobile device, and gives the waiter a tip for good food and smooth service.

–The mobile device gives the restaurant information about its location through Bluetooth connection, and the meal is ready when Simo enters the restaurant. The payment is done through Bluetooth connection as well.

• Enabling phenomena: location awareness, advanced human-computer interface for browsing and payment.

Simo is back in the office in good time, and has time to go through news on an adaptive notice-board.

–The mobile device is a remote control for an adaptive notice-board, and Simo can browse information stored in a news server with his device.

• Enabling phenomena: adaptive notice boards, WLAN connection between the news server and the device, personal profiling for news preferences.

User Device

General Environment

Local Environment

Connectivity

Application &Middleware

GUI/HCI

Network Connection

Lunch Assistant: Technology scenario assessment

Menu browse & order GUI

Lunch Reminder

Remind

Time

Lunch Manager

Menu & Table Info

Order

Menu & order through WLAN

Customer Record Info

Menu & Table Info

Customer-aware info organizing

UMTS

WLAN

Bluetooth

Location info through Bluetooth & WLAN

Location informerLoc

Menu & order through UMTS

Customer Record Info

Menu & Table Info

Customer-aware info organizing

Internetworking

Shopping and Wish List

Maria is wandering around shops and whenever she sees a nice item, she saves the information about the product onto her wish list, so that she can browse and choose the best ones later on.

–Maria selects certain products to be remembered. The code, the price, and the picture of each item are dowloaded, and transferred into her personal database from the store database.

• Enabling phenomena: Symbiam-based mobile device with color screen, bar code reader, Bluetooth connection to store database,and WLAN connection to a personal database for wish list.

Simo also has a wish list, he saves the information about a car onto his personal wishlist. During saving, Simo’s mobile device informs him about his wife’s (Maria) birthday. The device asks whether Simo wants to take a look at Maria’s personal wish list, so that he can choose a present for her.

–The device knows about Maria’s birthday based on his calendar information.

• Enabling phenomena: combining calendar and contex-information.

Shopping and Wish List

Simo browses Maria’s wish list, the mobile phone shows him his current location, and the shops nearby that have those items Maria has chosen onto her wish list. Simo chooses the nearest jewellery store and finds a necklace he likes. No more unfit presents for Maria.

–Simo has access to Maria’s wish list, so he can browse it with his personal mobile device. The device only suggests the stores that are near Simo’s current location.

• Enabling pehnomena: advanced human-computer interface for browsing, WLAN connection to Maria’s personal database, location awareness.

Shopping and Wish List Technology scenario assessment

See legend from next slide =>

General Environment

Local Environment

Application &Middleware

GUI/HCI

Network Connection

Sense location

Recognizecontext

Services: shopping, navigation, localization, map databases, information databases etc.

General Knowledge(Rules, service providers’

locations, maps etc.)

Personal Knowledge(Rules, profile, calendar

etc.)

Payment,request

Selection, products browse

ProductInformation

Customer &Request Info

Customer Account Info

User

Landline connection

Shopping mode ?on-off

GPRS connection

User Device

WLAN connection

Context Aware Well Being

Simo is interested in his own well-being. He purchases an active well-being application from a service provider.

–Terminal side Java/MIDP Application is downloaded to Simo’s UMTS mobile phone.

–Application can access (with Simo’s approval) personal profile and properties that are related to his well being from Simo’s profile manager.

The application starts to monitor Simo immediately.

–The application identifies devices Simo is wearing. The application identifies devices (BlueTooth, UPnP, Jini, Salutation) it can use to improve its functionality. (eg. wrist watch that provides detailed energy consumption, blood pressure and pulse information)

The application makes automated daily nutrition plans and produces a shopping list for Simo

–The application monitors Simos eating to chart the energy supply and suggests scheduled meals and snacks to help the user to achieve the target body weight and/or fitness level.

Context Aware Well Being

The application makes an exercise program for Simo.

–The application makes a daily exercise plan for Simo.

–The application monitors Simo’s physical condition and the results of Simo’s exercises, and adjusts the exercise plan according Simo’s development. The results are stored to Simo’s personal exercise calendar.

–The application can be used to book time from the gym web service (WSDL/SOAP over UMTS)

The application works in collaboration with local grocery shop

–The well being application can inform the shop that Simo is coming there and has just finished his exercise. The shop can give Simo special offers.

Context Aware Well BeingTechnology scenario assessment

User Device

General Environment

Local Environment

Connectivity

Application &Middleware

GUI/HCI

Network Connection

•Access to other wearable devices•Network access•Push content

Context knowledge(preferred gym, shops, eating habits)

Service catalog(Service provider

accessibility, 3rd party)

Install application

Connect to 3rd party

application

Connect to 3rd party hardware

Context Recognition

Monitor user, habits,

consumption

•Disconnected operation•Connected operation

Ad Hoc Networking

Simo works as a project manager in a software company. As a first thing tomorrow morning Simo has to inform his co-workers about the new project so he decides to prepare a short presentation at home in the evening. Simo prepares few slides and saves them into the his personal database. Simo dresses himself up overcoat, puts the mobile phone into his pocket and leaves the house. Audio device, TV, PC and coffee maker and lights are turned off when Simo leaves. The house enters into the economy mode.

– When user is leaving the house Local Bluetooth server informs Context server about user’s actions. General environment server turns off the audio device, TV, PC and coffee maker .

– Connection between mobile phone and Local Bluetooth server is closed.

• Bluetooth connection (Local Bluetooth server), Landline connection (Context server),

• Mobile phone and home appliances (audio device, coffee machine, lights, PC, TV etc.) must have a Bluetooth chip.

Ad Hoc NetworkingThe next morning Simo arrives at his office meeting room, other participants of the meeting are already sitting

around the table. Simo selects an icon in his mobile phone and the mobile phones shows him a menu. Mobile phone presents a list of available services that he can use in this meeting room. Simo sees a list: audio device, data projector, lights and resources.

– User’s mobile phones Bluetooth monitoring is automatically activated.

– In the meeting room mobile phone observes the local environment around it via Bluetooth and notices Local Bluetooth server’s access point which continuously sends Bluetooth signals (Bluetooth base station feature). The phone starts to communicate with it. The phone gets to know that access point is a meeting rooms local access point and the phone tells this local point who it is. Phone identification is made by SIM-card recognition.

– Local Bluetooth server gives the mobile phone the list of activities (data projector, resources etc.). The mobile phone filters the list according the context and user preferences and shows it to the user, and in addition adds ”Communication” to the list for signalling direct Bluetooth connection between other Bluetooth devices.

• Bluetooth connection (Local Bluetooth server), Landline connection (Context server), System needs context recognition capability tool in order to sense the arrival of Simo’s mobile phone in the meeting room.

• Mobile phone and office appliances (data projector, printer, laptop etc.) must have a Bluetooth chip.

Ad Hoc Networking

Simo selects ”Projector” from the list. The projector is turned on. Next Simo selects “Lights” and adjust the lights a bit.

Simo selects ”New minutes” from the list and watches the minutes and especially the list of present and absent people appearing.Next Simo selects resources and the mobile phone shows the files that have been edited last. Simo selects the slide set he wrote the previous evening. Simo’s slides appear on the wall and Simo goes on with his presentation.

–Mobile phone connects to personal database and shows the list of the files that have been edited last.

–Mobile phone sends info from the selected task and the selected document to the Local Bluetooth server. Bluetooth connection transmits the document wirelessly from Local Bluetooth server to the data projector. No cables are needed for transmission.

–Bluetooth connection (Local Bluetooth server), Mobile phone and office appliances (data ptojector, printer, laptop etc.) must have a Bluetooth chip.

Ad Hoc Networking :Technology scenario assessment

General Bluetooth Environment

User

User Device Local Bluetooth Environment

Bluetooth monitoring on

Mobile phone

Local Bluetooth server Context

server

Personal Knowledge(Rules, profile, calendar etc.)

Bluetooth compatible device availability &

user info

Bluetooth

Data projector(Bluetooth)

presentation

Landline

inquiry and data saving

List of Ad Hoc Services

1

2

Documentselection

3 4

Presentation

5

6

Thank You!

For further information please contactmarika.leskela@oulu.fi