ubiquitous computing ii: research projects

Ubiquitous Computing II:Research Projects

Ubiquitous Computing II:Research Projects

POSMIS Lab. POSTECHPOSMIS Lab. POSTECH

POSTECH Strategic Management of Information Systems Lab.


ContentsContents EasyLiving by Microsoft

e-Class by Georgia Tech.

Paper++ by DCI (Disappearing Computing Initiatives)

EasyLiving by MicrosoftEasyLiving by Microsoft



EasyLivingEasyLiving Project for developing a prototype architecture and

technologies for building intelligent environments

Key features• Computer vision for person-tracking and visual user interaction• Multiple sensor modalities combined• Use of a geometric model of the world to provide context• Automatic or semi-automatic sensor calibration and model building• Fine-grained events and adaptation of the user interface• Device-independent communication and data protocols• Ability to extend the system in many ways



Elements of EasyLivingElements of EasyLiving

Sensing andSensing andWorld ModelingWorld Modeling

DistributedDistributedSystemSystem

ArchitectureArchitecture

UI andUI andApplications ofApplications of

Computing in theComputing in thePhysical WorldPhysical World



Person Tracking System (1/2)


Stereo processing with commercial software

Background subtraction and person detection

Reports sent to central person tracker about 7 Hz

colorcolor depthdepth patchespatches peoplepeople

TriclopsColor Stereo

Cameras

Stereo Processingand Person Detection

PersonTracking





Multiple sensors of different modalitiesMultiple sensors of different modalities

Explicit reasoning about occlusionExplicit reasoning about occlusion

No more expensive multi-camera headsNo more expensive multi-camera heads

One vision computer per roomOne vision computer per room

VoxelVoxelrepresentation of 3D srepresentation of 3D space in roompace in room



Active BadgesActive Badges Badges for person tracking:

• Larger area, less precision• Fusion via generic person tracker

Ultrasound badges (AT&T) in lab use

Diffuse-IR system from Arial Systems

Signal strength from Wireless LAN

Bluetooth beacons

http://www.arialsystems.com/products/TAN_Detail.html



Keyboard DetectionKeyboard Detection Match color and approximate shape

11

22

33



EasyLiving Service ScenarioEasyLiving Service ScenarioTom is at home. He enters the living room sits down at a P

C in the corner. He surfs through a selection of MP3's, and adds them to a playlist. He gets up and sits down on the couch. His session follows him to the large wall screen across from the couch. This screen is selected because it is available and in Tom’s field of view. Tom picks up a remote control sitting on the coffee table and uses the trackball on it to request the room controls. They appear in a window on the wall screen, showing a small map of the room with the controllable lights. He uses this interface to dim the lights. Tom opens up his play list and presses play. The music comes out of the room's large speaker system.

Sally enters the living room from the sliding doors to the outside and walks over to the PC. She has to manually log in, since she hasn't previously identified herself. She brings up a Word document that is an invitation to a shindig she and Tom are hosting. Wanting Tom’s input, she asks him if she can use the large room display. He uses the remote to release control of the wall screen, and she uses the room's controls on her PC to move her session to that display.

Tom is at home. He enters the living room sits down at a PC in the corner. He surfs through a selection of MP3's, and adds them to a playlist. He gets up and sits down on the couch. His session follows him to the large wall screen across from the couch. This screen is selected because it is available and in Tom’s field of view. Tom picks up a remote control sitting on the coffee table and uses the trackball on it to request the room controls. They appear in a window on the wall screen, showing a small map of the room with the controllable lights. He uses this interface to dim the lights. Tom opens up his play list and presses play. The music comes out of the room's large speaker system.

Sally enters the living room from the sliding doors to the outside and walks over to the PC. She has to manually log in, since she hasn't previously identified herself. She brings up a Word document that is an invitation to a shindig she and Tom are hosting. Wanting Tom’s input, she asks him if she can use the large room display. He uses the remote to release control of the wall screen, and she uses the room's controls on her PC to move her session to that display.

e-Class by Georgia Tech.e-Class by Georgia Tech.



ObjectiveObjective An attempt to study the

impact of Ubiquitous Computing on education

Support both teaching and learning in a university through the introduction of automated support for lecture capture



Stage 1: Feasibility Studies (1/4)


Capture enough of the lecture experience to provide students both short & long term benefits

Four phases of Architectural Scheme• Preproduction• Live Capture• Postproduction• Access





LiveBoard• A 67-inch diagonal upright pen-bas

ed computer• Integrate document and video conf

erencing and multimedia presentation with whiteboard





ClassPad• The pen can be used to write on the slide as the lecture progress• Every time a slide was visited, ClassPad would record the time• After class, the lecturer and student slides were converted into a series of HTM

L





Comparison of teacher and student notes



Stage 2: A Living Laboratory (1/6)






Zen★ (“zen-star”) System• An Electronic Whiteboard System in Java• ZenMaster: Central Server• ZenPad





Failed miserably as an access interface• Return to the HTML only access interface of Stage 1 • Time-stamp information for every pen stroke• Draw every pixel• Begin capturing Web browsing behavior during classes• Stream Weaver: make use of client-side image maps





HTML only access interface by StreamWeaver





Manual effort needed to coordinate activities• Effort was needed to take a prepared presentation and import it into

the system• Need to synchronize• Log of visited URLs and the digitally encoded audio needed to be

placed in the central data repository• Stream Weaver had to be invoked manually to produce the HTML

notes



Stage 3: Recent Enhancements (1/2)


Supporting nontraditional classroom use.• DUMMBO (Dynamic Ubiquitous Mobile Meeting Board)





Allow the students once again to take their own electronic notes in class

Paper++ by DCIPaper++ by DCI



Disappearing Computing InitiativesDisappearing Computing Initiatives

Mission• “see how information technology can be diffused into everyday

objects and settings, and to see how this can lead to new ways of supporting and enhancing people’s lives that go above and beyond what is possible with the computer today”

http://www.disappearing-computer.net/



Paper++Paper++ EU project to develop and investigate technologies

for the integration of paper and digital media• Paper, printing and reader devices• Server technologies to link information sources

Technology Overview• Use inductive ink to print a grid of invisible position information on

paper• Inexpensive, pen-like reader device• Support flexible, dynamic authoring of two-way links between digital

and paper documents



The Myth of the Paperless Office

The Myth of the Paperless Office

For decades, people have predicted the office of the future as a paperless office• Documents generated electronically• Documents published and distributed electronically• Documents read electronically

What has happened to this imminent Revolution?• It is much simpler to scan a book by rapidly flicking through pages

than to browse through a digital document• Paper also supports forms of collaboration and interaction that are

difficult to mimic in current digital environments

Toward an integration of paper and digital technologies



Data IntegrationData Integration



The Concepts of Shapes, Layers, Users (1/2)


Shape• A selectable area of a page

Layer• Every shape is positioned on a specific layer.

Formally, • there are k layers numbered 1, 2, . . . , k, and that each shape Si is associated with exact

ly one layer i.e. layer(Si) = j where 1 ≤ j ≤ k.• For a given point (x, y), we write (x, y) ∈ Si to denote that the point (x, y) is contained withi

n the shape Si. If two shapes Si and Sj are overlapping, i.e. there exists some point (x, y) such that (x, y) ∈ Si and (x, y) ∈ Sj , then the shapes must be on different layers i.e. layer(Si) ≠ layer(Sj ).

• At any point in time, the set of active layers is a subset of all layers. Let A denote the set of all active layers. Then A ⊆ {1, 2, . . . , k}.

• For a set of shapes S = {S1, S2, . . . , Sn}, the selected shape associated with the given point (x, y) is given by the function s

• s(x, y) = Sm • Where• layer(Sm) = max{j|Si ∈ S, (x, y) ∈ Si, layer(Si) = j and j ∈ A}



Mapping of page positions to defined shapes

Mapping of page positions to defined shapes



Dynamic Links bet’n DocumentsDynamic Links bet’n Documents



Paper++ ServerPaper++ Server Consists of three main components

• The barcode decoder and visualization component• XIMA component• OMS Java data management component

eXtensible Information Management Architecture (XIMA)