1 mobil computing © 2010-2013 daniel p. siewiorek end-to-end application considerations daniel p....

1Mobil Computing© 2010-2013 Daniel P. Siewiorek

End-to-End Application End-to-End Application ConsiderationsConsiderations

Daniel P. SiewiorekDaniel P. SiewiorekCarnegie Mellon Carnegie Mellon

UniversityUniversity

February 2013February 2013


Outline

Introduction Individual

» Wearable Computers

» Augmented Reality

Infrastructure» Museums

» Hospital

» e-Display

Ad Hoc» Wildfire

» Volcano

Concluding Lessons


Outline





» Hospital

» e-Display

Ad Hoc» Wildfire

» Volcano

Concluding Lessons


Wearable Applications and Architecture

Procedures - upload at completion Work Orders - incremental updates Collaboration - real time interaction

» Client-Server

– Thin Client Legacy Systems

– Interactive Electronic Technical Manuals (IETMs)


Time Rate of Change of Data Taxonomy

Procedures. Maintenance and plant operation applications are characterized by a large volume of information that varies slowly over time.

A typical request consists of approximately ten pages of text and schematic drawings. Changes to the centralized information base can occur on a weekly basis.


Savings Using Tactical Information Assistants in Marine Heavy Vehicle MaintenanceCurrent Practice

Current Practice

SAVINGS FACTOR

VuMan 3 Field Trials

VuMan 3 Field Trials

Personnel

2:1

Inspection time

40% less

SAVINGS FACTOR



(continued)

Work Orders. The trend is towards more customization in systems.

Manufacturing or maintenance personnel receive a job list that describes the tasks and includes text and schematic documentation. This information can change on a daily or even hourly basis.



(continued)

Collaboration. An individual often requires assistance. In a “Help Desk” an experienced person is contacted for audio and visual assistance. The Help Desk can service many people simultaneously.

Information can change on a minute-by-minute and sometimes even a second-by-second basis.


C-130 Help Desk


Lessons Learned From Usage

Maximum Size, weight, energy consumption before change user behavior

No fixed relationship between input/output/display

User less patient, expect instant response Intuitive to use, no user’s manual Information overload, user may focus on

computer rather than physical world Potential to lose initiative, user does exactly

what the computer tells them to do


Based on a LectureDan Siewiorek, Thad Starner, Asim Smailagic

Morgan & Claypool

U - User C - Corporal

» Interface physically without discomfort or distraction

A - Attention» Divided between physical and virtual world

M - Manipulation» Controls quick to find, easy to operate

P - Power


Major Factors in Portable Electronic Systems and Their Relationship to Design Disciplines

Power

Corporal

Manipulation

User

Attention


UCAMP

User


Sensor-based models of human activity and situation

Example:» Using just the sensors in a laptop we have predictive

model that determines “now is a bad time to interrupt” more accurately than human observers

Full Sensor Sets Only aLaptop

No Camerasor Microphones

76.9%Human

Observers

All

Sub

ject

s

Man

ager

s

Res

earc

hers

Inte

rns

Man

ager

s

Res

earc

hers

Man

ager

s

Res

earc

hers

Inte

rns

Better than human observers with just sensors already in laptop!


Cognitive Load


fMRI Experiment Configuration


JUL 31, 2001

Car Calls May Leave Brain Short-Handed

By SANDRA BLAKESLEE

Scientists have bad news for people who think they can deftly drive a car while gabbing on a cell phone.

The first study using magnetic resonance images of brain activity to compare what happens in people's heads when they do one complex task, as opposed to two tasks at a time, reveals a disquieting fact: the brain appears to have a finite amount of space for tasks requiring attention.

The study, published in the Aug. 1, 2001 issue of the journal NeuroImage, was led by Dr. Marcel Just, a psychology professor and co- director of the Center for Cognitive Brain Imaging at Carnegie Mellon University in Pittsburgh...


Right Temporal

Left Temporal Right Parietal

Left Parietal

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

Sentences Alone Rotations Alone Sum of Single Tasks Dual Task

Capacity constraint in association areas:Activation volume is less in dual task compared to single tasks, even for tasks without cortical overlap (auditory comprehension and mental rotation)


Assessing measures of cognitive load

while performing tasks in divided attention

• Cognitive load can be a measure for whether, when and how to present information while performing a number of different tasks

• Need to manage human attention and understand when users can attend to new information

• Necessary while driving, learning new skills• Particularly an issue with elders


Assessing measures of cognitive load

• We are working to develop a reliable, real-time and objective measure of cognitive load for contexts of divided attention

• Current techniques not appropriate• Post-hoc, so not real time

• Not accurate

• Subjective measures


Study of cognitive load

• Six ECTs: visual perception and cognitive speed• Four sensors• Gaze tracking


Assessing Cognitive Load

• 20 young adults• Compared sensors for determining cognitive load

• Best feature: > 74%

• Sensors/features: median heat flux, ECG

• > 81% across all participants

• Older adults• Same sensors useful, although heat flux also seems important

• BodyMedia device seems to be sufficient to capture cognitive load with elementary tasks

• Need to see how this transfers to real settings• Then, use information about cognitive load in virtual coaches


User Lessons

“The user is not I”» Built two dozen systems to identify repeated patterns (e.g.

procedures, work orders, team collaboration) Mobile Users are more impatient than desktop users

» Must operate more like a flashlight than a computer rebooting Institutional Review Board

» “But madam, they are Marines and they do what they are told” Be prepared to reduce functionality

» “It’s a fine goal” Instant Creditability

» “I served with you on the Vinson’s first world cruise”


UCAMP

Corporal


“If it looks good,it will fly well”

MoCCA and VuMan 3 received the prestigious Industrial Design Excellence Awards (IDEA) from award co-sponsors Business Week magazine and the Industrial Designers Society of America (IDSA).

VuMan 3MoCCA


NATO Soldier Using CMU TIA-P Language Translator in Bosnia


Corporal Lessons

Air Logistics Command, Sacramento CA» The vapor barrier on a hot August day

Fort Gorden Battle Command» “Does this bend?!”

Six units to Bosnia ….. none returned


UCAMP

Attention


On

A KC-135 Aircraft was being pressurized at ground level. The outflow valves which are used to regulate the pressure of the aircraft were capped off during a 5 year overhaul and never opened back up. The post-investigation revealed: that a civilian depot technician who, "had always done it that way," was using a homemade gauge, and no procedure.

The technician's gauge didn't even have a max "peg" for the needle and so it was no surprise he missed it when the needle went around the gauge the first time. As the technician continued to pressurize the aircraft, and as the needle was on its second trip around the gauge the aircraft went "boom" - the rear hatch was blown over 70 yards away, behind a blast fence!

But, I’ve Always Done it This Way!


KC-135 Pressure Test Results


Attention Lessons

The more information and more engaging the virtual world, the less interaction with the physical world

Use systems for reference (e.g. information dashboard) then focus on physical world


UCAMP

Manipulation


Selection of “hot links” with CMU’s Wheel/Pointer


Manipulation Lessons

Strong user interaction mental model» Not only trained in the field in less than ten minutes but

also able to train the next user» No users manual» Training without access to what the user is seeing

Without stationary reference, users easily become disoriented and confused with traditional input devices

» “Which is the left button” Control devices have to be flexible to mount

on different parts of the body» Boeing – shoulder holster» Sailors on aircraft carrier – shoulders used to carry tie-

down chains


UCAMP

Power


Power Lessons

It’s the peak, not the average » Muffled pops

Warning labels are no substitute for good design

More to be gained on reducing demand than increasing source

» Current batteries have half the energy density of dynamite

» Current user interactions demand all resources to reduce response time

Land Warrior» Army standard operating system

» Nine different battery types


Speech Enabled Augmented Reality for Maintenance [Goose]

SEAR – Speech Enabled Augmented Reality 3 D Augmented Reality graphical view with

location-sensitive 3 D speech-driven interface 3 D component specific vocabulary triggered by

proximity sensors» Display component specific commands when in vicinity

» Key word activated

Simultaneous 3 D “parameterized” synthesized speech streams

» For current values from physical plant

Coded visual markers for tracking, location» 4 X 4 matrix with dots for over 10,000 combinations


Augmented Reality - Maintenance of Nuclear Power

Plant Components

Room “augmented” with the CAD drawing of a floorplan


Speech Enabled Augmented Reality for Maintenance [Goose]

System Architecture and User Interface


Outline





» Hospital

» e-Display

Ad Hoc» Wildfire

» Volcano

Concluding Lessons


Interactive Museums [Fleck]

San Francisco Exploratorium» Several hundred interactive exhibits

» Frequently rotated off the floor

Prior work» PDA, Acoustic Guides, IR beacons for pointing


Interactive Museums - Services

Informer – detailed information on exhibit Suggester – what to try Remember – build record of experiences, selects what to

record Guider – suggests an order of exhibits* Communicator – instant messaging, leave notes at exhibits*

* Not implemented


Interactive Museums – Lessons Learned

Overall positive responses Not enough hands to hold PDA and operate exhibit

interfering with exploration Undesirable demand on user attention Lost in hyper reality – focus on device rather than

exhibit Wow factor – part of positive feedback Beacons OK but sometimes picked up other beacons Browser interface – people unfamiliar with stylus Content design – did not know what was clickable Forgetting to use Remember – application complexity


Pervasive Technologies in a Hospital [Hansen]

Hospital scheduling, coordination, and awareness system

» Location tracking, context awareness, large interactive displays, mobile phones

Awareness Media» Status of work in Operating Rooms

» Video stream for accessing current state

» Progress bar

» Chat area

» Schedule

» Location tracking of who is in Operating Rooms


Pervasive Technologies in a Hospital - Scenario

Acute patient Find/schedule Operating Room Find surgeon not in Operating Room Send message Notify patient ward Scheduled patient notified of their surgery

postponement


Pervasive Technologies in a Hospital – Hardware

Lessons


Pervasive Technologies in a Hospital – Software

Lessons


Pervasive Technologies in a Hospital – User Lessons


e-Campus Display [Storz]

Three deployments

Conference Signage

Art Center

Underground Bus Station


e-Campus Display [Storz]

Conference Signage» Dynamic schedule of time/location of conference events

Art Center Exhibition» 60 year celebration of VE day

» Three displays, video diary, web based diary, interactive artifact exhibit

» News footage, images of artifacts, visitors record own memories

Underground Bus Station» Three screens

» Artistic material, textual information, videos

» Context triggered


e-Campus Display – Lessons Learned


e-Campus Display – Lessons Learned Details – Part 1

Deployments are costly» Time, money, is there a cheaper way to get the same results

Environmental Challenges can be significant» Fire alarm tests and power cycling, diesel fumes clogging

projector air filter

After deployment comes maintenance» Underpass projector failure required road closure, insure

accessibility. Shadow system in lab

Follow the rules» Accessibility for people with disabilities. Work with local site

manager

See what public sees» Remotely monitor what public perceives. Add cameras to

view projector output



Build white boxes, not black» Be able to monitor whole system state (what displayed

where, when) rather than just observing projected images

Content is king» Content is the system. Considered late in cycle led to

mismatch between content support requirements and what was provided

Content is expensive» Generating compelling content is nontrivial requiring

specialist skills and domain knowledge

Manage your assets» Difficulties in managing content once it is created including

encodings, archiving, previewing, approval



Define the user experience» Precisely define user experiences – carefully coordinated

performances

Provide transactions» Transaction level atomicity – because multiple components

involved there should be no partial transitions

Manage expectations» Testing confused the viewing public. Communicate access

policies. Blank displays create expectation of content to come

Be accountable» Entering public domain makes work subject to increased

levels of scrutiny


Outline





» Hospital

» e-Display

Ad Hoc» Wildfire

» Volcano

Concluding Lessons


Monitoring Weather Conditions in Wildland

Fires [Hartung] Predictions based on current conditions,

previous weather, and predicted weather Thermal belts, temperature inversions

» Cold air moves into valleys forcing war air to rise which is trapped by continuously moving air above ridge tops

» Fire stays more active in warmer air with lower relative humidity

Current approach» Manual Kit – 5 to 10 minutes to collect at current location

each hour

» Remote Automated Weather Stations – measure temperature, wind speed/direction, relative humidity, precipitation, barometric pressure, fuel moisture/temperature, soil moisture



Fires -Background

Inversion Layer

Manual Measurement



Fires -Architecture


Monitoring Weather Conditions in Wildland Fires

– Lessons Learned 1 Fire activity changes rapidly with even small

changes at low Relative Humidity Conform to existing infrastructure/logistics

» Powered by AA batteries – quickly accepted since nearly all electronics devices operate by crews used AA batteries

Battery temperature has significant effect on battery power available


Monitoring Weather Conditions in Wildland Fires

– Lessons Learned 2

Housing vents and open bottom» Allows airflow but protects from rain

» Prevent heat trapping

» Painted white to protect from radiation heating

» Mount 1.8 meters above ground to protect from heat damage caused by fire

Large change in elevation increased radio range by 4 to 10 times

» Further from “ground” less interference

» Fresnel Zones - measure of phase difference between reflection of waves between transmitter and receiver that cause cancelling effect


Air Dropped Sensors for Volcano Monitoring [Song]

Goals» Synchronized sampling – time synchronized within 1 msec

» Real Time continuous raw data

» One year robust operation – harsh conditions including heavy rain, snow, ice, wind gusts to 120 mph

» On-line configurable

» Fast deployment – air drop sensors (70 pounds), network self starting and organizing

Sensors» Seismic

» Infrasonic

» Lightning – RF pulse detector


System for Volcano Monitoring


Air Dropped Sensors for Volcano Monitoring -

Lessons Signal strength does not reflect signal quality,

rather signal to noise ratio Problems with connectors System data delivery 91.7% Station tipped over, damaged voltage regulator Infrasonic – pressure sensor also recorded

heavy wind gusts Outage for 20 hours since data base could not

handle daylight savings time Large data loss due to low battery voltage


Outline





» Hospital

» e-Display

Ad Hoc» Wildfire

» Volcano

Concluding Lessons


Common Lessons Learned

Demands on User attention Risk of user becoming engaged in virtual world

and become less aware of physical context Mobile users impatient Strong interaction mental model Fit into current procedures/logistics Environmental influences Power especially battery characteristics

1 mobil computing © 2010-2013 daniel p. siewiorek end-to-end application considerations daniel p....

Documents

siewiorek slide

mobil computing

siewiorek savings

siewiorek lessons

siewiorek outline

siewiorek time rate

help desk slide

lecture dan siewiorek