Research Issues in Adapting Research Issues in Adapting Computing to Small DevicesComputing to Small Devices

Francis C.M. Lau

Department of Computer Science

The University of Hong Kong

APPT 2005 • Hong Kong

fcmlau@cs.hku.hk

APPT 05 (27/10/05, HK) 2

Popular ComputingPopular Computing

• 70’s: one computer, many users• 80’s: one computer, one user• 90’s: many (connected) computers,

many users – distributed computing• Now: many computers, one user

• What next? Ubiquitous computing? Grid computing? Service-oriented computing? …– Do we need that many computers?– Do we need a “computer”?

APPT 05 (27/10/05, HK) 3

Windows PanicWindows Panic

• Continue or quit or kill the window or details?

• How would my mother know!?

• The best computing is without the computer

APPT 05 (27/10/05, HK) 4

Imagine …Imagine …

• APPT 2015 (Disneyland, Hong Kong) …

APPT 2015APPT 2015

APPT 05 (27/10/05, HK) 5

My Future HandheldMy Future Handheld

• All in one: phone, PDA, wallet, game console, remote terminal, ad-hoc networking …

• That means a lot of software – big and fat!?

• But what I really want is lean and thin and mean– Lean – simple software which does only what I want

– Thin – that’s all the space in my pocket, or around my neck!

– Mean – affordable

LTMLTM

APPT 05 (27/10/05, HK) 6

LTM ComputingLTM Computing

• The (thin-) client-server paradigm, works but in limited ways

• Too thin: a browser-only device, a remote terminal …

• Future devices are more powerful than just that– But how powerful is

powerful?

APPT 05 (27/10/05, HK) 7Subramanian, 1999

O my god!

Software complexity

APPT 05 (27/10/05, HK) 8

Rich ComputingRich Computing

• My hard disk and VM can be as big as the Internet

• Not just contents, but software come and go on demand

• Pay-per-use subscription model• Browser plug-in’s and Java applets –

not lean enough, and some require just too much resources

• Software in terms of “functionalities”

The Networkis the Computer ™

APPT 05 (27/10/05, HK) 9

The Network Behind Every DeviceThe Network Behind Every Device

• Future digicams– Snap and go

• The network is the memory• The network is the photo album• (The network is NOT the power)

– Context aware

APPT 05 (27/10/05, HK) 10

Edge ComputingEdge Computing

• Person .. device .. middleware (proxies) .. Internet

• The abstract cloud moves with the client – personalized “cuddleware”, nomadic computing

Internetproxies united

client

metropolis

APPT 05 (27/10/05, HK) 11

DEVICEEDGE

GRID

The continuum: WHAT SHOULD BE DONE The continuum: WHAT SHOULD BE DONE WHERE?WHERE?

APPT 05 (27/10/05, HK) 12

A ContinuumA Continuum

• Your device .. other devices (peers) in the vicinity .. server(s) nearby .. the grid– “Neighborhood” is still important

• network bandwidth can’t be infinite; latency will never be 0; network will never be trouble-free

• What should be done where?– The end-device: completely passive (HAS) to c

ompletely active (DIY)

APPT 05 (27/10/05, HK) 13

““Hire a Server”Hire a Server”

• Completely passive (remote terminal)– A waste of bandwidth– Responsiveness and interactivity (esp. for

games)• Does it make sense?

– MP3, camera, PDA, …

• Something must be done locally– Adaptation

APPT 05 (27/10/05, HK) 14

Limits You Can’t BeatLimits You Can’t Beat

• Power?• Screen size: foldable, rollable, we

arable, projectable? Or do without the screen (audible?)– Convenience and privacy; and po

wer consumption– Resolution helps a little bit

• Why I don’t use a PDA (I just ordered an 24” LCD; but I still look at my watch)– Ergonomics is very important– Scrolling is a nuisance (now)– User experience!

APPT 05 (27/10/05, HK) 15

Wearable Wearable ComputersComputers

server

display

APPT 05 (27/10/05, HK) 16

AdaptationAdaptation

• Data– = not code

• Content– A special kind of data– Text, paged, non-text– Pixel level and/or higher

• UI– Higher level, structural, interactive– Buttons, icons, menus, panels and panes … content

• Code

APPT 05 (27/10/05, HK) 17

Componentized SoftwareComponentized Software

• Executable code as puzzle pieces

• Fast start, like contents• Software download

pipelining – first-come-first-execute, the most needed functionalities first

APPT 05 (27/10/05, HK) 18

The Need for Code On DemandThe Need for Code On Demand

• How many functions have you ever used?

A simplified view of Microsoft Word

APPT 05 (27/10/05, HK) 19

Remember UNIX?Remember UNIX?

• Small is beautiful• Make each program do one

thing well• Choose portability over

efficiency• Store data in flat ASCII files• Avoid captive user interfaces• Make every program a filter• …

APPT 05 (27/10/05, HK) 20

UNIX Philosophy AppliedUNIX Philosophy Applied

• Small software components implementing single functionalities

• Bigger functions runtime-composable from small components

• Download on demand, and disposable

• Standard formats and interfaces (eg. XML)

• Shared GUI

APPT 05 (27/10/05, HK) 21

Service-Oriented ComputingService-Oriented Computing

Can Web Services Scale Down?

- for small device apps

APPT 05 (27/10/05, HK) 22

Adapting X to YAdapting X to Y

• Many issues– Psychology– Aesthetics– Readability, convenience, privacy …– Economics

• Context awareness– Because small devices are likely mobile– The user moves around

APPT 05 (27/10/05, HK) 23

Awareness and AdaptationAwareness and Adaptation

• An intelligent supporting architecture

• Aware of the come and go of mobile users

• Choices of functions, depending on situation: the location, the device, the user, …

• Fast reacting nearby proxy• Proxies united, within certain

locality

APPT 05 (27/10/05, HK) 24

Summary 1: Summary 1: BasicBasic Problems (I Problems (I think) Worth Pursuingthink) Worth Pursuing

• Edge computing → “seamless”– New protocols for the edge

• The continuum → the network is the computer– Collaborative models and mechanisms, esp. at the edge

• The global grid → invisible, “PC” disappearing• The device

– Adaptation• On-demand code composition

– The SOC approach?

• Content– HTML

• UI description languages

– New paradigms for user interaction in small devices (input and output)

APPT 05 (27/10/05, HK) 25

Some HKU ProjectsSome HKU Projects

• The Sparkle Project – “code adaptation”• PDF document content adaptation system

– “content adaptation”• HTML adaptation – “UI/content

adaptation”

APPT 05 (27/10/05, HK) 26

SparkleSparkle

• Mobile code on demand– Proxy to device, device to device

• Features– Code On Demand– Context Based Adaptation– Proxy Caching on Code– Code & State Migration– Ontology– Sparkle View & Universal Browser

Application = “facet” composition

APPT 05 (27/10/05, HK) 27

Universal BrowserUniversal Browser

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

Time (ms)

Add Noise AddShadow

FindEdges

FlipHorizontal

FlipVertical

GaussianBlur

OpenImage

Save asJpeg

Sharpen

APPT 05 (27/10/05, HK) 28

PDF Document AdaptationPDF Document Adaptation

APPT 05 (27/10/05, HK) 29

APPT 05 (27/10/05, HK) 30

APPT 05 (27/10/05, HK) 31

APPT 05 (27/10/05, HK) 32

Web Document AdaptationWeb Document Adaptation

APPT 05 (27/10/05, HK) 33

(a) The original content

(b) Only the bottom level is “compressed”

(c) The two lowest levels are “compressed”

APPT 05 (27/10/05, HK) 34

Handling SegmentsHandling Segments

APPT 05 (27/10/05, HK) 35

Summary 2: When I Grow OlderSummary 2: When I Grow Older

• I want a simpler life (others can enjoy all the complexities)– One device, LTM– No computer please; terminal is fine; plug and

play– Single data storage by hire, access anywhere– Single sign-on, single environment (adaptive)

APPT 05 (27/10/05, HK) 36