Ubiquitous Intelligence,Ubiquitous Intelligence,Smart uSmart u--Things & World, Things & World, Real World ChallengesReal World Challenges

Jianhua MaMUSE Lab (Multimedia Ubiquitous Smart Environment)

CIS, Hosei University, Japanhttp://cis.k.hosei.ac.jp/~jianhua/

2005

Computer, Computing and Computer, Computing and Communication Tech. EvolutionCommunication Tech. Evolution

* Sputnik

1960 1970 1975 1980 1985 1990 1995 2000

* ARPANET* Email

* Ethernet* TCP/IP * IETF

* Internet Era

* WWW

* Mosaic* XML

* PC Clusters*Crays *MPPs

* Mainframes

* HTML* W3C

*P2P*P2P**Grid Grid

* XEROX PARC wormCom

pute

rCo

mm

unic

atio

n

* * Semantic WebSemantic Web

* Minicomputers* PCs

* WS Clusters

* PDAs* Workstations* HTC

**UbiquitousUbiquitous**PervasivePervasive

**Web ServicesWeb Services

*Mobicomp

Com

putin

g

* Parallel Comp**WebcompWebcomp

* Distributed Comp

* M-Media Comp*IDC/SOC*IDC/SOC

2005

**AmIAmI

IDC/SOCIDC/SOC: Intersection of : Intersection of WbSWbS, P2P, Grid, P2P, Grid

Internet Distributed Computing (IDC), Service Oriented ComputingInternet Distributed Computing (IDC), Service Oriented Computing (SOC)(SOC)-- unified and common platform of Web Services, P2P, Grid computinunified and common platform of Web Services, P2P, Grid computing, etc.g, etc.

P2PP2P GridGridWbSWbS

IDC IDC Cyber WorldCyber Worlde/d/ve/d/v--WorldWorld

SOCSOC

Cyber Computing: eCyber Computing: e--Thing & eThing & e--ActivityActivityCyber World Cyber World -- Built on web/cyberspace in digital/virtual formBuilt on web/cyberspace in digital/virtual form

Computers & Networks/InternetComputers & Networks/Internet

Web, Web, WbSWbS, Grid, P2P, Agent, , Grid, P2P, Agent, SemSem., etc.., etc.

ee--ThingThing

ee--ActivityActivity

Cyber WorldCyber WorldReal WorldReal World

Real ThingReal Thing

Real ActivityReal Activity

International International Conference Conference on Cyber Worlds (CW) on Cyber Worlds (CW) (co(co--founder, since 2002)founder, since 2002)

WeiserWeiser’’ss VisionVision (1990):(1990):

Ubiquitous Computing Ubiquitous Computing (UC, (UC, UbicompUbicomp))

Industry VisionIndustry Vision (1999, IBM, etc.):(1999, IBM, etc.):

Pervasive Computing Pervasive Computing ((Percomp/PercomPercomp/Percom))

EUEU’’ss VisionVision (2001):(2001):

Ambient Intelligence Ambient Intelligence ((AmIAmI))

Ubiquitous/Pervasive Computing, Ubiquitous/Pervasive Computing, AmIAmI: : Real Thing & ActivityReal Thing & Activity

ee--ThingThing

ee--ActivityActivity

Cyber WorldCyber WorldReal WorldReal World

Real ThingReal Thing

Everyday ActivityEveryday Activity

Sensor/M/NEMS, Comps & Per. NetsSensor/M/NEMS, Comps & Per. Nets

UC, ID, Context, Middleware, etc.UC, ID, Context, Middleware, etc.

Ubicomp/Percomp/AmIUbicomp/Percomp/AmI Real world with digital enhanced real things Real world with digital enhanced real things

Computers & Networks/InternetComputers & Networks/Internet

Web, Web, WbSWbS, Grid, P2P, Agent, , Grid, P2P, Agent, SemSem., etc.., etc.

Ubi-comp: a field on a physical world richly and invisibly interwoven with sensors, actuators, displays, and computational elements, embedded seamlessly in everyday objects of lives and connected through a continuous network.

- Mark Weiser in his last article in IBM Sys. Journal, 1999

Real World ComputingReal World Computing

WeiserWeiser’’ss View View –– Three RelationshipsThree Relationships

mm--toto--11 11--toto--11 11--toto--mm

Trend One: Device MinistrationTrend One: Device Ministration-- TinyTiny Chips and MicroChips and Micro--Devices Devices --

μ- chip

Small Small Tiny Tiny Dust Dust InvisibleInvisible DisappearDisappearThey are Attachable, Embeddable, BlendThey are Attachable, Embeddable, Blend--ableable

By Hitachi, 2003

Trend Two: Ubiquitous CommunicationTrend Two: Ubiquitous Communication-- Wired Wired && Wireless Pervasive Networks Wireless Pervasive Networks

All Things ConnectedAll Things Connected Able to TalkAble to Talk

Trend 3: uTrend 3: u--ThingsThings’’ boomingboomingAttachment, Embedment, BlendingAttachment, Embedment, BlendingThree fundamental Technology TrendsThree fundamental Technology TrendsContinuing miniaturization of electronic chips & electroContinuing miniaturization of electronic chips & electro--mechanical devices mechanical devices (Moore(Moore’’s law, new material, nanotech., s law, new material, nanotech., ……))

Various computers/devices Various computers/devices small small tiny tiny invisibleinvisible

Available interconnections by ubiquitous/pervasive networks espeAvailable interconnections by ubiquitous/pervasive networks especially using cially using wireless communicationswireless communications

All computers/devicesAll computers/devices connectable connectable talktalk--able able interactinteract--ableable

Many real things can be integrated with attached, embedded, and/Many real things can be integrated with attached, embedded, and/or blended or blended computers, networks, and/or some other devices such as sensors, computers, networks, and/or some other devices such as sensors, actors, eactors, e--tags and so on.tags and so on.

uu--ThingsThings: : Real things with some kind of Real things with some kind of attachmentattachment, , embedmentembedment, , blendingblending((AEBAEB) of computers, networks, and/or other devices) of computers, networks, and/or other devices

ee--booming booming uu--boomingbooming

Following the above trends, what the future world will be?Following the above trends, what the future world will be?

Essential QuestionsEssential QuestionsCan machines think ?Can machines think ? AIAI-- By A. Turing in By A. Turing in ““Computing Machinery and IntelligenceComputing Machinery and Intelligence”” in 1950 in 1950 -- HeHe believes machines think and also believes that believes machines think and also believes that ““at the end of the century the use of at the end of the century the use of words and general educated opinion will have altered so much thawords and general educated opinion will have altered so much that one will be able to speak t one will be able to speak of machines thinking without expecting to be contradictedof machines thinking without expecting to be contradicted””..

Can things think ?Can things think ? Intelligent real objectIntelligent real object-- CConsortium of Things That Think (TTT) in MIT Media Lab since 1995onsortium of Things That Think (TTT) in MIT Media Lab since 1995-- ““When things start to thinkWhen things start to think”” by Neil by Neil GershenfeldGershenfeld of MIT in 1999. of MIT in 1999. -- Thing thinking means the thing has some intelligence.Thing thinking means the thing has some intelligence.

Can thinking things be everywhere ?Can thinking things be everywhere ? UI, Smart World (SW),UI, Smart World (SW), AmIAmI-- Ubiquitous Intelligence (UI) or Pervasive Intelligence ( ) sinUbiquitous Intelligence (UI) or Pervasive Intelligence ( ) since 2003 ce 2003 –– by usby us-- Being a fact/phenomenon of ubiquitous existence not only as metBeing a fact/phenomenon of ubiquitous existence not only as methods/toolshods/tools-- Residing in everyday objects, environments, systems, ourselves,Residing in everyday objects, environments, systems, ourselves, ……-- Extending to both manExtending to both man--made and natural thingsmade and natural things

Information Explosion Information Explosion Intelligence Intelligence Explosion !Explosion !-- The Intelligence RevolutionThe Intelligence Revolution, Interview by Wise Media, ID People Magazine, Apr./05, Interview by Wise Media, ID People Magazine, Apr./05-- Ubiquitous Intelligence Summit 2005Ubiquitous Intelligence Summit 2005,, A Think TankA Think Tank, , OuluOulu, Finland, June 15, Finland, June 15--17, 200517, 2005-- Journal of Ubiquitous Computing and Intelligence (Journal of Ubiquitous Computing and Intelligence (EiCEiC), ), American Scientific PublishersAmerican Scientific Publishers

π

Intelligent Computing WavesIntelligent Computing Waves

33rdrd: Agent : Agent (Social Comp)(Social Comp)-- Autonomous softwareAutonomous software-- Multi/Massive agentsMulti/Massive agents-- Agent languageAgent language-- Agent negotiation & cooperationAgent negotiation & cooperation-- Personal/social behaviorPersonal/social behavior-- Web intelligence/semanticsWeb intelligence/semantics

44thth: UI (?): UI (?) (Real things(Real things’’ intelligence)intelligence)-- Physical/everyday thingsPhysical/everyday things’’ intelligenceintelligence-- Atop of the above 3 intelligent compAtop of the above 3 intelligent comp-- Scale,dynamic,heterogeneous,spontaneousScale,dynamic,heterogeneous,spontaneous-- Predictable, controllable, adaptable, Predictable, controllable, adaptable, manageable, ethic, manageable, ethic, ……

-- OthersOthers--aware aware && selfself--awareaware mind/spirit?mind/spirit?

11stst: AI : AI (Logic/KL(Logic/KL--based)based)-- Machine learningMachine learning-- NLP & CompNLP & Comp--VisionVision-- Robot & game theoryRobot & game theory-- Expert systemExpert system-- Knowledge/ReasoningKnowledge/Reasoning-- DAI (Distributed AI)DAI (Distributed AI)

22ndnd: Soft/Natural Comp: Soft/Natural Comp-- Probabilistic computingProbabilistic computing-- Fuzzy logicFuzzy logic-- Neural networkNeural network-- GA/Evolutionary computingGA/Evolutionary computing-- Chaotic/Swarm computingChaotic/Swarm computing-- Biologic computingBiologic computing

HyperworldHyperworld and Smart Worldand Smart WorldHyperworldHyperworld-- ““HyperworldHyperworld ModelingModeling””, Keynote in VIS, Melbourne, Feb. 1996 (Kunii, Ma, Huang), Keynote in VIS, Melbourne, Feb. 1996 (Kunii, Ma, Huang)-- ““Its basic characteristic is direct mapping between virtual and rIts basic characteristic is direct mapping between virtual and real worlds via eal worlds via

active devices including sensors, actuators, microactive devices including sensors, actuators, micro--machines, robots, etc.machines, robots, etc.”” -- 19961996-- ““A Study on A Study on a a HyperworldHyperworld System of OneSystem of One--toto--Many InteractionMany Interaction””,, ICAIICAI’’97 (197 (1--toto--m relationship)m relationship)-- ““Towards Direct Mapping betTowards Direct Mapping betweenween InforInformation mation Worlds Worlds andand Real WorldsReal Worlds””, , LNCS1LNCS1306306, , 19971997

Smart World (SW)Smart World (SW)-- Based on ubiquitous computers, networks, information, services,Based on ubiquitous computers, networks, information, services, etc.etc.-- Created in both real and cyber spacesCreated in both real and cyber spaces-- Characterized by ubiquitous intelligence in the real worldCharacterized by ubiquitous intelligence in the real world-- Pervaded with intelligent or smart ubiquitous things/uPervaded with intelligent or smart ubiquitous things/u--thingsthings

Smart eSmart e--ThingThing

ee--ActivityActivity

Cyber WorldCyber WorldSmart WorldSmart World

Smart uSmart u--ThingThing

Real ActivityReal Activity

HyperworldHyperworld

Roads Towards Smart World & UIRoads Towards Smart World & UIUniversal IdentificationUniversal Identification-- RFID, ERFID, E--Tag, AV/BiomedicalTag, AV/Biomedical-- Location aware computingLocation aware computing

Physical & Social AwarenessPhysical & Social Awareness-- Sensor networks Sensor networks -- Context based computingContext based computing-- Privacy/trust computingPrivacy/trust computing

Digital Enhanced ObjectDigital Enhanced Object-- Embedded computingEmbedded computing-- Wearable/augmented computingWearable/augmented computing-- Everyday computingEveryday computing (Georgia Tech)(Georgia Tech)

-- Sentient computingSentient computing (AT&T)(AT&T)

Platform & MiddlewarePlatform & Middleware-- WS, UPnP, WS, UPnP, JiniJini, SLP, , SLP, OSGiOSGi, , ……-- SmartSmart--Its, TIts, T--Engine, Engine, eTRONeTRON, , ……-- Aura, Gaia,, Aura, Gaia,, iROSiROS, RCSM, , RCSM, MetaGlueMetaGlue,, ……

Interface & InteractionInterface & Interaction-- Calm technologyCalm technology ((WeiserWeiser & Brown)& Brown)

-- Invisible/disappear computingInvisible/disappear computing((UoWUoW, CMU, DARPA & EU), CMU, DARPA & EU)

-- Palpable computingPalpable computing ((PalComPalCom, EU FP6), EU FP6)

-- Proactive computingProactive computing ((TennenhouseTennenhouse, CHI, CHI’’0404--P)P)

Deployment & ManagementDeployment & Management-- Sustainable pervasive compSustainable pervasive comp (SPC(SPC’’04)04)

-- Autonomic computingAutonomic computing (IBM, ICAC(IBM, ICAC--04)04)

-- Organic computingOrganic computing (GI(GI--WorkshopWorkshop’’04, ARCS)04, ARCS)

Model & DesignModel & Design-- Massive multi agentMassive multi agent ((MMASMMAS’’04, Kyoto04, Kyoto))

-- CW Axioms & Cellular Modeling CW Axioms & Cellular Modeling (Kunii)(Kunii)

-- Amorphous computingAmorphous computing (MIT)(MIT)

-- Spray computingSpray computing ((ZambonelliZambonelli))

Intelligent EnvironmentIntelligent Environment-- Ambient intelligence (Ambient intelligence (AmIAmI,, AmIAmI--03,03, IEIE--0505))

Smart uSmart u--things things -- Essential ElementsEssential Elementsin Smart World with Ubiquitous Intelligencein Smart World with Ubiquitous Intelligence

Smart Object (Smart Object (SmartifactSmartifact))-- A physical object with AEB and some smartness/intelligenceA physical object with AEB and some smartness/intelligence-- Device, card, label, sensor, artifact, appliance, goods, furnituDevice, card, label, sensor, artifact, appliance, goods, furniture, textile, robot, re, textile, robot, …………

Smart Space/EnvironmentSmart Space/Environment-- A physical spatial environment integrating smart objects &/ usuA physical spatial environment integrating smart objects &/ usual devicesal devices-- Smart services via these objects/devices and their Smart services via these objects/devices and their communcommun./cooperation./cooperation

Smart SystemSmart System-- May be a real system like a network, traffic system, May be a real system like a network, traffic system, ……-- May be a platform middleware for a kind of smart object/environMay be a platform middleware for a kind of smart object/environmentment-- May be a general one supporting a class of smart applicationsMay be a general one supporting a class of smart applications

(Future) (Future) Ubicomp/PercompUbicomp/Percomp computing of smart ucomputing of smart u--thingsthings11stst IntInt’’l l WkspWksp on Ubiquitous Smart World (USWon Ubiquitous Smart World (USW--05, Taiwan, March 2005)05, Taiwan, March 2005)22ndnd IntInt’’l Sym. on Ubiquitous Intelligence & Smart World (UISWl Sym. on Ubiquitous Intelligence & Smart World (UISW--05, Japan, Dec. 2005)05, Japan, Dec. 2005)33rdrd IntInt’’l Conf. on Ubiquitous Intelligence & Computing (UICl Conf. on Ubiquitous Intelligence & Computing (UIC--06, China, Sept. 2006)06, China, Sept. 2006)

Smart & Smartness: Terms and CharactersSmart & Smartness: Terms and Characters

Smart related computing termsSmart related computing terms-- ““intelligentintelligent””-- Sentient, Aware, ContextSentient, Aware, Context--aware, Active, Reactive, Proactive, Assistive, Adaptive, aware, Active, Reactive, Proactive, Assistive, Adaptive,

Automated, Autonomic, Perceptual, Cognitive, Thinking, Automated, Autonomic, Perceptual, Cognitive, Thinking, SelfwareSelfware, , ……

Smartness features in generalSmartness features in general-- Some kind/level of intelligence, but softer, wider and flexibleSome kind/level of intelligence, but softer, wider and flexible than than ““intelligenceintelligence””-- From simple reactive functions to complex intelligent behaviorsFrom simple reactive functions to complex intelligent behaviors-- Anything from real to e/virtual ones, but with emphases of realAnything from real to e/virtual ones, but with emphases of real/physical ones/physical ones-- Felt relatively not absolutely Felt relatively not absolutely two cotwo co--exist facets: smart and stupid!exist facets: smart and stupid!-- Complicated or abstruse philosophical, culture social, ethical Complicated or abstruse philosophical, culture social, ethical and other implicationsand other implications

Ultimate goal of UI and SWUltimate goal of UI and SW-- To make uTo make u--things things ““calmcalm ““ or behave or behave trustworthily in contexttrustworthily in context--/self/self--awarenessawareness-- To move To move Ubi/PerServicesUbi/PerServices from from ANYANY place/time/meansplace/time/means to to RIGHTRIGHT place/time/meansplace/time/means

Smart uSmart u--thing challengesthing challengesTechnology ComplexityTechnology Complexity: discussed widely and realized relatively well: discussed widely and realized relatively wellReal World ComplexityReal World Complexity: addressed rarely and realized not very well: addressed rarely and realized not very well

Smart uSmart u--Things Things –– Technology ChallengesTechnology Challenges

Three basic types of functions for smart uThree basic types of functions for smart u--thingsthings-- Computation & storage for processing and memoryComputation & storage for processing and memory-- Networking & naming for interconnection and communicationNetworking & naming for interconnection and communication-- Sense & effect for perception and interactionSense & effect for perception and interaction

Technology challenges (not exhausted) Technology challenges (not exhausted) Both in hardware and software Both in hardware and software ddevices miniaturization, power evices miniaturization, power management, sensor nets, universal ID, ubiquitous/pervasive management, sensor nets, universal ID, ubiquitous/pervasive networks, ad hoc mobility, open service architecture, sensed networks, ad hoc mobility, open service architecture, sensed information overload & database, context semantics & information overload & database, context semantics & management, autonomic system administration, user interface, management, autonomic system administration, user interface, operating system, language, middleware, integration, operating system, language, middleware, integration, cooperation, scalability, heterogeneity, dependability, avaicooperation, scalability, heterogeneity, dependability, availability, lability, security, privacy, test, evaluation, standards, etc.security, privacy, test, evaluation, standards, etc.

““Global Ubiquitous Computing: Design and ScienceGlobal Ubiquitous Computing: Design and Science””, UC Grand Challenge: Manifesto, UK, June 2005 , UC Grand Challenge: Manifesto, UK, June 2005 BeyondBeyond--TheThe--Horizon Workshop on Pervasive Computing and Communications (PCC)Horizon Workshop on Pervasive Computing and Communications (PCC), Vienna, July 2005, Vienna, July 2005

Smart uSmart u--Things Things –– Real World ChallengesReal World Challenges

Smart uSmart u--things are emergingthings are emerging-- Many scenarios: Many scenarios: WeiserWeiser’’ss Sal, Sal, AmIAmI’’ss Maria/Maria/DimitriosDimitrios/Carmen, Aura/Carmen, Aura’’s Jane/Fred, s Jane/Fred, ……-- Various prototypes but yet widely practical onesVarious prototypes but yet widely practical ones-- A highly possible boom of smart uA highly possible boom of smart u--things in near futurethings in near future

Ideal smart uIdeal smart u--things expectedthings expectedAble to act adaptively and automatically according toAble to act adaptively and automatically according to1. Surrounding Situations 1. Surrounding Situations 2. Users2. Users’’ NeedsNeeds3. Things3. Things’’ RelationsRelations4. Common Knowledge4. Common Knowledge5. Self Awareness 5. Self Awareness 6. Looped Decisions6. Looped Decisions

Above challenges from real world intrinsic characteristicsAbove challenges from real world intrinsic characteristics-- RW == physical + social + natural + RW == physical + social + natural + …… uncertainuncertain, , unpredictableunpredictable, , changingchanging, , ……-- RW computing: complicated/abstruse RW computing: complicated/abstruse philphil., social, ethical & other implications., social, ethical & other implications-- Understanding real world (RW) diversity and complexity Understanding real world (RW) diversity and complexity Extremely Hard !!Extremely Hard !!-- Novel cyber dimensions newly added in physical/digital combinedNovel cyber dimensions newly added in physical/digital combined uu--things things

Challenge 1 Challenge 1 –– Situation Approximation Situation Approximation ContextContext-- Information characterizing the situation of an entity Information characterizing the situation of an entity ((DeyDey)) or 5Ws or 5Ws ((AbowdAbowd))-- Whole contexts are a collection of various values from sensors Whole contexts are a collection of various values from sensors or other sourcesor other sources

C(tC(t) = {) = {CCii(B(Bii, L, , L, tthh), ), ii∈∈I(tI(t), ), tthh∈∈[t[t--hh, t]}, t]}

SiturationSituration-- Relatively compact, more semantic, directly used for judgment/dRelatively compact, more semantic, directly used for judgment/decisionecision

S(tS(t) =) =φφ[C(t[C(t)] = {)] = {SSjj(W(Wjj, t), , t), jj∈∈J(tJ(t)})}Designing a ContextDesigning a Context--aware System to Detect Dangerous Situations in School Routes foraware System to Detect Dangerous Situations in School Routes for Kids Outdoor Safety Care, LNCS, Proc. of EUC'05Kids Outdoor Safety Care, LNCS, Proc. of EUC'05..

C(tC(t), ), S(tS(t)) approximations to a real environmentapproximations to a real environment-- Are the contexts Are the contexts C(tC(t)) sufficient and precise enough to characterize a real environmensufficient and precise enough to characterize a real environment?t?-- How correctly be How correctly be S(tS(t)) determined using available but incomplete & uncertain contexts?determined using available but incomplete & uncertain contexts?-- What are consequences of situation judgment errors to contextWhat are consequences of situation judgment errors to context--aware/situated uaware/situated u--things?things?

NyquistNyquist Sampling Theorem Sampling Theorem && Shannon Entropy/Shannon Entropy/R(D)R(D) Theory Theory Theory for context/situation?Theory for context/situation?

Keep incompletion, uncertainty and misjudgment as basic design aKeep incompletion, uncertainty and misjudgment as basic design assumptions !ssumptions !

Challenge 2 Challenge 2 –– Knowing UsersKnowing Users’’ NeedsNeedsInteractive MechanismInteractive Mechanism-- A requestA request--response dialog process response dialog process betwbetw. users (activator) & computer (passive). users (activator) & computer (passive)-- 3 user limits: 3 user limits: too small/many/complex computers to be visible/interacttoo small/many/complex computers to be visible/interact--able/manageableable/manageable

Proactive and Autonomic MechanismsProactive and Autonomic Mechanisms-- Take actions proactively by anticipating usersTake actions proactively by anticipating users’’ needs needs ((TennenhouseTennenhouse, Intel, 2000), Intel, 2000)-- Manage themselves under human supervisions/needs Manage themselves under human supervisions/needs (Paul Horn, IBM, 2001)(Paul Horn, IBM, 2001)

"Proactive Control of Group Revision Assistance Management Using P2P Technology" (Takata & Ma, ISCIT04)Journal of Autonomic and Trusted Computing (JoATC, EiC), ASP, USA1st Int’l Wksp on Trusted and Autonomic Ubiquitous & Embedded Systems (TAUES-05, Japan, Dec. 2005)2nd International Workshop on Trusted and Autonomic Computing Systems (TACS-06, Austria, Apr. 2006)

33rdrd IntInt’’l Conf. on Autonomic and Trusted Computing (ATCl Conf. on Autonomic and Trusted Computing (ATC--06, China, Sept. 2006)06, China, Sept. 2006)

UsersUsers’’ NeedsNeeds-- Multi aspects and subtle affection by many factorsMulti aspects and subtle affection by many factors

N(t) = {NN(t) = {Nuu,,vv[P[Puu,,vv(t), S(t), S(t(t), S(t), S(t--ΔΔ)], u)], u∈∈U(t), vU(t), v∈∈V(t)}V(t)}

Hardness of knowing usersHardness of knowing users’’ true needstrue needs-- HHow much can be known correctly and promptly for usersow much can be known correctly and promptly for users’’ true needs in changing situationstrue needs in changing situations??-- ““Know you and know your face, but donKnow you and know your face, but don’’t know your mindt know your mind”” -- Chinese sawChinese saw

Challenge 3 Challenge 3 –– Complex ThingsComplex Things’’ RelationsRelations

Complex Relations among uComplex Relations among u--ThingsThings-- uu--things are connected and interactedthings are connected and interacted

-- ““Everything will be connected to everything elseEverything will be connected to everything else”” -- by R. Lucky, 1999by R. Lucky, 1999Cyber dimensionsCyber dimensions, except spatial/temporal/other conventional dimensions, except spatial/temporal/other conventional dimensionsComplex dynamic relations among users, uComplex dynamic relations among users, u--things, and other thingsthings, and other things

R(t) = {RR(t) = {Rqq(T(Tqq, U, Uqq, , tt), U), Uqq∈∈U(t), qU(t), q∈∈Q(t)}Q(t)}

-- How to define, find, describe and use complex relations necessaHow to define, find, describe and use complex relations necessary for ury for u--thingsthings’’ systems?systems?Smart Smart HyperspaceHyperspace: a set of interconnected smart spaces with situational: a set of interconnected smart spaces with situational--spatialspatial--temporal relationstemporal relations((““Modeling Interface with a Multimedia Modeling Interface with a Multimedia HyperworldHyperworld””,, HISHIS’’96, 96, ““Smart Hyperspaces and Project Smart Hyperspaces and Project UbikidsUbikids””, , USW05USW05))

usersusers

other thingsother thingsuu--thingsthings

Challenge 4 Challenge 4 –– Common Knowledge Common Knowledge

Why common knowledge needed?Why common knowledge needed?-- Knowledge is the base of Knowledge is the base of analysis, reasoning, anticipation and judgmentanalysis, reasoning, anticipation and judgment (Ref AI(Ref AI))-- SSmart umart u--things are in the physical environments and serve peoplethings are in the physical environments and serve people’’s daily life and works daily life and work-- Smart uSmart u--things need some common knowledge about physical world, human sothings need some common knowledge about physical world, human society, etc.ciety, etc.

K(tK(t) = {) = {KKmm(F(Fmm, , tthh), ), mm∈∈M(tM(t), ), tthh∈∈[[tt00, , tt]}]}

Issues areIssues are-- What knowledge is necessary for smart uWhat knowledge is necessary for smart u--things? things? -- What knowledge should be initially set? What knowledge should be initially set? -- What knowledge can be added later on?What knowledge can be added later on?-- What knowledge may be selfWhat knowledge may be self--learned during uses?learned during uses?-- How knowledge is used for rich and varied real situations?How knowledge is used for rich and varied real situations?

CChallengehallenge -- how to abstract, learn, use complex knowledge about human & worhow to abstract, learn, use complex knowledge about human & world.ld.

How about DAI, swarm, How about DAI, swarm, softcompsoftcomp, agent, semantics, etc.?, agent, semantics, etc.?

Challenge 5 Challenge 5 –– SelfSelf--Aware uAware u--Things Things

Why smart uWhy smart u--things should be selfthings should be self--awareaware-- Physical, not virtual, things with attached/embedded/blended (APhysical, not virtual, things with attached/embedded/blended (AEB) computers/etc.EB) computers/etc.-- AEB devices are function parts or components of the real physicAEB devices are function parts or components of the real physical thingsal things-- AEB is for enhancing original functions of real things followinAEB is for enhancing original functions of real things following common rulesg common rules-- Smart uSmart u--things should be aware their roles and function without against things should be aware their roles and function without against the rulesthe rules

A(t) = {AA(t) = {Azz(G(Gzz, , tt), z), z∈∈Z(Z(tt)})}

ContextContext--awareness awareness vsvs selfself--awarenessawareness-- ContextContext--awareness: knowing othersawareness: knowing others-- SelfSelf--awareness: knowing selvesawareness: knowing selves

““Knowing others is wisdom, knowing yourself is enlightenmentKnowing others is wisdom, knowing yourself is enlightenment”” -- Tao TzuTao Tzu

-- What this exactly means? How to equip selfWhat this exactly means? How to equip self--awareness to uawareness to u--things?things?

Extreme Challenging and rare researchExtreme Challenging and rare research““we seem so ill equipped to understand ourselveswe seem so ill equipped to understand ourselves””, The Society of Mind, , The Society of Mind, MinskyMinsky, 1986, 1986WWorkshop on Selforkshop on Self--Aware Computer Systems Aware Computer Systems -- ChaudhriChaudhri & McCarthy, SRI/DARPA, 2004& McCarthy, SRI/DARPA, 2004

Challenge 6 Challenge 6 –– Looped DecisionsLooped Decisions

Decisions cannot be correct always in 100%Decisions cannot be correct always in 100%Imprecise decision & exception are commonImprecise decision & exception are common

Loop mechanism is indispensableLoop mechanism is indispensable !!How many loops?How many loops? Layered models?Layered models?

Context - Processing - Management - Database

Supervision - Task & Plan - Policy & Rule - Background

Network Devices

Internet, WebCyber Space/WorldE-Service/Grid/etc.

Interface

Service/Middleware

Real Space/World Model and Semantics

Network Devices

Network Devices

Application - Awareness - Assistance - Advice

SpaceInterface

Space

UbicKidsUbicKids general architecturegeneral architecture

Challenge is how to adaptively correct Challenge is how to adaptively correct errors & make necessary adjustmentserrors & make necessary adjustmentsHow related to automatic control theory?How related to automatic control theory?

D(t) =ψ[S(t), N(t), R(t), K(t), A(t), D(t-Δ), E(t)]

ψS(t)N(t)R(t)

D(t)

K(t) A(t)

actionsactions

D(t-Δ)

E(t)χ

SummarySummary

Trend toward Ubiquitous Intelligence Trend toward Ubiquitous Intelligence (UI)(UI) & Smart World & Smart World (SW)(SW)

Smart uSmart u--things: essential SW elements things: essential SW elements core of core of ubi/percompubi/percomp

RW complexity challenges in making uRW complexity challenges in making u--things truly smartthings truly smart-- To examine the possible hard issues to suggest some potential reTo examine the possible hard issues to suggest some potential research lines search lines -- To let researchers in this field coolheaded and being aware of To let researchers in this field coolheaded and being aware of the hardnessthe hardness

WeiserWeiser’’ss words in his last dayswords in his last days““If the computational system is invisible as well as extensive, iIf the computational system is invisible as well as extensive, it becomes hard to t becomes hard to

know what is controlling what, what is connected what, where infknow what is controlling what, what is connected what, where information is ormation is flowing, how it is being used, what is broken, what are the consflowing, how it is being used, what is broken, what are the consequences of any equences of any given action. Maintaining simplicity and control simultaneously given action. Maintaining simplicity and control simultaneously is still one of the is still one of the major open questions facing ubiquitous computing research.major open questions facing ubiquitous computing research.””

-- IBM System Journal, Vol. 38, No. 4, 1999.IBM System Journal, Vol. 38, No. 4, 1999.

Our Research Our Research –– Smart HyperspaceSmart Hyperspace(Situational Principle) Context Awareness ((Situational Principle) Context Awareness (SchilitSchilit et al, 1994)et al, 1994)

(Spatial Principle) Boundary Principle ((Spatial Principle) Boundary Principle (KindbergKindberg & Fox, 2002)& Fox, 2002)

(Temporal Principle) Continuous Interaction ((Temporal Principle) Continuous Interaction (AbowdAbowd et al, 2002)et al, 2002)

(Situational/Spatial/Temporal Principle) Interrelation (Ours, 19(Situational/Spatial/Temporal Principle) Interrelation (Ours, 1996, 2003)96, 2003)HyperworldHyperworld Modeling (VISModeling (VIS’’96), Modeling Interface with a Multimedia 96), Modeling Interface with a Multimedia HyperworldHyperworld (HIS(HIS’’96)96)

-- A present situation may be related to events in the past/futureA present situation may be related to events in the past/future probably at other spacesprobably at other spaces-- A current event may result in a sequence of followA current event may result in a sequence of follow--up events in different places/time up events in different places/time -- A user may frequently move between different spaces in daily acA user may frequently move between different spaces in daily activities tivities -- A user may be sometimes interested in what happen at other placA user may be sometimes interested in what happen at other places in a particular timees in a particular time

Current statusCurrent status: focused on a variety of individual and isolated smart spaces : focused on a variety of individual and isolated smart spaces but seldom or without fully addressingbut seldom or without fully addressing space interrelations !space interrelations !

Need of smart hyperspaceNeed of smart hyperspace, interrelated , interrelated && connected smart spaces!connected smart spaces!

InternetInternet Interconnect NetworksInterconnect NetworksWWWWWW Interconnect FilesInterconnect FilesP2PP2P Interconnect ClientsInterconnect ClientsGridGrid Interconnect ResourcesInterconnect ResourcesWbSWbS Interconnect ApplicationsInterconnect ApplicationsHyperspaceHyperspace Interconnect SpacesInterconnect Spaces

Smart Hyperspace IssuesSmart Hyperspace IssuesHyperspace abstraction and modelHyperspace abstraction and modelHyperspace semantics and representationHyperspace semantics and representationConnections of heterogeneous smart spacesConnections of heterogeneous smart spacesContext interrelations and sharing across places/timeContext interrelations and sharing across places/timeSmoothness of uneven spaces or space jitters (smartness differenSmoothness of uneven spaces or space jitters (smartness differences)ces)Coordination and management of associated smart spacesCoordination and management of associated smart spacesScalability and manageability of a hyperspaceScalability and manageability of a hyperspaceSecurity, privacy, and trust in multiSecurity, privacy, and trust in multi--spacesspacesInterface and Interaction of a hyperspaceInterface and Interaction of a hyperspaceHyperspace network infrastructureHyperspace network infrastructureHyperspace middleware and interoperationsHyperspace middleware and interoperationsHyperspace social/economic/cultural/ethic implications and impacHyperspace social/economic/cultural/ethic implications and impactsts……

No answer yet ! The issues themselves need to be further clarifiNo answer yet ! The issues themselves need to be further clarified !!ed !!

Two fundamental research waysTwo fundamental research ways::General thinking and systematic theoretical study is essentialGeneral thinking and systematic theoretical study is essentialProper and representative concrete case study is necessaryProper and representative concrete case study is necessary

Why A Smart Hyperspace for Kids CareWhy A Smart Hyperspace for Kids CareScope CriteriaScope Criteria-- Includes a set of different but interrelated spaces capable of Includes a set of different but interrelated spaces capable of being smartbeing smart-- Covers core issues related to the smart hyperspaceCovers core issues related to the smart hyperspace-- Involves both technical and nonInvolves both technical and non--technical factorstechnical factors

Feasibility CriteriaFeasibility Criteria-- Complexity controllable and improvable continuouslyComplexity controllable and improvable continuously-- Privacy relatively acceptable and can be enhanced graduallyPrivacy relatively acceptable and can be enhanced gradually-- Cost is reasonable, especially in the beginningCost is reasonable, especially in the beginning

Other CriteriaOther Criteria-- UsefulUseful-- NovelNovel-- FunFun

A ubiquitous kids care system likely matches the above criteria.A ubiquitous kids care system likely matches the above criteria.AA unique one comparable with Personalized Instrumented Health Sysunique one comparable with Personalized Instrumented Health System for elders (PIHS,tem for elders (PIHS, UR/MIT/GT/UFUR/MIT/GT/UF))

A survey recently made in Japan, says thatA survey recently made in Japan, says that72.5%72.5% parents worried about their kids, parents worried about their kids, 82.3%82.3% parents felt tired in caring their kids, andparents felt tired in caring their kids, and91.9%91.9% parents had no enough time to satisfactorily take care of theirparents had no enough time to satisfactorily take care of their kids.kids.

IT gifts to kids: toy, game, animation, what else are specially IT gifts to kids: toy, game, animation, what else are specially designed for them?designed for them?-- Lacks enough research and nonLacks enough research and non--playable products for kids by ITplayable products for kids by IT-- Interaction Design and Children (IDC, annual conference since 2Interaction Design and Children (IDC, annual conference since 2002) 002) –– No kids care topics!No kids care topics!

Such system is not only proper for hyperspace study but valuableSuch system is not only proper for hyperspace study but valuable for a special type of usersfor a special type of users

UbicKidsUbicKids –– Ubiquitous Care for KidsUbiquitous Care for KidsUbicKidsUbicKids ObjectivesObjectives-- TTo develop a set of ubiquitous applications for assisting parentso develop a set of ubiquitous applications for assisting parents to take care of their to take care of their

kids with more convenient, prompt, reliable, precise, skids with more convenient, prompt, reliable, precise, secure and trust services.ecure and trust services.-- To build a representative smart hyperspace for probing and reseTo build a representative smart hyperspace for probing and researching ubiquitous arching ubiquitous

hyperspace related issues, models, technologies, etc.hyperspace related issues, models, technologies, etc.-- To study impacts and solutions of nonTo study impacts and solutions of non--technical factors to both ubiquitous systems and technical factors to both ubiquitous systems and

users, especially children, their growth, character devusers, especially children, their growth, character development, etc.elopment, etc.

UbicKidsUbicKids AssumptionsAssumptions-- Usual family with one or more normal children (disabled, singleUsual family with one or more normal children (disabled, single par., grand par., par., grand par., ……))-- Typical spaces such as home, yard, park, street, station, car, Typical spaces such as home, yard, park, street, station, car, school, office, clinic, school, office, clinic, ……-- Kids ages: ways in caring kids vary for kids in different ages,Kids ages: ways in caring kids vary for kids in different ages, (families, cultures, (families, cultures, ……))

UbicKidsUbicKids Functions Functions –– 3A3A-- Kids Kids AAwareness: knowing kids current & past statuswareness: knowing kids current & past status-- Kids Kids AAssistance: helping kids in doing somethingssistance: helping kids in doing something-- Kids Kids AAdvice: advising/reminding kids and parentsdvice: advising/reminding kids and parents

0 2 4 6 12years old

8

Towards a Smart World and Ubiquitous Intelligence: A WalkthroughTowards a Smart World and Ubiquitous Intelligence: A Walkthrough from Smart Things to from Smart Things to Smart Hyperspaces and Smart Hyperspaces and UbicKidsUbicKids, Journal of Pervasive Comp. and Comm., 1(1), March 2005. , Journal of Pervasive Comp. and Comm., 1(1), March 2005.

UbicKidsUbicKids -- NonNon--Technical FactorsTechnical Factors

NonNon--Technical FactorsTechnical FactorsCommon characteristics shared by many kids

Special characteristics for individual kids

Characteristic changes along with growing kids

Relationships and roles of family members

Features of kids care activities

Heterogeneity in kids care

Cultures and laws in kids care

Psychological behavior in a smart space

Child’s personality development

Child’s habit and moral cultivation

Child’s independence improvement

Child’s intelligence increase

Feeling/love enhancement of parents-kids

Special care to disabled/incapacitated children

Family of single parent, with nurse, etc.

…

DoubleDouble--edged Swordedged SwordSeamless integrations of physical and digital world,

- “a strange new world” (by Mattern, 2004)

Kids learn from everyday interactions with the environments

help forming their characteristics, behaviors, habits, personalities etc.,

influence their whole lives.

A truly useful UbicKids system

full considerations on non-technical factors, i.e., human, society, culture, physiology, psychology, moral, feeling, etc.

Positive and negative impacts to kids be seriously investigated

solutions must be discovered to overcome the negative aspects.

Final Final ……

Care some special Care some special ““kidskids””: dog, cat, : dog, cat, …… (pets)! (pets)! UbicPetsUbicPets, , UbicUbic~ ?~ ?UbiUbi--others?others?

From virtual From virtual ee--thingsthings to to mm--thingsthings to real to real uu--/p/p--/s/s--/i/i--thingsthingsee--, e, e--, e, , e, …… mm--, m, m--, m, m--, , …… uu--, u, u--, u, u--,, …… ??

pp--, p, p--, p, p--,, …… ??ss--, s, s--, s, s--, , …… ??ii--, i, i--, i, i--, , …… ??

Lesson: Lesson: εε(concrete/small)(concrete/small) ∞∞ (general/great)(general/great)

WhatWhat’’s s UbiUbi--next next ?? Beyond Beyond UbiUbi/Per/Am/Per/Am-- !!