54

ChallengesThe most immediately striking challenge

in designing wearable computers is creatingappropriate interfaces. However, the issues ofpower use, heat dissipation, networking, andprivacy provide a necessary framework inwhich to discuss interface. Part 1 of this arti-cle covers the first two of these issues; Part 2begins with the networking discussion.

NetworkingAs with any wireless mobile device, the

amount of power and the type of servicesavailable can constrain networking. Wearablecomputers could conserve resources throughimproved coordination with the user inter-face. For example, the speed at which a giveninformation packet is transferred can be bal-anced against latency, energy costs, and finan-cial costs. Often, bits per second per watt is amore meaningful measure of a particular wire-less networking technology than maximum

throughput. Another serious issue is openstandards to enable interoperability betweendifferent services. For example, only one long-range radio should be necessary to providetelephony, text messaging, Global PositioningSystem (GPS) correction signals, and so on.

For wearable computers, networkinginvolves communication off body to the fixednetwork, on body among devices, and nearbody with objects near the user. Each of thesethree network types requires different designdecisions. Designers must also consider pos-sible interference between the networks.

Off-body communications. Wireless commu-nication from mobile devices to fixed infra-structure is the most thoroughly researched ofthese issues. On the consumer side, analog cel-lular phones and digital amateur-radiorepeaters provided the first glimpse of futureproblems; these systems would often dropconnections as the user moved. Communica-

Thad StarnerGeorgia Institute of

Technology

WEARABLE COMPUTING PURSUES AN INTERFACE IDEAL OF A CONTINUOUSLY

WORN, INTELLIGENT ASSISTANT THAT AUGMENTS MEMORY, INTELLECT,

CREATIVITY, COMMUNICATION, AND PHYSICAL SENSES AND ABILITIES.

MANY CHALLENGES AWAIT WEARABLE DESIGNERS. PART 2 BEGINS WITH THE

CHALLENGES OF NETWORK RESOURCES AND PRIVACY CONCERNS. THIS

SURVEY DESCRIBES THE POSSIBILITIES OFFERED BY WEARABLE SYSTEMS AND,

IN DOING SO, DEMONSTRATES ATTRIBUTES UNIQUE TO THIS CLASS OF

COMPUTING.

0272-1732/01/$10.00 2001 IEEE

THE CHALLENGES OF WEARABLECOMPUTING: PART 2

tions researchers developed systems based onseveral standards—cellular digital packet data(CDPD), Global System for Mobile Com-munications (GSM), time-division multiple-access (TDMA), and code-divisionmultiple-access (CDMA)—to help this prob-lem. Today, next-generation communicationssystems (2.5G and 3G) will further improveconnection reliability and aggregate through-put. The “Cellular phones” sidebar discussesnetworking and interface issues applicable towearable computers and their design.

No matter what technologies finally dom-inate, some challenges will remain. First, noneof the current networking systems will beubiquitous. Users will always face situationsin which mobile devices will not be in rangeof a network cell. Although additional celldeployment and satellite use is slowly address-ing this problem, it will remain unprofitableto provide coverage for some areas. However,an interesting concept is to employ automo-biles as repeaters for the wearable user’s wire-less data traffic.1 Although a wearablecomputer has a relatively small battery andantenna, cars can carry much larger equip-ment. In addition, drivers rarely stray furtherthan a few miles from their automobiles. Fur-thermore, even when driven into a remotelocation, a car is often within communicationrange of other cars. For example, imagine aseries of cars along rural highways acting asrepeaters to route wireless data to a local, fixednetworking center, such as a US post office.This wireless service does not need to be realtime to provide value; store-and-forward net-working has been used successfully with theInternet for decades.

For such a wireless participatory network-ing scheme to succeed, developers mustaddress standardization, security, quality of ser-vice, and synchronization. This style of ad hocnetworking of mobile devices complicates tra-ditional issues of resource discovery and rout-ing. Furthermore, mobile nodes traveling atvariable speeds create difficulties for manywireless systems. Fortunately, this area ofresearch has become very active in recent years.

Another way to alleviate coverage issues is toemploy aggressive caching. By observing thewearer’s network use, the wearable computercan speculate about what the user will accessnext and cache material using spare network

bandwidth. When the user is working offline,the system employs this cache and updates anychanges when network connectivity becomesavailable.2 But what happens when a conflictoccurs? For example, suppose a businessmanupdates his calendar while disconnected, and,during the same time, his assistant also sched-ules an appointment. In practice, such conflictsare rare, but this problem raises the issue ofwhere to locate a file’s correct, or master, copy.

Most wireless mobile devices by natureadhere to a thin-client approach to comput-ing. In other words, the device provides justenough processing power, user interface, anddata storage to access services that are basedon a fixed-infrastructure server located else-where. Mobile devices without wireless access,such as the original PalmPilot, provide asyn-chronous services where docking the devicewith the fixed network updates the file’s mas-ter copy. However, the rapid increase inmobile mass-storage capacity, increasing at arate that surpasses that of Moore’s law, makesa strong case for the mobile device maintain-ing the master copy. Today’s ruggedized, pock-et-size hard drives can store 48 Gbytes. Soon,mobile users will maintain a terabyte on body,making local storage space a nonissue. In fact,users might prefer keeping all their data withthem, making the file data physically secure,accessible at any time, and always the author-itative copy.

55JULY–AUGUST 2001

Cellular phones Although not wearable computers, the original analog cellular phones provide a study in

design constraints related to networking and interfaces of devices. Initially, one of the mostimportant features of these phones was a user interface that resembled, as closely as possi-ble, a push button telephone. This feature provided a key improvement over the half-duplex,push-to-talk walkie-talkies of the time.

As the technology matures, a variety of fundamentally different interfaces are becomingpopular, such as instant messaging. Alternative networking paradigms will also appear becauseof the expense of deploying cellular towers, upgrading infrastructure continuously, and shrink-ing available bandwidth. These new paradigms will take advantage of asynchronous com-munication and other modes of human-to-human interaction, such as those used in Internetstore-and-forward networks. Even today, cellular phones are becoming more like wearable com-puters. Higher-end microprocessors, multitasking operating systems, and bitmapped displaysnow exist in cellular phones. Cellular phone manufacturers embed sensors in these devicesto determine if the user is in a meeting or walking on a noisy street. The cellular phone usesthis information to determine if the user is interruptible and the mode to use as an incoming-call alert. Indeed, researchers are developing some systems in which the cellular phone com-municates a worker’s current task to a remote expert who can provide advice.

Caching, revision control, and intelligentagents can emulate remote access to the user’son-body personal directories when wirelessconnectivity is unavailable. Although this strat-egy could prove inconvenient for the occasionalthird-party user, it gives the wearer the mostconvenient access. For applications like calen-dars, the wearer knows her copy is definitive.

Retaining information locally could alsohelp conserve battery power. In terms ofpower, wireless network access requests aregenerally more expensive than local accessrequests. Thus, if the user is responsible for themajority of the data accesses, maintaining alocal copy will conserve power. Modeling useractivities can also conserve power. Many wire-less networks scale transmission power to theminimum necessary to maintain a connection.Thus, transmitting a 10-Mbyte e-mail mes-sage when the wearer is near a receiver couldrequire significantly less power than when onlya weaker signal is available. If the wearablecomputer understands a particular message’surgency level and can predict the wearer’sfuture location, it could delay transmissionuntil the wearer is closer to a receiver. In fact,with multiple wireless services to choose fromand pricing schemes that depend on time oftransmission, the evaluation criteria of whento transmit could extend to include financialcost as well as efficiency. Expected latencycould also help determine when and whichreal-time or interactive services to use.

Interoperability. Unfortunately, the hardwareneeded to access more than one wireless ser-vice burdens the user with extra equipment.Software radios could improve this situation.3

Many wireless-modem components can beemulated using digital signal processing.Downloading the appropriate software to theradio can change its communication standardsand protocols (of course, wireless devices willstill require certain analog components such asantennas). If this vision proves practical, wear-able computers will not only use differentwireless networking services based on geo-graphical location, cost, and power, but couldalso replace certain common, portable con-sumer electronics, such as GPS receivers,radios, and televisions.

As with mobile ad hoc networking, suchwireless services between devices on the body

will also need standards for resource discov-ery and node arbitration to enable communi-cation. Solving this problem for wearablescould prove easier than for ubiquitous com-puting because a wearer will only occasional-ly add or remove a device from his bodynetwork.

Although on-body communication requiressignificantly less energy than off-body net-works, energy use becomes critical becauseeach device must have its own, relatively smallbattery. This is a current challenge for theBluetooth and IEEE 802.15 communities.Some experimental systems, such as BBN’sBodyLAN, require as little as 4-nJ/bit to trans-mit, while maintaining moderate bandwidth.4

With such energy conservation, low-band-width, on-body sensors and interface devicescould last for a year on one charge. As a sidebenefit, such low transmission power makesUS Federal Communications Commissionapproval significantly easier.

Privacy and contention between differentwearers’ body networks becomes an issue withon-body wireless networks. Zimmerman’s Per-sonal Area Network addresses this problem byexploiting near-field effects. In a sense, thewearer’s body contains the transmissions, anddevices must touch the user to receive a con-nection.5 Post and Orth take a different tackby researching clothing that has embeddedelectrical components.6 By careful use of tra-ditional fabrics and connectors and the designof new conductive threads, these researchersdemonstrated functional, washable clothingwhere the dressing process makes electricalconnections. For example, snaps attachedwith conductive thread can serve as connec-tions among electronics in the shirt, belt, andpants. Such experimentation has also led tonew interfaces, including keyboards embroi-dered onto the user’s jacket.

Communicating with near-body objects. Com-munication with near-body objects providesyet another set of challenges to wearable com-puter design. Many consumer electronicsmanufacturers are now proposing network-ing standards based on mature radio andinfrared transceivers. Most of these standardsassume that the device has access to a signifi-cant energy supply. However, Hull, Neaves,and Bedford-Roberts propose embedding an

56

WEARABLE COMPUTING

IEEE MICRO

RFID tag reader in the user’s shirt cuff andpassive RFID tags in devices with which theuser wishes to communicate or track.7 When-ever the user reaches for a tagged device, ener-gy is transmitted wirelessly from the user’sshirt cuff to the device, which collects thepower and responds with a few bytes ofinformation. This information can be thedevice’s unique ID or, in the future, the cur-rent state of the device’s low-power sensors.The wearable computer can then use its high-er-power wireless connection to transmit theseresults to the network at large. Thus, a taggeddevice’s location and state is uploaded to thenetwork each time it is moved by a user witha tag reader. This scheme realizes some of theadvantages of ubiquitous sensors and net-working without the inconvenience and costof ubiquitous batteries.

The Locust positioning system, shown inFigure 1, provides another example of animplementation of a power-restrictive net-work scheme. While several technologies provide communication and location infor-mation outdoors, the Locust’s primary pur-pose is to provide location information insidea building. Each Locust consists of a micro-controller, infrared transmitter, and infraredreceiver. To avoid the maintenance overheadof batteries, each Locust is mounted near anoverhead light and generates power with asmall solar panel. By mounting these devicesclose to light sources, the infrared transmit-ter/receiver pair receives a view of the work-space in a room. The infrared transmitterrepeats a unique ID every second with an off-set determined by its ID to avoid repeatedcollisions with other Locust. By listening forthese IDs and having a corresponding mapof the area, wearable computers can deter-mine their location. The wearable can thenrepeat this information to networks in theenvironment, depending on the wearer’s pri-vacy preferences. A user can upload smallamounts of data to a particular Locust, andthe Locust relays this data along with itsunique ID. Thus, a user can annotate a givenarea with specific information for discoveryby another, later user. Note that the infor-mation can stand alone or act as a pointer tomore data, such as sound files, animations,or programs stored on a traditional networkand accessed by a user’s real-time wireless con-

nection. Although researchers have not yetimplemented a secure system, a user couldtarget information to one or several users byincluding a cryptographic key in an uploadto a given Locust. In this way, the system cre-ates a simple form of low-power, location-based networking.

Privacy

Those who design systems which handle

personal information therefore have a spe-

cial duty: They must not design systems

which unnecessarily require, induce, per-

suade, or coerce individuals into giving up

personal privacy in order to avail them-

selves of the benefit of the system being

designed.8

These words, written by Leonard Foner,seem especially applicable to wearable com-puters, which could become storehouses ofusers’ most intimate information. Indeed,designers of early ubiquitous computing sys-tems often cite privacy as one of the key userconcerns in adopting their technology.9 Userprivacy concerns are not equivalent to secu-rity concerns. Security involves the protec-tion of information from unauthorized users;privacy is the individual’s right to control thecollection and use of personal information.8

When considering security and privacy, sys-tem designers must consider what threats thesystem might face, such as those posed bycrackers, employees, employers, the courts,and so on. For an example of an early wear-able computer and how its design addressed

57JULY–AUGUST 2001

Figure 1. The Locust infrared transpondersystem mounted in an overhead fluorescentlight fixture. The solar cell on the right pro-vides power.

the need for privacy, see the “Thorp/Shan-non wearable roulette predictor” sidebar.

Depending on the perceived threat, securi-ty and privacy concerns can conflict. Forexample, consider an active-badge system thata company deploys as a security measure.Employees must wear badges at all times to

identify themselves to other employees andsecurity personnel, and to unlock variousdoors. Each badge continually announces itspresence to the environment through radiotransmissions. Receivers in the environmentreport the badge’s location to a central system.When combined with other sensors, this sys-tem can determine when an individual hasentered a restricted space without a badge orappropriate authorization. Although this sys-tem could be a highly effective security sys-tem, it raises several privacy concerns.

A common user perception about such secu-rity systems is that employers could spy onthem or monitor activities such as time spent inthe restroom or the length of coffee breaks. Infact, employers could install additional, hiddenreceivers to monitor employee actions covert-ly. Even if a concerned badge wearer removesthe badge for a given situation, the aggregateinformation collected over several days ormonths can still reveal behavior patterns.

Assuming that the user does not mind suchintrusions or is sufficiently recompensed forthem, the security measures a business uses toprotect its employee’s privacy is still an issue. Asan extreme example, consider an employeewho has a former spouse stalking her. Does thebusiness sufficiently protect data collectedfrom the security system so that the stalker can-not determine the employee’s work hours?

Technically, it is possible to make activebadges secure. A badge system can use encryp-tion technology that only gives a master oper-ator access to a given badge’s descrambledsignature. However, this master operatormight be bribed or manipulated into unwit-tingly revealing critical information.

Legislation could provide another privacythreat to employees. For example, the USFreedom of Information Act and similar statelaws might require releasing information thata government employee mistakenly thoughtwas private. In addition, court subpoenasmight demand information recovered andstored by ubiquitous computing systems.Already, US authorities have tapped automaticautomobile toll-pass systems to help provecourt cases. For the active-badge security sys-tem, a simple solution is to erase or overwriterecords when they are no longer needed. Evenso, employees must still trust employers to bediligent in this regard.

58

WEARABLE COMPUTING

IEEE MICRO

Thorp/Shannon wearable roulette predictorProbably the first electronic wearable computer mentioned in the literature is also one of

the most carefully designed with regard to privacy and unobtrusiveness. Edward Thorp andClaude Shannon designed, built, and field-tested an analog wearable computer that yield-ed an expected gain of 44 percent in the game of roulette by predicting the most favoredoctant for the ball to land.

Roulette is a game of chance, popular in many casinos. A small ball spins on a shallowsloped track mounted above a spinning wheel that has numbered pockets to receive the ball.Players bet on the pockets, trying to pick the one in which the ball will land. As the ball andwheel spin, gamblers continue placing bets until it appears that the ball will fall from the trackin the next two or three revolutions.

The game was long considered unpredictable and became a challenge to the two Mass-achusetts Institute of Technology professors who were interested in probability and statis-tics. Thorp and Shannon determined that by timing the speed of the ball and the wheel andcalculating the differential equations that determined their movement, they could determinethe ball’s path with fair accuracy. By 1961, Thorp and Shannon had created an analog wear-able computer to aid a gambler’s bets.

Because of the potential serious consequences of being discovered with an electronicdevice at a gaming table in the early 1960s, unobtrusiveness and privacy were primary con-cerns. Significantly, not only was it necessary to keep private the information generated bythe wearable, but Thorpe and Shannon needed to hide the computer’s existence from onlook-ers. Thus, power, networking, and interface considerations were subservient to minimizingthe probability of detection.

The system was divided into separate parts used by an observer and a bettor. The observ-er wore the computer and timing system, while the bettor wore an earphone so he couldreceive instructions regarding how to bet. To camouflage the system, the speaker on the bet-tor’s system was placed in one ear canal, and the wires to it were colored to match skin andhair. The observer timed the ball and rotor using microswitches in his shoes. Thus, both theinput and output devices were unobtrusive. The computer was placed in a cigarette-pack-age-size box; it remained on the observer. To minimize the risk of detection, an inductivewireless scheme connected the observer’s computer to the bettor’s receiver. Only surveil-lance devices that were very close to the system could detect its communications. In design-ing primarily for privacy, Thorp and Shannon purposely crippled their system; compared to ageneral-purpose wearable computer; it was a single purpose machine with an awkwardinterface, limited power system, and rudimentary connectivity. The system was used suc-cessfully in a Las Vegas casino in the mid-1960s, but the machine was fragile, preventinglarge-scale winnings. Worried about getting caught, they delayed revealing their systemuntil 1969, when they described it in a statistics journal.1

Reference1. E. Thorp and Anonymous, “Optimal Gambling Systems for Favorable Games,”

Rev. of the Int’l. Statistics Inst., vol. 37, no. 3, 1969.

Even if technology and policy address theseconcerns, active-badge systems are vulnerableto yet another form of attack. Simply moni-toring the amount of traffic from variousbadge-receiving stations provides data on aperson’s path through a building on howmany people are in a given area. This trafficanalysis can be aggregated or combined withother sources of information to reveal poten-tially damaging data.

An alternative to active badges is to designsystems in which the user solely controls theresultant information. In other words, theuser’s wearable computer would concentrate,process, and filter any data collected or dis-tributed about the user. In this way, the usercontrols the degree of functionality and canbalance it against the amount of informationrevealed.

The GPS and Locust (see Part 1) positionsystems demonstrate this design philosophy.Both systems place transmission beacons inthe environment: low earth orbit for GPS andthe local environment for Locust. By carryinga compatible receiver operating independentlyfrom the infrastructure, users can benefit fromthese systems without revealing their where-abouts. However, certain functionality, such asletting others inquire about a user’s location,would not be available without the userretransmitting his position to the environ-mental infrastructure.

Although such a retransmission schememight seem gratuitous, it provides certain ben-efits. The first is that the user has finer con-trol over what information is revealed. Evenif the user does not reveal any information,the wearable computer knows his location—an important piece of contextual information.Another benefit is that the worn componentsimply listens to beacon signals instead of con-tinuously broadcasting one itself, which givessecurity badges longer battery life. Also, usersmight more willingly accept technology in thisform. By giving employees explicit controlover personal information, an employer showsrespect and confidence in employees’ use ofthe technology.

In general, wearable computing could pro-vide users with a sense of control with respectto privacy. This issue will gain in importanceas sensing systems become common through-out homes and offices. By limiting a sensor’s

physical range and network connectivity tothat provided through the user’s wearablecomputer, the wearable computer becomes anatural control point for all user-related infor-mation. In some cases, an electric field gen-erated by the wearable computer couldwirelessly power the sensors, following themodel of passive RFID tags. In this manner,sensors and objects “wake-up” as the userpasses through their environment. Without auser with an appropriate power system near-by, the sensors are unavailable. This methodhelps limit abuse of such systems by a remotethird party. When such a scheme is not pos-sible, the user could give explicit permissionfor the sensing to occur by turning on thesensor’s power. The sensor should turn offautomatically when it can no longer sense theuser or the user’s network connection.Although not preventing potential abuses,such schemes help make abuse more incon-venient.

One major challenge to preserving privacyin wearable computing is the disseminationof information and ideas on potential abusesand protection schemes. Wearable comput-ing system designers should have a neutralforum to discuss techniques and introducestandards. Or perhaps the community shoulddevise a privacy protection scheme designedwith ratings displayed on various systems andcomponents. For example, a wearable com-puter component could protect privacy byway of the following barriers:

• Physical. In this approach, some mecha-nism maintains a physical barrierbetween data and potential abusers. Thebarriers could range from a system inwhich users always carry crucial data ontheir bodies to a methodology thatsecures data in a safe when not in use.Other physical safeguards could includeshielding the wearable computer to min-imize unintentional wireless emissions.

• Technological. These approaches use secu-rity methods such as encryption and bio-metric identifiers—fingerprints, irisscans, and so on—as barriers.

• Legislative. Laws could specify conditionsunder which privacy is considered vio-lated. The law could tailor associatedpenalties to particular technologies,

59JULY–AUGUST 2001

ensuring relevance and avoiding misin-terpretation in court.

• Social. Wearable systems could use exist-ing social conventions to build barriers.For example, systems could store sensi-tive data in physical articles—such asdata repositories resembling diaries orwallets—that a particular culture wouldnormally consider personal.

• Obscuring. The wearable computer couldhide sensitive information in directorieswith large quantities of nonsensitiveinformation. Thus, a casual investigatorcould not look at all the files to determinewhich are the most revealing.

Although some of these protections are veryweak, different combinations allow a user tospecify various levels of inconvenience for awould-be interloper. In some cases, the usermight want the barriers to be breached givenenough persistence. For example, if the user hasa serious accident, the user might want col-leagues to discover his medical history, where-as in normal circumstances he would considerthis information private. Perhaps a medicalinformation memory card stored in the user’swallet would provide an appropriate level ofprotection against casual prying. Such a cardwould let bystanders help the user in an emer-gency. However, if the main concern is exploita-tion by the user’s medical insurance company,more levels of security would be necessary.With care, wearable designers can specify bar-rier combinations to adapt to the changingpolitical, technical, and social climates in dif-ferent markets and geographic areas.

Interface designIn the following discussion, the term interface

is used as a generalization to refer to the numer-ous fields that address human and computerinteraction. This includes, but isn’t limited to,human-computer interfaces, psychophysics,human factors, ergonomics, industrial design,and fashion. Wearable computing interfacesare a topic for an entire textbook; this articlecannot begin to summarize the work in thefield to date or even the areas yet to be explored(for an overview, see the Proceedings of the IEEEInternational Symposium on Wearable Comput-ers).10 Instead, this discussion is meant to stim-ulate curiosity in the field.

Clothing, design, and fashion. Wearable com-puting represents an unusual intersection ofscience, engineering, design, and fashion. Thesame basic computer components found in amainframe also comprise a wearable, butdesign decisions for wearables must accountfor the restrictions of portability and usabili-ty over the need for speed and throughput.

In addition, there is a surprising social aspectto wearable computing. The design, tailoring,and expense of a wearable computer can reflectthe user’s taste and importance, as does abanker’s choice of business suits. Natick ArmyResearch Lab researchers have indicated to theauthor that in the armed forces, wearers’ per-ception of design and expense affects theiracceptance and opinion of the most basic sup-plies, such as mountaineering boots.

To make a computer wearable for an extend-ed period of time, designers must carefullydetermine size, body placement,11 clothingtype,6 and wired interconnections to othercomponents. This situation could reverse thenormal design process for computers; withwearables, developers might first decide on aform factor and then determine functionalityand associated electronics. Such a philosophyresulted in the PalmPilot when other stylus-based handheld devices were failing.12 Unlikemuch of computer science and electrical engi-neering, where ungainly prototypes might notaffect an experiment’s quality, simple variationsin the form factor of a research-grade wearablecomputer could prove significant.

Peripheral interfaces: Making simple things sim-ple and complex things possible. To improveportability, peripherals for wearable computerstend to be small. For example, the Twiddlerone-handed keyboard can fit in a pocket, andthe flat panel display in a pair of MicroOpti-cal eyeglasses is about the size of a grain of rice.Unfortunately, unlike computer processors,there is a limit to how small these devices canbecome. The resolution of the human eye lim-its a usable display’s size and resolution. Simi-larly, the size of the user’s fingers limits theplacement and number of keys on a keyboard.Wearable peripheral designers must considertrade-offs between usability, portability, andunobtrusiveness for every device they make.

As previously discussed, one vision of wear-able computers involves a body-centered wire-

60

WEARABLE COMPUTING

IEEE MICRO

less network through which the user can inte-grate peripherals simply by pocketing them.Such a system eliminates redundancy inportable consumer electronics; it could alsoenable interfaces that would otherwise provetoo costly to implement because of the repeat-ed design cost for each independent device.More importantly, such a system structureencourages the design of specialized periph-erals for particular tasks. The “A day in thelife” sidebar presents a scenario that illustratesthis point.

Some wearable computer users might pre-fer general-purpose equipment that allowsmaximum flexibility and constant availabili-ty. Unfortunately, such general-purpose sys-tems provide a bit of a conundrum socially.Onlookers do not know if the wearer is usingthe system as a camcorder, a cellular phone, atext editor, or a game system. Thus, anonlooker does not know if she can conve-niently interrupt the user. Of even greater con-cern to the onlooker is whether or not thewearer records their conversation covertly.While virtually undetectable video and audiorecording equipment has been available forsome time, as evidenced by investigative newsprograms, the similarity of head-up displaysto camcorder eyepieces causes confusion withonlookers. In the past, the form factor ofportable devices helped constrain their per-ceived uses. For example, a microphone hang-ing from a box at a reporter’s side designatesthe box as an audio recording device. How-ever, with wearable computers, the form fac-tor and uses are not yet commonly known.Providing some external cue about the tasksbeing performed by the wearer from minuteto minute could prove important as wearablecomputers become more widespread.

Intellectual toolsOne of the early applications of computers

was to calculate ballistic trajectories, a task forwhich the human mind is not well suited. Inartificial intelligence, research efforts try to cre-ate machines that perform tasks the humanmind and body do perform well. An interest-ing challenge that serves as a compromise is tocreate systems that augment a user’s natural abil-ities through computational components. Thisidea is not new; the history of computing isfilled with systems and philosophies described

by scientists who approached this problem insome form, for example, Bush’s Memex,Wiener’s Cybernetics, Licklider’s Man-MachineSymbiosis, and Englebart’s implementation ofNLS/AUGMENT, to name a few.

In the pursuit of such an interface, wear-able computing provides a set of advantagesnot available before. Wearable computers arephysically close to the user, highly portable,quickly accessed, and designed to consume afraction of the user’s full attention. PDAs, thecommercial devices most similar to a wear-able, provide a contrast to these traits. Usersoften store PDAs in a pocket or carrying case.So, although PDAs are physically close, it cantake users a significant amount of time toaccess the interface. In addition, the use of astylus-based PDA requires both hands, andthe interface requires most of the user’s visu-al attention. Thus, it is difficult to use PDAswhile, for example, walking down the street,repairing an automobile, or even having a

61JULY–AUGUST 2001

A day in the lifeA fictitious police detective’s typical day demonstrates how wearable computers may

benefit from specialization of peripherals: When documenting a crime scene, the detectiveremoves a small camera and microphone from his car’s glove compartment and fastens themto his lapel. The camera stores video on the detective’s pocket wearable. With a small key-board attached to his belt, the detective privately adds text annotations to the video stream.At the office, the detective stashes the camera and keyboard in his coat pocket, and approach-es a large screen and interface area festooned with a variety of knobs normally associatedwith video editing. The wearable interfaces with this equipment, and the detective spendsthe next few hours splicing together segments of his field observations to make his report.Given an insight by his editing, the detective decides to question a suspect. He holsters hisgun, which can only be activated to fire when it is kept within two meters of his wearable.The detective borrows a pair of sunglasses with a small display, camera, and microphoneembedded in them. He also picks out a longer-range network transceiver so that the othersat the station can monitor his progress. As he questions the suspect, the detective comparesthe suspect’s answers to his report and images from the crime scene displayed on the sun-glasses. As a result, he discovers an inconsistency in the suspect’s story and decides to arresthim. Although not as convenient as the lapel camera to manipulate, the sunglasses’ cameraperforms well, inconspicuously documenting and reporting the suspect’s resistance to arrest.Back at the police station, another police officer, who monitors all detectives’ progress viatheir long-range transmitters in the field, alerts nearby units to the situation. Back-up offi-cers respond to the location, and the suspect is arrested safely.

The peripherals in this scenario were designed for specific tasks and form factors. Usingeach device was simple—any software or hardware reconfiguration on the wearable wasperformed transparently when the user approached the peripheral. All the peripherals linkedinto the detective’s personal machine, allowing maintenance of crucial information on thedetective’s body. The detective makes an active choice in what functionality he wishes to carryversus the inconvenience of added weight and bulk the peripherals might cause.

conversation. However, a wearable equippedwith a display built into eyeglasses and a one-handed or speech-driven interface lets the userconcentrate on a primary task while the wear-able provides information support. For exam-ple, in repairing a car, a mechanic only has tomake a small eye movement to refer to thecar’s technical manual in a head-up display.

User attention is the scarcest resource forwearable computing. In particular, hand-eyecoordination is at a premium as the user livesin the physical world while accessing virtualinformation. Interfaces should provide themost support for the smallest investment ofattention diverted from the user’s primarytask. This idea is at odds with the window-,icon-, menu-, and pointer-driven interfacesgenerally found on desktops; desktop appli-cations often assume that the application itselfis the primary focus of the user’s attention.Hence, wearable computer systems might notuse traditional GUIs for many tasks.

Note taking and immediacy of interface. Awearable equipped with a head-up display andone-handed keyboard allows rapid note takingin most situation. In some cases, the wearableprovides a less obtrusive and more efficientmethod of recording information than is pos-sible with traditional means, such as pen andpaper. For example, in the author’s case, typ-ing on a one-handed keyboard takes half asmuch time as writing by hand.

Wearable computer users who use theirmachines during conversation tend to opti-mize their systems such that they can begintaking notes within two seconds of realizingthe need. Given this level of speed and access,everyday users generally take a huge variety ofnotes, ranging from how to fix a given piece ofequipment to what they need to do in the next10 minutes. Researchers have noted infor-mally that in some domains, access to theinterface in under two seconds results in sig-nificantly more use than systems that requirea longer delay.13 This “two second rule” pro-vides an initial heuristic for defining theacceptable delay in accessing the interface forwearable systems.

Perception and context. A wearable can alsoretrieve the context in which notes were taken;such context is useful for indexing. Given a

user and a set of goals, context is defined asthose environmental features not createdexplicitly for input to the system.14 Lammingshowed how context could be effectively usedwith Xerox’s PARCTabs.15 His Forget-me-notsystem sensed particular office activities, suchas personal location, encounters with others,workstation activities, telephone calls, fileexchanges, and printing. It demonstrated theuse of complex queries to augment memory.For example, suppose the user remembers thathe discussed a business plan with a colleaguea week ago and was interrupted by a telephonecall. However, he does not remember whocalled or why it was important enough tointerrupt the conversation. Forget-me-not letsthe user search these events from the past and,from the context of the interruption, providethe user with the name of the person whocalled and a record of the user’s actions after-ward, which might illuminate the importanceof the call.

More sophisticated on-body perception sys-tems can capture a more complete sense ofcontext. Through sensors placed near wherethe user’s natural senses are located, the wear-able receives a first-person view of the wear-er’s interactions with others and the world.Recently, computer vision researchers havebegun experimenting with wearable comput-ers. Clarkson uses unsupervised learning onwearable-based audio and low-resolutionvideo to identify interesting events during theuser’s everyday life.16 These characterizationscan act as additional features when lookingfor particular pieces of information. Althoughnot wearable, Moore has demonstrated avision system that identifies objects, such as abook or keyboard, by tracking how the usermanipulates these objects.17 Schiele and Starn-er have developed wearable gesture and objectrecognizers.18 Although these are research sys-tems, they demonstrate how a wearable com-puter’s first-person perspective could capturethe day-to-day experiences of wearers withoutburdening them with having to specify explic-itly what to capture.

Just-in-time information. Capturing informa-tion does not have much meaning unless it isindexed and retrievable in a timely fashion. Amajor question in the wearable computingcommunity is how to present information to

62

WEARABLE COMPUTING

IEEE MICRO

a wearer. Researchers at Carnegie Mellon Uni-versity have reported several successes usingwearable computers for inspection, mainte-nance, and repair, especially in industrial andmilitary domains.19 However, in her doctoralthesis at the Georgia Institute of Technology,Ockerman reports that her wearable task guid-ance system inhibits experienced pilots’ per-formance in inspecting their aircraft.20 Theapparent discrepancy is more than likely due tothe users’ relative experience levels with thetask and the authority they assign to the wear-able computer. When given a checklist, Ock-erman’s experienced users tend to over rely onit. To help prevent this problem, the wearableinterface should encourage the user to exercisejudgment and physically touch the equipment.Adapting the user interface based on activesensing of the inspected object might alleviatethis problem. Providing overviews of the task’spurpose and each step’s context in the check-list has been shown to improve performance.In short, with experienced users, wearablecomputers should provide formal structure yetencourage independent thought and adapta-tion of the interface to the situation.

While Ockerman’s task guidance systemsexpect explicit user interaction, such as whenthe wearer notes an observation or signals thata subtask is complete, another style of inter-face suggests pieces of information based oncurrent context. Rhodes describes these asjust-in-time information retrieval agents or

“software that proactively retrieves and pre-sents information based on a person’s localcontext in an accessible yet nonintrusive man-ner.”14 Creating these mobile, nonintrusivecomputer interfaces is a distinct challenge andresearch focus.

A particular just-in-time information retrievalagent of interest is the Remembrance Agent.14

Although information retrieval systems exist formany applications, almost all of these systemsconcentrate on written text and query-basedinformation retrieval on demand. For example,they can answer questions such as “When is thatconference’s paper deadline?” or “Who’s anexpert on this particular algorithm?” However,they do not help the user remember to ask aquestion or what question to ask. The Remem-brance Agent addresses these problems. Its asso-ciative forms of recall might remind a user thatan important conference exists, or that there arereferences to a particular algorithm the usermight have missed.

The Remembrance Agent, shown in Figure 2, performs this task by continuouslydisplaying relevant information to an indi-vidual user in his current context. In its cur-rent form, the Remembrance Agent softwareuses text that a user is reading or writing as aquery into its relevance engine. At the bottomof the user’s screen, one-line summaries ofpotentially useful documents (e-mails, papers,books, notes, and so on) appear as a list inorder of potential relevance. If a document is

63JULY–AUGUST 2001

Figure 2. Remembrance Agent. Every 10 seconds, the current text in the top buffer forms anautomatic query that returns the one-line summaries of potentially interesting documents inthe bottom buffer.

indeed interesting, this summary alone maybe a sufficient reminder for the user of thedocument’s existence. Alternatively, if thesummary intrigues the user, he can view a fullannotation with a simple sequence of key-strokes.

Although early wearable prototypes of theRemembrance Agent attempted to coupleaudio and visual perception to informationretrieval, the resulting systems were not prac-tical for everyday use.18 However, recentadvances in desktop-based systems that rec-ognize gesture, faces, and speech indicate thatthese types of mobile systems could becomefeasible in the near future.

Facilitating collaborationMany occupations are inherently mobile,

with spontaneous meetings often happeningaway from any desktop. In such cases, wear-able computers might aid communication andcollaboration. Kraut et al. found that remoteassistance significantly improves task perfor-mance in wearable applications.21 In this study,a technician wore a head-mounted camera anddisplay combination to enable a remote expertto see the technician’s workplace and to dis-play appropriate manual pages. Kortuem iden-tified several collaborative primitive functionsthat wearable computers can exhibit, includingremote awareness, presence, presentation,pointing, and manipulation. Kortuem exper-imented with body-stabilized spatial informa-tion displays to support 3D collaboration.Although the state of the current hardwareoften interferes with exploring these princi-ples, an intuitive collaborative interface thatdoes not overwhelm the wearer’s attentionseems feasible.22

An intriguing collaboration idea exploitscommunities of wearable agents acting onbehalf of their owners.23 These agents negoti-ate for cooperation during physical encoun-

ters between wearers with selfish, and possiblyconflicting, goals. For example, if two packagedelivery drivers encounter each other at a drop-off point, their wearables can compare deliveryschedules and determine if, by exchangingpackages, the drivers can minimize theirroutes. Fickas et al. simulated such negotia-tions in large-scale wearable communities andexplored the role of deception as well as meth-ods for building such communities.23

Tailoring augmented-reality systemsMany systems presented in this discussion

involve information interfaces that areportable and personal, and demand as littleuser attention as necessary. In some forms ofaugmented reality, however, the informationhas immediate bearing on a physical objectand its properties. In such cases, coupling thevirtual interface to the hand-eye coordinationand visual attention needed to interact withthe physical object is appropriate and evendesirable. Examples of this are augmentedrealities that provide sensory enhancement orthat help compensate for physical handicaps.

As stated earlier, augmented reality overlaysinformation on the physical world. For exam-ple, a doctor could wear a head-up display tosee the results of an X-ray overlaid on an actu-al patient. In such situations, the informationprovided by the head-up display, in this case,the skeleton, is not incongruous with thepatient’s body presented to the doctor’s unaid-ed eyes. In fact, the head-up display needs toaccurately track the patient with low latencyto maintain the doctor’s illusion of seeingthrough the patient. This visualization’s accu-racy and stability is especially important whenperforming surgery. During surgery, updat-ing the visualization in a timely manner is alsoimportant. Thus, coupling the visualizationto the surgeon’s actions is beneficial.

On the other end of the spectrum, some aug-mented realities might need only the loosestcoupling. For example, augmented realitiesmight trigger audio events based on which roomthe wearer visits or even go so far as to commu-nicate to the wearer primarily through ambientinterfaces. Between these two extremes lies acontinuum of interfaces for further research.

As mentioned earlier, one particularlyintriguing idea in augmented reality is toextend the Web to physical reality. Figure 3

64

WEARABLE COMPUTING

IEEE MICRO

Augmented reality overlays

information on the physical

world.

demonstrates an early implementation of sucha system using computer vision and head-updisplays.18 The user wears a combinationhead-up display and camera. Computer visionalgorithms continuously search the videoimages for unique tags that indicate an objecthas information associated with it. Comput-er graphics then overlay the user’s visual fieldto communicate this information. In the sys-tem shown in Figure 3, when the system firstlocates a tag, an arrow is rendered to indicatea hyperlink. If the user shows interest, the sys-tem displays the appropriate text labels. Fig-ure 3b shows the overlying text rotating inconjunction with the tag, demonstrating thatthe system can recover rotation with respectto the user’s head. If the user approaches theobject, the system displays 3D graphics ormovie sequences, as shown in Figure 3c.

In the past decade, several researchers havebegun addressing how to improve the sensingand human-computer interface problems ofaugmented-reality systems. The problemsassociated with a “real World Wide Web” arean area of active research, but a true field testcannot occur until the requisite equipmentbecomes more accessible and a sufficientlylarge population of users is supportable.

In many senses, this augmented-realityextension of the Web combines many prop-erties of the intellectual tools described earli-er. A wearer can attach annotations to physicalobjects, thus performing a form of context-dependent note taking. Letting others accessthese notes achieves a form of asynchronouscollaboration. Indeed, coupling notes to aphysical context or to a set of actions that trig-

ger them creates a form of just-in-time infor-mation retrieval agents. Augmented realitycould also assist synchronous collaborationbetween Web users by enabling shared visu-alization of file systems, design tools, andinformation searches.

Hardware and software engineeringIn creating wearable systems, many trade-

offs occur with respect to the challenges dis-cussed in this article. Balancing thesecharacteristics during design, runtime, andmaintenance is the domain of software andhardware engineering. Academic and indus-trial groups are beginning to organize collab-orations and workshops to explore whatsoftware and hardware engineering mean inthe wearable computing domain, but muchwork remains.

Most general-purpose wearable computersdescribed in the literature do not address thecomplicated design choices implicit in suchsystems. Devices like those discussed in thepacemakers (see Part 1) and cellular phonessidebars of this article, use special-purposewearable computers and related infrastructurehardware. Each design prioritizes one attributesignificantly more than others. Althoughprogress in hardware will enable more func-tionality in a wearable computer, the balancebetween privacy, power, networking, andinterface will be continually revisited witheach new development in the field.

Wearable computing pursues an interfaceideal in which the computer persists

and provides constant access to information

65JULY–AUGUST 2001

Figure 3. Augmenting reality with hypertext links. When a computer vision system first locates a tag, it renders an arrow ontop of the live video in the wearer’s head-up display; the arrow indicates a hyperlink (a). If the user shows interest by staring atthe object, the system displays appropriate text labels (b). If the user approaches the object, the system shows moviesequences or 3D graphics (c).

(a) (b) (c)

services, senses and models context, augmentsand mediates the user’s interactions with theenvironment, and interacts seamlessly with theuser. Much work remains. Perception on thebody is a relatively new endeavor since appro-priate sensors are just now becoming available.While much study has centered on low-atten-tion interfaces for automobiles and aircraft, lit-tle has been done for users of personal head-updisplays. Most ambitiously, wearable comput-ing will enable and encourage development ofintelligent agents that model the user’s minute-by-minute behavior in an effort to predictfuture needs and goals. In the words of Lick-lider, if even partly successful, “the resultingpartnership will think as no human brain hasever thought.” MICRO

AcknowledgmentsThanks to the Spring 2001 Mobile and

Ubiquitous Computing class at the GeorgiaInstitute of Technology, which field-testedmany of this article’s examples and its moreesoteric ideas. Thanks are also due to theGeorgia Institute of Technology ContextualComputing Group, the Massachusetts Insti-tute of Technology Wearable Computing Pro-ject, and the members of wearables list. Inparticular, Brad Rhodes, Lenny Foner, RobMelby, Kent Lyons, Dan Ashbrook, John Nis-sen, and Chip Maguire contributed their timeto help guide structure and provide details andreferences.

This article is dedicated to the memory ofClaude Shannon, hacker par excellence.

References1. T. Kanter, G. Maguire, and T. Smith,

“Rethinking Wireless Internet with SmartMedia,” Proc. Nordic Radio Symp. (NRS 01),Nordic Radio Soc., Sweden, 2001.

2. M. Satyanarayanan, “Fundamental Chal-lenges in Mobile Computing,” Proc. Symp.Principles of Distributed Computing, ACMPress, New York, 1996.

3. V. Bose, “The Impact of Software Radio onWireless Networking,” Mobile Computingand Comm. Rev., vol. 3, no. 1, Jan. 1999, pp. 30-37.

4. P. Carvey, “Technology for the WirelessInterconnection of Wearable Personal Elec-tronic Accessories,” VLSI Signal ProcessingIX, IEEE Press, Piscataway, N.J., 1996, pp.13-22.

5. T. Zimmerman, Personal Area Networks(PAN): Near-Field Intra-Body Communica-tion, master’s thesis, Media Laboratory,Massachusetts Inst. Technology, Cam-bridge, Mass., 1995.

6. E. Post et al., “E-broidery: Design and Fabri-cation of Textile-Based Computing,” IBMSystems J., vol. 39, no. 3, 2000, pp. 840-850.

7. R. Hull, P. Neaves, and J. Bedford-Roberts,“Towards Situated Computing,” Proc. Intl.Symp. Wearable Computers, IEEE CS Press,Los Alamitos, Calif., 1997, pp. 146-153.

8. L. Foner, Political Artifacts and Personal Pri-vacy: The Yenta Multi-Agent DistributedMatchmaking System, doctoral dissertation,Media Laboratory, Massachusetts Inst.Technology, 1999.

9. R. Want et al. “An Overview of the PARCTabUbiquitous Computing Experiment, IEEEPersonal Communications, vol. 2, no. 6, Dec.1995, pp. 28-33.

10. Proc. Int’l Symp. Wearable Computers(ISWC), IEEE CS Press, Los Alamitos, Calif.,1997-2001.

11. F. Gemperle et al., “Design for Wearability,”Int’l. Symp. Wearable Computers, IEEE CSPress, Los Alamitos, Calif., 1998, pp. 116-122.

12. P. Dillon, “The Next Small Thing,” Fast Com-pany, vol. 15, June 1998, pp. 93-113.

13. B. Shneiderman, Designing the User Inter-face, 3rd ed., Addison-Wesley, Reading,Mass.,1997, pp. 358-362.

14. B. Rhodes, Just-In-Time InformationRetrieval, doctoral dissertation, Media Lab-oratory, Massachusetts Inst. Technology,Cambridge, Mass., 2000.

15. M. Lamming and M. Flynn, Forget-Me-Not:Intimate Computing in Support of HumanMemory, tech. report RXRC TR 94-103,Rank Xerox Research Center, Cambridge,UK, 1993.

16. B. Clarkson and A. Pentland, “RecognizingUser’s Context from Wearable Sensors:Baseline System,” tech.report TR-519,

66

WEARABLE COMPUTING

IEEE MICRO

Further resources

•http://www.cc.gatech.edu/ccg•http://www.media.mit.edu/wearables•http://www.charmed.com•For a complete bibliography for this article, visit http://computer.org/micro.

Media Laboratory, Massachusetts Inst.Technology, Cambridge, Mass., 2000.

17. D. Moore, Vision-Based Recognition ofActions Using Context, doctoral dissertation,Dept. Electrical Engineering, Georgia Tech,Atlanta, 2000.

18. T. Starner and et al., “Augmented RealityThrough Wearable Computing,” Presence,vol. 6, no. 4, Winter, 1997, pp. 386-398.

19. A. Smailagic and D. Siewiorek, “The CMUMobile Computers: A New Generation ofComputer Systems,” Proc. IEEE Int’l. Com-puter Conf. (COMPCON), IEEE CS Press, LosAlamitos, Calif., 1994, pp. 467-473.

20. J. Ockerman, Preventing Operator Over-Reliance on Task Guidance Systems, doctor-al dissertation, Dept. Industrial and SystemsEngineering, Georgia Tech, Atlanta, 2000.

21. R.E. Kraut, M.D. Miller, and J. Siegel, “Col-laboration in Performance of Physical Tasks:Effects on Outcomes and Communication.”Proc. ACM Conference on Computer Sup-ported Cooperative Work, ACM Press, NewYork, 1996, pp. 57-66.

22. M. Billinghurst et al., “A Wearable SpatialConferencing Space,” Proc. Intl. Symp.Wearable Computers, IEEE CS Press, LosAlamitos, Calif., 1998, pp. 76-83.

23. S. Fickas et al., “When Cyborgs Meet: Build-ing Communities of Cooperating WearableAgents,” Proc. Intl. Symp. Wearable Com-puters, IEEE CS Press, Los Alamitos, Calif.,1999, 124-132.

Thad Starner is an assistant professor at theGeorgia Institute of Technology. His researchinterests include intelligent agents, wearablecomputing, computer vision, and patternrecognition. Starner has a PhD from theMassachusetts Institute of Technology MediaLaboratory and has been wearing general-pur-pose computers as part of his daily life foreight years. He is a member of the IEEE Com-puter Society, the ACM, and the AAAS.

Direct questions and comments about thisarticle to Thad Starner, Georgia Institute ofTechnology, College of Computing, 801Atlantic Dr., Atlanta, GA 30332-0280;thad@cc.gatech.edu.

67JULY–AUGUST 2001

Next-generation coursesfor the

next

generation

of computer

professionals

Influence what our

students learn. Review the

latest draft of Computing Curricula 2001.

http://computer.org/education/curricula2001

Prepared by the

IEEE Computer Society/ACM joint task force

on Computing Curricula 2001