happy healthy home - semantic scholar...happy healthy home ellen yi-luen do a,1 and brian d. jones b...

16
Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute b Interactive Media Technology Center, Aware Home Research Initiative Georgia Institute of Technology, USA Abstract. This chapter introduces the idea of a smart living environment in which the home tracks and supports happy healthy living for the residents. We start with the introduction of the concepts of wellness and the Aware Home Research Initiative at Georgia Institute of Technology. We then present several interesting projects to illustrate the approaches and implementations toward health, awareness and entertainment and conclude with some reflections and discuss possible future research directions. Keywords. Ubiquitous Computing, Ambient Intelligence, Aware Home Research Initiative 1. Introduction – Toward a Smart Living Environment It’s a spring day in the year 2050. As you finish your breakfast your table displays a picture of the medicine and vitamin to take after the meal. When you go to the kitchen to get yourself a glass of water, you notice the handle of the kettle is red to remind you that the water is hot and ready for tea. Glancing out the window, you see fresh snow accumulated on the ground over night and feel thankful that the heat was automatically turned up while you were asleep. Meanwhile the living room starts playing the music for your exercise routine. You may be sixty, eighty or a hundred years old now. You are happy and healthy. You are aging gracefully and living alone with your life style partner, the Aware Home. We have entered the age of ubiquitous/pervasive/ambient computing. Increasingly we are seeing computing and information processing diffused into everyday life, and become invisible. The question is, can our homes help us stay active, alive and vital? This vision is already becoming a reality. Nowadays there is a wide variety of network sensors and computers that can be installed in a home. A computer is no longer a desktop machine, instead, it is becoming part of the room, part of the building, and constantly present. Can we imagine the world with things that think, spaces that sense, and places that play? Can we employ computing creatively to enhance our lives? Can we use technological innovation to unlock and augment human potential? Design and Human-Computer Interaction are crucial components of information technologies in daily life and they color our experience of computation and communication. 1 Corresponding Author: Ellen Yi-Luen Do, Georgia Institute of Technology, Atlanta, GA 30332, USA; E-mail: <[email protected]> The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Upload: others

Post on 19-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b

a College of Architecture and School of Interactive Computing, Health Systems Institute b Interactive Media Technology Center, Aware Home Research Initiative

Georgia Institute of Technology, USA

Abstract. This chapter introduces the idea of a smart living environment in which the home tracks and supports happy healthy living for the residents. We start with the introduction of the concepts of wellness and the Aware Home Research Initiative at Georgia Institute of Technology. We then present several interesting projects to illustrate the approaches and implementations toward health, awareness and entertainment and conclude with some reflections and discuss possible future research directions.

Keywords. Ubiquitous Computing, Ambient Intelligence, Aware Home Research Initiative

1. Introduction – Toward a Smart Living Environment

It’s a spring day in the year 2050. As you finish your breakfast your table displays a picture of the medicine and vitamin to take after the meal. When you go to the kitchen to get yourself a glass of water, you notice the handle of the kettle is red to remind you that the water is hot and ready for tea. Glancing out the window, you see fresh snow accumulated on the ground over night and feel thankful that the heat was automatically turned up while you were asleep. Meanwhile the living room starts playing the music for your exercise routine. You may be sixty, eighty or a hundred years old now. You are happy and healthy. You are aging gracefully and living alone with your life style partner, the Aware Home.

We have entered the age of ubiquitous/pervasive/ambient computing. Increasingly

we are seeing computing and information processing diffused into everyday life, and become invisible. The question is, can our homes help us stay active, alive and vital?

This vision is already becoming a reality. Nowadays there is a wide variety of

network sensors and computers that can be installed in a home. A computer is no longer a desktop machine, instead, it is becoming part of the room, part of the building, and constantly present. Can we imagine the world with things that think, spaces that sense, and places that play? Can we employ computing creatively to enhance our lives? Can we use technological innovation to unlock and augment human potential? Design and Human-Computer Interaction are crucial components of information technologies in daily life and they color our experience of computation and communication.

1 Corresponding Author: Ellen Yi-Luen Do, Georgia Institute of Technology, Atlanta, GA 30332, USA;

E-mail: <[email protected]>

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 2: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

Computing that is aware of what people are doing and what they want would significantly impact our life.

At Georgia Tech, the Aware Home Research Initiative (AHRI), an interdisciplinary group of researchers, is exploring emerging technologies and services based in the home. Since 1998, faculty and students involved in the initiative have focused their efforts on solving problems of significant social and economic impact, particularly in the areas of wellness and health. Core to this research and teaching is an understanding of individual needs; how individuals perceive and interact with different devices; and how differently individuals accept these technologies and devices in their homes and everyday lives. One of the facilities used in this research effort is The “Aware Home,” a two-story single-family house that serves as a living laboratory for ubiquitous computing research, built with sensing infrastructure that is capable of knowing information about itself and the whereabouts and activities of its inhabitants [16]. In this chapter, we focus on the idea of ambient assistive living, discuss and reflect on the development and deployment of these technologies in the context of a Happy Healthy Home.

2. Be Well – Happy Healthy Living at Home

The idea of wellness is related to human potential. What is wellness? Wellness is the presence of wellbeing. Wellness is about being active, alive and vital. It concerns individuals, communities and our environments. Wellness has multiple dimensions. A popular notion of the six dimensions of wellness consists of: physical, emotional, occupational, social, intellectual and spiritual [11]. The physical dimension concerns diet, nutrition and physical activities. The emotional dimension recognizes awareness and acceptance of feelings and behaviors. The occupational dimension realizes achievement and enrichment through work. The social dimension encourages contribution to the environment, community and the world. The intellectual dimension seeks creative and stimulating activities. The spiritual dimension recognizes the search for values, meaning and purpose in life.

Achieving wellness is a Grand Challenge. We are concerned about the quality of

life for ourselves and for our society. As human beings we want to develop and cultivate our untapped potential for a happy, healthy, creative and fulfilling life. Technological innovation may be just the key to unlock human potential for the Holy Grail of wellness.

The Georgia Institute of Technology aspires to the commitment to improving the

human condition through advanced science and technology. As a top technological university with genuine concerns about human conditions, Georgia Tech has created a culture of possibilities fostering the growth of interdisciplinary research centers such as the Aware Home Research Initiative, GVU Center, the Health Systems Institute and the Center for Music Technology.

The research and innovating projects produced by these centers are too numerous

to list them all. To briefly demonstrate the spirit and the scope of the types of the research efforts, the rest of this paper will focus on three areas of interest – health,

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 3: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

awareness, and entertainment, each illustrated by a couple projects, to form part of the picture of the theme of “happy, healthy living at home with ambient intelligence.”

3. Health: for the old and the young, with capturing, recording and notification

Besides supporting healthcare services and capabilities ranging from surgery planning to diagnosing and treating chronic disease, many opportunities exist to help people live a more carefree and independent life.

3.1. Digital Family Portrait

The Digital Family Portrait [19, 24] helps family members at a distance to “keep an eye” out for their family members in a casual, lightweight manner. Figure 1 shows a picture of Grandma displayed together with other family portraits. Displayed on a LCD monitor, the picture is surrounded by the images of the butterflies that change daily, reflecting some portion of Grandma’s activities recorded by the non-obtrusive sensors installed in the house. In the Aware Home, the Digital Family Portrait uses motion sensors to collect the activity data on the first floor of the home, while other installations have used strain sensors on the joists of the house to get activity information. A server in the home collects the information and serves it up as a portal from which the client systems retrieve the data.

Figure 1. Digital Family Portrait of Grandma’s activities as butterfly icons

An earlier version of the picture frame provides many icons for users to understand

the activity level of the individual in the sensed home. Study participants found the interface too complex to understand what was going on. Thus, as a result, four levels of information (two representing low and high average activity, one low activity and one high activity) were portrayed in the size of the butterflies, based on the typical activity level of the individual on that day of the week over the last month. Multiple days were included on the screen as this feature helped users to be able to compare different days

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 4: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

(e.g., today vs. the past days) on the frame. A touchscreen option allows the user to “scrub” or replay the days’ activity for a better understanding of when and where the activity occurred in the home.

3.2. Cook’s Collage

Although for privacy reasons, one may appreciate not being constantly under the surveillance camera, there are times that a recording and monitoring service may come in handy. Take Cook’s Collage [26] for example, a capture system installed under the kitchen cabinets, that provides a visual summary of recent cooking activity can serve as a memory aid. Imagine you are in the middle of making a cake and you stop to answer the phone. When you get back, you wonder if you have already put in either three or four cups of flour in your mixing bowl. The display shows visual snapshots (from two or three mounted webcams under the cabinets providing different perspectives) arranged as a series of comic strip panels for you to touch and review past events. Figure 2 shows a person touching Cook's Collage display to check to see how many cups of flour he already put into the bowl.

The system as shown in Figure 2 was part of a user study conducted in the Aware

Home. The researcher used a Wizard of Oz approach, with an application created to simulate a computer vision system that can quickly pick out images (e.g., when an object being touched or moved) that would best indicate the latest steps in the recipe and display those on the kitchen display.

Figure 2. Cook’s Collage captures cooking activities for later review

3.3. Pervasive Remote Asthma Monitoring

Living well at home is a concern for people of all ages. Children with asthma enjoy playing outdoors as much as others. Parents are concerned about their children’s wellbeing even when they are out of sight. With the Pervasive Remote Asthma Monitoring [4] added to a cell phone or a cute animal pendant worn on their necks,

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 5: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

children’s coughing and wheezing can be recorded no matter where they are located. This monitoring could alert caregivers remotely and enable them to quickly supply medical attention in the event of an emergency. Figure 3 shows a diagram of the system. With embedded electronics and software, the voice can be processed through digital signal processing, sent through the network and incorporated into patients’ electronic medical records.

Figure 3. Pervasive Asthma Monitoring helps detect patients’ coughing and wheezing patterns

3.4. Personal Robotic Assistant

If you are disabled or confined to a wheelchair at home, Personal Robotic Assistant [15] can turn your world into a clickable interface with a laser pointer. If you want your robotic assistant to pick up a toy or a remote control on the floor and deliver it to your friend on the other side of the room, all you need to do is to point the green laser at the object first, and then at the person. Your robotic helping hand will then follow your command and deliver the object. Figure 4 illustrates how the robot would perceive objects and a person, instructed using a green laser pointer.

In order to complete this task, the robot must perform a number of computations in

parallel. It must be capable of detecting the laser pointer when positioned in any orientation. This was accomplished with a camera oriented vertically and monitoring a 360-degree mirror, allowing it to see the laser pointer at any location. Then, the robot must determine the path to reach the object safely and identify the object in order to determine the best manner to retrieve that object (where to grip it, how much force to apply, etc.). Once the object is retrieved, it will turn toward the individual (a social expectation by participants) and wait for a location to which it should deliver the object (human or table top). While the price tag for a full-scale robot is quite hefty, this research has lead to smaller, more affordable versions that can really make a difference in the independence of people who are limited in their mobility.

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 6: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

Figure 4. Instructing the Personal Robotic Assistant by clicking on highlighted objects with a laser pointer

3.5. Technology Coach

The in-home personal assistant does not have to look like a robot. It could be an invisible fairy or a guardian angel that watches over your shoulder and guides you through unfamiliar or difficult tasks. For example, the Technology Coach [22] provides feedback to assist older adults in using medical devices (such as a blood glucose meter) for the first time, or when the system detects the user did not follow the procedure for using the device. Figure 5 shows that the Technology Coach using a computer vision system to recognize user actions and to recognize potential errors and provide appropriate guidance.

Figure 5. Technology Coach “watches” user actions to provide feedback on home medical device operation

3.6. ClockReader

Worried about your memory and cognitive function but don’t want to (or can not) get an appointment with the doctors? Want to self-administer cognitive impairment test at

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 7: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

home just like how you could monitor weight with a scale or blood sugar level with a glucose meter? The Clock Drawing Test is one of the simplest, but most commonly used screening tools to detect Alzheimer’s disease and related disorders [8]. The task is to draw a clock with a pencil on a given sheet of paper and set a specific time (e.g., 11:10, 1:45). Neurologists or neuropsychologists then spend hours analyzing and scoring the test for diagnosis and treatment. The ClockReader system [17] provides automatic recording and scoring of the Clock Drawing Test on a tablet PC so you could perform the test at the comfort of your own home without the need for the visit of a specialist or a trip to the hospital. Figure 6 left shows the screen of the ClockReader system with the result of the recognition shown in small window. Figure 6 right shows the interaction of drawing a clock using a stylus on a tablet PC.

Figure 6. ClockReader interface and scoring (left) with a stylus-and-tablet (right)

There are many scoring systems with differing emphases on visual-spatial, executive, quantitative, and qualitative issues [14]. Missing or extra numbers, reversal of the minute and clock hand proportion or misplaced clock hands often appear in clock drawings from people with cognitive impairment. The ClockReader system has three main components: data collection, sketch recognition, and data analysis. Each stroke drawn on a Tablet PC is given a bounding box. Every coordinate of the cusps and intersections of each stroke are stored in the memory for character recognition processing. The recognition engine takes into consideration that some characters have than one stroke and consequently more than one bounding box. The system then automatically analyzes the drawing and reports the result based on the scoring criteria.

4. Awareness: of self & the environment, for physical, emotional, social & spiritual wellness

Information technology can serve as a medium to help us connect with ourselves, with other people, and with our environment. Applications in a hospital setting may be designed to help relieve anxiety of the patients and their family members. Applications in a household that inform people about energy consumption patterns may help residents monitor their energy use and modify their behavior patterns. Application in a prayer practice may even help people connect with their spiritual needs. Custom home sensor networks using multiple cameras or existing residential power line could

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 8: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

provide location and event detections to support a variety of applications in our daily activities. Let’s look at some examples here.

4.1. Patient Interactive Communication & Learning System

Being sick is no fun. It is uncomfortable and stressful. While confined in a hospital bed, a patient may feel overwhelmed and alone. Being able to learn about one’s own illness and medical procedures, or to communicate with clinicians, friends and families may help alleviate the stress. The Patient Interactive Communication and Learning System (PILS) simplifies the patient's hospital communication experience by combining communication tools into one easy-to-use system mounted on the patient bed [7]. As shown in Figure 7, PILS has a one-touch call button that connects to nurse’s videophone. Family video conferencing, educational video, entertainment media, and vital signs information are all just “one-touch away.” The entertainment system also doubles as a display for medical records so that the clinicians could use it to explain medical conditions and treatment options to the patients and concerned family members.

Figure 7. Patient Interactive Communication & Learning System mounted on the bedside connects patients,

care givers & family members at home, monitors vital signs and provides learning opportunities

4.2. Energy Puppet

Or consider energy awareness – we are all concerned about climate change and our impact on the environment and the world. Would you like to be aware of the energy consumption at your home and change how you use energy? Would you adopt an electronic pet for this purpose? The Energy Puppet [1] is an ambient display device that provides peripheral awareness of energy consumption for individual home appliances. The display produces different “petlike” behavioral reactions according to energy use patterns of the appliances to give homeowners an indication of their energy consumption. The puppet would raise its “arms” in victory to display normal

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 9: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

consumption rate (see Figure 8), or its “eyes” would change color to red and “roar” to warn the homeowners when the specific appliance reaches high consumption rates.

Figure 8. Energy Puppet raises its arm to celebrate low energy consumption in the home.

4.3. Power Line Positioning

Besides energy monitoring, many home automation, entertainment and healthcare applications with activity sensing require large numbers of sensors or extensive installation procedures. Wouldn’t it be nice if we could have low-cost and easy-to-deploy location and sensing technologies at home by simply leveraging the existing infrastructure? Power Line Positioning [20] is an inexpensive technique that uses fingerprinting of multiple tones transmitted along the power line for location detection within the home space, offering an lower-cost alternative to location aware systems.

The system is based on a popular wire-finding technique used by electricians to

locate or trace hidden wires behind a wall or underground. The diagram in Figure 9 illustrates the components of the positioning system. A custom wireless tag can detect signals generated by plugging in two signal generator modules into electrical outlets at extreme ends of a house. The power lines essentially act like antennas, but since the signal attenuates over the length of all the power lines in the home, the strength of the signal at every point in the home is the contribution of all the power lines in the vicinity. Thus, after a look up list of signal signatures is generated for the home, the sensor tag is able to sense and transmit the values to determine its location within the home. The detection was able to determine location at better than a 1 meter x 1 meter resolution. With this resolution, the system can be used for many applications like “where are my keys?” or assist in determining the location of an individual in the home to help in understanding the context of their needs.

A sister project called Power Line Event Detection uses a single plug-in sensor to

detect a variety of electrical events throughout the home using machine learning techniques to recognize electrically noisy events, such as switching on or off a particular light switch, a television set, or an electric stove and classify various electrical events with accuracies ranging from 85-90% [21]. This technology would be very helpful for low-cost monitoring of activity in the home that may have health implications. Further research is being conducted into the use of water line sensing [11]

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 10: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

using a pressure sensor to determine which water tap or appliance is currently using water. This too could be used to monitor certain aspects of health or activities of daily living.

Figure 9. Power Line Positioning system: two signal-generating modules (top right) at extreme ends of a house (left), the location tag with a receiver and antenna connected to a handheld computer (bottom right)

4.4. Vis-a-Viz

An approach to understand human activity in natural environments over long periods of time is to use camera and visualization tools to facilitate longitudinal in-situ behavioral analysis. Vis-a-Viz is a visualization tool (see Figure 10) that interprets data from the overhead cameras, computes motion and blob tracking, aggregates over space and time and provides interactive navigation of the results [23]. By having interpretation and visualization in a spatial or geographical information system, where the floor plan of the inhabited space serves as the geography and time is stacked as layers on top of the plane, the tool helps contextualize video analysis in the space and time to facilitate rapid overview, filtering and zooming, and details on demand of large volumes of video data, that helps identify potentially sparse target behaviors.

Built in Google Sketchup, the system allows the researcher to interactively

manipulate the 3D volume (2D floorplan on x and y axis and time on z) to find video of interest. Being able to monitor and analyze behaviors in space could contribute to understanding of the wellness of the inhabitants. For example, Vis-a-Viz data and analysis could be the back end for Grandma’s Digital Family Portrait.

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 11: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

Figure 10. Vis-a-Viz system processes data from overhead camera and turns it into blobs, activity data and

activity map for further analysis.

4.5. Sun Dial

Sun Dial [28] is an application that supports Muslims’ prayer practices by showing the natural environmental cues of the movement of the sun through the sky pictorially in the screen of a mobile phone. The ritual involves several cycles and prayer takes place during a “window of opportunity,” instead of an exact time. Therefore, an alarm clock function is not appropriate. With a simulated sun movement through the sky on a cell phone, even people in an interior space with no view to the sky can perceive the different phases of the day for prayer opportunities. This project provides an example of how prayer as an activity can be supported with technology. Figure 11 shows the sketch used in the study with the Muslim community and the resulting screen shot for Sun Dial showing a mosque in profile and the sun locations in the blue sky background.

Figure 11. Sun Dial provides environmental cues on a cell phone for prayer opportunities

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 12: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

5. Entertainment – video, music and technology

Watching TV or listening to stereo is a favorite passtime for many people. Finding the remote control can be a chore. You can’t quite remember where you put it. You want to change the channel or turn the volume up but you just can’t find the remote. Some remotes may be too complicated and difficult to use. Or you may be listening to music when the phone rang and you just wanted to wave at your music player to turn it off.

5.1. Gesture Pendant

The Gesture Pendant project [25] allows control of ordinary household devices, literally, with the wave of a hand (see Figure 12). Acting as an input device, the pendant enables a number of applications that can use a gesture in place of the remote control. By making gestures in front of the pendant you could control anything from a home theater system, to lighting, to the kitchen sink. In order to detect gestures, a wireless camera is embedded in the small pendant you wear and infrared LEDs illuminate objects (hands) in front. Researchers also investigated the use of the system to analyze user movements and detect loss of motor skill or tremors in the hand that might indicate an illness or problems with medication. Other applications include monitoring of regular eating or activity patterns for detection of health conditions or enabling elderly and disabled people in achieving greater independence in their homes.

Figure 12. Waving a finger in front of the Gesture Pendant could control a device at home.

5.2. Videotator

Interested in editing and annotating video for personal diary or to send a commentary along with a video program to your friends? With Videotater you can play, cut segments of digital video and tag them by using a pen on a Tablet PC [6]. The tool provides efficient and intuitive navigation, selection, segmentation, and tagging of digital video. It automatically segments the video into meaningful categories (i.e., by characters or events) and signals to the user where appropriate segment boundaries should be placed. It also allows rapid review and refinement of manually or

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 13: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

automatically generated segments, all with simple pen gestures. Figure 13 shows the editing interface of videotater.

Figure 13. Videotater supports video segmentation and tagging with a pen-based interface

5.3. Music

Music is an important part of life. We hear it everywhere: in elevators, vehicles, concert halls and shopping malls. Music is an art form that helps us express ourselves, and our feelings. Music is fun. An upbeat happy tune can cheer you up. A soft melody can soothe you. Appreciation of music is more than just entertainment. It also has social, cultural, and emotional implications. Music helps promote wellness, manage stress, alleviate pain, express feelings and improve communication [2, 3]. Hand drums have been used to help Alzheimer's patients improve their short-term memory and increase social interaction, help autistic children increase their attention spans, and to aid Parkinson's patients and stroke victims to regain their movement control of or increase their gaits [5, 9].

5.4. Haile the Robotic Percussionist

Imagine finding a musician who can jam with a group of players and produce inspiring rhythms in various varying speed and intensity that go beyond human player’s ability? Haile is an interactive robotic percussionist that can listen to live players, analyze their music in real-time, and use the results of the analysis to play back in an improvisational manner [27]. Besides being able to imitate human musicians, Haile can sense, analyze, and react to perform with acoustic diversity and dynamic range, and with varying velocities between the two arms that are difficult for human players. The experience of playing with an interactive robotic percussionist may facilitate a musical experience that is inspiring and encourages novel expressions and interactions or serves as effective music therapy [2, 3, 5, 9]. Figure 14 shows a drumming session with the interactive robotic percussionist.

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 14: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

Figure 14. Drumming with an interactive robotic percussionist

5.5. Piano Touch

Too busy and can’t find time to practice piano? How about wearing a glove that can help you learn to play music while you’re at your desk or on the move. The Piano Touch [12] project provides a new way for people to learn to play the piano. Wirelessly synchronized with an iPod, cell phone or other music playing device, the Piano Touch is a light-weight glove outfitted with little vibration motors to cue the musicians about which finger they need use to play the next note. Figure 15 shows a converted golf glove with vibration electronics playing on a lighted keyboard. A pilot study shows that students learned the songs that they were practicing with the Piano Touch glove with fewer fingering mistakes than the songs that they were practicing without the glove. Extending Piano Touch’s passive haptic learning, we are currently working on making Mobile Music Touch an engaging, pervasive hand rehabilitation aid for patients who suffer loss of functionality of their hands [13, 18].

Figure 15. Piano Touch is a lightweight glove that cues which fingers to play.

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 15: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

6. May You be Happy and Healthy at Home!

In this chapter we have covered topics of human potential and the dimensions of wellness and presented a holistic view of ambient assistive living. We have seen technological innovation projects addressing different aspects of wellbeing in our everyday lives - about health, awareness and entertainment. Researchers in these projects came from the disciplines of computer science, electrical engineering, human-computer interaction, health systems, industrial engineering, digital media, architecture, industrial design, and music. These projects would not have been possible without the culture of interdisciplinary collaboration, the synthesis of research across many domains, and the aspiration to support fundamental human needs.

To quote from an old Irish Blessing here: “May the road rise up to meet you, may

the wind be always at your back. May the sun shine warm upon your face and the rain fall softly on your fields.” Extending ambient assistive technologies to understand and support human needs, our home, and the larger built environment could make happy healthy living a reality.

7. References

[1] S.M. Abdelmohsen and E Y-L Do (2008) “Energy Puppet: An Ambient Awareness Interface for Home Energy Consumption,” in SID 08, 7th International Workshop on Social Intelligence Design, Designing socially aware interactions, Dec 3-5, Universidad de Puerto Rico, San Juan, PR http://cdr.uprrp.edu/SID2008/default.htm

[2] AMTA (2008) American Music Therapy Association, http://www.musictherapy.org/ [3] D. Campbell (2000) The Mozart Effect for Children: Awakening Your Child's Mind, Health and

Creativity With Music, William Morrow Publishing [4] J. Chhabra, A. Asmi, L. Ragavan, (2007). Pervasive Asthma Monitoring, in Pediatric Center of the

Future class, taught by Do, E. YL., Zimring, C., Cowan, D., Lamb, G. Lindgren, J., Jones, J., http://www.hsi.gatech.edu/pedcenter

[5] M.M. Chiang (2008) Research on Music and Healing in Ethnomusicology and Music Therapy, Master Thesis, University of Maryland, College Park

[6] N. Diakopoulos, and I. Essa. (2006) Videotater, An Approach for Pen-Based Digital Video Segmentation and Tagging. Symposium on User Interface Software and Technology (UIST) pp. 221-224

[7] T. Fischer, T, Davis, C. Vargas (2006) Patient Interactive Communication and Learning System, http://cool.coa.gatech.edu/patientroom, the Patient Room of the Future class at Georgia Tech, taught by Do, E. YL, Zimring, C. Cowan, D., Lamb, G., Kahn, S., Mullick, A., Winegardend, C

[8] M. Freedman, L, Leach, E. Kaplan, G. Winocur, K. Shulman, D. C. Delis (2004) Clock Drawing: A Neuropsychological Analysis. Oxford University Press, USA.

[9] R. L. Friedman (2000) "The Healing Power of the Drum", White Cliffs Media, Inc. [10] J. E. Froehlich, E. Larson, T. Campbell, C. Haggerty, J. Fogarty, and S. N. Patel. (2009) HydroSense:

infrastructure-mediated single-point sensing of whole-home water activity. In Proceedings of the 11th international conference on Ubiquitous computing (Ubicomp '09). ACM, pp. 235-244. DOI=10.1145/1620545.1620581 http://doi.acm.org/10.1145/1620545.1620581

[11] B. Hettler, (1976) Six Dimensions of Wellness, National Wellness Institute (www.nwi.org), and http://www.hettler.com/sixdimen.htm

[12] K. Huang, E Y-L Do, T. Starner, (2008) “PianoTouch: A Wearable Haptic Piano Instruction System For Passive Learning of Piano Skills,” in ISWC 2008, 12th IEEE International Symposium on Wearable Computers, http://www.iswc.net/, pp 41-44, Sep 28 - Oct 1, Pittsburgh, Pennsylavania

[13] K. Huang, T. Starner, E. Do, G. Weiberg, D. Kohlsdorf, C. Ahlrichs, and R. Leibrandt. 2010. Mobile music touch: mobile tactile stimulation for passive learning. InProceedings of the 28th international conference on Human factors in computing systems (CHI '10). ACM, New York, NY, USA, 791-800. DOI=10.1145/1753326.1753443 http://doi.acm.org/10.1145/1753326.1753443

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands

Page 16: Happy Healthy Home - Semantic Scholar...Happy Healthy Home Ellen Yi-Luen DO a,1 and Brian D. JONES b a College of Architecture and School of Interactive Computing, Health Systems Institute

[14] E. Kaplan (1990) The process approach to neuropsychological assessment of psychiatric patients. Journal of Neuropsychiatry. 2, 72-97

[15] C. C. Kemp, A. Cressel, H. Nguyen, A. Trevor, and Z. Xu, (2008) A Point-and-Click Interface for the Real World: Laser Designation of Objects for Mobile Manipulation, 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI), 241-248

[16] C.D. Kidd, R. J. Orr, G.D. Abowd, C.G. Atkeson, I.A. Essa, B. MacIntyre, E. Mynatt, T.E. Starner and W. Newstetter. (1999) "The Aware Home: A Living Laboratory for Ubiquitous Computing Research," in the Proceedings of the Second International Workshop on Cooperative Buildings - CoBuild'99. Position paper, October 1999. http://awarehome.imtc.gatech.edu/publications

[17] H. Kim, Y. S. Cho, E. Y-L. Do, (2011) “Computational clock drawing analysis for cognitive impairment screening,” in TEI '11 Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction Pages: 297-300 doi>10.1145/1935701.1935768

[18] T. Markow, N. Ramakrishnan, K. Huang, T. Starner, M. Eicholtz, S. Garrett, H. Profita, A. Scarlata, C. Schooler, A. Tarun, and D. Backus, (2010) “Mobile Music Touch: Vibration Stimulus as a Possible Hand Rehabilitation Method,” in Proceedings of the 4th International Pervasive Health Conference, Munich, Germany, March 2010.

[19] E. D. Mynatt, J. Rowan, S. Craighill, and A. Jacobs (2001). "Digital family portraits: Providing peace of mind for extended family members." Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2001) Seattle, Washington: ACM Press, pp. 333-340

[20] S.N. Patel, K.N. Truong, and G.D. Abowd. (2006) PowerLine Positioning: A Practical Sub-Room-Level Indoor Location System for Domestic Use. Proceedings of Ubicomp 2006. Pp. 441-458

[21] S. N. Patel, T. Robertson, J.A. Kientz, M.S. Reynolds, G.D. Abowd (2007) At the Flick of a Switch: Detecting and Classifying Unique Electrical Events on the Residential Power Line. Proceedings of Ubicomp 2007, pp. 271-288

[22] W.A. Rogers, I. Essa & A. D. Fisk (2007). “Designing a technology coach”. Ergonomics in Design. A Publication of the Human Factors and Ergonomics Society, pp 17-23

[23] M. Romero, J. Summet, J. Stasko, G. Abowd, (2008) “Viz-A-Vis: Toward Visualizing Video through Computer Vision,” IEEE Transactions on Visualization and Computer Graphics, vol. 14, no. 6, pp. 1261-1268, Nov./Dec. 2008, doi:10.1109/TVCG.2008.185

[24] J.T. Rowan, (2005) Digital family portraits: support for aging in place, PhD Dissertation, Georgia Institute of Technology Atlanta, GA, USA ©2005, ISBN:0-542-43423-7

[25] T. Starner, J. Auxier, D. Ashbrook, M. Gandy (2000) The Gesture Pendant: A Self-illuminating, Wearable, Infrared Computer Vision System for Home Automation Control and Medical Monitoring. ISWC 2000: 87-94

[26] Q. Tran, G. Calcaterra, E. Mynatt (2007) "Using Memory Aid to Build Memory Independence." Proceedings of HCII: Human Computer Interaction International, 959-965

[27] G. Weinberg and S. Driscoll (2007) The interactive robotic percussionist: new developments in form, mechanics, perception and interaction design, Proceedings of the ACM/IEEE international conference on Human-robot interaction, Arlington, Virginia, USA, Pages: 97 - 104

[28] S.Wyche, K.E. Caine, B. Davison, M. Arteaga, R.E. Grinter, (2008) Sun Dial: Exploring Techno-Spiritual Design through a Mobile Islamic Call to Prayer Application, CHI extended abstract: 3411-3416

The Handbook of Ambient Assistive Technologies for Healthcare, Rehabilitation and Well-being J. Maitland (ed) IOS Press, Amsterdam, The Netherlands