A Human Caregiver Support System in Elderly Monitoring Facility

M. Anwar Hossain and Dewan Tanvir AhmedCollege of Computer and Information Sciences (CCIS)

King Saud UniversityRiyadh, KSA

{mahossain, dtahmed}@ksu.edu.sa

Abstract—The number of elderly population is increasingworldwide and often they need assistance in their daily ac-tivities. In many situations, these elders are placed in elderlycare facilities in order to receive continuous assistance fromthe human caregivers. The caregivers usually keep a watchfuleye on the elders and help them in their activities of dailyliving. However, study shows that the human caregivers oftensuffer from boredom for being engaged in monitoring theelderly, which also compromises the care and assistance neededfor the vulnerable elderly. In order to address this issue,we propose a human caregiver support system that aimsto comprehend elderly persons’ activities and decides whatservices to provide them in different situations and when tonotify the human caregiver about any incident that happensin the care facility. Our preliminary experiment shows thepotential of such system.

Keywords-elderly support system; human caregiver; healthand well-being; elderly monitoring facility;

I. INTRODUCTION

Worldwide elderly population is increasing, so is the needto support this group of population. Many elders sufferfrom physical and cognitive impairment, and remain insome elderly care facilities where they receive assistancefrom human caregivers. The caregivers continuously monitorthese elderly and help them in their activities of daily livingwhen needed.

It has been reported that the continuous monitoring ofelderly also puts burden on the human caregivers [1] andincrease their cognitive overload. This not only hampers thetimely assistance needed for the elderly, but also compro-mises the quality of assistance they receive. Therefore, it isimportant to develop effective tools and techniques for thehuman caregivers so that they can provide the much neededassistance to the elderly in monitoring their health andwell-being. However, developing such tools and techniquesis challenging since it involves understanding the currentsituation and activities of the elderly.

Existing work address this issue from different corners:deliver information and entertainment services to the elderpatients [2], [3]; provide robotic assistance to the elderly [4],[5]; offer reminding services for task completion [6] and soon. Although these works contribute significantly to elderlymonitoring domain, they lack the focus on minimizinghuman caregivers burden and boredom. Unlike these works,we aim to develop a human caregiver support system, which

will help them in effectively addressing the needs of theelderly.

The cornerstone of our approach is to develop a virtualsupport system for the human caregiver to assist them foreffectively monitoring the elderly. The system is able tocapture the context of the environment and elderly activities,determine on-the-fly what services to provide to the elderly,decide whether to invoke those services automatically, andintervene human caregiver for additional support or involveexternal parties when necessary. Our contribution in this pa-per is two-fold. First, we introduce human caregiver supportsystem in the context of elderly monitoring, which worksby considering the surrounding context and activities of theelderly people. Second, we illustrate the different modesof interaction of the human caregiver support system formonitoring effective the elderly and reducing the cognitiveload of the human caregiver.

The remainder of this paper is organized as follows. Wecomment on existing literature in Section II; The proposedhuman caregiver support system is illustrated in SectionIII. This is followed by the implementation highlights inSection IV and experimental results in Section V. The paperis concluded in Section VI with a glimpse of future workdirections.

II. RELATED WORK

We survey related research in this section. In particular,we investigate what tools and support services have beenproposed for monitoring the elderly and whether those toolsand services work in mediation with the human caregiver toreduce his/her cognitive load that is felt in monitoring theelderly.

The CMU Pittsburgh NurseBot project [7] is one endeavortowards developing robotic assistance for the elderly. Itfocused on robotic platform that was used for testing sev-eral functionalities such as intelligent reminding functions,tele-presence applications, surveillance and assistance forphysically impaired people. The robot in this project usedto observe elderly and learn their patterns of movement toprovide the required services.

Several other works also prompted for developing roboticassistance. Tamura et al. [4] described the use of enter-tainment robot that responds to spoken commands from

2012 IEEE International Conference on Multimedia and Expo Workshops

978-0-7695-4729-9/12 $26.00 © 2012 IEEE

DOI 10.1109/ICMEW.2012.82

435

�������

��������

������

����

���������

�������

������

��������

����������

�����

����

��������

��������

��������

Figure 1. High-level task and interaction flow of the human caregiver support system

the people with severe dementia. The authors convincinglydemonstrated the robot-based approach as an effective reha-bilitation tool for this group of people. Similarly, robots wereused by the authors [5] as a communication activator basedon a recreational game call ”Nandoku”. They stressed theneed for improving communication, especially group com-munication for the cure of elderly suffering from depressingand dementia.

An agent-based architectural framework has been pro-posed in [8], which aimed to facilitate the development offlexible elderly care services. Other researchers has estab-lished CareLab [9] at the Philips High Tech Campus inEindhoven, which is an one bedroom hi-tech apartment as asenior-care facility. It is equipped with myriad sensors anddevices to study different context situations in which theelderly will use several applications related to their healthand well-being. The work reported in [2] describes AVANTIproject, which facilitates the adaptation of web content anduser interface for individuals including elderly.

Many works have focused on elderly care in the homeenvironment. E. Hollywood et al. [10] created an artificialcompanion called Program Alleviating Loneliness (PAL)for the home-bound elderly. The system is engaging andentertaining to be elderly at the same time. The OLDESproject [11], as realized in Italy and Czech Republic, pro-vides entertainment services to the elderly through differentchannels and animator-based special interest forums. An-other work with a broader perspective has been reported asiCare [3], which addresses the social and behavioral aspectsof aging services. The system supports access to ambientservices, unbound information reachability and seamlesssocial connection through the iCare home portal.

A recent work [12] addressed the development of in-home assistance application for the elderly. This work aimsat sensing, predicting, reasoning, and acting in response to

the elderly activities at home. The authors developed a C-AmI system architecture by synthesizing various sensors,activity recognition, case-based reasoning (CBR) and elderlyin-home assistance customized knowledge.

The above works focused on different tools and systemsthat help the elderly in their daily lives. However, mediationbetween the tools and the human caregiver was hardlypresent. Unlike those, our approach is to develop an elderlymonitoring system as a support service for the humancaregiver so that they can effectively monitor and assist thoseelderly.

III. PROPOSED HUMAN CAREGIVER SUPPORT SYSTEM

The goal of the proposed human caregiver support systemis to assist the caregiver to monitor the elderly person fromhealth and well-being perspective. We developed this systemas a means of providing services to the elderly based ontheir current activities and context. Thus, such a systemcan reduce both cognitive and physical load of a humancaregiver. Figure 1 shows a high-level interaction flow ofthe system. This figure shows that the system becomes awareof the situation based on the sensing data. The system thendecides whether to invoke relevant services, interact withhuman caregiver or call for external assistance in criticalsituations. In the following section, we describe these steps.

A. Sensing

The sensing module collects data from multiple sensorsplaced in the environment, processes the data and identifiesdifferent context parameters such as who is around, what ishappening, the location and the like. In the overall systemcontext, this module independently works to process thecontext parameters. Other modules obtain these contextparameters when required. Therefore, the operation of othermodules is not affected even if there is an underlying changein the sensing architecture and devices.

436

Table IACTIVITY STATE TRANSITION AND THE CORRESPONDING ACTIONS

Normal Socializing Lying

Normal NA visitor entered fall downinvoke service intervene caregiver, inform external members

Socializing visitor left NA NAinvoke service

Lying standup visitor enterNAinvoke service invoke service

B. Situation analysis

From Figure 1, it is evident that the situation analysistask is based on the outcome of the sensing tasks. The mainobjective of the situation analysis task is to determine thecurrent context of the environment and the activities theelderly people are engaged in. Depending on the situation ofthe the elderly monitoring environment, the human caregiversupport system will carry out relevant interaction to supportcare-giving. The following scenarios illustrate some caseswhen such interaction may occur:

• Elderly person falls down (drop suddenly or collapse)and remains down on floor for more than a specifiedperiod of time with minimal body movement.

• Elderly person falls asleep but some environmentaland entertainment devices (e.g. lights, TV, DVD) arerunning.

• Somebody enters the room occupied by the elderlyperson but the person is unknown to the elder or theelderly facility.

C. Human caregiver support system

Based on the current situation of elderly activities, thehuman caregiver support system determines what level ofassistance is required by the elderly or the human caregiver.In Table I, we provide a snapshot of the elderly activities,and list the different types of support the system can provide.In a similar fashion, we can determine the required level ofassistance in different situations. In this table, the left-mostcolumn and the top-most row shows sample activity names.The content of each cell describes a transition betweentwo activities from left column to top row. For example,a state change from sleeping to normal state occurs whenthe elderly person wakes up and that will require the supportsystem to invoke services relevant to that situation. In thefollowing, we describe human caregiver support systemin terms of several important aspects, which are 1) ruledefinition for activity transition, 2) interaction modes, and3) service invocation.

1) Rule definition: We model the transition between twoactivity states and the corresponding actions to be performedby the support system using the Event Condition Action(ECA) rules. The ECA is a short-cut to represent the struc-ture of active rules especially for event-driven architectures

and active database systems. ECA languages are indeed anintuitive and powerful model of programming that dealswith reactive systems. In recent days, ECA rule enginesuse several variations on rule structure. Augusto et al. [13]and Liu et al. [14] show the use of ECA rules and somevariations for applications in Smart Homes and assistedliving for elderly people. The basic construct of ECA isreactive rules, which have the following form:

On <event expression>If <condition>Do <action>

This means that On detecting certain events (i.e. elderlyactivity), If certain conditions are verified (i.e. true), thenspecific actions should be executed (i.e. Do). Based onsuch ECA rule construct, we can model the cases definedin Section III-B, for example the following:

On elderly person in bed for a long period of timeIf unexpectedly in bed during this period of timeDo intervene the human caregiver.

2) Interaction modes: According to the situation, theproposed support system acts based on three different modesthat are explained below.

• Automatic service invocation: In this mode, the systeminteracts with service invocation manager to implicitlyinvoke relevant services for the elderly. The service in-vocation manager identifies potential conflicts betweenthe already running services and the newly prescribedservices. It then schedules the execution of the sug-gested services.

• Human caregiver intervention: According to the changeof the activity states, the system might intervene humancaregiver in critical cases. For example, an elderlymight fall on the floor and remain there for a specifiedperiod of time, which might require human caregiver’sattention. The human caregiver in response can explic-itly activate any services; give command to the systemto perform certain actions or ask for external assistanceif needed.

• External service request: In some critical situations, likethe human caregiver, the system might as well call forexternal assistance. For example, calling for emergencyassistance in case of a fire.

437

���������

���� ��

����������

���������

����������������������������

���������� �������������

��������

��������������������������

�������

����������������

�� �� �� � � �

Figure 2. The core modules of the proposed support system

3) Service invocation: The Service Invocation modulereceives implicit command from the system when there is aneed to invoke services relevant to a situation. The servicescould vary from switching light ON/OFF to adjusting lightlevels, selecting media, locking/unlocking doors, managingalarm ON/OFF etc. that have an effect on the elderlymonitoring environment.

D. Algorithm

This algorithm for human caregiver support system takescontext with an attribute set as input and provides elderlyservices as output. There are several steps in this process,which are listed below.

• Determine the list of possible services based on thecurrent context that can be provided in the environment.

• Identify the list of currently running services that isneeded to avoid any conflict for running the newlysuggested services.

• Determine the modes of interactions for providing theelderly support services based on current context andelderly activity. The proposed system defines threemodes of interaction, which suggests whether to uti-lize automatic Service Invocation, to intervene humancaregiver and/or to call for external assistance.

• Identify the services that would conflict with the newlysuggested services and make an schedule to execute therelevant services.

• Intervene the human caregiver if there is a need forhuman caregiver’s attention.

• Call for external assistance based on the selected modeof interaction.

E. Architecture

The architecture of the proposed system is given inFigure 2. It is developed using a service-oriented architecture(SOA), where each of the modules acts as a service. Aservice can be a producer, a consumer or both. For example,

the Sensor Data Processor in this figure is the producer ofcontext-related information, whereas the Situation Analyzeris the consumer of such information. This figure also showsthat the elderly monitoring environment is equipped withdifferent sensors and actuator devices. The sensors are usedto sense the environment activities whereas the actuator isused to activate different services in the environment.

IV. IMPLEMENTATION

The proposed framework is implemented on the .NETdevelopment framework. Our implementation adopts webservice-based approach in order to provide an interop-erable technology platform. Also, we used web servicediscovery (WS-Discovery) [15] and web service eventing(WS-Eventing) [16] mechanism to make the developedservices discoverable on the fly and to send notificationof events based on a publish-subscribe model. Our imple-mented framework is deployed in an instrumented smartenvironment, where we placed several sensors such as IPcameras and X10 motion sensors, actuators, X10 lamp andappliance modules, phidget’s temperature sensor, wirelessweight scale, RFID tags, RFID reader, and VLC mediaplayers.

V. EXPERIMENT

For the experiment, we considered a two-room instru-mented apartment where we strategically placed differentsensor and actuator devices. We invited five elderly testparticipants as volunteers and two human caregivers whotakes care of their older parents at home.

A. Demonstrating situation analysis

We used camera picture and motion sensor data to identifysome situations when caregiving is needed. The sensing dataprocessor first processes the sensor data that it receives andpass its observation to the situation analyzer. In our experi-ment, we used the openCV’s face detection and recognition

438

Figure 3. Results showing the satisfaction level of the elderly

Figure 4. Results showing the acceptance rate of the human caregiver

(http://opencv.willowgarage.com/wiki/FaceRecognition) al-gorithm to identify the visitor coming in the environment.Also the X10 motion sensor that we used provides RF-based motion sensing from which we could understandthe movement level in the environment. Combining thesetwo we understood the socialization activity the elderly isengaged. The system also identified whether the elderly iswatching TV, which was determined based on RFID; andwhether the elderly was sleeping, which was determinedthrough motion sensing activity.

B. Demonstrating service invocation

Based on the situation analysis data, we demonstratethe interaction of the caregiver support system. In one testscenario, we had an intruder coming in the room the elderlylives, who was not registered in the database. As soonas the system became aware of this event, it sent SMSto the human caregiver. The human caregiver immediatelycame over to see what happened. We repeated this test tosee whether the system performs the prescribed actions. In

addition to this, we also tested with known visitors whoseprofiles were registered in the system. In that case, thesystem instructed the service invocation manager to providethe identified services, for example switching on TV withsome selected news sources. Similarly other situation andinteraction modes can be demonstrated. In the case of a reallife deployment, one should implement a fully functionalautomated activity detection mechanism and integrate it withthe proposed system.

C. Evaluating satisfaction and acceptance

In order to measure the satisfaction of elderly people, weasked questions to the elderly to know whether they aresatisfied by implicit interaction assistance provided by thesystem. We took a similar approach to know the level ofacceptance of the human caregiver for the caregiving assis-tance he received from the system. Figure 3 and Figure 4show the results of the responses provided by the elderly andthe human caregiver, respectively. The degree of freedom ofthe responses was based on a scale between 1 to 7. Theresults show clear advantages of the system as accepted byboth the elderly and the caregiver.

VI. CONCLUSIONS

This paper proposes a human caregiver support systemthat provides assistance to the elderly as well as supportshuman caregiver to keep a watchful eye on the elderly.The system is aware of the monitoring environment and itdynamically determines whether to provide implicit services,intervene human caregiver and/or call for external assistancedepending of the activity state change of the elderly. Ourexperimental results show that the elderly participants werehighly satisfied with the system’s assistance. The samewas noticed from the human caregiver’s positive feedback.However, more works need to be done. For example de-veloping robust and distributed sensing data processing forunderstanding elderly activities and experimenting in realeldercare facility.

REFERENCES

[1] H. Brodaty et al., “Towards harmonisation of caregiveroutcome measures,” Brain Aging, vol. 2, pp. 3–12, 2002.

[2] J. Finka, A. Kobsaa, and A. Nill, “Adaptable and adaptiveinformation provision for all users, including disabled andelderly people,” New Review of Hypermedia and Multimedia,vol. 4, pp. 163–188, 1988.

[3] W. Chang, S. Yuan, and E. Y. Li, “iCare home portal: anextended model of quality aging e-services,” Communicationsof the ACM, vol. 52, pp. 118–124, 2009.

[4] T. Tamura et al., “Is an entertainment robot useful in thecare of elderly people with severe dementia?,” The journalsof gerontology. Series A, Biological sciences and medicalsciences, vol. 59, pp. 83–85, 2004.

439

[5] Y. Matsuyama, H. Taniyama, S. Fujie, and T. Kobayashi,“System design of group communication activator: an en-tertainment task for elderly care,” in 4th ACM/IEEE inter-national Conference on Human Robot interaction, 2009, pp.243–244.

[6] H. Si, S.J. Kim, N. Kawanishi, and H. Morikawa, “A context-aware reminding system for daily activities of dementia pa-tients,” in IEEE/RJS International Conference on IntelligentRobots and Systems (IROS 2011), 2011, pp. 2430–2437.

[7] M. Montemerlo, J. Pineau, N. Roy, S. Thrun, and V. Verma,“Experiences with a mobile robotic elderly guide for theelderly,” in National Conference on Artificial Intelligence.August 2002, AAAI.

[8] L. M. Camarinha-Matos and H. Afsarmanesh, “A multi-agent based infrastructure to support virtual communities inelderly care,” International Journal of Networking and VirtualOrganisations, vol. 2, pp. 246–266, 2004.

[9] B. de Ruyter and E. Pelgrim, “Ambient assisted-livingresearch in carelab,” Interactions, vol. 23, pp. 30–33, 2007.

[10] E. Hollywood, G. O’Brien, and S. Lennon, “SIGCHI project:user centered design of a program alleviating loneliness(pal),” in CHI 2005 Extended Abstracts on Human Factorsin Computing Systems, 2005, pp. 2089–2093.

[11] M. Busuoli et al., “Entertainment and ambient: a newoldesv́iew,” in 4th international Conference on Universal

Access in Human-Computer interaction: Applications andServices, 2007, pp. 511–519.

[12] F. Zhou, J. Jiao, S. Chen, and D. Zhang, “A case-drivenambient intelligence system for elderly in-home assistanceapplications,” IEEE Transactions on Systems, Man, andCybernetics-Part C: Applications and Reviews, vol. 41, pp.179–189, 2011.

[13] J.C. Augusto, J. Liu, P.J. McCullagh, H. Wang, and Y. Jian-Bo, “Management of uncertainty and spatio-temporal aspectsfor monitoring and diagnosis in a smart home,” InternationalJournal of Computational Intelligence Systems, vol. 1, pp.361–378, 2008.

[14] J. Liu, J.C. Augusto, and H. Wang, “Consideration on uncer-tain spatio-temporal reasoning in smart home systems,” in 7thInternational Conference on Applied Artificial Intelligence,2006, pp. 817–824.

[15] J. Beatty et al., “Web servicesdynamic discovery (ws-discovery),”http://specs.xmlsoap.org/ws/2005/04/discovery/ws-discovery.pdf, Accessed on January 10, 2012.

[16] D. Box et al., “Web services eventing (ws-eventing),”http://www.w3.org/Submission/WS-Eventing/, Accessed onJanuary 14, 2012.

440