remote monitoring and - mdh · remote monitoring and ... it is always nice to be surrounded by...

Malardalen University Licentiate ThesisNo.??

Remote Monitoring andAutomatic Fall Detection for

Elderly People at Home

Gregory Koshmak

March 2015

Department of Computer Science and EngineeringMalardalen University

Vasteras, Sweden

Copyright c©Gregory Koshmak, 2015ISSN ????-????ISBN ??-?????-??-?Printed by Arkitektkopia, Vasteras, SwedenDistribution: Malardalen University Press

Abstract

Aging population is a one of the key problems for the vast majority of socalled ”more economically developed countries” (MEDC). The amountof elderly people who suffer from multiple disease and require permanentmonitoring of their vital parameters has increased recently resulting inextra healthcare costs. Modern healthcare systems exploited in geri-atric medicine are often obtrusive and require patients presence at thehospital which interferes with their demand in independent life style.Recent developments on telecare market provide a wide range of wire-less solutions for distant monitoring of medical parameters and healthassistance. However, most of the devices are programmed for spot check-ing and operate independently from each other. There is still a lack ofintegrated framework with high interoperability and on-line continuousmonitoring support for further correlation analyses. The current studyis a step towards complete and continuous data collection system for el-derly people with various types of health problems. Research initiative ismotivated by recent demand in reliable multi-functional remote monitor-ing systems, combining different data sources. The main focus is madeon fall detection methods, interoperability, real-life testing and correla-tion analyses. The list of main contributions contains (1) investigatingcommunication functionalities, (2) developing algorithm for reliable falldetection, (3) multi-sensor fusion analyses and overview of the latestmulti-sensor fusion approaches, (4) user study involving healthy volun-teers and elderly people. Evaluation is performed through a series ofcomputer simulation and real-life testing in collaboration with the localmedical authorities. As a result we expect to obtain a monitoring sys-tem with reliable communication capabilities, inbuilt on-line processing,alarm generating techniques and complete functionality for integrationwith similar systems or smart-home environment.

i

Sammanfattning

En aldrande befolkning utgor ett av de viktigaste problemen for de allraflesta sa kallade ”mer ekonomiskt utvecklade lander” (MEDC). Mangdenaldre manniskor som lider av multi-sjukdomar och kraver standig overvakningav vitala parametrar har okat pa senare tid, vilket resulterar i okadesjukvardskostnader. Geriatrikens moderna sjukvardssystem kraver oftaatt patienterna ar narvarande pa sjukhuset, vilket kraftigt begransaren sjalvstandig och oberoende livsstil. Den senaste utvecklingen patelemedicinomradet erbjuder ett brett utbud av tradlosa losningar inomhalsovard for distansovervakning av medicinska parametrar. De flestalosningarna innebar punktkontroll av enskilda parametrar och arbetaroberoende av varandra. Det saknas fortfarande integrerade losningarmed hog interoperabilitet och kontinuerlig on-line overvakningsstod foratt kunna genomfora ytterligare korrelationsanalyser. Detta arbete utgorett steg mot ett fullstandigt och kontinuerligt datainsamlingssystem foraldre personer med olika typer av halsoproblem. Forskningsinitiativetmotiveras av senaste tidens efterfragan pa tillforlitliga multifunktionellasystem for distansovervakning, som kombinerar olika datakallor. Huvud-fokus utgors av falldetektionsmetoder, interoperabilitet, verkliga testeroch korrelationsanalyser. Listan over de framsta bidragen innehaller(1) att undersoka kommunikationsfunktionaliteter, (2) utveckla en al-goritm for tillforlitlig falldetektion, (3) multisensor-fusion-analyser ochoversikt over multisensor-fusion-strategier, (4) en anvandarstudie medfriska frivilliga aldre. Utvarderingen sker genom en serie av datorsimu-leringar och tester i verklig miljo i samarbete med lokala halso- ochsjukvardsmyndigheter. Malet ar ett overvakningssystem med tillforlitligakommunikationsmojligheter, inbyggd on-line-bearbetning, tekniker forlarmgenerering och funktionalitet for integration med liknande systemeller i en smart hemmiljo.

iii

To my relatives

Acknowledgement

First and foremost let me offer my sincerest gratitude to supervisorsMaria Linden, Amy Loutfi and Matts Bjorkman, who have constantlybeen a great inspiration for me. From the very moment I have startedmy academic carrier I always know where to go in times of doubt anddesperation. Your expert knowledge, academic experience and wise ad-vice is guiding me through the dark forest of research life. I am lucky towork with such professional researches and extremely pleasant people.

I have not been spending in Vasteras every day during the past years,nut I can honestly enjoy IDT team and appreciate all the moments weshared. I would like to personally thank my ”old” office room matesNikola, Marcus, Jimmy, Martin and my current desk neighbors Sara,Per, Arash and . It is always nice to be surrounded by intelligent and atthe same time cheerful people with a great sense of humor. Thanks forbeing brave to participate in my experiments and thanks for challengingme with intellectual discussions. I can not help mentioning our annualjourneys with IFT group, this was something I will remember for a longtime!

Thanks to all the members of the GiraffPlus project, especially Fil-ippo, Ales, Jonas, , with whom I was collaborating a lot during theproject lifetime. Additional words of gratitude go to Stig and Uno, whoin spite of their advanced age have charged me with unconditional en-ergy and positive attitude towards life.

A huge thanks to my family. There are no such words to describe howmuch they all mean to me. My parents and grandparents have alwaysbeen a source of unlimited inspiration and strong support. Researchwork can be tough sometimes, but it is not even close to what they hadto go through during the last 1.5 yeah. And yet they always managed tofind some encouraging words, wise advice and just a funny joke to share.

vii

List of Publications

Papers included in the licentiate thesis1

Paper A Evaluation of the Android-Based Fall Detection System with Phys-iological Data Monitoring. Gregory Koshmak, Maria Linden, AmyLoutfi, 35th Annual International Conference of the IEEE EMBSOsaka, Japan, July 3-7, 2013.

Paper B Dynamic Bayesian Networks for Context-Aware Fall Risk Assess-ment. Gregory Koshmak, Maria Linden, Amy Loutfi, A Special Is-sue of Ambient Assisted Living (AAL): Sensor, Architectures andApplications.

Paper C Challenges and Issues in Multi-Sensor Fusion Approach for FallDetection: Review Paper. Gregory Koshmak, Maria Linden, AmyLoutfi. Submitted to Journal of Sensors.

1The included articles have been reformatted to comply with the licentiate layout

ix

x

Additional papers, not included in the licen-tiate thesis

Conferences

• A smart-Phone Based Monitoring System with Health Device Pro-file for Measuring Vital Physiological parameters. Gregory Koshmak,Martin Ekstrom, Maria Linden. World Congress on Medical Physicsand Biomedical Engineering, Beijing, China, May 26-31, 2012

• Heart Rate Measurement as a tool to quantify Sedentary Behav-ior. Anna Akeberg, Gregory Koshmak, Anders Johansson, MariaLinden, International Conference on Wearable Micro and NanoTechnologies for Personalized Health, Vasteras, Sweden, June 2-4,2015.

Contents

I Thesis 1

1 Introduction 31.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Problem description . . . . . . . . . . . . . . . . . . . . . 41.3 Research Hypothesis . . . . . . . . . . . . . . . . . . . . . 51.4 Research Questions . . . . . . . . . . . . . . . . . . . . . . 51.5 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Wireless Monitoring 72.1 Wearable Sensors . . . . . . . . . . . . . . . . . . . . . . . 72.2 Fall Detection . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2.1 Fall Characterisitcs . . . . . . . . . . . . . . . . . . 92.2.2 Context-Aware Fall Detection . . . . . . . . . . . . 112.2.3 Smartphone-based Fall Detection . . . . . . . . . . 12

2.3 Smart Home Environment . . . . . . . . . . . . . . . . . . 13

3 Methodology 153.1 Research Approach . . . . . . . . . . . . . . . . . . . . . . 153.2 GiraffPlus Project . . . . . . . . . . . . . . . . . . . . . . 16

4 Research Contribution 194.1 Paper A . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.2 Paper B . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.3 Paper C . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5 Conclusions and Future Work 255.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . 255.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 26

xi

xii Contents

Bibliography 29

II Included Papers 35

6 Paper A:Evaluation of the Android-Based Fall Detection Systemwith Physiological Data Monitoring 376.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 396.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . 40

6.2.1 Fall Detection Algorithm . . . . . . . . . . . . . . 406.3 Experiments and Results . . . . . . . . . . . . . . . . . . . 426.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 46Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

7 Paper B:Dynamic Bayesian Networks for Context-Aware Fall RiskAssessment 537.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 557.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 56

7.2.1 Context-Aware Fall Detection . . . . . . . . . . . . 567.2.2 Mobile Healthcare Integration . . . . . . . . . . . . 58

7.3 Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 597.3.1 Mobile-Based Fall Detection System . . . . . . . . 597.3.2 Context Recognition . . . . . . . . . . . . . . . . . 627.3.3 Dynamic Bayesian Network . . . . . . . . . . . . . 63

7.4 System Integration . . . . . . . . . . . . . . . . . . . . . . 657.5 System Evaluation . . . . . . . . . . . . . . . . . . . . . . 69

7.5.1 Matlab Simulated Model . . . . . . . . . . . . . . 697.5.2 Demonstration Model . . . . . . . . . . . . . . . . 707.5.3 Fall Risk Probability Estimation . . . . . . . . . . 72

7.6 Conclusions and Future Work . . . . . . . . . . . . . . . . 75Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

8 Paper C:Challenges and Issues in MultiSensor Fusion Approachfor Fall Detection: Review Paper 838.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 858.2 Fall detection . . . . . . . . . . . . . . . . . . . . . . . . . 86

Contents xiii

8.2.1 Fall Characteristics and Popular Approaches . . . 868.3 Sensor fusion in Fall Detection . . . . . . . . . . . . . . . 89

8.3.1 Context-aware sensors fusion . . . . . . . . . . . . 918.3.2 Wearable sensors fusion . . . . . . . . . . . . . . . 948.3.3 Wearable/Ambient sensor fusion . . . . . . . . . . 98

8.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . 1038.4.1 Challenges . . . . . . . . . . . . . . . . . . . . . . 1038.4.2 Future Trends . . . . . . . . . . . . . . . . . . . . . 104

8.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 105Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

I

Thesis

1

Chapter 1

Introduction

1.1 Motivation

Aging population has been one of the main concerns in most developedcountries during the last decade [1]. Most elderly people suffer fromwider spectrum of various diseases and more emergency situations suchas fall are likely to occur [2]. As a result, they need to be urgentlytransported to the hospital, where they will be observed and providedwith medical help if health condition is at risk. At the same time,the amount of elderly people choosing to maintain their independentlifestyles is growing rapidly, which makes it harder for medical profes-sionals to follow changes and trends in patient’s health conditions outsidehospital environment. However, remote monitoring can help to preventdescribed scenario, significantly reduce healthcare costs and at the sametime maintain patient’s independent lifestyle [3]. Therefore, there isa clear demand in reliable multi-functional remote monitoring systemsfor elderly people, which collect and combine different sources of med-ical data corresponding to everyday routine of the monitored patient.In many cases, different components comprising the systems are dis-integrated and operating separately from each other. However, if wecombine monitoring components (e.g. sensors, actuators) into smart en-vironments, we will be able to carry out observations for people withmultiple chronic conditions at home. It will help to improve elderly pa-tient’s level of freedom and safety, which is one of the main issues inhealthcare industry.

3

4 Chapter 1. Introduction

At teh same time fall incidents are considered to be one of the mostcommon and dangerous risks among elderly population, with nearly halfof nursing home residents and 30% of independently living people fallingeach year. Therefore, modern healthcare systems tend to integrate re-liable fall detection functionality into general monitoring framework.With the recent development on ICT market wearable sensors are of-ten deployed in conjunction with environmental devices to improve falldetection rates and minimize false alarms. In this case a multi-modalsystem requires a special fusion algorithm to combine all the active com-ponents.

Another source of motivation is a lack of contextual data in a vastmajority of modern healthcare systems. Therefore, smart homes, with acapability to unobtrusively collect contextual data (e.g. radio-frequencyidentification (RFID) tags, pressure mats, switcher sensors and etc.) areessential sources of information. These data can be processed afterwardsand infer real activities (e.g. cooking, sleeping, exercising), giving an ex-tra insight on physiological processes happening with elderly. However,there are no obvious solutions for integrating medical sensors into a smarthome environment, which makes this area open for further research in-vestigations.

1.2 Problem description

As a response to the aging population, modern healthcare market pro-vides a wide range of medical devices for remote measuring of vital healthparameters. Most of the equipment is programmed and exploit for spotchecking and is not able to give a continuous overview of the patient’shealth conditions. Moreover, different parameters are measured sepa-rately and monitoring process is not synchronized. At the same time,modern smartphones are equipped with advanced sensor functionality,which has a great potential for the healthcare, but is mostly exploitedin game industry.

Assuming mentioned circumstances and previously conducted studyfocused on android based monitoring system for patients with chronicobtrusive disease [4] we can formulate two main challenges of presentedresearch: (1) a lack of knowledge and sufficient expertize for continuousmedical data analyses and (2) inefficient insight on the correlation be-tween measured parameters.

1.3 Research Hypothesis 5

Assuming formulated problems, current research investigates the pos-sibilities of continuous remote monitoring of elderly people in their homeenvironment, collecting vital parameters and subsequent analyses to-wards correlation between acquired measurements. A special focus ismade on multiple sources of medical data, different communication pro-tocols and variety of processing algorithms for alarm generation.

1.3 Research Hypothesis

We believe the future development of wireless monitoring is based onintegration of unrelated data sources into a multi-modal system andswitching between the components depending on particular situation.The result can be achieved by gradual development of independent com-ponents with a subsequent integration into a common framework with ageneric processing algorithm. We believe these types of system are ableto replace modern spot-checking health sensors, can be implementedwith a low power consumption rate, improve reliability and increase theoverall acceptance of health monitoring systems.

1.4 Research Questions

During development and investigation process we expect to answer anumber of research questions to check the formulated hypothesis. Eachquestion corresponds to particular challenges and demands in terms ofremote monitoring domain and healthcare industry in general.

Question AWhich combination of sensors, actuators and measured parametershave the potential to provide sufficient amount of data for on-lineand posterior analyses. We are looking for research solutions whichwill help to establish unobtrusive monitoring and at the same timemeasure vital parameters both outdoors and in home environment.

Question BFollowong elaborated data collection process, we want to investi-gate what types of algorithms should be involved to perform anal-yses of the collected measurements.

6 Chapter 1. Introduction

1. First part of the question will concern the ”on-line” processingperformed during the monitoring stage

2. Alternatively we will investigate possible algorithms for pos-terior analyses applied to previously collected data

Question CAnother important research issue concerns the list of optimal out-puts determined to provide competence assistance to medical staffor notify patients directly. In this case, we want to investigatevarious user cases and monitoring scenarios which require differentapproaches in information delivery. Possible options can include:instant alarm indication, visualization feature or diagnose assis-tance.

Question DAdditionally we conduct a review study to give a better insighton multi-fusion based monitoring and fall detection solutions. Alarge number of healthcare device are already available for long-term monitoring and can be potentially integrated into a commonnetwork. It is not clear however which integration mechanism orfusion tecnique is the most applicable for this approach.

1.5 Thesis Outline

The rest of the thesis is organized as following:Part I consists of five chapters. Chapter 1 provides a motivation for ini-tiated studies, gives a brief problem description, formulate main hypoth-esis and research questions. Chapter 2 presents background knowledgeon research topic and describes the latest publications within wirelesshealthcare systems with a special focus on fall detection and smart homeenvironments. In Chapter 3 we provide details on research methodologydeployed to achieve goals and answer announced questions. Chapter 4contains main thesis contributions listed according to their relevance tothe published papers. Finally we draw conclusions and discuss possiblefuture work in Chapter 5.Part II presents technical contributions of the thesis in the form of threepapers that are organized in Chapters 6 to 8.

Chapter 2

Wireless Monitoring

Particular circumstances including aging of population in developed coun-tries, increasing costs of primary healthcare and strong demand in inde-pendent living, have already evoked an intensive research work in remotemonitoring area. Normally, these types of systems are subdivided intothree major sections: sensor layer, communication layer and caregiver.We are particularly interested in the first two categories responsible forcollecting, transferring and processing of the streaming data. Usuallymonitoring process involves both wearable and environmental sensors,collecting data for further processing and visualizing. However, thesetwo types of information channels are managed separately and rarelypresented as a combined structure.

2.1 Wearable Sensors

Latest wearable medical devices often operate in conjunction with smart-phones, which are playing a major role in the modern healthcare systems[3]. The list of possible applications has been growing along with marketdevelopment: early detection of Alzheimer’s disease [5], face-to-face com-munication between doctor and patient [6], complex activity recognition[7] and medicine in-take assistance [8]. Modern smartphones, operatingas wearable sensor can also be deployed as a communication entity forother medical devices providing a link between different types of datasources [9]. This particular idea was implemented and described in an ar-ticle by Kohei Arai [10], where blood pressure, body temperature, pulse

7

8 Chapter 2. Wireless Monitoring

rate EEG, calorie consumption and other sensors are attached to thehuman body. Measured data are transferred to mobile devices throughBluetooth and further to the Information Collection Center with the helpof WiFi or Wireless LAN. Alternatively, we can establish connection toa smartphone device via ZigBee, which is a standard communicationprotocol for low-cost, low-power, wireless sensor and control networks.Matthias Wagner et al [11] developed a two-approach telemedical sys-tem focused on the measurement and evaluation of vital parameters, e.g.ECG, heart rate, heart rate variability, pulse oximetry, plethysmographyand fall detection. According to the first approach, all the obtained pa-rameters are transferred to coordinator node via ZigBee technique, whichas Bluetooth operates on the GHz radio frequency, but has a maximumdata rate of 0.25 Mbps. Both protocols have been additionally improvedto be used by medical devices via universal communication standardsincluding ”Health Care” for ZigBee and Health Device Profile (HDP)for Bluetooth.

Apart from their communication functionality, mobile devices canserve as preprocessing tool during the monitoring. This feature allowsto avoid overloading potential caregiver with unnecessary informationor even trigger an early alarm in case of emergency [12]. Kozlovsky etal [13] developed an Android based mobile data acquisition (DAQ) so-lution, which collects personalized health information of the end-user,store, analyze and visualize it on the smart device and optionally sendsit to the data center for further processing. The software even enablescorrelation analysis between the various sensor data sets. This option,however, is not feasible for all the mobile devices due to some differencesin processing time and memory consumption.

In our research study we tend to investigate potential benefits of fus-ing unrelated sorts of data into a multi-modal framework. Therefore, inaddition to medical entities based on smartphones, we deploy environ-mental sensors and involve them into a monitoring process. We selectemergency situations associated with falls as a main application for de-veloping system, which can be potentially expanded to a larger scale ofhealthcare problems.

2.2 Fall Detection 9

2.2 Fall Detection

As it was previously discussed in Chapter 1, falls are among major prob-lems in modern healthcare and a serious threat for elderly population.As a result, most of the wireless monitoring systems tend to includeautomatic fall detection into their functionality. Modern smartphonesare often equipped with a set of powerful sensor technology and start toplay a significant role in healthcare development. Recent studies provedthat accelerometer, gyroscope and magnetometer can comprise an inde-pendent fall detecting tool or be a part of the fall detection framework.Commonly, acceleration data is collected and stored on the smartphonewith subsequent on-line or off-line processing depending on the currentcircumstances. Alternatively, some of the studies propose algorithmswhere contextual or visual data collected by environmental sensors isdeployed to detect a fall. In this case obtrusiveness of the process isrelatively low since patients do not require to wear any devices. At thesame time, these type of systems are often facing privacy issues and re-quire additional ethical approve. Due to these reasons and complexity ofthe fall process in general several attempts were made to combine bothtypes of data to improve overall performance of fall detection systems.In the following section we provide main fall characteristics, describepopular approaches and explain how fall detection can be included in ageneral monitoring model implemented in a smart home environemt.

2.2.1 Fall Characterisitcs

A fall is commonly defined as ”unintentionally coming to rest on theground, floor, or other lower level”. Losing the balance and subsequentfalling with the help of an assistant also considered as a fall [14]. Basedon possible scenarios 4 main types of falls can be distinguished: (1) fallfrom sleeping, (2) fall from sitting, (3) fall from walking/standing and(4) fall from standing on support tools such as ladder. Each type hasits’ own unique characteristics, which can help developers to adapt falldetector platforms to a wider spectrum of user requirements.

Typically all the modern fall detection systems can be split into 3main classes depending on the sensor technology deployed for monitor-ing: wearable sensors, ambient sensors and vision-based sensors (see Fig-ure 2.1). Most of the wearable fall detectors are based on accelerometer


data and operating with posture and motion of the patients body.

Vision-basedsensorsWearable Sensors Ambient sensors

Posture Motion

Body shape change

3-D head change

InactivityPosture Presence

Fall Detection

Figure 2.1: Fall detection classification

They can additionally be subdivided into thresholding or machinelearning methods according to the processing algorithm they deploy. Ac-celeration data collected during the fall in different directions is demon-strated on Figure 2.2. Each line represents raw, pitch or yaw of thesmartphones coordinate axis, has its unique variation and can distinc-tively depict three different fazes of every fall motion: (1) pre-fall, (2)impact, (3) after-fall phase.

50 100 150 200 250 300

−10

0

10

Time, hours

Avera

ge a

ctivity

50 100 150 200 250 300 350 400

−10

0

10

Time, hours

Avera

ge a

ctivity

50 100 150 200 250 300

−10

0

10

Measurements

Avera

ge a

ctivity

Figure 2.2: Smartphone coordinate axis

Alternative fall detection methods are based on contextual data and

2.2 Fall Detection 11

deploy modern vision or ambient techniques to detect a fall. In thiscase collected measurements (video stream, sound etc) are transfered toa remote device and inspected for possible emergency situations associ-ated with falls. In the vast majority of the context-aware or vision-basedsystems falls are detected off-line with the help of statistical or machinelearning algorithms.

Each of the presented approaches still gives a significant amount offalse positive alarms while operating independently. It is therefore im-portant to integrate additional sensor functionality in order to improvereliability of fall detection systems. This trend is becoming popular andaddressed as multi-sensor fusion based fall detection. In this case, sev-eral sensor channels are deployed to collect data which is later fusedon a processing level. In the following sections we continue to describeeach type of fall detection approach in particular and discuss possiblesolutions for a multimodal framework fusing both techniques.

2.2.2 Context-Aware Fall Detection

Contextual data has been recently deployed to perform fall detection andactivity recognition of elderly people in their home environment. Brulinet al.[15] described an approach, with the main idea to fuse differenttypes of data source channels into a special architecture, acquiring infor-mation from RIP detectors, thermopile or cameras. This system can alsoperform on-line or posterior processing to derive posture or orientation ofthe user and trigger an alarm in case of fall risk. Various attempts weremade to improve this process by introducing additional sources of datalike surrounding audio captured by microphone arrays [16, 17] or cur-rent location of the user [18]. Alternatively, RGBD-camera is deployedin study by BinBing Ni et al [19] for hospital fall prevention. Anotherrecent study demonstrates how fall prevention system makes use of col-lected data from sensors in order to control and advice patients or evento give instructions to treat an abnormal condition and reduce the fallrisk [20]. In this case monitoring and processing data from sensors isperformed by a smartphone that will issue warnings to the user and inemergency situations send them to a caregiver. Moreover, relationshipbetween acceleration of body’s center of gravity during sit-to-walk mo-tion and a process of falling is investigated by Shiozawa, N et al. [21].The result of discriminant analysis by using indexes with a significantdifference revealed a 90.3% correct prediction rate for falling. However,


there is still a relatively high level of false alarm generated associatedwith the context-based detection and prevention systems, which can bepotentially improved by integrating with a wearable device.

It was previously discussed that wearable sensors with inbuilt ac-celerometer can serve as an effective healthcare device. Most of themhave recently been deployed for accurate fall detection [6] demonstrat-ing dignified results during evaluation process. We are interested inaccelerometer-based sensor both as an individual component and as apart of the multi-modal system, where it is combined with environmen-tal sensors. In our case a wearable device is replaced with a smartphone,which can serve as fall detector sensor and gateway mode at the sametime.

2.2.3 Smartphone-based Fall Detection

With the recent development on mobile market, smartphones start toplay an important role in modern healthcare systems [3]. Latest ver-sions equipped with an accelerometer sensor are commonly used as falldetection tools [22, 23, 24, 25]. In this case they replace both process-ing mode and a communication tool while maintaining relatively smallsize. A choice of processing algorithm depends on final application ofthe system and varies in different studies. Some of the recent imple-mentation methods apply Gaussian distribution of clustered knowledge[26], neural network [27] and machine learning techniques [28]. However,most of them are initially based on three essential parameters associatedwith falls: impact, velocity and posture. According to the recent article,combining impact and posture while analyzing the fall case is enough tocreate a reliable algorithm [29].

Based on research questions formulated earlier, we tend to developa fall detection system and investigate its further integration into afull-scale monitoring system with additional sensor functionality includ-ing medical devices and environmental sensors. Similar approach wasadopted in several studies with intention to combine contextual datawith essential accelerometer measurements exploiting inertia and loca-tion sensors [30]. Qiang Li et al in [31] investigate a novel fall detec-tion method that utilizes acceleration, posture and context information,where context can be presented by environmental sensors (room loca-tion or furniture positions) and personal profiles (e.g. health status andage). Wireless accelerometer, 3-D camera and microphone are being si-

2.3 Smart Home Environment 13

multaneously processed by Leone et al to reach a better result in fallrisk assessment [32]. All the presented studies, however, are lacking areliable fusing technique to combine processing results from independentcomponents. In work by Zhang et al [33] an off-the-shelf programmablesensing platform called Sun SPOT is used for data recording. Contextinformation is presented in several categories, covering main aspects ofelderly living: (1) physical activity; (2) physiological condition; (3) per-sonal health record; and (4) location. Each category represents a sep-arate variable or processing result (including fall alarm from isolatedalgorithm) and merged into a Bayesian network for further statisticalanalyses. In the current research, we make an attempt to develop an in-dependently operating fall detection system, which can be easily mergedwith other types of sensors and integrated into a larger monitoring en-vironment. Therefore in Paper B, we describe a method for managingmulti-variant healthcare data with Dynamic Bayesian Network, and pro-pose the framework of the developed system.

2.3 Smart Home Environment

Reliable fall detection is one of the steps towards universal monitor-ing system for elderly people with different types of disease. It shouldbe combined with other sources of medical information channels in or-der to give a better insight on patients medical conditions. This canbe successfully implemented via modern smart home environments as amulti-modal platform with fusing capabilities. Smart homes in generalare a part of the Ambient Assisted Living area, responsible for continu-ous monitoring of elderly people in comfortable home environment. Inrecent years, an increasing number of projects have been based on thisapproach implying various components and applications. Special nutri-tion advisor was proposed as an attempt to improve physical conditionfor elderly people with diabetes [34]. It demonstrates the possibility tomake nutritional management much more effective through deployingAmbient Intelligence systems. Moreover, Juan A Botia et al. [35] pre-sented their Necesity system with adaptive monitoring capabilities andexhaustive evaluation methodology which was integrated in the develop-ment process. Jer-Vui Lee et al. [36] make an attempt to build a smartelderly home based on an android device, which is utilized as a 3-axialaccelerometer device to detect a fall of the carrier.


We tend to utilize smart home environment in order to build a long-term monitoring platform for elderly people. At the same time, someof its capabilities can be used to collect contextual data, which makesit an additional source of information. Villarrubia et al. [37] make anattempt to incorporate image processing and artificial techniques basedon PANGEA (Platform for the Automatic Construction of Organiztionsof Intelligent Agents) plaform. The system is presented in a case studydesigned using different agents and sensors responsible for providing usersupport at home in the event of incidents or emergencies. Developers ofthe eCAALYX project [3] (Enhanced Complete Ambient Assisted Liv-ing Experiment) take 24/7 monitoring of healthy older people one stepfurther by refining it and making it available to older people with multi-ple chronic disease. A particular effort is made on communication withthe user deploying various sorts of interactive devices: TV-based Set TopBox system, Customer-Premises Equipment and interactive TV. The keyidea is to extend independent life at home and avoid hospitalization forlonger periods.

The number of publications in healthcare domain is expected to growrapidly. There are both successful implementations and studies whichrequire additional research effort. The obvious trend observed in mostof the recent publications can be characterized by combining unrelatedsource of data into integrated framework. However, it is still not clearhow to process collected measurements or which set up will be mosteffective in the biggest amount of monitoring instances. Therefore, webelieve it is important to (1) combine both wearable and contextualdata to be able to adjust developed system for different types of usersand monitoring scenarios, (2) create a flexible integration platform to beable to add/remove sensors depending on particular user requirements.In this case an efficient and flexible algorithm is required for fusing var-ious sources of data before the processing stage. Therefore, as a partof the research process, we conducted a literature search resulted in areview paper on recent studies within the multi-sensor fusion based falldetection. Combination of contextual and wearable data based on smarthome platform for effective detection of emergency situations associatedwith falls is a step towards the full-scale monitoring system for elderly.

Chapter 3

Methodology

3.1 Research Approach

As it was previously formulated in Section 1.2, main challenges of theproposed study include (1) a lack of knowledge and sufficient expertizefor continuous medical data analyses and (2) a lack of efficient fusingalgorithm for managing unrelated data sources (3) inefficient insight onthe correlation between various parameters. All challenges represent dif-ferent types of analyses interacting between each other, and therefore canbe approached with cross-disciplinary research. A lack of knowledge forcontinuous data analyses can be overcome by introducing additional datasources to the monitoring process. In order to provide a better insight ona correlation between measured parameters we can deploy multi-sensorfusion algorithm and combine different types of data into a single sourcechannel. The choice of algorithm in this case can be based on literaturesearch and review of the recent studies within multi-modal monitoringarea. Therefore, we commenced a multi-disciplinary research work in-cluding components like sensor management, signal processing, androiddevelopment, artificial intelligence and medical knowledge expertise.

Each area corresponds to a particular stage of the monitoring pro-cess and therefore requires independent research approach, hardware andsoftware configurations. The vital component of any remote diagnosticsystem is a sensor layer, which is responsible for the continuous datacollection. Rapidly growing market of medical devices allow us to de-ploy different types of sensors and expand the number of vitals sings to

15

16 Chapter 3. Methodology

capture. Moreover, to establish reliable communication channel betweensensors and processing device, Bluetooth technology is utilized includingHealth Device Profile protocol especially designed for medical applica-tions. Various algorithms and schemes from ”signal processing” area aredeployed for managing sensor data on both pre and post-processing levelwith a particular focus on sensors fusion and fall detection techniques.In order to develop the most efficient algorithm for integration of col-lected measurements into a common framework we conduct a literaturestudy with a particular focus on multi-sensor fusion techniques for falldetection. It will help us to get a better insight on recently demandedapproach and systematize the current efforts in this novel area. At thesame time, all on-line analyses and other methods are implemented onandroid device using android API and latest available software. Addi-tional effort will be dedicated to the choice of potential recipient andexploitation of the output (i.e. external storage, alarm notification, datavisualization) and elderly perception of the developed technologies. Fi-nally, evaluation of the current studies is implemented through a seriesof simulations and real-life tests for both on-line and off-line data col-lection. Particular focus is made on active involvement of elderly pop-ulation into evaluation process in collaboration with the local medicalauthorities. This initiative will help to promote distance healthcare forelderly people and improve the level of cooperation. After preliminaryconsultations and mutual consent, we launched a series of online moni-toring process involving elderly person with reported risk of falling and agroup of caregivers. We managed to establish a reliable communicationbetween the patient and medical personnel, perform daily monitoringand generate alarms in case of emergency. Collected data can be laterdeployed for further investigations and give a better insight on physi-cal activity of the elderly person who belongs to a risk group. We willdiscuss preliminary evaluation results in Section 7.6 of the thesis.

3.2 GiraffPlus Project

A number of research investigations in this thesis is a part of GiraffPlusproject, financed by the EU-FP-7 foundation, which helps us to covermost of the mentioned research questions. It can also allow us to per-form various types of data collection essential for evaluation part of thefinal thesis. The main agenda of the project coincides with our research

3.2 GiraffPlus Project 17

aims and requires deployment of physiological and environmental sen-sors for continuous monitoring of elderly people in their homes. Severaladditional functionalities are planned to be implemented to enhance thesystem [38] including continuous data collection of daily activity andphysiological parameters from distributed sensors, long-term trend anal-ysis, data presentation via a personalized interface and social interactionbetween primary users (elderly people) and secondary users (caregivers).This interaction is implemented through Giraff robot that is teleoper-ated by secondary users and can move and see everything inside thehome and talk to elderly patient (see Figure 3.1). The list of secondaryusers include family, friends, informal and formal care givers and healthprofessionals, who can access the system through their personal com-puter. A special category is represented by medical personnel, who isable to access the information collected on-line and analyze trends indata.

Figure 3.1: GiraffPlus system with extensions

Data collection is organized with the help of physiological and envi-

18 Chapter 3. Methodology

ronmental sensors, which are discreetly placed inside the apartment andnot visible. The list of deployed environmental actuators can vary de-pending on monitoring requirements, but normally include RFID tags,pressure mats, switcher sensors etc. Collected data is continuously pro-cessed with context recognition technique [39], which is able to inferparticular patient’s activity (cooking, exercising, taking a shower, sleep-ing), performed during the day [40]. The choice of physiological sensorsdepends entirely on user’s health conditions and, therefore, strictly in-dividual. Currently deployed devices are pulse oximeter, blood pressuresensor and electronic weights, which perform occasional spot checkingand send collected data to the global database for further processing.

We utilize GiraffPlus system as a primary platform and make anattempt to improve its capabilities by introducing android-based contin-uous monitoring of physiological parameters. This research methodologycan help us to combine unrelated sources of data under developed ar-chitecture, test various integration algorithms on real-life dataset andassess major parameters of created system (i.e. reliability, interoperabil-ity, user acceptance). The main focus is made on incorporation of thefall detection algorithms based on acceleration data into a monitoringframework. Basically, we are required to perform integration into a pre-viously developed architecture, which can also help to assess flexibilityof chosen approach in terms of data synchronization. As a further work,we plan to develop a smartphone-based algorithm, deploying contextualdata from the the smart home in conjunction with physiological measure-ments. The theoretical structure of the proposed method is presented inPaper B.

Chapter 4

Research Contribution

In this research work we plan to present a number of relevant contri-butions. Firstly, we explore communication protocols including bothstandard Bluetooth Serial Port Profile and novel Health Device Profile(HDP) with the main focus on reliability, interoperability and contin-uous connectivity. As a result a user API is developed with a numberof functionalities providing reliable connection between multiple medicalsensors and mobile device for further extended data collection.

Following the previous contribution we initiate a series of long-termdata collection to obtain patient’s vital parameters. At this stage wealso make an attempt to involve the inbuilt sensor functionality of theandroid device into the monitoring process for subsequent multi-sensoranalyses. It is a step towards the correlation analyses between varioussources of data (i.e. patient’s activity, pulse, body orientation, bloodpressure). As a result a number of major contributions is obtained in-cluding fall detection algorithm which is able to integrate the physiolog-ical parameters (i.e. pulse rate, oxygenation) into a monitoring process.We consequently incorporate the context information collected from en-vironmental sensors to improve a mobile-based fall detection process. Inthis case a novel technique is developed, which deploys statistical methodfor time series data and adopt it for multi-sensor fusion. A probabilisticmodel based on dynamic Bayesian network (DBN) is described in Pa-per B. Moreover, it is required to investigate the state-of-art in fusionalgorithms for multi-modal healthcare systems. Therefore, we initiatedan extensive scientific search procedure resulted in a review paper de-

19

20 Chapter 4. Research Contribution

scribing most of the latest studies within multi-sensor fusion based falldetection approach. These results will be highly valuable for a plannedfurute work, when additional sensor functionality will be included intothe system.

Evaluation part of the thesis will be implemented through a series ofreal-life experiments and case studies involving elderly representatives.The first trial was initiated for testing a fall detection algorithm andinvolved healthy volunteers performing different types of falling duringice-skating (Paper A). Additional data collection was performed at a lo-cal medical facility involving elderly people in their home environment.Patients performed their daily life activities while being monitored bywearable and environmental sensors. This type of experimental set upis still rare in healthcare research connected with falling and thereforecan contribute to further improvement of fall detection algorithms andremote monitoring in general. It is planned to utilize these data forfuture publications. Specific contributions corresponding to particularpublication are discussed in the following sections.

4.1 Paper A

Evaluation of the Android-Based Fall Detection System with Physiolog-ical Data Monitoring. Gregory Koshmak, Maria Linden, Amy Loutfi,35th Annual International Conference of the IEEE EMBS Osaka, Japan,3 - 7 July, 2013.

Short Summary: Healthcare costs are expected to grow rapidlyduring the following decades due to the aging of ”baby boomer’s” gen-eration. At the same time, emergency situations associated with fallsare considered to be one of the major problem among elderly. This pa-per proposes a framework which uses mobile phone technology togetherwith physiological data monitoring in order to detect falls. The systemcarries out collecting, storing and processing of acceleration data withfurther alarm generating and transferring all the measurements to re-mote caregiver. To perform evaluation, an experimental setup involvingnovice ice-skaters were carried out to obtain realistic fall data and ex-amine the effects of falling on physiological parameters. A fall detectionalgorithm has been designed to cope with large variations of movementin the torso. The on-line algorithm operating showed performance re-sults of 90% specificity, 100% sensitivity and 94% accuracy.

4.2 Paper B 21

ContributionsIn the following study we address the first (Questions A) and second(Questions B) research questions and make a first attempt to combineunrelated sorts of data for reliable fall detection. Physiological measure-ments collected from pulse oximeter are merged with acceleration valuesobtained through the sensors inbuilt into a smartphone. A special mon-itoring framework is designed based on android device which allows tocollect, store and analyze data on-line during the long-term monitoringprocess. At the same time, we utilize smartphone processing capabili-ties and enable data synchronization, fall detection algorithm and alarmgeneration. Thresholding approach is deployed for fall detection dueto overall simplicity and low battery consumption comparing to othermethods. The main focus is done on evaluation part where we test de-veloped algorithm in a series of real-life experiments involving healthyvolunteers. In this way data collection contains natural falls instead ofsimulation, which helps to test reliability of the system in situations closeto real life.

My Contribution: I am the main author of the paper and proposeddesign of the fall detection algorithm with two types of communicationbetween medical sensor and processing device. I have also organizedevaluation procedure including data collection, on-line processing andposterior analyses of collected data.

4.2 Paper B

Dynamic Bayesian Networks for Context-Aware Fall Risk Assessment.Gregory Koshmak, Maria Linden, Amy Loutfi, A Special Issue of Am-bient Assisted Living (AAL): Sensor, Architectures and Applications.

Short Summary: Aging population is considered to be a majorproblem in modern healthcare. At the same time, fall incidents oftenoccur among elderly and cause serious injuries affecting their indepen-dent living. Recent studies propose various algorithms for reliable andautomatic fall detection, employing wearable sensors or smart home en-vironments as a main source of data. We make an attempt to combineboth data sources into a single algorithm based on Dynamic BayesianNetwork. DBN is fusing mobile phone operated fall detector together

22 Chapter 4. Research Contribution

with contextual data received from smart home environment and out-puts fall risk probability. Elaborated method helps to minimize falsepositive alarms, which are likely to occur after processing each of themethods separately. Evaluation of the developed system is performedthrough matlab simulation, where monitoring process is activated for100 time steps. Each time step is represented by a single user activityand interferes with fall alarm, received from mobile fall detector every10th time slice. Joint probability is calculated for each case involvingcontextual component and filtered with a simple threshold. As a result,4 out of 10 alarms were classified as false during simulation, which suc-cessfully demonstrates high sensitivity of the designed approach.

Contributions: This study is mainly focused on the second (Ques-tion B) and third (Question C) research questions and expand previouslydescribed data collection including contextual sensor functionality into amonitoring process. Information collected by environmental actuators isdeployed to perform context-aware fall detection. Raw sensor data col-lected from motion detectors, pressure mats or switchers are processedby GiraffPlus context recognition module and converted into user activ-ities. We design a statistical model based on dynamic Bayesian networkto combine inferred activities with mobile-based fall detection alarm andperform multi-sensor fusion (see Paper B). As a result developed methodprovides a fall risk probability based on both data sources which increasesreliability of the system and eliminates false positive alarms.

My Contributions: I am the main author and developed previ-ously mentioned statistical model for automatic fall risk evaluation. Ihave also conducted a number of test simulations, which are included inthe experimental part of the paper.

4.3 Paper C

Challenges and Issues in Multi-Sensor Fusion Approach for Fall Detec-tion: Review Paper. Gregory Koshmak, Maria Linden, Amy Loutfi.

Short Summary: Emergency situations associated with falls areamong the major problems in modern healthcare. Following the recentdevelopment on ICT market significant number of solutions were pro-posed to track body movement and detect falls in daily life with themost effective approaches being recently reviewed. Based on analyzed

4.3 Paper C 23

reviews combination of sensor platforms from unrelated categories is con-sidered to be the most efficient approach and among the future trendsin the area. In the following publication we review and systematizestudies with a special focus on multi-sensor fusion based fall detectionsystems. Our goal is to explain main differences compared to the singlesensor-based approach, identify typical categories, provide informationon possible issues and discuss trends for future work. We also give abrief overview of the recently developed method for multi-sensor fusionfall detection deploying Dynamic Bayesian Networks.

Contributions: In this paper we address the last research question(Question D) and conduct an explicit search in fall detection methodsbased on multi-modal sensor fusion. The main goal of the study is topresent a better insight on state of the art within modern multi-sensorfusion systems developed for elderly people in general. Additionally weinitiate a discussion regarding benefits, challenges and open issues ofproposed approach based on search results.

My Contributions: I am the main author and established basicsearch criteria to obtain optimal results. I also propose a particularclassification structure in order to present all search results in the mostconvenient manner.

Chapter 5

Conclusions and FutureWork

5.1 Conclusions

In presented thesis work we establish a monitoring framework as a partof the cross-disciplinary research towards a full-scale monitoring networkfor elderly people at their home environments. In attempt to answer thefirst (Question A) and second (Question B) research questions, we startby developing smartphone-based fall detection system with basic on-lineprocessing algorithms. According to the preliminary set up, the systemis able to retrieve data from different types of medical sensors deployingseveral communication protocols, store these data for further processing,apply fall detection analyses and notify caregivers in case of emergency.Additionally, up to seven devices should be able to establish simulta-neous stream channels via Serial Port Profile or Health Device Profile,which provides reliable data transfer and re-connection capability in caseof channel interruption.

As a next step, developed system is upgraded to serve as a supple-mentary platform integrated into a smart home. It can operate as an ex-tra sensor device providing continuous physiological measurements (e.g.pulse, oxygenation and physical activity) on a long-term basis. Workingon answers to the second (Question B) and third (Question C) researchquestions, we expand the initial set up and include environmental sen-

25

26 Chapter 5. Conclusions and Future Work

sors as a part of monitoring network. In this case, initial pre-processingstep of collected measurements is followed by more sophisticated anal-yses including multi-sensor fusion and fall detection. The exploitationenvironments are represented by patients home or medical care centerand provide contextual data, which is incompatible with the rest of themeasurements. Therefore, sensor and information fusion is an impor-tant component of the monitoring system in terms of data processing.A special focus is made on applying multi-variant analyses for accurateand reliable fall detection. Theoretical model of the proposed systemresulted in a journal publication described in Paper B. Prior to its prac-tical implementation we conduct a review study in order to investigatethe forth research question (Question D). Search results provide a bet-ter insight on recent multi-fusion algorithms for healthcare systems ingeneral and fall detection components in particular.

Finally, we perform a real-life data collection with developed systemin its current form involving elderly population and local care center.During established data collection patient is being daily monitored bywearable device with inbuilt fall detector functionality. In case fall-likeevent has been registered by the system, medical staff receives instantnotifications and provides required assistance. Described scenario is stillconsidered to be ”work in progress” as well as discussion on possibleoutput form, corresponding to research Question C of the thesis. Mainfactors affecting the choice of the final output are working environmentor monitoring purposes. It can potentially be represented by databasestorage, early alarm generation or trend analyses.

5.2 Future Work

Presented thesis work is focused on remote monitoring and automaticfall detection for elderly people in their homes. We conducted a multi-disciplinary research and made an attempt to answer announced researchquestion, however, there is still a fair amount of work to be done in fu-ture. Firstly, it is expected to replace currently employed fall detectionalgorithm based on thresholding approach with machine learning tech-nique. It will help to improve reliability of the system and decrease theamount of false positive alarms. Moreover, it is essential to incorporateadditional sensor functionality, providing physiological data in order to

5.2 Future Work 27

cover wider spectrum of emergency situations. As a result, this willcause further investigations on data fusion techniques including on-lineimplementation of previously designed scheme based on DBN. Finally, itis required to extend evaluation procedure based on ”patients-caregiver”communication and include more elderly volunteers into experimentalwork. This will help to assess previously created frameworks and giveexplicit feedback on user demands and requirements.

Bibliography

[1] Department of Economic Population Division and United NationsSocial Affairs. World Population Ageing. 2009.

[2] Gunnar Akner. Nutrition och fysisk funktion/fysisk aktivitet hosaldre personer. Technical report, Orebro Universitet, April 2009.

[3] Maged N Kamel Boulos, Steve Wheeler, Carlos Tavares, and RayJones. How smartphones are changing the face of mobile and par-ticipatory healthcare: an overview, with example from eCAALYX.Biomedical engineering online, 10(1):24, January 2011.

[4] Gregory Koshmak. An android based monitoring and alarm systemfor patients with chronic obtrusive disease. Master thesis, OrebroUniversity, 2011.

[5] Juan Cheng, Student Member, Xiang Chen, and Minfen Shen. AFramework for Daily Activity Monitoring and Fall Detection Basedon Surface Electromyography and Accelerometer Signals. 2012IEEE Journal of Biomedical and Health Informatics, pages 38 –45, December 2012.

[6] Marcin Bajorek and Jedrzej Nowak. The role of a mobile device ina home monitoring healthcare system. Proceedings of the FederatedConference on Computer Science and Information Systems, pages371–374, 2011.

[7] Stefan Dernbach, Barnan Das, Narayanan C. Krishnan, Brian L.Thomas, and Diane J. Cook. Simple and Complex Activity Recog-nition through Smart Phones. 2012 Eighth International Conferenceon Intelligent Environments, pages 214–221, June 2012.

29

30 Bibliography

[8] Peihusan Tsai Jane W.S. Liu John K. Zao, Mei-Ying Wang. Smartphone based medicine in-take scheduler, reminder and monitor.2010 12th IEEE International Conference on e-Health NetworkingApplications and Services (Healthcom), pages 162–168, June 2010.

[9] Dominik Kobylarz and Jacek Danda. A Common Interface forBluetooth-based Health Monitoring Devices. 2013 29th SouthernBiomedical Engineering Conference, pages 153–154, May 2013.

[10] Kohei Arai. Wearable physical and psychological health monitoringsystem. Science and Information Conference (SAI), 2013, pages133–138, 2013.

[11] Matthias Wagner and Benjamin Kuch. Android based Body AreaNetwork for the evaluation of medical parameters. 10th Inter-national Workshop on Intelligent Solutions in Embedded Systems,pages 33–38, 2012.

[12] Zhang Aihua and Hu Linghao. The system of pulse monitoringbased on Windows Mobile. Business Management and ElectronicInformation (BMEI), 2011.

[13] M Kozlovszky and L Bartalis. Personal health monitoring with An-droid based mobile devices. Information & Communication Tech-nology Electronics & Microelectronics (MIPRO), pages 326–330,2013.

[14] N. Noury and A. Fleury. Fall detection-principles and methods. Pro-ceedings of the 29th Annual International Conference of the IEEEEMBS, pages 1663–1666, January 2007.

[15] Damien Brulin and Estelle Courtial. Multi-sensors data fusionsystem for fall detection. Proceedings of the 10th IEEE Interna-tional Conference on Information Technology and Applications inBiomedicine, pages 1–4, November 2010.

[16] A Leone and G Diraco. A multi-sensor approach for people falldetection in home environment. Workshop on Multi-camera andMulti-modal Sensor Fusion Algorithms and Applications, pages 1–12, 2008.

Bibliography 31

[17] Charalampos N. Doukas and Ilias Maglogiannis. Emergency fallincidents detection in assisted living environments utilizing motion,sound, and visual perceptual components. IEEE Transactions onInformation Technology in Biomedicine, 15(2):277–289, 2011.

[18] Mitja Lustrek, Hristijan Gjoreski, Simon Kozina, BozidaraCvetkoviu, Violeta Mirchevska, and Matjaz Gams. DetectingFalls with Location Sensors and Accelerometers. Proceedings ofthe Twenty-Third Innovative Applications of Artificial IntelligenceConference, (2006):1662–1667, 2010.

[19] Bingbing Ni, Nguyen Chi Dat, and Pierre Moulin3. RGBD-Camerabased get-up event detection for hospital fall prevention. IEEE In-ternational Conference on Acoustics, Speech and Signal Processing(ICASSP), pages 1405–1408, 2012.

[20] Bruno Neves Ferreira, Vania Guimaraes, and Hugo Sereno Ferreira.Smartphone based fall prevention exercises. 2013 IEEE 15th In-ternational Conference on e-Health Networking, Applications andServices (Healthcom 2013), (Healthcom):643–647, October 2013.

[21] Naruhiro Shiozawa and Shin Arai. Gait analysis of sit-to-walk mo-tion by using portable acceleration monitor device for fall preven-tion. Proceedings of the IEEE-EMBS International Conference onBiomedical and Health Informatics (BHI 2012), 25(Bhi):563–564,2012.

[22] Carlo Tacconi, Sabato Mellone, and Lorenzo Chiari. Smartphone-based applications for investigating falls and mobility. 2011 5thInternational Conference on Pervasive Computing Technologies forHealthcare (PervasiveHealth) and Workshops, pages 258–261, 2011.

[23] Stevan Marinkovic and Riccardo Puppo. Implementation and test-ing of a secure fall detection system for Body Area Networks. Proc.27th International Conference on Microelectronics (MIEL 2010),pages 16–19, 2010.

[24] Patrick Crilly and Vallipuram Muthukkumarasamy. Using smartphones and body sensors to deliver pervasive mobile personalhealthcare. 2010 Sixth International Conference on Intelligent Sen-sors Sensor Networks and Information Processing, pages 291–296,2010.

32 Bibliography

[25] Hamed Ketabdar and Tim Polzehl. Fall and emergency detectionwith mobile phones. In Proceeding of the eleventh internationalACM SIGACCESS conference on Computers and accessibility AS-SETS 09, page 241. ACM SIGACCESS, ACM Press, 2009.

[26] Bruce Moulton Mitchell Yuwono, Steven W. Su. Fall Detectionusing a Gaussian Distribution of Clustered Knowledge , AugmentedRadial Basis. Proceedings of the 6th International Conference onBroadband Communications Biomedical Applications, pages 145–150, November 2011.

[27] Christoph Dinh and Matthias Struck. A new real-time fall detectionapproach using fuzzy logic and a neural network. 6th InternationalWorkshop on Wearable Micro and Nano Technologies for Personal-ized Health (pHealth), pages 57–60, June 2009.

[28] Mark V Albert, Konrad Kording, Megan Herrmann, and ArunJayaraman. Fall classification by machine learning using mobilephones. PLoS ONE, 7(5), 2012.

[29] A. K. Bourke, P. van de Ven, M. Gamble, R. O’Connor, K. Mur-phy, E. Bogan, E. McQuade, P. Finucane, G. Olaighin, and J. Nel-son. Evaluation of waist-mounted tri-axial accelerometer basedfall-detection algorithms during scripted and continuous unscriptedactivities. Journal of biomechanics, 43(15):3051–3057, November2010.

[30] Hristijan Gjoreski, Mitja Lustrek, and Matjaz Gams. Context-Based Fall Detection Using Inertial and Location Sensors. AmbientIntelligence, 2012.

[31] Qiang Li and John A Stankovic. Grammar-based, posture- andcontext-cognitive detection for falls with different activity levels.Proceedings of the 2nd Conference on Wireless Health WH ’11, 2011.

[32] Alessandro Leone, Giovanni Diraco, and Pietro Siciliano. Context-Aware AAL Services through a 3D Sensor-Based Platform. Journalof Sensors, 2013:1–10, 2013.

[33] J.W.Zheng, T.H.Wu, and Y.Zhang. A Wearable Mobihealth CareSystem support in Real-time Diagnosis and alarm. Med Bio EngComput, pages 877–885, September 2007.

[34] Topi Pulkkinen, Young-Sung Son, Joohyung Lee, Yann-Hang Lee,Mikko Sallinen, and Jun-Hee Park. Progressive monitoring andtreatment planning of diabetes mellitus in smart home environ-ment. 2013 IEEE International Conference on Consumer Electron-ics (ICCE), pages 206–207, January 2013.

[35] Juan a. Botia, Ana Villa, and Jose Palma. Ambient Assisted Livingsystem for in-home monitoring of healthy independent elders. ExpertSystems with Applications, 39(9):8136–8148, July 2012.

[36] JV Lee, YD Chuah, and KTH Chieng. Smart elderly home monitor-ing system with an android phone. Int. J. Smart Home, 7(3):17–32,2013.

[37] Gabriel Villarrubia, Juan F De Paz, Javier Bajo, and Juan M Cor-chado. Ambient agents: embedded agents for remote control andmonitoring using the PANGEA platform. Sensors (Basel, Switzer-land), 14(8):13955–79, January 2014.

[38] C Silvia, A. Cesta, L. Coraci, and L. Karlsson. GiraffPlus: Com-bining social interaction and long term monitoring for promotingindependent living. 2013 6th International Conference on HumanSystem Interactions (HSI), pages 578–585, June 2013.

[39] Lars Karsson, Lia Silva-lopez, and Jonas Ullberg. D 3 . 1 ContextInference and Configuration Planning Prototypes. Technical ReportJanuary 2012, 2014.

[40] Federico Pecora, Marcello Cirillo, Francesca Dell Osa, Jonas Ull-berg, and Alessandro Saffiotti. A constraint-based approach forproactive, context-aware human support. Journal of Ambient In-telligence and Smart Environments, 4:347–367, 2012.

II

Included Papers

35

Chapter 6

Paper A:Evaluation of theAndroid-Based FallDetection System withPhysiological DataMonitoring

Gregory A. Koshmak, Maria Linden, Amy Loutfi35th Annual International Conference of the IEEE Engineering in Medicineand Biology Society (EMBC), pages 1164 – 1168, Osaka, Japan, 2013

37

Abstract

Aging population is considered to be major problem in modern health-care. At the same time, fall incidents often occur among elderly andcause serious injuries affecting their independent living. This paper pro-poses a framework which uses mobile phone technology together withphysiological data monitoring in order to detect falls. The system carriesout collecting, storing and processing of acceleration data with furtheralarm generating and transferring all the measurements to remote care-giver. To perform evaluation, an experimental setup involving noviceice-skaters were carried out to obtain realistic fall data and examine theeffects of falling on physiological parameters. A fall detection algorithmhas been designed therefore to cope with large variations of movement inthe torso. The on-line algorithm operating showed performance resultsof 90% specificity, 100% sensitivity and 94% accuracy.

6.1 Introduction 39

6.1 Introduction

Aging population has been one of the main concerns in most developedcountries during the last decade [1]. Most elderly people suffer fromwider spectrum of various diseases and more emergency situations suchas fall are likely to occur [2]. As a result, they need to be transportedto the hospital, observed and provided with medical help if health con-dition is at risk. However, remote monitoring can help to prevent de-scribed scenario, significantly reduce healthcare costs and at the sametime maintain patient’s independent lifestyle [3].

Fall injury is considered to be one of the most common and dan-gerous risks among elderly population. The estimated fall incidence forboth hospitalized and independently living people over 75 is at least 30%every year. Nearly half of nursing home residents fall each year, with40% falling more than once [4]. These accidents can have both physical[5] (often head injury) and psychological [6] (fear of falling) effect.

With the recent development on mobile market, smart-phones startto play an important role in modern healthcare systems [3]. New featurescreate new opportunities to use smart-phones or tablets for managingand presenting medical data. The list of possible applications has beengrowing along with market development: early detection of Alzheimer’sdisease [7], face-to-face communication between doctor and patient[8],complex activity recognition [9] and medicine in-take assistance [10].Modern smart-phones equipped with an accelerometer sensor are alsocommonly used as fall detection tools [11][12][13][14]. They replace bothprocessing mode and a communication tool while maintaining relativelysmall size. A choice of processing algorithm depends on final applicationof the system and varies in different studies. Some of the recent imple-mentation methods apply Gaussian distribution of clustered knowledge[15], neural network [16] and machine learning techniques [17]. However,most of them are initially based on three essential parameters associatedwith falls: impact, velocity and posture. According to the recent article,combining impact and posture while analyzing the fall case is enough tocreate a reliable algorithm [18].

In the current research we intend to develop a mobile-based fall de-tection system with physiological data monitoring. This work is a partof the collaborative project1 focusing on independent living for elderlypeople. The main idea is to to perform activity monitoring (using phone

1http://www.giraffplus.eu/

40 Paper A

accelerometer, figure 6.1), detect the fall and notify a nursing personnel.For evaluation of the algorithm we asked 7 healthy young people with aminor ice-skating skills to wear fall detection system and perform regularskating activity. Afterwards, we calculate the sensitivity and specificityof the algorithm based on collected data. Moreover, we perform an off-line analysis danger point and anomaly detection to demonstrate thecorrelation between major physiological parameters and fall incident.

In the rest of the paper we discuss some of the recent publications andscientific background within the healthcare systems and related areas.It will be followed by the systems design overview and implementationof the proposed ideas.

Figure 6.1: Smart-phone coordinate axis

We finally describe experimental part, demonstrate and discuss cur-rent results and propose some ideas for a future work.

6.2 Implementation

6.2.1 Fall Detection Algorithm

As it was previously discussed in Introduction, three main parametersare associated with falls: impact, posture and velocity. Combining twoof them, however, will provide accurate results and high level of reliabil-

6.2 Implementation 41

ity [18]. A fall, including impact, has been defined to have four distinctphases [19]: (1) the pre-fall phase, which is where most normal activ-ity of daily living (ADL) occur, but may contain some instability; (2)the critical phase, when the body experiences a sudden movement to-ward the ground, ending with a vertical shock; (3) a post-fall phase,when the body comes to rest and the body is lying and (4) a recov-ery phase. We deploy embedded 3-axis accelerometer sensor and designmulti-functional application based on Android operating system. Thisapplication will carry out both activity and physiological data monitor-ing, which means that more than one process will be ran on the phone atthe same time. Therefore our goal is to create a simple, but yet sufficientalgorithm with a low power consumption rate.

The mobile device is located on the central part of the waist, corre-sponding to the body center of gravity. We split the process into threemain stages. The system starts to receive data from accelerometer andcalculate the overall acceleration value (empirically developed activitymeasure) Act :

Act = E[|v2a − E[v2

a]|]; va =√a2x + a2

y + a2z, (6.1)

where ax, ay, az correspond to acceleration along 3 axis of the smart-phone coordinate system (see figure 6.1). If the impact is registered (Act≥ threshold) during ADL, we consider it as a potential fall risk, pausemonitoring process and check orientation of the phone. Using the sameacceleration values ax, ay, az we determine devices’ Euler angles withrespect to the earth’s gravitational attraction:

roll = asin(ax

gravity), (6.2)

pitch = asin(az

gravity). (6.3)

The very same method is used in games or bubble level applicationsfor smart-phones. A combination of thresholds for roll and pitch anglescorresponding to horizontal position of the body triggers an alarm sig-nal informing user about the fall. Before forwarding this message to asmart-home database system or directly to a nursing personnel, user isallowed to cancel an alarm if he/she thinks the fall was false positive andno assistance is required. Otherwise, algorithm complements an alarmwith additional data.

42 Paper A

Firstly we deploy accelerometer values occurred during the impactin order to determine falls direction (forward, backward, left-side, right-side). Each direction corresponds to a particular combination of acceler-ation numbers along the axis (see figure 6.2). We attach a specific valueto each direction and include this data in the alarm message. Further-more, physiological measurements are collected during the monitoringprocess as a complementary unit. Every time system detects a fall, thelast pulse and oxygen saturation value is attached and sent along withthe person’s location and fall direction number.

50 100 150 200 250 300

−10

0

10

Data, samples

Fo

rwa

rd F

all

roll

yaw

pitch

50 100 150 200 250 300 350 400

−10

0

10

Data, samples

Ba

ckw

ard

s F

all

50 100 150 200 250 300

−10

0

10

Data, samples

Rig

hts

ide

Fa

ll

Figure 6.2: Roll, Pitch, Yaw variation during different types of falls

After all, data sample consists of alarm index, current geographi-cal location and physiological measurements. Simplicity of presentedapproach does not affect its sufficiency which was proven in the experi-mental part of the paper (see Section 6.3).

6.3 Experiments and Results

Algorithm evaluation is a primary step preceding stable operating ofthe system. The most common approach is simulation process when

6.3 Experiments and Results 43

young and healthy volunteers are asked to perform falls multiple timesin different directions. Experiments with elderly people would be morerelevant, but can obviously lead to an injury and therefore inappropriate.We found a fair solution which evaluate the system in a real environmentand contains both regular movements and fall incidents. After a personalconsent we ask 7 volunteers with minor winter sport skills to wear mobilefall detection system while performing ice-skating activity. Due to thelack of experience in this area, falls occurred consistently, together withfrequent and random motions of the torso. No one was injured or hurtand experimental part finished successfully.

12:00 12:05 12:10 12:15

95

100

normal min

normal max

Time, hours

O2

−sa

tura

tio

n

O2−saturation

O2 (mean)

12:00 12:05 12:10 12:1580

100

danger pointdanger point

danger point

Time, hours

Pu

lse

ra

te

pulse rate

pulse (mean)

anomalies

12:00 12:05 12:10 12:150

100

200

Time, hours

Ave

rag

e a

ctivity

12:00 12:05 12:10 12:150

0.5

1

Time, hours

De

tecte

d f

all

Figure 6.3: Monitoring process and fall detection results (Test 5). Con-tains (1) oxygenation variation, (2) pulse rate variation with processedanomaly and danger point detection, (3) overall acceleration value of theperson and (4) registered falls with corresponding time stamps

The above proposed approach has a number of benefits. Firstly, weare able to test the system in a situation close to real life when falls

44 Paper A

appear unintentionally. Secondly, a fair amount of physical activity isinvolved in ice-skating which can be considered as a stress test to avoidfalse positive results. Additionally, we perform physiological data moni-toring during the experiment collecting heart rate and oxygen saturationmeasurements depicted on figure 6.3. Although it is nearly impossibleto build a reliable fall prediction algorithm based on these data, it cangive a better insight on the nature of falling behavior and help in thefuture research. All the volunteers performed from 15 up to 30 min ofice-skating during each session resulted in 50 falls overall.

Table 6.1: Test Summary

AverageTP FN FP TN Pulse, BPM SPO2, %

Test 1 3 2 0 4 107.1 95.5Test 2 3 0 0 5 96.3 94.5Test 3 9 0 0 4 91.1 98.6Test 4 7 2 0 5 109.4 97Test 5 7 0 0 5 94.8 96.3Test 6 7 0 0 5 83.4 93.7Test 7 9 1 0 5 112.4 93.6Totall 45 5 0 33

Assuming the individual approach during the test, we split data andillustrate personal statistic for each participant including number of falls(1) detected (true positives TP) (2) not detected (false negatives FN) bythe algorithm, and number of activity of daily living (3) detected (falsepositive FP) (4) not detected (true negative TN) as fall events. ADLin our case was represented by various exercises and activities on icewhich were not recognized as falls. An extra column displays the aver-age value of pulse/oxygen saturation variation during experimental part.It is important to mention that fall detection procedure was performedin on-line mode during the monitoring process and no post-processingwas involved at this stage.

Subsequent evaluation is based on computing sensibility and sensi-tivity of the algorithm according to the following formulas:

Sensitivity =TP

TP + FN∗ 100%. (6.4)

6.3 Experiments and Results 45

and

Specificity =TN

TN + FP∗ 100%. (6.5)

Out of 50 falls 45 were successfully detected (TP) and only 5 were ignoredby the algorithm (FN) which resulted in 90% sensitivity. None of theregular skating activity or exercises were recognized as falls, giving 100%specificity. An extra accuracy parameter

Accuracy =TP + TN

TP + TN + FN + FP∗ 100%, (6.6)

corresponding to percentage of true discrimination between falls andADL resulted in 94%. It shows that it is possible to achieve a high levelof reliability for the algorithm combining only two parameters (impactand orientation) when performing the task. Every time a high accel-eration value occurred system considered it as a potential fall case andinitiated the orientation check.

It is important to mention the difference in acceleration levels be-tween ice-skating and elderly people activity. An idea was to test thesystem in rough environment with a lot of disturbances and show thatit can only react when a real fall occurred. Therefore, it is importantto preliminary set the impact threshold to a bigger number to avoid theunnecessary processing. However, the average acceleration lever of el-derly is expected to be significantly lower which requires to adjust thethreshold accordingly. At the same time, we believe system performancewill improve when it comes to distinguishing between falls and regularactivities in real life.

According to overall statistics (see Table 6.1), there were 5 false neg-ative cases when the fall was not detected during the monitoring. Wetry to clarify the reason why system failed and examine both TP and FPcases, where both acceleration values correspond to a fall behavior (seefigure 6.4). In occurred situation, algorithm triggers further processingand orientation check. However, in the second case, posture was notidentified as horizontal and therefore, alarm was not fired. We explainthis exception by singularity of the developed algorithm and particularbehavior of volunteers during the test. The system is set to provide afew seconds time gap between the impact and orientation check to sepa-rate two different phases. In many cases, a person would quickly changethe orientation of the body (e.g stand up) precisely after fall occurredwhich system identifies as true negative. A simple and straightforward

46 Paper A

500 1000 1500 2000 2500 3000 3500

100

150

200

250

Data, samples

Av

era

ge

act

ivit

y

19:53:00 19:53:15 19:53:30 19:53:45 19:54:00 19:54:15 19:54:30 19:54:450

0.2

0.4

0.6

0.8

1

Time, hours

De

tect

ed

fa

ll

FALSE NEGATIVE

Figure 6.4: A data cut demonstrating True Positive (TP) and FalseNegative (FN) algorithm results

approach to avoid this issue is to reduce an above mentioned time gap.Moreover, this will not be a problem in case of elderly people, when thefall is more fixed and recovery period is significantly longer. In eithercase if the person was able to stand up immediately after the fall, thissituation can still be detected from activity variation but does not haveto be reported as an emergency alarm.

Besides from alarm generation, system can be configured to performthe consistent monitoring of pulse and oxygen saturation. These datacan be subsequently transfered to the server, stored in the database andaccessed by caregivers. In this case, medical personnel can not only ob-serve the alarm list, but also follow the variation of physiological dataand analyze its behavior before or after the fall.

6.4 Conclusion

Following the recent demand in independent living for elderly people inthe modern healthcare, we developed an algorithm for automatic fall de-tection system with physiological data monitoring. The technique was

6.4 Conclusion 47

implemented on the smart-phone platform running android operatingsystem and carries out collecting, storing and processing of accelerationdata. In case of fall incident, system generates an alarm and informsremote caregiver about patients current location. High reliability ofthe developed system is proven during evaluation part where algorithmshowed 90% sensitivity, 100% specificity and 94% accuracy. Monitoringprocess is complemented with physiological data such as heart rate andoxygenation collected via external sensor. This features gives a betterinsight on fall tendency and tracks essential medical parameters beforeand after the incident.

However, the fact that certain amount of falls were not detected bythe system opens possibilities for further improvements. One possibleapproach is to tune impact and orientation thresholds depending on themonitoring purposes and conditions. Adding extra parameter such asvelocity can improve the algorithm performance, but affect the compu-tational demand at the same time. One of the most reasonable paths forfuture development is integration with a smart home system, a part ofthe ambient assisted living area with a specific aim of monitoring health,safety and well-being of the patient. For example a fall alarm messagecan trigger the system to check current location of the patient insidesmart home, how much time passed since he/she was moving and whichservice was enabled before the fall occurred (e.g. television, microwaveor training simulator). It will improve the overall performance of thesystem, decrease emergency response time and give a better insight onfalling behavior in general. We also plan to proceed the experimentalpart and perform data collection in different study cases.

Bibliography




[4] Jiangpeng Dai, Xiaole Bai, Zhimin Yang, Zhaohui Shen, and DongXuan. Mobile phone-based pervasive fall detection. Personal andUbiquitous Computing, 14(7):633–643, 2010.

[5] Siv Sadigh, Anne Reimers, Ragnar Andersson, and Lucie Laflamme.Falls and fall-related injuries among the elderly: a survey ofresidential-care facilities in a Swedish municipality. Journal of com-munity health, 29(2):129–40, May 2004.

[6] B J Vellas, S J Wayne, L J Romero, R N Baumgartner, and P JGarry. Fear of falling and restriction of mobility in elderly fallers.Age and ageing, 26(3):189–93, May 1997.


49

50 Bibliography









Chapter 7

Paper B:Dynamic BayesianNetworks forContext-Aware Fall RiskAssessment

Gregory Koshmak, Maria Linden and Amy LoutfiA Special Issue of Ambient Assisted Living (AAL): Sensor, Architecturesand Applications, pages 9330-9348, 2014.

53

Abstract

Fall incidents among elderly often occur in the home and can cause se-rious injuries affecting their independent living. This paper presents anapproach where data from wearable sensors integrated in a smart homeenvironment is combined using a Dynamic Bayesian Network. The smarthome environment provides contextual data, obtained from environmen-tal sensors and contributes to assessing a fall risk probability. The eval-uation of the developed system is performed through simulation. Eachtime step is represented by a single user activity and interacts with a fallsensors located on a mobile device. A posterior probability is calculatedfor each recognized activity or contextual information. The output ofthe system provides a total risk assessment of falling given a responsefrom the fall sensor.

7.1 Introduction 55

7.1 Introduction

Aging populations have been one of the main concerns in most developedcountries during the last decade [1]. Most elderly people suffer from awider spectrum of various diseases and more emergency situations suchas falls are likely to occur [2]. As a result, they need to be transportedto the hospital, observed and provided with medical help if their healthcondition is at risk. However, remote monitoring can help to preventdescribed scenario, significantly reduce healthcare costs and at the sametime maintain patient’s independent lifestyle [3].

A growing trend in ICT is to combine monitoring components (e.g.sensors, actuators) into smart environments, and carry out observationsfor people with multiple chronic conditions at home. Such systems arepromoted in order to improve elderly patient’s level of freedom andsafety, which is one of the main issues in healthcare today. As fall injuryis considered to be one of the most common and dangerous risks amongelderly population it is reasonable that such fall detectors will also beintegrated components in future smart homes. Today, the estimated fallincidence for both hospitalized and independently living people over 75is at least 30% every year. Nearly half of nursing home residents falleach year, with 40% falling more than once [4]. These accidents canhave both physical [5] (often head injury) and psychological [6] (fearof falling) effect. Various methods has been proposed including smarthome approach and wearable sensor monitoring. However, fall risk min-imization is a complex problem and therefore requires a combination ofmeasures to be applied.

In the current research we deploy a wearable fall sensor in a smarthome environment containing environmental sensors and merge the datainto a single system and perform reliable fall detection. The approachadheres to information fusion, which is the process of dealing with combi-nation of information collected from disparate sources into one coherentstructure, and can be deployed by a system to make better decisions thanfrom single source [7]. In this paper, both components operate indepen-dently from each other but are fused using a Dynamic Bayesian Network(discussed in Section 7.3.3). The premise of the proposed method is anisolated fall detection algorithm which is based on android mobile phoneand operates as a wearable sensor. It has additional functionality forcapturing physiological measurement such as pulse and oxygen satura-tion of the patient. A smart home environment represents the second

56 Paper B

component of the system and covers various aspects of a patient’s dailymonitoring. In this case we are particularly interested in contextual datacollected from the RIF, pressure mats etc. These sensor readings are fur-ther transformed into separate activities, performed by users, fused withwearable sensor data and processed resulting in context-aware fall de-tection system. Work on detection of the activities is presented in [8].This paper’s contribution is on the sensor fusion method implementedthrough Dynamic Bayesian Network, which is commonly used for timeseries and statistical data.

This work is part of the collaborative project GiraffPlus1, which isfocused on independent living for elderly people. As an example of Am-bient Assisted Living (AAL) system, Giraffplus implies various types offunctionalities including continuous monitoring, alarm generation or di-rect communication with remote caregiver [9]. It contains a combinationof environmental and physiological sensors intended to capture most ofthe patients activities and changes in lifestyle or health conditions. Allthe data are stored in the global database through a special middleware,connecting different components of the system. GiraffPlus system is amulti-functional model that can cover different types of monitoring sce-narios and also includes a focus on fall detection.

The rest of this paper is organized as following: we start with relatedwork describing the latest developments in wearable sensors, smart homesystems and their role in fall detection area. We proceed with the mainframework and system overview, where we present a sensor fusion algo-rithm based on Dynamic Bayesian Network. This part is followed byevaluation work and consequent results. Finally we conclude with thesystem functionality and future perspectives in Section 7.6.

7.2 Related Work

7.2.1 Context-Aware Fall Detection

Smart homes are part of the Ambient Assisted Living area, responsiblefor continuous monitoring of elderly people in comfortable home environ-ment. In recent years, an increasing number of projects have been basedon this approach utilizing various components and application purposes.

1The study was partly financed by the EU-FP-7 project GiraffPlus and the Knowl-edge Foundation’s research profile Embedded Sensor Systems for Health (ESS-H)

7.2 Related Work 57

For example, a special nutrition advisor was proposed as an attemptto improve physical condition for elderly people with diabetes[10]. Itdemonstrates the possibility to improve efficiency of nutritional manage-ment through deploying Ambient Intelligence systems. Moreover, JuanA Botia, Ana Villa and Jose Palma presented their Necesity system withadaptive monitoring capabilities and exhaustive evaluation methodol-ogy that were integrated in the development process[11]. Researches ineCAALYX project (Enhanced Complete Ambient Assisted Living Ex-periment) take 24/7 monitoring of healthy older people one step furtherby refining and making it available to older people with multiple chronicdisease. A particular effort is made on communication with the userdeploying various sorts of interactive devices: TV-based Set Top Boxsystem, Customer-Premises Equipment and interactive TV [12]. It be-comes clear that smart environments equipped with various sensors canpotentially cover a significant number of health issue, providing essentialinformation about patients life status.

In the current research, however, we are particularly interested indeploying contextual data for accurate fall detection. Similar approachwas described in the article by Brulin et al.[13], where main idea is tofuse different types of data source channels into a special architecture,that allows us to ensure a constant acquaintance of different informa-tion from RIP detectors, thermopile or cameras. This developed systemalso performs on-line or posterior processing to derive posture or ori-entation of the user and triggers an alarm in case of fall risk. Variousattempts were made to improve this process by introducing additionalsources of information like surrounding audio captured by microphonearrays [14, 15] or current location of the user [16]. Alternatively, RGBD-camera is deployed in study by BinBing Ni et al [17] for hospital fallprevention. To prevent potential falls, once the event of patient getsup from the bed is automatically detected via Microsoft Kinect sensor,nursing personnel is alarmed immediately for assistance. However, con-textual data approach is lacking accelerometer measurements captureddirectly from the patients body which makes the system incomplete. Ina recent effort by Ferreira et al. [18], fall prevention system makes use ofcollected data from sensors in order to control and advice the patient oreven to give instructions to treat an abnormal condition to reduce thefall risk. Monitoring and processing data from sensors is performed by asmartphone that will issue warnings to the user and in gravity situationssend them to a caretaker. Moreover, relationship between acceleration

58 Paper B

of body’s center of gravity during sit-to-walk motion and a process offalling is investigated by Shiozawa, N et al. [19]. The result of discrim-inant analysis by using indexes with a significant difference revealed a90.3% correct prediction rate for falling.

Wearable sensors with inbuilt accelerometer can play an essentialrole as a complementary component for smart home environments andhas recently been deployed for accurate fall detection [20]. In our casea wearable device is replaced with a smartphone, which can serve asaccelerometer sensor and gateway mode at the same time.

7.2.2 Mobile Healthcare Integration

With the recent development on mobile market, smartphones begin toplay an important role in modern healthcare systems [3]. New featurescreate new opportunities to use smartphones or tablets for managingand presenting medical data. The list of possible applications has beengrowing along with market development: early detection of Alzheimer’sdisease [21], face-to-face communication between doctor and patient[20],complex activity recognition [22] and medicine in-take assistance [23].Modern smartphones equipped with an accelerometer sensor are alsocommonly used as fall detection tools [24, 25, 26, 27]. They replaceboth processing mode and a communication gateway while maintainingrelatively small size. A choice of processing algorithm depends on finalapplication of the system and varies in different studies. Some of therecent implementation methods apply Gaussian distribution of clusteredknowledge [28], neural network [29] and machine learning techniques [30].However, most are initially based on three essential parameters associ-ated with falls: impact, velocity and posture. According to the recentarticle, combining impact and posture while analyzing the fall incidentis enough to create a reliable algorithm [31].

In our research we deploy isolated fall detection system, implementedon the mobile device as an element of the integrated smart home envi-ronment. Similar approach was adopted in several studies with intentionto combine contextual data with essential accelerometer measurementsexploiting inertia and location sensors [32]. Qiang Li et al in [33] investi-gate a novel fall detection method that utilizes acceleration, posture andcontext information, where context can be presented by environmen-tal sensors (room location or furniture positions) and personal profiles(e.g. health status and age). Wireless accelerometer, 3-D camera and

7.3 Framework 59

microphone are being simultaneously processed by Leone et al to reacha better result in fall risk assessment [34]. All the presented studies,however, are lacking a reliable fusing technique, combining processingresults from independent components. In work by Zhang et al [35] anoff-the-shelf programmable sensing platform called Sun SPOT is used fordata recording. Context information is presented in several categories,covering main aspects of elderly living: (1) physical activity; (2) phys-iological condition; (3) personal health record; and (4) location. Eachcategory represents a separate variable or processing result (includingfall alarm from isolated algorithm) and merged into a Bayesian networkfor further statistical analyses. System’s output represents a probabil-ity of the fall to occur, given the contextual information. Formulatedapproach provides convenient tool for managing different data channels,but fails to consider time reference in the process. In many cases mostof the system elements directly depend on the their previous states,which should be taken into account. This requirement can be coveredby introducing an extension to regular probabilistic networks commonlyaddressed as dynamic Bayesian networks, a statistical model, that hasbeen previously deployed in a number of studies including sensor fusionvehicle localization and road matching [36] [37]. In Section 7.3, we showthe method of managing multi-variant healthcare data with DynamicBayesian Network, and describe the framework of the developed system.

7.3 Framework

7.3.1 Mobile-Based Fall Detection System

The first component of the integrated platform is isolated fall detectionsystem, which is implemented on the android-based mobile device andoperates as a wearable sensor. In our previous study [38] we deployedembedded 3-axis accelerometer, depicted on Figure 7.1 and designedmulti-functional application based on Android operating system device.This application is able to carry out both activity and physiological datamonitoring, with more than one process running simultaneously on thephone. Therefore simple, but yet sufficient algorithm with a low powerconsumption rate was created to satisfy these constraints.

The mobile device is located on the central part of the waist, corre-sponding to the body’s center of gravity. For maintaining accelerationaxis, the device is fixed in a special phone case attached to the users

60 Paper B

Figure 7.1: Smartphone coordinate axis

belt. The process is split into three main stages. The system starts toreceive data from accelerometer and calculate the overall accelerationvalue (empirically developed activity measure) Act :

Act = E[|v2a − E[v2

a]|]; va =√a2x + a2

y + a2z,

where ax, ay, az correspond to acceleration along 3 axis of the smart-phone coordinate system. If the impact is registered (Act ≥ threshold)during activity of daily living, we consider it a potential fall risk, pausemonitoring process and check orientation of the phone. Using the sameacceleration values ax, ay, az we determine devices’ Euler angles withrespect to the earth’s gravitational attraction:

roll = asin(ax

gravity),

pitch = asin(az

gravity).

The similar approach is used in games or bubble level applications forsmartphones [39]. Unlike methods utilizing only registered impact in ac-celeration, developed algorithm deploys current orientation of the body,which allows to avoid alarms during high-impact activities (e.g., sitting

7.3 Framework 61

chair, stumble, rising bed, etc.). A combination of thresholds for roll andpitch angles corresponding to a horizontal position of the body triggersan alarm signal informing the user about the fall. Before forwarding thismessage to a smart home database algorithm complements an alarm withadditional data. First we deploy accelerometer values occurred duringthe impact in order to determine falls direction (forward, backward, left-side, right-side). Each direction corresponds to a particular combinationof acceleration numbers along the axis (see Figure 7.1). Furthermore,physiological measurements are collected during the monitoring processas a complementary unit. Every time the system detects a fall, the lastpulse and oxygen saturation value is attached and sent along with theperson’s location and fall direction number. The combination of the ad-ditional data can vary depending on application purposes and currentset up.

The evaluation process in fall detection area is based on commonapproach, where young and healthy volunteers are asked to simulatefalls multiple times in different directions [40]. We found a fair solutionwhich evaluates the system in a real environment and contains both reg-ular movements and fall incidents. After a personal consent we ask 7volunteers with minor winter sport skills to wear mobile fall detectionsystem while performing ice-skating activity. Due to the lack of experi-ence in this sport, falls occurred consistently, together with frequent andrandom motions of the torso. The proposed approach helps to test thesystem in situations close to real life when falls appear unintentionally.Besides, a fair amount of physical activity is involved in ice-skating whichcan be considered as a stress test to avoid false positive results. All thevolunteers performed from 15 up to 30 min of ice-skating during eachsession which resulted in 50 falls overall. No participant was injured andexperimental part finished successfully. The evaluation process resultedin 90% sensitivity, 100% specificity and 94% accuracy of the developedalgorithm.

However, these results can not be entirely objective assuming thatthe final target-users of the current project are elderly people. In thiscase, we have to deal with a different level of body motions, higher riskof false positive alarms and data misinterpretation. Therefore, in orderto guarantee reliability of the final system and account for the mentionedconstraints, we integrate contextual data, collected in the smart homeenvironment into the common framework.

62 Paper B

7.3.2 Context Recognition

In our case contextual data are represented by a set of sensors locatedthroughout an apartment including PIR motion, door contact, pressuremats and power usage detectors. A special context recognition system,designed and described by Ullberg et al [41] infers human activities fromraw sensor data. The context recognition model consists of several inde-pendent modules including preprocessing, inference and extraction mod-ule.

The preprocessing module is fetching samples from the sensors lo-cated in the environment and build a higher lever representation of theevents that take place at home. For instance a timestamped series oftemperature readings can be ”down-sampled” to a single or a set oftemporal intervals indicating that the temperature was ”High” duringthe corresponding periods of time. The preprocessing is the only modulethat is fetching samples from the remote database [42].

Figure 7.2: Inferring ”Cooking” activity

The inference module infers activities from the intervals, previouslygenerated by preprocessing module. For instance, it can deploy the datacoming from the temperature sensor mounted to a warm-water pipeto deduce that inhabitant is showering. In this hypothetical case a rulecould define Showering as an activity that occurred During In Bathroomand Contains Temperature High Shower.

The last, extraction module is responsible for generating time-linesthat can be visualized to the user. Currently this module supports onlyone type of extraction method, which simply extracts the maximum du-ration interval for an activity. We can illustrate a context recognitionprocess on a simple example of how activity of ”Cooking” can be in-

7.3 Framework 63

ferred from intervals representing sensed data (see Figure 7.2). In orderto infer the context, the generated set of intervals is combined with amodel consisting of rules of the form:

Cooking = Stove ∧CookingDuringKitchen

These types of rules define abstract patterns of constrains in Allen’sinterval algebra [43]. The final output of the context recognition block isa set of activities of daily living (ADL), which are subsequently mergedtogether with the isolated fall detection algorithm results (described inSection 7.3.1). We deploy Dynamic Bayesian Network and design aspecific system, configured according to our case to perform multi-sensorfusion.

7.3.3 Dynamic Bayesian Network

As it was previously mentioned in Section 7.1, information fusion impliescombination, association and correlation of multiple data sources for abetter decision making. Generally, this process can be described withthe following formula:

θ = F (S1, S2, ..., Sn),

where Si denotes disparate sensor readings, F is a fusion function re-sponsible for merging data and θ represents the final output utilized fordecision-making [7]. In our particular case dynamic Bayesian network(DBN) is deployed as a fusion method with probabilistic inference as afusion function. The output θ is posterior probability of the fall risk,which we try to infer. Unlike standard probabilistic networks, DBNsare suitable for modeling dynamic events and represented by directedgraphical models of stochastic processes which are defined by means ofdirected acyclic graph (DAG) as follows:

P (Xt|Yt−1) =

T∏i=1

P (Xit |π(Y i

t )),

where X(i, t), Y (i, t) represent hidden and observed nodes (variables) ac-cordingly. A DBN model is made up of interconnected two time slices ofa static BN, with parent nodes Pa(Y(i,t)) either at time slice t or t − 1and the transition of BN, that satisfies the Markov process. The simplest

64 Paper B

kind of DBN is a Hidden Markov Model (HMM), which has one discretehidden node and one discrete or continuous observed node per slice. Itis depicted on Figure 7.3, where clear circles denote hidden layer andshaded represent observed nodes.

Figure 7.3: Hidden-Markov model (HMM)

The model is unrolled for 3 initial time slices, but structure is assumedto repeat as the system is unrolled further. Normally, to specify a DBN,we need to define the intra-slice topology (connections between vari-ables within one slice) and inter-slice topology (between variables fromdifferent slices), as well as parameters for the first two slices. The mostcommon task within the Bayesian networks, is to perform probabilis-

tic inference using Bayes theorem P (A|B) = P (A)P (B|A)P (B) . At the same

time, graphical model mentioned above also specifies a complete jointprobability distribution (JPD) over all the variables. Given the JPD,we can answer all possible inference queries by marginalization (sum-ming out over irrelevant variables). In case of using DBN the generalinference problem is to compute P (X(i, t0)|y(:, t1 : t2)), where X(i, t)represents the i′th hidden variable at time and t, Y (:, t1 : t2) repre-sents all the observed variables ( evidence) between times t1 and t2. Inother words, we try to calculate the current state of hidden variable,given the evidence. Several special cases of interest can be used for thispurpose including filtering, Viterbi, prediction, fixed-lag smoothing andfixed interval smoothing. Final effort is made to determine the posteriorprobability representing a fall risk for a subsequent decision-making. Inthe following section we describe representation and inference in DBNon our particular case of multi-sensor fusion for home monitoring.

7.4 System Integration 65

7.4 System Integration

Two independent components of the system described in Section 7.3 rep-resent different source channels and provide incompatible outputs (seeFigure 8.2). Wearable sensor operates accelerometer data and generatesalarms based on fall impact and orientation angles. Context recognitionretrieves data from environmental sensors and infers regular ADLs.

We deploy a statistical method described in Section 7.3.3 and com-bine both components into a single platform, based on Dynamic BayesianNetwork. In this case each activity derived from context or fall detectionalgorithm is represented by specific variable and comprise the evidenceof the network. Preliminary list of ADL, that can be performed by el-derly and detected by context recognition is introduced in Table 7.1 andcontains: (1) SL - sleeping, (2) Sh - taking a shower, (3) C - cooking,(4) TV - watching a TV, (5) Tr - training, physical exercise, (6) FA -fall alarm.

DBNSystem

Conext Recognition Fall Detector

Raw Sensor Data Accelerometer Data

ADL Fall Alarm

Figure 7.4: Integrated System

One extra variable is derived from the isolated fall detection algo-rithm and included into the system as FE - fall event. In this way, wetransform an alarm, based on accelerometer data into a statistical unitand merge it with the rest of the system. In a similar way we are able tointroduce another variable based on physiological data collected from themedical sensor. Nevertheless, the research work in the GiraffPlus projectis still in progress and final list of activities is yet to be confirmed. The

66 Paper B

flexibility of the system, however, allows us to adjust parameters and in-clude new variables as they are confirmed. Based on the outlook of theintegrated platform, it is possible to establish the essential goal of thestudies which implies estimating fall risk probability or, in other words,calculating FA.

Table 7.1: Preliminary list of activities

activity abbreviation detailsSleeping Sl pressure bed sensor + presence in the bedroom

Showering Sh tap sensor + presence in the bathroomCooking C stove sensor + presence in the kitchen

TV watching TV TV sensor + presence in the living roomFall Alarm FA final indication of the fallTraining Tr exercise machine sensor + accelerometer sensor

Fall Event FE fall indication received form the phone-based detector

As the initial step, we deploy matlab DBN toolbox [44] provided byKevin Murphy for building a statistical model. Commonly, DBN consistsof the graph structure (visualized on Figure 7.5) and parameters (see Ta-ble 7.2). Following the standard approach specified in Hidden MarkovModel DAG, shaded circles represent observed variables, whereas clearones are hidden nodes. As an initial step we define all the intra andinter connections between each element within the network. Once theconnectivities are set we adjust parameters for the listed variables beforeexecuting the inference process. The standard procedure implies formingsimple Conditional Probability Tables (CPT), which define the proba-bility distribution of a node (e.g. random variable) given its parents. InMatlab CPTs are stored as multidimensional arrays, where dimensionsare arranged in the same order as the nodes. In this way, conditionaltables for Sl (see Table 7.2), for instance, are first indexed by FA andthen Sl itself. Hence the child is always the last dimension. If a nodedoes not have a parent (as node FA in our case), the CPT is representedby the prior vector containing the initial probability distribution. Wealso make a convention that false = 1 and true = 2, which makes asimple example of the conditional dependency for Sl activity look as the

7.4 System Integration 67

Sl C Sh TV FE Tr

FA

FA

TrFETVShCSl

inter

intra

Figure 7.5: System’s DAG unrolled for 2 time slices

following: Pr(Sl = true|FA = true) = 0.1. This equation correspondsto a common life experience indicating an extremely low chance of fallingwhile the patient is asleep.

Table 7.2: Conditional probabilities

Sl FA probability1 1 0.42 1 0.61 2 0.92 2 0.1

TV FA probability1 1 0.52 1 0.51 2 0.872 2 0.13

Unlike simple Bayesian Networks, DBN can contain variables that areinter-connected with their parents. In this case we introduce additionalCPT (see Table 7.3) for the nodes from the second time slice, which aremodified based on the conditional dependencies between variables. Inthe following example Sh activity is indexed by Tr, FA and then by Shitself. The main challenge while forming CPTs is to assign the correct

68 Paper B

probability values for each variable. There are no precise statistical datacovering the probability of falling, given that the person is performingany of the corresponding ADLs. We can consider a simple approach andassume that any frequent movement implies the higher chances of fallsto occur. In other words, if the person is training, there is a higher riskfor falling than if the same person is watching a TV. Therefore we canutilize the accelerometer data and calculate the probabilities for eachactivity in relation to an initially set value. This approach, however, isnot taking into account patients personal information and parametersmay vary from one individual to another. Alternatively, it is possible todeploy DBN learning features and perform system learning for parame-ter adjustment. In this way we will be able to assign CPT values basedon a previous experience, which is considered as a future work of thestudies.

Once the graphical structure and parameters are set we can pro-ceed with inferencing and use previously developed network to find anupdated state of hidden variables subset, when other variables are ob-served.

Table 7.3: Conditional probabilities including inter connections

Tr FA Sh probability1 1 1 0.92 1 1 0.21 2 1 0.92 2 1 0.51 1 2 0.52 1 2 0.81 2 2 0.12 2 2 0.1

Tr FA C probability1 1 1 0.62 1 1 0.21 2 1 0.62 2 1 0.61 1 2 0.42 1 2 0.81 2 2 0.42 2 2 0.4

In DBN it also means to compute the probability P (X(i, t0)|y(:, t1 :t2)), where X(i, t) represents the i′th hidden variable at time and t,Y (:, t1 : t2) represents the evidence between times t1 and t2. If all thehidden nodes are discrete like in our case, it is reasonable to apply thejunction tree algorithm to perform inference. In order to avoid numer-

7.5 System Evaluation 69

ical underflow and decrease processing time jtree is applied to pairs ofneighboring slices at a time; this is implemented in jtree dbn inf enginefrom the same DBN toolbox [39].

7.5 System Evaluation

7.5.1 Matlab Simulated Model

After creating a statistical model we can perform the evaluation stepwhich is split into simulation and demonstration part. In the first casewe deploy matlab functionality and unroll a previously created modelfor 100 time slices (first 11 slices are shown on Figure 7.6) to simulatethe monitoring process.

Figure 7.6: Simulation Process

This approach allows us to test the system in terms of a continuousmonitoring process and check its efficiency in avoiding the actual risk offalling. We are not interested in fall mechanics or health conditions of thepatient, but the fall frequency and system reactions instead. Our goal

70 Paper B

is to show the flexibility of the algorithm regardless of the monitoringobject or test environment. Therefore, all the elements and parametersof the model can be easily removed or adjusted depending on systemsconfigurations and current monitoring state.

The choice of time slice unit is entirely subjective and can representone minute, one hour or several hours depending on current purposes andmonitoring circumstances. Every slice contains a number of activitiesthat could happen simultaneously during this period of time includingfall event or every possible ADL. Our simulation scenario generates onesingle activity from context recognition per time unit with fall event in-terfering every 10th time slice. In other words, every time step we obtainonly one observed variable and seven hidden nodes, which correspond toa system with known structure, but partial observability. In this casewe can use the Expectation Maximization (EM) algorithm to find a (lo-cally) optimal Maximum Likelihood Estimate (MLE) of the parameters.So each step, we compute the expected values of all the nodes using aninference algorithm, and then treat these anticipated values as thoughthey were observed (distributions). As previously mentioned, simulationmethod can help us to recreate continuous monitoring and avoid thehealth risks during the experimental part of the studies. Nevertheless, itis technically randomized, and therefore, can not completely correspondto the real-life scenario.

7.5.2 Demonstration Model

We propose an alternative approach representing monitoring procedureto overcome the random aspect of the matlab simulation process. Itimplies a short system demonstration, where raw acceleration data arecollected, while a healthy volunteer performs regular ADLs.

These data are subsequently deployed as a ground material for theevaluation process. Unlike simulation method, all the activities are per-formed according to the natural behavior and therefore allow us to avoidunrealistic sequences. In total, 4 different ADLs were executed duringa short-term demonstration with 2 fall alarms triggered by the mobilephone based algorithm, which is depicted on Figure 7.7. The first fallwas registered during the ”watching TV ” activity (see Table 7.1) andsupposed to represent a false positive alarm. A Similar scenario is likelyto occur if the mobile device is exploited inappropriately by the patient


13:50 13:55 14:00−10

0

10

20

30

Data, samples

Average activity

13:50 13:55 14:000

0.2

0.4

0.6

0.8

1

Time, hours

Detected fall

Cooking Watching TV Training Sleeping

Figure 7.7: System Demonstration

or a technical error is affecting the algorithm. Therefore, it is essentialto take into account context information and check the current activityof the user. The second (actual) fall was registered during the ”training”activity and was performed by a volunteer without any health damage.After short-term data collection and minor preprocessing we deploy mat-lab functionality and proceed with sensor fusion.

As a result from both evaluation scenarios, we obtain a full observ-ability of the system, indicating the probability of every event, regis-tered in the network for a particular time slice. However, our maininterest is a fall risk assessment, and therefore, it is essential to cal-

culate P (FA|ADL) = P (FA)P (ADL|FA)P (ADL) , the probability of FA, given

any of ADL, using the Bayes rule from Section 7.3.3. Or in case of

72 Paper B

activity overlap, when more than 1 node becomes observed, we com-pute Pr(FA|C, TV, Tr, Sh, Sl, FE), which is also known as ”explainingaway” or Berkson’s paradox in statistics [45]. Moreover, computing thefall risk associated with context information and isolated fall detector isnot the only option provided by the system. We are able to estimate theprobabilities for each event to occur, based on previously observed data,which gives an opportunity to perform parameter learning.

7.5.3 Fall Risk Probability Estimation

The final step of algorithm evaluation process implies calculating theFA probability value for each time step after performing simulation anddemonstration scenario. The estimated result after matlab simulationprocess is summarized in Table 7.4. The outcome from both evaluationprocesses is depicted as a bar graph on Figure 7.10, where blue compo-nents are representing the regular time slices and every 10th element (inyellow) demonstrates how the probability is affected by the FE activity.

We also set a fall risk level to indicate potentially emergency situa-tions. This level is a flexible parameter, which can be adjusted dependingon the current user or monitoring scenario. According to the developedconfiguration, integrated system is managing only activity data, obtainedfrom the context recognition module. At this stage, inference algorithmprovides probability estimation for the set of activities, based on theirprevious values and current configuration.

Certain activities are depending on their previous state or previousstate of their inter-connected parent, which is reflected in correspondingconditional dependencies and specified in Conditional Probability Ta-bles (see Table 7.2 and 7.3 in Section 7.4). For instance, in our case”sleeping” is likely to occur right after ”shower” (Sh→ Sl in Figure 7.5)”shower” and ”cooking” after ”training” (Tr→ Sh, Tr→ C). Besides,it is also highly unrealistic that two falls happen consequentially aftereach other, which is covered by FA→ FA connection.

Whenever FE activity is received, it implies that physiological sensorregistered an alarm. Subsequently, DBN network performs the multi-sensor fusion process, takes into account previously collected contextualdata and calculates the final output based on both components of themonitoring system.

In case of simulation process, 10 alarms were received from the falldetector located on the phone and only 4 of them were confirmed by the


.5

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

10 20 30 40 50 60 70 80 90 100

Time slices

0

Es

tim

ate

d p

rob

ab

ilit

ies

Fall risk level

Figure 7.8: Simulation results

.51 2 3 4

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Fall risk level

Es

tim

ate

d p

rob

ab

ilit

ies

Time slices

Figure 7.9: Demonstration results

Figure 7.10: Evaluation results

74 Paper B

Table 7.4: Simulation process summary

activity amount min. FA max. FA average alarmsSh 28 0.01 0.53 0.14 0.2, 025, 0.53Sl 24 0.05 0.2 0.14 0.38, 0.4, 0.6C 17 0.09 0.8 0.29 0.8

TV 12 0.03 0.23 0.13 -Tr 18 0.24 0.92 0.58 0.85, 0.9, 0.92

Table 7.5: List of confirmed alarms

time slice # formula probability prev.event follow.event30 P (FA|Tr) 0.92 Sh Sh50 P (FA|C) 0.79 C Sh70 P (FA|Tr) 0.85 Tr Sh80 P (FA|Tr) 0.89 C TV

system. The list of confirmed emergency cases is laid out in Table 7.5together with probability outcome and neighboring activities. In thisway we are able to eliminate all the false positive alarms generated bythe wearable sensor (including alarms triggered during ”showering” and”sleeping” activity) and improve the level of reliability of the fall detec-tion algorithm.

After processing the data collected during the demonstration, weperformed similar procedure for calculating the posterior probability ofthe fall risk. In this case, 2 alarms were subsequently triggered by theindependent fall detector and one of them was canceled after applyingthe DBN based multi-sensor fusion. The first fall occurred during thewatching TV activity and was faked by the user. Therefore, it receiveda low probability value and was classified as a false positive. The secondalarm was registered during training activity and represented an actualfall. It was confirmed by the system and can be subsequently processedand used to notify a caretaker. Apart from that, various sources of datacombined into a single system provide different types of measurementsand, therefore, processing opportunities for future research, which is dis-

7.6 Conclusions and Future Work 75

cussed in Section 7.6.

7.6 Conclusions and Future Work

The initial goal of the studies implied combination of various sensorsbased on a single platform to provide reliable and stable fall detec-tion. We were able to deploy phone-based wearable sensor together withcontextual data and fuse results from both sources based on DynamicBayesian Network. This paper is an attempt to explore the benefits ofapplying a statistical methods for continuous monitoring of the physi-ological and contextual information for elderly home care. Therefore,structure of the network is corresponding to the predefined monitoringscenario and parameters are set according to the subjective assump-tions.

Evaluation part of the studies demonstrates that suggested multi-sensor fusion approach is applicable for on-line monitoring and fall de-tection disregard the sensor features involved in the process. Assumingthe flexibility of the algorithm, it can be equally applied to differentsources of data after simple adjustment of the appropriate parametersor introducing supplementary variables. With the current set up, wewere able to detect false positive alarms received from the isolated falldetection algorithm and provide thorough statistical information regard-ing the fall risk probability for the user.

We also consider a number of improvements, that can be addressedas a future work of this studies. First possibility implies investigating thelearning capabilities of the Dynamic Bayesian Networks. Using its broadfunctionality we will be able to assign CPT values automatically, detectsimilar patterns during monitoring and predict different types of alarmsbased on these data. Another opportunity to enhance the system can befound in deploying physiological data received from the medical sensors.In this case, professional doctors will be able to see correlations betweendaily activity of the person, amount of movement and essential healthparameters. We believe this approach leads to a better understandingof the cause, nature and the consequences of falling among elderly.

Bibliography







[7] Yongmian Zhang and Qiang Ji. Active and Dynamic InformationFusion for Multisensor Systems With Dynamic Bayesian Networks.IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cy-bernetics, 36(2):467–472, 2006.

[8] J Ullberg, A Loutfi, and F Pecora. Towards continuous activitymonitoring with temporal constraints. Proc of the 4th Workshop on

77

78 Bibliography

Planning and Plan Execution for Real-World Systems (at ICAPS-09), 2009.

[9] C Silvia, A. Cesta, L. Coraci, and L. Karlsson. GiraffPlus: Com-bining social interaction and long term monitoring for promotingindependent living. 2013 6th International Conference on HumanSystem Interactions (HSI), pages 578–585, June 2013.

[10] Topi Pulkkinen, Young-Sung Son, Joohyung Lee, Yann-Hang Lee,Mikko Sallinen, and Jun-Hee Park. Progressive monitoring andtreatment planning of diabetes mellitus in smart home environ-ment. 2013 IEEE International Conference on Consumer Electron-ics (ICCE), pages 206–207, January 2013.

[11] Juan a. Botia, Ana Villa, and Jose Palma. Ambient Assisted Livingsystem for in-home monitoring of healthy independent elders. ExpertSystems with Applications, 39(9):8136–8148, July 2012.

[12] Maged N Kamel Boulos, Ricardo Castellot Lou, Athanasios Anasta-siou, Chris D Nugent, Jan Alexandersson, Gottfried Zimmermann,Ulises Cortes, and Roberto Casas. Connectivity for healthcareand well-being management: examples from six European projects.International journal of environmental research and public health,6(7):1947–71, July 2009.




[16] Mitja Lustrek, Hristijan Gjoreski, Simon Kozina, BozidaraCvetkoviu, Violeta Mirchevska, and Matjaz Gams. Detecting

Bibliography 79

Falls with Location Sensors and Accelerometers. Proceedings ofthe Twenty-Third Innovative Applications of Artificial IntelligenceConference, (2006):1662–1667, 2010.


[18] Bruno Neves Ferreira, Vania Guimaraes, and Hugo Sereno Ferreira.Smartphone based fall prevention exercises. 2013 IEEE 15th In-ternational Conference on e-Health Networking, Applications andServices (Healthcom 2013), (Healthcom):643–647, October 2013.

[19] Naruhiro Shiozawa and Shin Arai. Gait analysis of sit-to-walk mo-tion by using portable acceleration monitor device for fall preven-tion. Proceedings of the IEEE-EMBS International Conference onBiomedical and Health Informatics (BHI 2012), 25(Bhi):563–564,2012.





80 Bibliography









Bibliography 81

[32] Hristijan Gjoreski, Mitja Lustrek, and Matjaz Gams. Context-Based Fall Detection Using Inertial and Location Sensors. AmbientIntelligence, 2012.

[33] Qiang Li and John A Stankovic. Grammar-based, posture- andcontext-cognitive detection for falls with different activity levels.Proceedings of the 2nd Conference on Wireless Health WH ’11, 2011.

[34] Alessandro Leone, Giovanni Diraco, and Pietro Siciliano. Context-Aware AAL Services through a 3D Sensor-Based Platform. Journalof Sensors, 2013:1–10, 2013.


[36] Cherif Smaili. Multi-Sensor Fusion Method using Dynamic BayesianNetwork for Precise Vehicle Localization and Road Matching. 19thIEEE International Conference on Tools with Artificial Intelligence,2007.

[37] Cindy Cappelle and ME El Najjar. Multi-sensors data fusion us-ing dynamic bayesian network for robotised vehicle geo-localisation.11th International Conference on Information Fusion, pages 1829–1836, 2008.

[38] Gregory a Koshmak, Maria Linden, and Amy Loutfi. Evaluationof the android-based fall detection system with physiological datamonitoring. Proceedings of Annual International Conference of theIEEE Engineering in Medicine and Biology Society, pages 1164–8,January 2013.

[39] Jim Nelson. Using an accelerometer to classify motion. Technicalreport, 2001.

[40] Daniel Teng Amitoz Ralhan Li Chen Vanina Dal Bello-Haas JennyBasran Seok-Bum Ko Carl McCrowsky Anh Dinh, Yang Shi. A falland near-fall assessment and evaluation system. The Open Biomed-ical Engineering Journal, pages 1–7, 2009.

[41] Lars Karsson, Lia Silva-lopez, and Jonas Ullberg. D 3 . 1 ContextInference and Configuration Planning Prototypes. Technical ReportJanuary 2012, 2014.


[43] James F. Allen. Towards a general theory of action and time. Ar-tificial Intelligence, 23(2):123–154, July 1984.

[44] FX Breu, S Guggenbichler, and JC Wollmann. The Bayes NetToolbox for Matlab. Technical report, 2008.

[45] R S Roberts, W O Spitzer, T Delmore, and D L Sackett. An empir-ical demonstration of Berkson’s bias. Journal of chronic diseases,31(2):119–28, February 1978.

Chapter 8

Paper C:Challenges and Issues inMultiSensor FusionApproach for FallDetection: Review Paper

Gregory Koshmak, Amy Loutfi, Maria LindenSpecial Issue of Hindawi Journal of Sensors: Healthcare Sensors for DailyLife, Paper submitted

83

Abstract

Emergency situations associated with falls are among the major prob-lems in modern healthcare. Following the recent development on ICTmarket a significant number of solutions were proposed to track bodymovement and detect falls in daily life with the most effective approachesbeing recently reviewed. Based on these reviews combination of sensorplatforms from unrelated categories is considered to be the most effi-cient approach and among the future trends in the area. In the presentpaper we review and systematize studies with a special focus on multi-sensor fusion based fall detection systems. Our goal is to explain maindifferences compared to the single sensor-based approach, identify typi-cal categories, provide information on possible issues and discuss trendsfor future work. We also give a brief overview of the recently developedmethod for multisensor fusion fall detection deploying Dynamic BayesianNetworks.

8.1 Introduction 85

8.1 Introduction

According to the latest United Nation statistic reports, the mean age ofthe population is expected to grow rapidly in developed countries withinthe next several decades [1]. This will subsequently increase the cost ofthe healthcare and result in significant loss for the national budgets. Atthe same time fall injury is considered to be one of the most commonrisks among elderly population. The estimated fall incidence for bothhospitalized and independently living people over the age of 75 is at least30% every year. Close to half of nursing home residents experience fallseach year, with 40% falling more than once [2]. These accidents can oftenhave both physical [3] (often head injury) and psychological [4] (fear offalling) consequences. Other current issues associated with falls includeunconscious after falling, recovery time, injury related death, etc., andcan be overcome by improving medical response level and rescue time.

The recent development in information and communication technol-ogy triggered an intensive research towards detection and prevention ofemergency situation associated with falls. This area is commonly consid-ered as a part of Ambient Assisted Living (AAL) which is an emergingmulti-disciplinary field exploiting ICT in personal healthcare for coun-tering the effects of aging population [5]. Modern AAL systems can alsohelp to promote independent lifestyle for elderly people with multiplechronicle disease in a situation of rapidly increasing healthcare costs [6].

Commonly fall detection systems are categorized into three differ-ent classes depending on deployed sensor technology including wearabledevice based, ambient sensor based and vision based. The article byNoury et al. [7] from 2007 containing description of systems, algorithmsand sensors for automatic fall detection can be considered one of thefirst surveys in the field. Relatively recent status is described in pub-lications by Mubashir et al. [8] and Igual et al. [9] providing valuableknowledge about principles, trends, issues and challenges in fall detec-tion area. As the number of contributions continued to expand someauthors prefer to review a specific category within the field, i.e. articleby Bagala et al. [10], specifically evaluating worn sensors-based fall de-tection methods. In publication by Otanasap [11], the focus is made oncomputer vision exploiting various processing techniques to analyze crit-ical phase and post-fall phase. However, a recent trend is characterizedby combination of different data sources, which are processed by a singlemultisensor fusion algorithm [12]. This novel approach can potentially

86 Paper C

provide a significant improvement in reliability and specificity of fall de-tection system, but has never been reviewed before.

In this paper we present a systematic survey of fall detection studiesusing multisensor fusion as a main method. Our aim is to provide ageneral insight on this novel approach and show its benefits comparedto other methods in fall risk area. Unlike techniques with single sourcechannel, this approach deploys a combination of unrelated devices whichare later fused on data processing level. We conducted search process inall related databases and sort out available manuscripts based on tablerepresentation. All the reviewed studies are categorized depending onsensors used for monitoring: combination of wearable/ambient devices,only wearable or only ambient. Cases when monitoring is based on sen-sors both from the same category and of the same type (i.e. multiplecameras) are not considered. We also give an assessment to this novelapproach and discuss its perspective in the nearest future.

The rest of the paper is organized as following: in Section 2 we pro-vide a general definition of the fall and describe its major characteristics.Section 3 gives a brief overview of the popular trends and approachesin modern fall detection together with major benefits and challenges.We proceed with detailed survey on publications deploying multisensorfusion which is followed by discussion of the presented approach and itsfuture perspective in Section 5.

8.2 Fall detection

8.2.1 Fall Characteristics and Popular Approaches

In the following section we will demonstrate complexity of the fallingprocess, define various types of fall and discuss it’s main characteristic.A fall is commonly defined as ”unintentionally coming to rest on theground, floor, or other lower level”. Losing the balance and subsequentfalling with the help of an assistant also considered as a fall [13]. Basedon possible scenarios 4 main types of falls can be distinguished: (1) fallfrom sleeping, (2) fall from sitting, (3) fall from walking/standing and(4) fall from standing on support tools such as ladder. Each type has its’own unique characteristics, which can help developers to adapt fall de-tector platforms to a wider spectrum of user requirements. According tothe recent studies falls are more likely to occur inside patients’ room andin the bathroom or toilet during activities such as moving/transferring,

8.2 Fall detection 87

and showering/toileting [14, 13]. Weight, size and corpulence of the per-son have also a substantial impact while determining the cinematic offalling. The majority of patients in the risk group usually fall in theevening or at night. Therefore, falls databases are very limited due tothe lack of records made in real-life testing [15].

Fall detecting techniques, like other forms of telecare, can be catego-rized into three different generations: First-generation systems that relyon the user to detect the fall. Second-generation systems that are basedon the first-generation systems but have an embedded level of intelli-gence. Third-generation systems use data, often via ambient monitoringsystems, to detect changes (e.g. changes in activity levels) which mayincrease the risk of falling (or risk factors for other negative events). Thethird-generation systems are a more preemptive rather than reactive ap-proach [16]. We will mostly focus on fall detectors from the second andthird category, which are discussed in terms of sensor fusion applicabil-ity in Section 3. Typically all the modern fall detection systems can besplit into 3 main classes depending on the sensor technology deployed formonitoring: wearable sensors, ambient sensors and vision-based sensors[17].

Vision-basedsensorsWearable Sensors Ambient sensors

Posture Motion

Body shape change

3-D head change

InactivityPosture Presence

Fall Detection


Regardless specific categories vast majority of recently developed falldetection systems operate based on one general framework including (1)data acquisition (2) data processing/feature extraction, (3) fall detection,(4) caregiver notification. This framework can vary depending on num-ber of devices involved in the monitoring, communication protocols foralarm delivery and the end user, who is responsible for taking actions incase of emergency. In wearable sensor based systems data acquisition is

88 Paper C

often performed by accelerometer (sensing changes in orientation of wear-able device), gyroscope (detects angular momentum) and other types ofsensors like barometer, magnetometer or microphone. Ambient sensorapproach often includes infrared sensors, vibration or acoustic sensors.The first type can locate and track thermal target within sensor’s fieldof view [18]. Vibration sensors are able to differentiate vibration pat-terns acquired from Activity of Daily Living (ADL) and falls meanwhileacoustic sensors use loudness and height of the sound to recognize thefall. Unlike wearable sensor techniques, this approach is considered tobe the least obtrusive as it implies minimum interaction with the patient[19]. The last fall detection method performs data acquisition via a setof video cameras embedded into monitoring environment [20]. Vision-based systems can carry out inactivity detection, analyze the body shapechanges or 3D head motion. They provide an unobtrusive way to monitorthe person of interest and rapidly decrease in price [21]. Several studiesmanaged to achieve significant results in reducing positive false alarmswhile using single sensor technology representing each of the categories.However, performance of the comprehensive indicates a significant raisein efficiency keeping reliability value over of 90%. Therefore combinationapproach is among the latest trends in fall detection/posture recognitionstudies listed by Augustyniak et al. [22]:

- building sensor networks instead of focus on a sensor set for aparticular disease

- promote multipurpose health prediction and prevention instead ofmonitoring patients with known medical records

- design monitoring process based on patients health conditions,habits and life-style

- unconstrained mobility of the monitored person

- real-time fusion and cooperation of ambient and wearable sensornetworks

Preliminary results [23] demonstrate significant improvement of falldetection system performance after deploying several sensor function-alities in one system. It can help to improve system performance andprovide significant reduction of false positive alarm rate.

8.3 Sensor fusion in Fall Detection 89

Sub-system Efficiency Reliability3D Vision 80.0% 97.3%Accelerometer 88.4% 79.3%Integrated system 94.3%% 90.9%

Assuming the overall complexity of the fall kinematics and diversityof fall characteristics, described in Section 2, we believe multisensor fu-sion approach is likely to become widely used in the fall detection area.Moreover, there is a strong demand in high standard of independent liv-ing for elderly people [24] and therefore particular focus should be ona unobtrusiveness of such a system. Single sensors based systems areoften not reliable enough and are only able to detect particular types offall in specific environments or circumstances. In the following sectionwe give a brief description of multisensor data fusion method, describeits adaption for fall detection area and suggest possible classification ofmain approaches.

8.3 Sensor fusion in Fall Detection

Multisensor data fusion is a technology to enable combining informationfrom several sources in order to form a unified picture [25]. Systemsbased on data fusion are now successfully exploited in various areas in-cluding sensor networks [26], image processing and healthcare [27], wherethey demonstrate enhanced performance in terms of accuracy and relia-bility compared to single source based systems [28]. Modern healthcaresystems commonly deploy data fusion algorithm to avoid intrinsic am-biguities caused by exploitation of unrelated type of sensors. In studyby Medjahed et al. [29] tele-monitoring system is proposed to integratephysiological and behavior data, the acoustic environment of the patientand medical knowledge. In this case data fusion approach is based onfuzzy logic with a set of rules corresponding to medical recommendationsand proved to increase the reliability of the whole system by detectingseveral distress situations. Another example of data fusion in health-care is proposed in article by Yang et al. [30], where Kinect and colorcameras are combined together to perform human tracking and identi-fication. Begum et al. [31] make an attempt to classify ”stressed” and”relaxed” individuals fusing data from various physiological sensors i.e.,Heart Rate, Finger Temperature, Respiration Rate, Carbon dioxide and

90 Paper C

Oxygen Saturation. In this case fusion algorithm performed on deci-sion and data level is additionally combined with Case-Based Reasoningfor further classification. The experimental results demonstrates an in-creased accuracy in comparison with an expert in the domain.

As it was previously mentioned in Section 2 fall detection systemsbased on single sensor technology are often lacking sufficient accuracyrates and require additional work to improve reliability . Multisensorfusion has proved its efficiency in various areas of the healthcare do-main [31] and therefore gained its popularity in fall detection domainwhere systems based on combination of monitoring devices perform sig-nificantly better. Moreover, with a recent development on ICT marketmore sensors are now available and can be combined to perform advancedlevel of activity tracking, which will increase number of publications inthis specific direction.

Ambient sensor fusionWearable Sen-

sor fusionWearable/Ambient

sensor fusion

Accelerometer

Gyroscope

Infrared sensors

Microphones

Vibration sensors

Accelerometer

IR sensors

3D Camera

Pressure sensors

Multisensor Fusion


In Section 2 fall detection methods were classified into 3 main cate-gories based on different types of sensor technology: wearable, ambientand vision-based. According to the vast majority of recent publica-tions within fall detection domain same types of sensors are involvedin multisensor fusion process with 2 major exceptions: (1) ambient andvision-based sensors are both integrated into environment and can beconsidered as a unified context-aware category and (2) wearable devicescan be combined together with context-aware sensors comprising addi-tional category. Assuming these corrections we propose an alternativeapproach to classify all fusion systems operating in the fall detectiondomain. Unlike single based methods, the choice of category does notdepend on utilized sensor technology but corresponds to a sensor type


which is being fused: context-aware sensors, wearable sensors and com-bination between context and wearable. We believe the main benefit offusion approach is its flexibility in terms of changing environment andpotential demands of the patient/user. Multisensor based systems canbe easily adjusted to the current monitoring instance, provide a broaderperspective on elderly falling problem and improve the fall preventionanalyses. At the same time, separately operating devices are often basedon different techniques and require additional processing before they canbe fused with the rest of the sensor set up, increasing computationalcosts. In the rest of the manuscript we review each category, present themost significant studies in multifusion fall detection domain and discussits possible challenges and limitations.

8.3.1 Context-aware sensors fusion

According to the recent review on fall detection methods, most of thesystems with multimodal approach are wearable sensor oriented and ex-ploit 3-axial accelerometer as a part of the process [32]. However, thereis a number of manuscripts providing solutions that are excluding wear-able sensors from the monitoring and fall detection in particular. Thesetypes of systems are effective when unobtrusiveness is the main require-ment and patients reject to wear any external devices on his/her body.They can detect persons movements, collect information regarding theusage of furniture or household items and answer questions regarding thepatients activity i.e. ”is the patient eating/exercising regularly?” [33].At he same time their operation capabilities are highly limited by thearea of distribution.

Typically sensors involved in the context-aware monitoring are rep-resented by cameras, vibration sensors, sound detector, pressure mats,floor or infrared sensors. Table 8.1 gives an overview of the most sig-nificant studies in this area. All the works are compared based on pub-lication year, sensors involved in the monitoring process, multifusionalgorithm, experimental part and evaluation results depending on theiravailability.

Unlike single sensor-based approach, where feature extraction is fol-lowed by data classification, multisensor systems perform independentdata analyses for each sensor technology with fusion method as a finalstep in fall detection [34]. Variation of the multisensor fusion techniquesin each category including context-aware systems is highly depended on

92 Paper C

sensors deployed in each study. Huang et al. [35] propose a new hu-man fall detection method based on fusing sensory information from avision system and a laser ranger finder (LRF). Both sensor platformsare attached to a three-wheeled omni-directional cane robot which is es-pecially designed for aiding elderly walking. In the operational state,the head position is tracked by CCD camera using color tracking algo-rithm and all the data from LRF is classified on-line by k-mean in totwo groups representing left and right leg. In order to obtain data fusionfrom two sensors, unrelated types of data are integrated into the imagecoordinate with a focus on the distance between the head and the cen-ter of two legs. Finally, the actual fall detection is based on a simplerule approach. Other fusion techniques deployed particularly in context-aware systems include: fuzzy logic [36], probability distribution function(PDF) [35], Bayes decision classifier [37], Winner-takes-all (WTA) deci-sion fusion algorithm [38] or Hidden Markov Models [39].

Initial sensor set up and preliminary processing plays an essentialrole in subsequent evaluation of the system. Brulin et al. [36] deployeda Health Smart Phone and recorded 15 video sequences illustrated situ-ations of everyday life or an emergency performed by two subjects. Inanother example experimental part is split into two related steps. First,the possibility distribution of ”normal walking” is investigated and fi-nally the validation of the fall detection method is performed. Otherexamples include dropping of ”Rescue Randy” doll, falling and speechsound generation, ADL and falls simulations. At this point, variabilityof trial approaches indicates an absence of common strategy for evalu-ation of the context-aware multisensor fusion systems. Due to the highvariability of devices deployed for multisensor it becomes complicated toanalyze and unify all the methods involved in the process or determinethe most reliable one. Further investigation and experimental work isrequired.


Artic

leY

ear

Basis

Deplo

yed

Deplo

yed

Evalu

atio

nP

erfo

rm

ance

Sensors

Alg

orit

hm

Bruli

netal.[3

6]

2010

Fusi

on

syst

em

arc

hit

ectu

refo

rfa

lldete

cti

on

PIR

,cam

era

,th

erm

opiles

Fuzzy

logic

+com

bin

ati

on

of

locati

on/p

ost

ure

dura

tion

15

vid

eo

sequences

record

ed

inH

ealt

hSm

art

Hom

e

Moti

on

dete

cti

on:

84%

Huang

etal.

[35]

2008

Inte

llig

ent

cane

fall

dete

cti

on

base

don

senso

rfu

sion

Lase

rra

nge

finder,

CC

Dcam

era

Pro

babilit

ydis

trib

uti

on

functi

on

wit

hre

levant

para

mete

r,ru

le-b

ase

dappro

ach

Norm

al

walk

ing/

Fall

dete

cti

on

exp

eri

ments

wit

hC

ane

rob

ots

Eff

ecti

veness

isconfirm

ed

thro

ugh

exp

eri

ments

Zig

eletal.

[37]

2009

Fall

dete

cti

on

base

don

dete

cti

on

of

vib

rati

on

and

sound

signals

Accele

rom

ete

r,m

icro

phone

Featu

reextr

acti

on,

Bayes

decis

ion

rule

cla

ssifi

er

Mim

ickin

gdoll

”R

esc

ue

Randy”,

40

dro

ps.

Oth

er

ob

jects

:80

dro

ps

SE

:97.5

%SP

:98.6

%

Yazaretal.

[38]

2014

Mult

i-se

nso

rsy

stem

for

fall

dete

cti

on

Vib

rati

on

senso

r,P

IRse

nso

rs

Win

ner-

takes-

all

(WT

A)

decis

ion

fusi

on

alg

ori

thm

Dem

oin

clu

din

g:

fallin

gp

ers

on,

hum

an

foots

tep,

hum

an

moti

on,

unusu

al

inacti

vit

ydete

cti

on

no

data

ispro

vid

ed

Toreyin

etal.

[39]

2014

Fall

dete

cti

on

usi

ng

mult

isenso

rsi

gnal

pro

cess

ing

Infr

are

d,

sound

senso

rsH

idden

Mark

ov

Models

2m

inute

sof

walk

ing

fallin

gand

speech

sounds

genera

tion

All

falls

are

dete

cte

dcorr

ectl

y

Aria

nietal.

[40]

2014

Unobtr

usi

ve

falls

dete

cti

on

wit

hm

ult

iple

pers

ons

PIR

and

moti

on

dete

cto

r,pre

ssure

mats

Decis

ion

tree

alg

ori

thm

3A

DL

scenari

os

12

typ

es

of

falls

SE

:100%

SP

:77.1

4%

Accura

cy:

89.3

3%

Yun

Lietal.

[21]

2013

Impro

vem

ent

of

acoust

icfa

lldete

cti

on

usi

ng

Kin

ect

depth

sensi

ng

FA

DE

(acoust

ic)

Kin

tect

segm

enta

tion,

thre

sh-

old

ing

record

ed

vid

eo

data

err

or

reducti

on

by

80%

Table

8.1

:C

onte

xt-

aw

are

senso

rsfu

sion

94 Paper C

Moreover, it is important to mention that none of the analyzed re-search works managed to perform experiments with elderly people, agroup, which is potentially in high risk of falling. This can be explainedby patients privacy, which is still a sensitive issue in terms of ambient andespecially vision-based fall detection systems. For historic reference, oneof the first projects in the area was forced to shift from image processingto body placed sensors due to privacy concerns [41]. We believe thisproblem can be partly solved by alternate deployment of camera-basedand ambient sensors depending on location, change in environment orcurrent emergency situation. In this way monitoring that violates pa-tients privacy is only performed when the calculated risk of falling issignificantly higher.

8.3.2 Wearable sensors fusion

With the recent development on ICT market, wearable devices start toplay an essential role in modern healthcare systems. Most of them arealready being utilized for automatic fall detection and showed its effi-ciency compared to other methods [42, 43, 44]. Unlike context-awaresensor technology, wearables attached to a patient’s body do not affecttheir privacy and can therefore perform monitoring on extended periodsof time.

The vast majority of fall detection systems based on fusing wearablesensors include accelerometer device as a main source of data. They areoften complemented by other types of wearables i.e. gyroscopes, mag-netometer, location tags, barometric pressure sensors. Moreover phys-iological devices combined with accelerometers can be considered as aseparate subgroups due to specific synchronization demands and pro-cessing of collected measurements. For example, Won-Jae Yi et al. [45]deploy temperature sensor and ECG together with accelerometer andperform individual data processing for each device later fused into a sin-gle alert message for medical staff.

Similarly to the previous category described in Section 3, multisensorfusion algorithms applied for wearables can vary depending on chosendevice type. In study by Felisberto et al. [41] a mash up of variousmethods including fuzzy logic, extended Kalmar filter (EKF), direct co-sine matrix (DCM) and control algorithm are applied in order to fuseaccelerometer, gyroscope and magnetometer. Fall detection based onmovement and sound data is performed by Doukas et al. [46, 47], where


accelerometer is deployed together with microphones and collected datais fused by Support Vector Machine technique. Some other examples in-clude heuristically trained decision tree classifier, rule-based reasoning,feature extraction and thresholding combination (see Table 8.3).

In the vast majority of fall related studies evaluation process is mainlyperformed by healthy volunteers or based on simulation [48, 49, 50]. Thisfact makes it almost impossible to give an accurate assessment for oper-ational capabilities of developed system or reliability of deployed algo-rithm. More experimental data received from elderly population shouldbe analyzed in order to improve sufficiency of developed fall detectionsystem. In study from 2012 Greene et al. [51] estimate the risk of fallingthrough multisensor assessment of standing balance. Pressure sensitiveplatform and body-worn inertial sensor are utilized during evaluation,which is based on monitoring 120 community dwelling older adults. It isone of few research studies where trials with elderly population are in-cluded as evaluation criteria. As a result the overall performance of thesystem was significantly affected demonstrating only 71.52% of classifi-cation accuracy, meanwhile the rest of the methods can reach 95%-97%for specificity, sensitivity and accuracy. Fall detection systems based onwearable devices is still a novel method and therefore lacking a unifiedapproach to effectively combine sensors due to different formats of col-lected data.

At the same time, additional number of digital devices attached topatient’s body is inconvenient for the users and can potentially lead to alow acceptance rate of this method. This issue can be solved if differenttypes of wearable sensors are incorporated in a single device performingcollection of unrelated types of data simultaneously. This will help re-duce data loss, improve processing time and at the same time maintainpatients independent life-style without affecting their privacy. Modernmobile phones are already equipped with advanced sensor functional-ity and can be suggested as a tool for synchronization and processingof collected measurements. However, modern smartphones and gadgetsare still poorly distributed among elderly people [52], which complicatesdeployment and further progress of proposed methodology.

96 Paper C

Artic

leY

ear

Basis

Deplo

yed

Deplo

yed

Evalu

atio

nP

erfo

rm

ance

Sensors

Alg

orit

hm

Feli

sb

erto

etal.

[41]

2014

Movem

ent

monit

ori

ng,

Accid

ent

Dete

cti

on

Base

don

Senso

rFusi

on

Accele

rom

ete

r,G

yro

scop

e,

Magneto

mete

r

Fuzzy

logic

+E

xte

nded

Kalm

an

Filte

r,D

irect

Cosi

ne

Matr

ix(D

CM

),C

ontr

ol

alg

ori

thm

Movem

ent

state

,O

rienta

tion

state

exp

eri

ment

wit

hpre

-collecte

ddata

Pass

ing

avera

ge:

84%

Doukasetal.

[47]

2008

Fall

dete

cti

on

base

don

movem

ent/

sound

data

Accele

rom

ete

r,m

icro

phones

Supp

ort

Vecto

rM

ach

ine

(SV

M)

2volu

nte

ers

:a)

sim

ple

walk

b)

walk

and

fall

c)

walk

and

run

All

fall

events

success

fully

dete

cte

dR

un

events

:96,7

2%

Bia

nchietal.

[53]

2010

Falls

event

dete

cti

on

wit

hbaro

metr

icpre

ssure

and

tria

xia

laccele

rom

ete

r

Accele

rom

ete

r,air

pre

ssure

senso

r

heuri

stic

ally

train

ed

decis

ion

tree

cla

ssifi

er

20

healt

hy

volu

nte

ers

:fa

lls/

AD

Lsi

mula

tion

Accura

cy:

96.9

%Sensi

tivit

y:

97.5

%Sp

ecifi

cit

y:

96,5

%

Lustrek

etal.

[54]

2011

Fall

dete

cti

on

wit

haccele

rom

ete

rand

locati

on

senso

r

Accele

rom

ete

r,lo

cati

on

tags

rule

-base

dre

aso

nin

g10

healt

hy

volu

nte

ers

,sp

ecifi

csc

enari

o

Meth

ods

uti

lized

both

conte

xt/

accele

rom

ete

r.accura

cy

incre

ase

:40%

Yietal.

[45]

2014

Weara

ble

Senso

rdata

fusi

on

for

fall

Dete

cti

on

Tem

pera

ture

,accele

rom

ete

rE

CG

senso

r

Data

ispro

cess

ed

indiv

idually

and

com

bin

ed

into

ale

rtm

ess

age

no

evalu

ati

on

pro

vid

ed

Hum

an

post

ure

ssu

ccess

fully

recogniz

ed.

Full

evalu

ati

on

isnot

perf

orm

ed


Tolk

iehn

etal.

[55]

2011

Fall

dete

cti

on

wit

haccele

rom

ete

rand

baro

metr

icpre

ssure

senso

r

Accele

rom

ete

r,baro

metr

icpre

ssure

senso

r

Featu

reextr

acti

on,

thre

shold

ing

com

bin

ati

on

12

healt

hy

volu

nte

ers

AD

L/fa

llsi

mula

tion,

297

data

sequences

Fall

identi

ficati

on

accura

cy:

94.1

2%

Greene

etal.

[51]

2012

Falls

risk

est

imati

on

thro

ugh

mult

isenso

rass

ess

ment

of

standin

gbala

nce

Pre

ssure

senso

r(p

latf

orm

),b

ody-w

orn

inert

ial

senso

r

SV

M120

com

munit

ydw

ellin

gold

er

adult

s

Cla

ssifi

cati

on

accura

cy:

71.5

2%

Table

8.3

:W

eara

ble

senso

rsfu

sion

98 Paper C

8.3.3 Wearable/Ambient sensor fusion

The last category is characterized by combination of previously presentedapproaches and can potentially help to detect wider spectrum of possibleemergency situations connected with falls. Context-aware fall systemscan provide long-term trend analyses describing patients behavior andrecognize abnormal patterns, but are often limited by their distributionarea. Wearable fall detection is becoming more and more available dueto cheap embedded sensors included in smartphones, demonstrate rel-atively high performance, but still produce significant number of fallalarms [56, 57] and has been mainly tested in laboratory environments.As a result, research studies which make an attempt to merge major ben-efits of both approaches into a self-complementing system is surpassingother methods by a number of publications (see Table 8.5). We reviewthe most significant studies to demonstrate the latest trends in multi-sensor fusion for context-aware and wearable sensors.

This approach is considered relatively new and therefore requiresthorough investigation and experimental work. As a result, the choiceof sensors can vary significantly from one study to another. Most of thesystems deploy accelerometers as a main device which are additionallycombined with either ambient sensors or 3-D cameras [58, 59]. Otherwearable devices can be represented by gyroscopes, microphones, physi-ological sensors, sound analyzer, infrared sensors or RFID tags. Toffolaet al. [60] in their study from 2011 pick up a different approach andcomplement a set of ambient and body-worn sensors with a home robotin order to improve fall detection. Slightly different concept is presentedby McIlwright et al. [61] and Kepski et al. [62] where accelerometerand gyroscope are accompanied by vision-based sensors. In the firstpublication surrounding vision sensors are deployed for accurate charac-terization of motion, and in the second case authors used commerciallyavailable microsoft Kinect camera instead and performed reliable fall de-tection. In some cases gyroscope can be replaced by microphones [47]or alternatively by RFID tags [56], with embedded tracking camera andaccelerometer still being part of the framework.

Due to high diversity in sensor technology deployed for fall detection,the choice of algorithms performing fusion function is still unique in eachresearch work. The most common approach to combine wearable andcontext-aware systems includes individual low-complexity algorithms forevery sensor technology, which are then followed by more advanced fu-


sion algorithm. None of the reviewed studies deployed thresholding tech-nique on individual or fusion level and the only example of rule-basedapproach was complicated by semantic web rule language. The mostpopular algorithm is fuzzy logic utilized as fuzzy inference system [62]or fuzzy logic decision tree [63]. Other methods include: evidential net-works, Dempster-Shafer theory or Hidden Markov Models. Similarly tothe previous categories, there is no possibility to determine a commonapproach or justify the choice of fusion methods since there is not enoughexperimental evidence to operate with.

Similarly to previous categories, variation in sensors and methods de-ployed for multimodal fusion has a significant effect on experimental partof research. The evaluation process can be characterized by two differentscenarios: (1) on-line testing with volunteers subsequently performingADL or falls, (2) off-line evaluation utilizing previously collected mea-surements. In both cases, combination of wearable and context-awareapproaches had a positive impact and resulted in increased specificity,sensitivity and accuracy. Doukas et al. [46] in their attempt to mergetracking camera, accelerometer and microphones managed to minimizethe amount of fall positive alarms to zero. Evaluation based on elderlypatients in real home-like environments is still a sensitive issue, assumingcomplexity of the sensor set up in this case.

In our previous studies we proposed a multisensor fusion systembased on Dynamic Bayesian Networks and combined wearable devicewith context-aware sensor framework [64]. All the accelerometer mea-surements were obtained from the android based smartphone and ana-lyzed for possible falls. Context-aware information was obtained fromenvironmental sensors network consisting of PIR motion, door contact,pressure mats and power usage detectors embedded into a smart homeand deploying a special context recognition algorithm to deliver useractivities. Physiological data was later interfered with ambient mea-surements and processed in Dynamic Bayesian Network performing falldetection. Evaluation process contains both simulation (matlab tools)and demonstration part (healthy volunteer). With the proposed tech-nique we managed to compliment 2 different fall detection approachesand improve the reliability of the fall detection system. However, it isstill far from deployment of developed or similar systems in every-daygeriatric practice or explicit examples of commercially successful appli-cations. Moreover, the vast majority of similar systems obtain highexperimental results in unrealistic or restricted conditions with a pure

100 Paper C

reference to real-life environments, which is among the issues of this ap-proach. Other challenges and limitations of multisensor fusion methodin fall detection are discussed in Section 4 of the review.


Artic

leY

ear

Basis

Deplo

yed

Deplo

yed

Evalu

atio

nP

erfo

rm

ance

Sensors

Alg

orit

hm

Aguil

aretal.

[65]

2011

Senso

rfu

sion

via

evid

enti

al

netw

ork

for

fall

dete

cti

on

RF

PA

T[6

5],

GA

RD

IEN

[66]

Evid

enti

al

netw

ork

Dem

pst

er-

Shafe

rT

heory

form

alism

Data

record

ed

at

Tele

com

SudP

ari

s

SE

:94%

Cavalc

ante

etal.

[67]

2012

Evid

enti

al

netw

ork

for

medic

al

data

fusi

on

inre

mote

monit

ori

ng

Weara

ble

senso

r,in

frare

dse

nso

rs,

sound

analy

zer

Dem

pst

er-

Shafe

rT

heory

Data

record

ed

at

Tele

com

SudP

ari

s

SE

:93,9

4%

Toff

ola

etal.

[60]

2011

Com

bin

ese

nso

rnetw

ork

sand

hom

ero

bot

toim

pro

ve

fall

dete

cti

on

Body

worn

senso

rs,

am

bie

nt

senso

rs,

hom

ero

bot

Futu

rew

ork

,floodin

gti

me

synch

roniz

ati

on

pro

tocol

for

nodes

pack

et

transm

issi

on

dela

ys

pow

er

consu

mpti

on

built

syst

em

suit

able

for

fall

dete

cti

on

McIl

wrait

hetal.[6

1]

2010

Weara

ble

and

am

bie

nt

senso

rfu

sion

for

hum

an

moti

on

dete

cti

on

Accele

rom

ete

r(e

-AR

),vid

eo

senso

rs,

gyro

scop

e

spati

al/

tem

pora

lH

MM

5acti

vit

ies

perf

orm

ed

by

volu

nte

ers

ina

const

rain

ed

manner

Accura

cy

incre

ase

:over

vis

ion

syst

em

:6.4

%over

gyro

scop

e:

17.2

%

Kepskietal.

[62]

2008

Fall

dete

cti

on

usi

ng

Kin

ect

and

accele

rom

ete

r

Kin

ect,

accele

rom

ete

r,gyro

scop

e

fuzzy

infe

rence

syst

em

Inte

nti

onal

falls

and

AD

Ls

perf

orm

ed

by

3volu

nte

ers

Fuse

dse

nso

rspro

ved

suffi

cie

ncy

toim

ple

ment

reliable

fall

dete

cti

on

syst

em

Leone

etal.

[34]

2008

Mult

isenso

rappro

ach

for

fall

Dete

cti

on

inhom

eenvir

onm

ent

3D

cam

era

,w

eara

ble

acc-t

er,

mic

rophone

mult

i-th

readin

gappro

ach

wit

hfu

zzy

logic

tech

niq

ue

under

develo

pm

ent

13

volu

nte

ers

perf

orm

450

events

inclu

din

g210

falls

3D

vis

ion

ala

rm:

81.3

%A

cc-t

er

ala

rm:

98%

Audio

ala

rm:

83%

Ale

mdaretal.

[68]

2010

Mult

i-m

odal

fall

dete

cti

on

wit

hin

WeC

are

fram

ew

ork

accele

rom

ete

r,em

bedded

cam

era

s,R

FID

tags

decis

ion

fusi

on

mech

anis

mV

olu

nte

er

perf

orm

ing

AD

L

Falls

are

success

fully

dis

tinguis

hed

from

AD

L

102 Paper CC

agnonietal.

[63]

2009

Fall

dete

cti

on

for

Ass

iste

dT

ech

nolo

gie

sA

pplicati

ons

Accele

rom

ete

r,vid

eo

cam

era

PSO

for

vis

ual

data

HT

Mfo

raccele

rati

on

Fusi

on

alg

ori

thm

sm

ult

iple

cla

ssifi

er

sets

fuzzy

logic

decis

ion

trees

Accele

rom

ete

r:conti

nuous

flow

of

real-

life

events

sim

ula

tion

Vid

eo

senso

r:lim

ited

set

of

image

sequences

Join

tsy

stem

isguara

nte

ed

topro

vid

ea

good

level

of

fault

-tole

rance

Doukas2

etal.

[46]

2011

Fall

dete

cti

on

uti

lizin

gm

oti

on,

sound

and

vis

ual

perc

eptu

al

com

ponents

Tra

ckin

gcam

era

,A

ccele

rom

ete

r,m

icro

phones

Sem

anti

cru

les

base

don

sem

anti

cw

eb

rule

language

(SW

RL

)

2m

ale

volu

nte

ers

perf

orm

ing

exp

eri

menta

lpro

tocol

Uti

lizati

on

of

rule

s-base

devalu

ati

on

min

imiz

efa

lse

posi

tives

tozero

Table

8.5

:W

eara

ble

/A

mbie

nt

senso

rfu

sion

8.4 Discussion 103

8.4 Discussion

8.4.1 Challenges

Most of the challenges specific for modern single-based fall detectionsystems are still valid in case of multimodal approach. Igual et al. [9]provides a number of typical problems which can affect final results in-cluding (1) lack of performance under real-life conditions, (2) limitedusability (mostly applies to wearable and smartphone-based fall detec-tors), (3) lack of publications regarding practicality and acceptability ofmodern fall detection technologies. Other suggested issues are connectedwith privacy concerns, lack of human contact and limited experimentalconditions.

After including additional sensor functionality a single-based fall de-tector becomes a multimodal system inheriting challenges typical toother frameworks with data fusion requirement. Khaleghi et al. [25] in-troduced these issues in their study starting with imperfection of the col-lected data and diversity or low reliability of sensor technologies. Basedon reviewed material we can complement the list of challenges in datafusion for fall detection with the following issues:

- Conflicting outputDuring the monitoring process similar activities can be interpretedin a different way by unrelated sensor platforms. The amount offalse alarms among modern fall detections is still relatively high.Therefore it is essential to give a priority to the technology whichis more reliable and can minimize unclassified falls or ADLs.

- Data correlationMeasurements collected during the monitoring process in case ofmultisensor fall detection are typically coming from different back-grounds and unrelated to each other. These data should not onlybe merged together in a most efficient way, but also analyzed forpossible common trends and similarities.

- Processing frameworkFirstly, majority of the systems analyze data for each compo-nent independently and deploy fusion algorithm as a final stepto combine acquired results [27]. However, in some cases raw datacollected from each sensor unit can be delivered to the common

104 Paper C

framework without preliminary processing. Alternatively, in caseof wearable and context-aware fusion, particular categories can beprocessed in conjunction (i.e. various types of ambient sensors)and later fused with sensors from unrelated category. As a result,it leads to unnecessary complication of the fusion algorithm andsubsequent increase in computational time.

Another drawback, which is particularly specific for modern multi-sensor fusion system is a lack of simplified evaluation procedure. In a vastmajority of articles evaluation method is often based on simulations orthis information is not available at all. It is partly caused by complexityof the monitoring set up in real environment. Sensor functionality shouldbe embedded into the regular apartment or specially designed test en-vironment. Moreover, similarly to regular fall detection systems fusionbased methods are evaluated on simulated falls performed by healthyvolunteers, which is far from he real-life scenario. Testing with real pa-tients who suffer from falling can help to improve the process, howeverit requires ethical content, additional complications and commonly notavailable in fall detection studies. Additional complexity is caused bydistinctive technological background of the sensor technology involved inthe monitoring process. This issue is specific to any fusion based systemand becomes essential when developing the multisensor fall detectionmechanism.

8.4.2 Future Trends

Based on majority of reviewed papers the main trend in multisensorfall detection can be characterized by merging sensor technologies fromdifferent categories and unrelated platforms. Systems developed withthis approach are fully interchangeable and can maintain monitoringeven when one of the components is inactive.

- Physiological sensorsMost of the elderly patients suffer from various health problemsincluding heart problem or Alzheimer which increases probabilityof falling in their daily life. Therefore, it is important to trackpatients activity in conjunction with significant physiological pa-rameters. Physiological sensors combined with fall detectors canhelp to understand correlation between patients activity and healthconditions and make monitoring process more detailed.

8.5 Conclusion 105

- Long-term analysesMonitoring people with high risk of falling on a regular basis dur-ing the long period of time will improve data analyses and helpto detect interesting patterns. In perspective we will be able todevelop an algorithm which can prevent the fall in case dangerousmeasurement sequence is repeating itself in time.

- Integration into smart home environmentsLong-term analyses is almost impossible without an appropriatesensor set up. The latest trend in building smart home environ-ments can be adopted for patient tracking and reliable fall detec-tion. In this case, smart homes can perform according to theirbasic functionality and have an extra option for fall detection.

- Patient-oriented systemsAssuming the individual approach in patient treatment most of themultimodal healthcare systems should be more patient-oriented.The choice of sensors and processing techniques should correspondto the actual patient demands and major health problems theyare suffering from. Otherwise, developed platforms should cover awide spectrum of healthcare problems or be as much universal aspossible.

Due to complexity of falls and variation in falling circumstances themost effective approach implies fusing information from sensors relatedto different categories. As a step towards a full-scale remote monitor-ing framework, fall detection components can be deployed in conjunctionwith other healthcare system to check patients well-being on a long termbasis. Following the recent trend, we suggest building a special envi-ronments with wearable, ambient and vision sensors, where fusion tech-niques can be effectively evaluated. At the same time, it is recommendedto complement these types of smart environments with additional sensortechnology only based on current patients’ demand or particular mon-itoring case in order to avoid data overload and unnecessary privacyviolations.

8.5 Conclusion

Fall detection systems play an essential role in modern healthcare sys-tems. Latest sensor technologies are deployed in order to distinguish be-

106 Paper C

tween falls and regular ADLs with a recent trend to combine unrelateddata sources. These types of systems has proven to be more reliablecompared to the single-based fall detectors and the number of publica-tions increased in recent years and is expected to grow. We conduct asearch within the latest works based on multisensor fall detection sys-tems, make an attempt to classify all systems into various categorize andanalyze challenges, issues and future trends. Most of the reviewed pub-lications still suffer from problems similar to single-based fall detectionmethods. However, we managed to formulate a number of challengesspecific to this domain, which can help to improve the whole approachand provide better results in the future.

Bibliography





[5] Mukhtiar Memon, Stefan Rahr Wagner, Christian Fischer Pedersen,Femina Hassan Aysha Beevi, and Finn Overgaard Hansen. Am-bient assisted living healthcare frameworks, platforms, standards,and quality attributes. Sensors (Basel, Switzerland), 14(3):4312–41, January 2014.

[6] Parisa Rashidi and Alex Mihailidis. A Survey on Ambient-AssistedLiving Tools for Older Adults. IEEE Journal of Biomedical andHealth Informatics, 17(3):579–590, 2013.


107

108 Bibliography

[8] Muhammad Mubashir, Ling Shao, and Luke Seed. A survey on falldetection: Principles and approaches. Neurocomputing, 100:144–152, January 2013.

[9] Raul Igual, Carlos Medrano, and Inmaculada Plaza. Challenges,issues and trends in fall detection systems. Biomedical engineeringonline, 12(1):66, January 2013.

[10] Clemens Becker, Angelo Cappello, Lorenzo Chiari, KamiarAminian, Fabio Bagala, Jeffrey M Hausdorff, Wiebren Zijlstra, andJochen Klenk. Evaluation of Accelerometer-Based Fall DetectionAlgorithms on Real-World Falls. PLoS ONE, 7(5):1–9, 2012.

[11] Nuth Otanasap and Poonpong Boonbrahm. Fall prevention usinghead velocity extracted from visual based VDO sequences. Proceed-ings of the 5th Augmented Human International Conference, 2012.

[12] Velislava Spasova and Ivo Iliev. A Survey on Automatic Fall Detec-tion in the Context of Ambient Assisted Living Systems. Interna-tional Journal of Advanced Computer Research, (1), 2014.

[13] Chen Xue-li, L I U Yun-hu, Daniel K Y Chan, Shen Qing, andHuong Van Nguyen. Characteristics associated with falls among theelderly within aged care wards in a tertiary hospital: a retrospectivecase-control study. Chinese Medical Journal, 123(13):1668–1672,2010.


[15] Jerome Boudy Dan Istrate H. Medjahed Bernadette Dorizzi J. CesarM. Mota Jean Louis Baldinger T. Guettari Imad Belfeki Paulo A.,Cavalcante A. Evidential Network-based Multimodal Fusion forFall Detection. 8th International Conference on Wearable Microand Nano Technologies for Personalised Health, 2012.

[16] Gillian Ward, Nikki Holliday, Simon Fielden, and Sue Williams. Falldetectors: a review of the literature. Journal of Assistive Technolo-gies, 6(3):202–215, September 2012.

Bibliography 109

[17] Xinguo Yu. Approaches and principles of fall detection for elderlyand patient. 10th International Conference on e-health Networking,Applications and Services, HealthCom, pages 42–47, 2008.

[18] Farrukh Hijaz, Nabeel Afzal, Talal Ahmad, and Osman Hasan. Sur-vey of Fall Detection and Daily Activity Monitoring Techniques. In-ternational Conference on Information and Emerging Technologies(ICIET), pages 1–6, 2010.

[19] Mohammad Habib, Mas Mohktar, Shahrul Kamaruzzaman, KhengLim, Tan Pin, and Fatimah Ibrahim. Smartphone-Based Solutionsfor Fall Detection and Prevention: Challenges and Open Issues.Sensors, 14(4):7181–7208, April 2014.

[20] S.K. Tasoulis, C.N. Doukas, V.P. Plagianakos, and I. Maglogiannis.Statistical data mining of streaming motion data for activity and fallrecognition in assistive environments. Neurocomputing, 107:87–96,May 2013.

[21] Yun Li, Tanvi Banerjee, Mihail Popescu, and Marjorie Skubic. Im-provement of acoustic fall detection using Kinect depth sensing.35th Annual International IEEE EMBS Conference, pages 6736–9,January 2013.

[22] Piotr Augustyniak, Magdalena Smolen, Zbigniew Mikrut, andEliasz Kantoch. Seamless tracing of human behavior using com-plementary wearable and house-embedded sensors. Sensors (Basel,Switzerland), 14(5):7831–56, January 2014.

[23] M. Grassi, A. Lombardi, G. Rescio, P. Malcovati, M. Malfatti,L. Gonzo, A. Leone, G. Diraco, C. Distante, P. Siciliano, V. Libal,J. Huang, and G. Potamianos. A Hardware-Software Framework forHigh-Reliability People Fall Detection. IEEE Sensors Conference,pages 1328–1331, October 2008.

[24] R. Bragg P. Catlin J. Carlier S. Brownsell, D. A. Bradley. Do Userswant Telecare and can It be Cost-Effective. Proceedings of TheFirst Joint BMES/EMBS Conference Serving Humanity, Advanc-ing Technology, page 714, October 1999.

[25] Bahador Khaleghi, Alaa Khamis, Fakhreddine O. Karray, andSaiedeh N. Razavi. Multisensor data fusion: A review of the state-of-the-art. Information Fusion, 14(1):28–44, January 2013.

110 Bibliography

[26] David L. Hall and James Llinas. An introduction to multisensordata fusion. Proceedings of the IEEE, 85(1):6–23, 1997.

[27] Kyungseo Park and Byoungyong Lee. Issues in Data Fusion forHealthcare Monitoring Categories and Subject Descriptors. 2008.

[28] Gabriel Villarrubia, Juan F De Paz, Javier Bajo, and Juan M Cor-chado. Ambient agents: embedded agents for remote control andmonitoring using the PANGEA platform. Sensors (Basel, Switzer-land), 14(8):13955–79, January 2014.

[29] Hamid Medjahed and Dan Istrate. Human activities of daily livingrecognition using fuzzy logic for elderly home monitoring. IEEEInternational Conference on Fuzzy Systems., 33:1466–1473, 2009.

[30] Mau-Tsuen Yang and Shen-Yen Huang. Appearance-based multi-modal human tracking and identification for healthcare in the digitalhome. Sensors (Basel, Switzerland), 14(8):14253–77, January 2014.

[31] Shahina Begum, Shaibal Barua, and Mobyen Ahmed. PhysiologicalSensor Signals Classification for Healthcare Using Sensor Data Fu-sion and Case-Based Reasoning. Sensors, 14(7):11770–11785, July2014.

[32] James T. Perry, Scott Kellog, Sundar M. Vaidya, Jong-Hoon Youn,Hesham Ali, and Hamid Sharif. Survey and Evaluation of Real-TimeFall Detection Approaches. 6th International Symposium on High-Capacity Optical Networks and Enabling Technologies (HONET),pages 158–164, 2009.



[35] Jian Huang, Pei Di, Kouhei Wakita, Toshio Fukuda, and Ko-suke Sekiyama. Study of Fall Detection Using Intelligent Cane

Bibliography 111

Based on Sensor Fusion. International Symposium on Micro-NanoMechatronics and Human Science, MHS, pages 495–500, 2008.


[37] Yaniv Zigel, Dima Litvak, and Israel Gannot. A Method for Auto-matic Fall Detection of Elderly People Using Floor Vibrations andSound — Proof of Concept on Human Mimicking Doll Falls. IEEETransactions on Biomedical Engineering, 56(12):2858–2867, 2009.

[38] Ahmet Yazar, Fatih Erden, and A Enis Cetin. Multi-Sensor Am-bient Assisted Living System for Fall Detection. 2014 IEEE Inter-national Conference on Acoustics, Speech, and Signal Processing,pages 1–3.

[39] B Toreyin, a Soyer, Ibrahim Onaran, and E Cetin. Falling PersonDetection Using Multi-Sensor Signal Processing. EURASIP Journalon Advances in Signal Processing, 2008(1):149304, 2008.

[40] Arni Ariani, Student Member, Stephen J. Redmond, David Chang,and Nigel H. Lovell. Simulated Unobtrusive Falls Detection WithMultiple Persons. IEEE Transactions on Biomedical Engineering,59(11):3185–3196, 2012.

[41] Filipe Felisberto, Florentino Fdez-Riverola, and Antonio Pereira. Aubiquitous and low-cost solution for movement monitoring and acci-dent detection based on sensor fusion. Sensors (Basel, Switzerland),14(5):8961–83, January 2014.


[43] Kohei Arai. Wearable physical and psychological health monitoringsystem. Science and Information Conference (SAI), 2013, pages133–138, 2013.

112 Bibliography

[44] Andrea Lombardi, Massimo Ferri, Gabriele Rescio, Marco Grassi,and Piero Malcovati. Wearable wireless accelerometer with em-bedded fall-detection logic for multi-sensor ambient assisted livingapplications. 2009 IEEE Sensors, pages 1967–1970, October 2009.

[45] Won-Jae Yi, Oishee Sarkar, Sivisa Mathavan, and Jafar Saniie.Wearable sensor data fusion for remote health assessment and falldetection. IEEE International Conference on Electro/InformationTechnology, pages 303–307, June 2014.


[47] Charalampos Doukas and Ilias Maglogiannis. Advanced patient orelder fall detection based on movement and sound data. 2008 SecondInternational Conference on Pervasive Computing Technologies forHealthcare, pages 103–107, January 2008.

[48] Gregory a Koshmak, Maria Linden, and Amy Loutfi. Evaluationof the android-based fall detection system with physiological datamonitoring. Proceedings of Annual International Conference of theIEEE Engineering in Medicine and Biology Society, pages 1164–8,January 2013.

[49] David Naranjo-Hernandez, Laura M. Roa, Javier Reina-Tosina, andMiguel Angel Estudillo-Valderrama. Personalization and Adapta-tion to the Medium and Context in a Fall Detection System. IEEETransactions on Information Technology in Biomedicine, 16(2):264–271, 2012.

[50] L. Schwickert, C. Becker, U. Lindemann, C. Marechal, a. Bourke,L. Chiari, J.L. Helbostad, W. Zijlstra, K. Aminian, C. Todd,S. Bandinelli, and J. Klenk. Fall detection with body-worn sensors.Zeitschrift fur Gerontologie und Geriatrie, 46(8):706–719, Novem-ber 2013.

[51] Barry R Greene, Denise Mcgrath, Lorcan Walsh, Emer P Doheny,David Mckeown, Chiara Garattini, Clodagh Cunningham, LisaCrosby, Brian Caulfield, Rose A Kenny, Intel Labs, Ulster Sports

Bibliography 113

Academy, and Northern Ireland. Quantitative falls risk estimationthrough multi-sensor assessment of standing balance. PhysiologicalMeasurement, pages 2049–2063, December 2009.

[52] Prasad Kuvalekar Shiv Kumar Goel, Nikita Haryani, Pramod Ti-wari, Abhishek Jain. Smart Phone for Elderly Populace. Interna-tional Journal of Research in Engineering and Technology, pages33–37, 2013.

[53] Federico Bianchi, Stephen J Redmond, Michael R Narayanan, Ser-gio Cerutti, and Nigel H Lovell. Barometric pressure and triaxialaccelerometry-based falls event detection. IEEE transactions onneural systems and rehabilitation engineering : a publication of theIEEE Engineering in Medicine and Biology Society, 18(6):619–27,December 2010.

[54] Mitja Lustrek, Hristijan Gjoreski, Simon Kozina, BozidaraCvetkoviu, Violeta Mirchevska, and Matjaz Gams. DetectingFalls with Location Sensors and Accelerometers. Proceedings ofthe Twenty-Third Innovative Applications of Artificial IntelligenceConference, (2006):1662–1667, 2010.

[55] Marie Tolkiehn, Louis Atallah, Benny Lo, and Guang-Zhong Yang.Direction sensitive fall detection using a triaxial accelerometer anda barometric pressure sensor. Conference proceedings : Annual In-ternational Conference of the IEEE Engineering in Medicine andBiology Society, 2011:369–372, January 2011.

[56] Rafael Luque, Eduardo Casilari, Marıa-Jose Moron, and Gema Re-dondo. Comparison and Characterization of Android-Based FallDetection Systems. Sensors, 14(10):18543–18574, October 2014.


[58] Sen Wang, Zhongwen Xu, Yi Yang, Xue Li, Chaoyi Pang, andAlexander G. Haumptmann. Fall detection in multi-camera surveil-lance videos: experimentations and observations. Proceedings of

114 Bibliography

the 1st ACM international workshop on Multimedia indexing andinformation retrieval for healthcare, pages 33–38, 2013.

[59] M Grassi, a Lombardi, G Rescio, M Ferri, P Malcovati, a Leone,G Diraco, P Siciliano, M Malfatti, and L Gonzo. An integratedsystem for people fall-detection with data fusion capabilities basedon 3D ToF camera and wireless accelerometer. 2010 IEEE Sensors,pages 1016–1019, November 2010.

[60] Luca Della Toffola, Shyamal Patel, Bor-rong Chen, Yalgin M. Ozse-cen, Alessandro Puiatti, and Paolo Bonato. Development of a Plat-form to Combine Sensor Networks and Home Robots to ImproveFall Detection in the Home Environment. 33rd Annual Interna-tional Conference of the IEEE EMBS, pages 5331–5334, 2011.

[61] D McIlwraith and J Pansiot. Wearable and ambient sensor fu-sion for the characterisation of human motion. 2010 IEEE/RSJInternational Conference on Intelligent Robots and Systems, pages5505–5510, October 2010.

[62] Michal Kepski, Bogdan Kwolek, and Ivar Austvoll. Fuzzy inference-based reliable fall detection using kinect and accelerometer. Artifi-cial Intelligence and Soft Computing, pages 266–273, 2012.

[63] Stefano Cagnoni, Guido Matrella, Monica Mordonini, FedericoSassi, and Luca Ascari. Sensor Fusion-Oriented Fall Detection forAssistive Technologies Applications. 2009 Ninth International Con-ference on Intelligent Systems Design and Applications, pages 673–678, 2009.

[64] Gregory Koshmak, Maria Linden, and Amy Loutfi. DynamicBayesian networks for context-aware fall risk assessment. Sensors(Basel, Switzerland), 14(5):9330–48, January 2014.

[65] P. A. C. Aguilar, J. Boudy, D. Istrate, B. Dorizzi, and J. C. M. Mota.A Dynamic Evidential Network for Fall Detection. IEEE Journalof Biomedical and Health Informatics, pages 281–285, 2014.

[66] Cavalcante Aguilar, Jerome Boudy, and Dan Istrate. A dynamicevidential network for fall detection. IEEE Journal of Biomedicaland Health Informatics, 18(4):1103–1113, 2014.

[67] P. A. Cavalcante, M. A. Sehili, M. Herbin, D. Istrate, F. Blanchard,J. Boudy, and B. Dorizzi. First Steps in Adaptation of an EvidentialNetwork for Data Fusion in the Framework of Medical Remote Mon-itoring. 34th Annual International Conference of the IEEE EMBS,pages 2044–2047, January 2012.

[68] Hande Ozgur Alemdar, Gokhan Remzi Yavuz, Mustafa Ozan Ozen,Yunus Emre Kara, Ozlem Durmaz Incel, Lale Akarun, and CemErsoy. Multi-modal Fall Detection within the WeCare Framework.Proceedings of the 9th ACM/IEEE International Conference on In-formation Processing in Sensor Networks, pages 436–437, 2010.

remote monitoring and - mdh · remote monitoring and ... it is always nice to be surrounded by...

Documents