RFID-based Object Localisation with aMobile Robot to Assist the Elderly with

Mild Cognitive Impairments

George BROUGHTON 1, Tomas KRAJNIK, Manuel FERNANDEZ-CARMONA,Grzegorz CIELNIAK and Nicola BELLOTTO

Lincoln Centre for Autonomous Systems, University of Lincoln, UK

Abstract. Mild Cognitive Impairments (MCI) disrupt the quality of life

and reduce the independence of many elderly people at home. Thosesuffering with MCI can become increasingly forgetful, hence solutions to

help them finding lost objects are useful. This paper presents a frame-

work for mobile robots to localise objects in a domestic environmentusing Radio Frequency Identification (RFID) technology. In particular,

it describes the development of a new open-source library for interacting

with RFID readers, readily available for the Robot Operating System,and introduces some methods for its application to RFID-based object

localisation with a single antenna. The framework adopts occupancygrids to create a probabilistic representations of tag locations in the

environment. A robot traversing the environment could then make use

of this framework to keep an internal record of where objects were lastspotted, and where they are most likely to be at any given point in time.

This information could be communicated directly to the elderly person

or used by the assistive robot for activity monitoring. Some preliminaryresults are presented, together with directions for future research.

Keywords. RFID, object localisation, assistive robotics, MCI

1. Introduction

Mild Cognitive Impairments (MCI) affect a significant number of people. MCIis often related to age-associated cognitive decline, which progressively has anincreasing effect on the elderly [1]. MCI can lead to diseases such as Alzheimers,which affects approximately 10% of the population at some point in their lives [2].The result of reduced cognition, particularly in the aged, is huge not only finan-cially, but in terms of the impact it has on those affected and their loved ones.Approximately 40% of these end up in full time care. The effect of an ageingpopulation is likely to make this problem worse in years to come [3]. Research has

1Corresponding Author: Lincoln Centre for Autonomous Systems, School of ComputerScience, University of Lincoln, LN6 7TS, Lincoln, UK; E-mail: gbroughton@lincoln.ac.uk.This work was partially supported by the H2020 EU Framework Programme, grant agreement

No. 643691, ENRICHME, and FP7 Programme, grant agreement No. 600623, STRANDS.

Figure 1. Left: ENRICHME assistive robot for the elderly with MCI, provided with various

sensors, including RFID antenna for object localisation. Right: ThingMagic M6e Development

Kit including RFID reader and tag sample.

shown that one of the best preventive strategies for those at risk is to maintain an

independent life style, although this can prove difficult for those living alone [4].

Assistive robotics looks at how the lives of the elderly can be improved, par-

ticularly in maintaining their independence, by providing support with people’s

daily activities. The main problem with this technology is that it is still very

difficult for robots to perceive and make sense of the world around them, which

makes solving practical problems, such as finding a set of keys, a challenging task.

This paper looks at the use of modern RFID technology as a solution to this prob-

lem. The emergence of this technology gives mobile robots an extra dimension

with which to sense their surroundings. Combined with the relatively low cost of

fitting a home with RFID tags, a mobile assistive robot can be made aware of

objects within a domestic environment, and therefore where they were last seen,

and where they are most likely to be in the future given past experiences.

The major contributions of this paper are the design and technical description

of a new software library2 for a mobile robot equipped with an RFID reader and

a single antenna, which is readily available within the Robot Operating System

(ROS) framework, and its application to the problem of indoor object localisation.

The research is also part of the European project ENRICHME3, one of the tasks

of which is to provide an assistive robot (see Figure 1) with the capability of

finding objects at home for the elderly with MCI.

The remainder of the paper is as follows: Sec. 2 covers relevant work in this

research area; Sec. 3 describes the framework and some innovative technical so-

lutions adopted in the proposed software library; Sec. 4 presents preliminary re-

sults towards the implementation of RFID-based object localisation; finally, Sec. 5

concludes the paper with insights into current achievements and future work.

2LibMercuryRFID – http://github.com/broughtong/LibMercuryRFID3ENRICHME: ENabling Robot and assisted living environment for the Independent Care

and Health Monitoring of the Elderly – http://www.enrichme.eu

2. Related Work

Combining mobile robots with RFID technology is an interesting and well knownarea of research, although it still poses many challenging problems. The abilityto remotely detect small passive tags, which are easily embeddable within anenvironment, has appealed to many, with uses ranging from self localisation, toobject detection and recognition [5,6]. However, the associated difficulty withusing RFID in these contexts stems from the noisy data received, particularlyas the radio waves emitted have a tendency to reflect in indoor environments,creating multipath signals which can confuse most localisation models [7,8].

The presence or absence of RFID tags in an environment can be used to givemobile robots hints about their environment for self-localisation and environmentmapping. However, information about tags is not limited to a simple binary de-tection, but also provides information about the tag’s identity. Each tag has itsown alpha-numeric serial number encoded into it, which is modulated into thesignal returned to the reader. RFID readers can then be used, for example, inindoor positioning systems [9], by including this technology as part of a sensorfusion model to help robots estimate their location using strategically positioned“landmark” tags at known points [10,5].

One of the most commonly used approaches to determine the proximity ofRFID tags is based on the Received Signal Strength Indication (RSSI). Someapproaches work by searching for robot locations and orientations with the highestRSSI, and use this to calculate the most likely location of the tag [11]. Otherapproaches involve predicting how the signal will behave within a given setting,then comparing the prediction to actual tag data [12].

Most of the RFID localisation algorithms tend to use multiple readings, takenfrom different antennas at several robot locations and orientations. The problemcan then be solved as a results of an optimisation process, or using some form ofBayesian estimation (e.g. Adaptive Monte Carlo Localisation) to find the mostprobable location of the tag [11,12]. Locating objects from a mobile robot with asingle antenna, however, is still an open problem.

One of the most sophisticated approaches to locating tags involved the use ofthe returned signal’s phase information to identify very small movements of thetag. This approach looks at how the signal phase is tied to the distance of the tag,and how this could be used with prior knowledge of range to detect very minorchanges in distance [13]. The approach has been used for robot localisation, whereodometry and other sensor information can be fused to reduce the estimationerror, but its application to objects localisation is still mostly unexplored.

3. System Framework

To maximise compatibility with existing robotics systems, a requirement of theproposed framework is that it must be compatible with the Robot Operating Sys-tem4 (ROS). This benefits both the current research, in terms of being able toreadily access existing robotic infrastructure, and also the general robotics com-munity, which will be able to take advantage of the RFID localisation framework

4Robot Operating System – http://www.ros.org

Figure 2. Modular framework design. To work with a new reader, a developer would need only

to add a node to output its data in the correct format. Alternative localisation algorithms (e.g.RSSI- or Phase-based) can be added to make use of this data and provide additional information

for the occupancy grid, which is then used to determine the most likely position of tags.

and the respective software library for other uses. The systems should also beable to concurrently track multiple tags, and handle multipath errors. The lat-ter are usually caused by reflections in the environment, where several differentpaths can be taken by the radio waves between the tag and the reader, withperformance-degrading effects on the algorithms estimating distance or direction.Finally, the framework should be compatible with the many different use-casesin the robotics community. This means, for example, a software design that isindependent from the specific RFID readers, and which allows future updates,improvements or porting across different platforms with ease. The modular designillustrated in Figure 2 addresses these issues.

Algorithmically, the framework consists of ROS nodes listening for the RFIDinformation provided by the reader driver. These nodes then use such informationto detect and localise tags in the environment. Based on this, they update a globaloccupancy grid, i.e. a discrete map representation of the environment. Many ofthese algorithms could be run in parallel, and the respective results combinedusing the most appropriate approach. As an example, if algorithm A performedbetter when the system had just started, it could receive a higher weighting,whereas if algorithm B performed better once it had refined the data over severaltag reads, its weighting could be gradually increased. Other approaches suchas probabilistic sensor fusion, e.g. Kalman-based [14], could also be adopted,although the topic is currently outside the scope of this paper. From the globaloccupancy grid, where the output of the framework algorithms is collated, themost likely tag position can be inferred and used for the specific application.

3.1. RFID Library Implementation

The current system has been implemented on a ThingMagic M6e (see Figure 1).The manufacturer, one of the largest producers of RFID readers worldwide, since2009 does not allow direct serial communication to their devices. By prevent-ing this protocol from being used, they in effect made many of the existing li-braries and previous ROS interfaces incompatible with their readers. These pack-ages include the hrl_rfid library for ROS, which was deployed on robots suchas PR2 [11]. Because hrl_rfid communicated via the low level serial protocol, itcannot be used on any new versions of the ThingMagic devices. Instead, a dedi-cated Application Programming Interface, the Mercury API, has been provided,so that programs using this API should work with any of their readers (from

Figure 3. RFID library pipeline. The C wrapper provides a middleware to handle the commu-nication between the RFID reader API and the ROS application interface.

M4 onwards, mandatory from M6), including future models. One of the benefitsof using this particular model is the large read range, which has been tested toexceed 6m in a noisy environment. The reader is also able to distinguish betweenup to 200 tags simulatenously. Each RFID tag has an associated alpha-numericidentification key. This key is then modulated into the return signal, which allowsit to be uniquely identified when multiple tags are present. RFID standards alsomake use of anti-collision algorithms which allow the reader to effectively readmultiple tags simulatenously. As such, cluttering an environment with many tagshas relatively little impact on the ability of the system to track an individual tag.

Unfortunately the Mercury API is available in C only and it is not directlyaccessible by many other programming languages. Normally, a workaround is pro-vided by other simple library commands. In Python, for example (which togetherwith C++ is the standard language for ROS), this functionality can be obtainedby using modules such as CTypes5. Unfortunately, this means that Python wouldhave to create and handle complex C data structures designed for the API, whichis infeasible. These problems motivated the development of a more robust andflexible RFID software library, compatible with the above system framework.

The solution adopted in the current software library includes an extra C mid-dleware, which is able to create instances of the reader’s API classes without wor-rying about reimplementing its complex data structures. As a result, a C librarywas written to abstract the difficulties of interfacing with the API. This greatlysimplified the task of managing memory across different languages, focusing onlyon useful information rather than data structures. The library works on a periteration basis, so it is not a problem for the same library to communicate withseveral different readers and antennas on the same machine. Each of these canthen track hundreds of tags independently.

A limitation of cross-language communication, like the one here presented,is the lack of protection from code incompatibilities (otherwise provided by acompiler able to check both code bases). As a result, incorrect parameters passedfrom Python to C, such as wrong number of arguments, could generate criticalerrors and segmentation faults. To avoid the risk, the library is designed withlightweight language-specific interfaces (see Figure 3). This has the advantageof allowing the library to be available to Python applications (and therefore toROS), rather than being accessible via complex CTypes. Furthermore, the libraryincludes an additional C++ interface to ease the development of applicationsin this language. Finally, an important feature of the library is that the driversfor other models of RFID readers can be plugged in and, with an appropriatewrapper, benefit from the same Python/C++ interfaces.

5A foreign function library for Python – http://docs.python.org/2.7/library/ctypes.html

RFID Tag

Figure 4. Left: Experimental platform with the RFID reader equipment and the RFID tag.Right: RSSI map with cardinal readings from set positions - tag position is in black, colour

slices denote relative RSSI intensities. Readings are taken in a grid with 1m spacing.

4. Experiments

In order to test the functionality of the RFID library and its application to objectdetection, two mobile robots were alternatively equipped with the ThingMagicM6e RFID reader depicted in Figure 1, and their ability to detect RFID tagsin an indoor environment evaluated. To this end, a platform based on the Me-traLabs SCITOS-G5 mobile robot (see Figure 4) was initially used. The robotwas equipped with a precise self-localisation system based on an on-board laserrangefinder and an Adaptive Monte Carlo Localisation software [15], which pro-vided position and orientation with 5cm / 5◦ accuracy.

In the first experiment, the dependence of the RSSI response on the directionof the RFID tag (with respect to the robot) was measured. In the second one,a probabilistic estimate of the RFID tag position was obtained by first creatinga coarse model of the antenna’s radiation pattern, and then using the latterin a Bayesian filtering scheme to map the tag’s position on a uniformly-spacedgrid while the robot rotates. Finally, an experiment with the Robosoft KOMPAIplatform (see Figure 1), having hardware and software configurations similar tothe previous one, was performed to simulate a simple object detection scenario.

4.1. Cardinal Points

In this experiment, an initial data set was created to analyse how the RSSI of asingle tag varies while the robot moves around the environment. and take RSSImeasurements. RSSI combined with the robot’s position and orientation was usedto establish how the signal varied around a real environment.

In order to do that, the robot was moved in a two-dimensional grid of equallyspaced navigation points; at each point, a series of readings was taken for eachcardinal direction (i.e. North, East, South, West). The median values of thesereadings were then used to create the RSSI map on the right part of Figure 4,where each circle represents one of the navigation points, and the four colourslices within it represent the signal strength in the respective cardinal directions.

As expected, the results of this experiment confirmed that most of the signalstrength concentrated in directions facing the tag, and quickly dropped as therobot’s orientation moved away from the tag’s bearing.

4.2. Grid-based Position Estimation

In order to implement the following object localisation method, a sensor modelfor the antenna needs to be created. This sensor model is the basis from which tagposition is derived, so its accuracy directly influences that of the final results. Ide-ally, such sensor model should contain the unique propagation distortions causedby the robot and the rest of the environment interfering with the RFID signal.Any change to the layout of the robot or the state of the environment can influencethe signal propagation and negatively affect the estimation process.

For the current experiment, therefore, an initial approximation of the an-tenna’s radiation pattern, provided by the manufacturer, was refined by usingreal data collected with the RFID reader. This was done by spinning the roboton the spot, with a tag at a fixed distance from the centre of rotation. As therobot rotated, the RSSI values were recorded together with the robot’s actual ori-entation (from its self-localisation). The approach was sufficient to yield a coarseapproximation of the sensor model, which was then used to implement the fol-lowing probabilistic occupancy grid. The above sensor model was used to deter-mine the likelihood of a particular RSSI reading, given that the position of thetag relative to the robot is known. This allows the implementation of a simpleBayesian update to compute the probability of the tag being at a particular lo-cation whenever a new RSSI reading is available. The approach adopted next isanalogous to the practice of building occupancy grid models from noisy sensorsin mobile robotics [16,17]. A uniform, 1000×1000 grid was created, with a 8mmresolution, representing the probability of the tag’s location in the robot envi-ronment. Each cell contained a value that indicates the probability of the tagbeing at that particular cell location. Whenever a reading was received, the grid’scells were updated using the aforementioned Bayesian approach, which allowedfor continuous integration of the sensor readings during the robot’s movement.

To test the approach, the robot was rotated on a set location while contin-uously gathering new RSSI readings. The robot localisation system provided aprecise position estimate in the global reference frame. The grid-based positionestimation of the tag, in the same global reference frame, was then refined duringthe rotation.

Figure 5 illustrates the evolution of the tag’s likely position as the robot ro-tates. The upper-left picture of Figure 5 shows the grid after the 360 degrees: notethat the robot is oriented towards the bottom left of the grid and the most proba-ble locations of the tag’s locations lie on an curve that correspond approximatelyto an isoline of the sensor model. This is expected, as a single measurement can-not provide the position of the tag, but it can at least constrain the tag’s possiblelocation around an isoline that corresponds to the strength of the RSSI reading.However, integration of the RSSI measurements during the robot rotation intothe probabilistic grid results in an improved estimate of the tag’s location. Thefigure indeed shows how the probability distribution of the tag’s position evolvesas more and more measurements are integrated in the grid – the individual images

Figure 5. Probability distribution of the tag’s estimated location during the grid experiment.

Robot’s position and orientation are in red; tag’s real location is the white mark located almostdirectly above the robot. The black color corresponds to a higher likelihood of the tag being

on the respective location. The top-left image reflects the situation after the first RSSI reading,

and the following images represent the probabilistic update after the robot rotated by 90, 180,360, 720, 1080 and 1440 degrees.

corresponds to the grid’s states after the robot rotated 90, 180, 360, 720, 1080and 1440 degrees respectively. After two rotations, the tag’s position estimate isalready good enough for many practical applications of the system.

4.3. Object Detection Scenario

A final test was performed by simulating a typical object detection tasks, withthe KOMPAI robot moving in a domestic-like environment and identifying sev-eral tagged objects. These include a mug, a stapler, a remote control, a kitchentable and a coffee table. The approach used in this case was simpler than theone described before: the robot simply counted how many times every tag wasdetected within a specific interval of time. If the detections were more than anempirically determined threshold (50 in this case), then the system would considerthe respective object as detected somewhere nearby the robot.

As illustrated in Figure 6, the robot was successful in locating the taggedobjects. The readings obtained during this experiment are also consistent withthe simpler one-tag scenario in Sec 4.1. A video of the whole test is also availableonline6. Although still relatively simple, the results validate the approach andshow that the proposed RFID system is able to deliver reliable information aboutobjects position.

6Object detection video – http://goo.gl/QSNWk0

Figure 6. Object detection. The three tagged objects (kitchen table, remote control, stapler)

are shown on the top-left. The remaining figures show the moment in which the objects werecorrectly detected by the robot navigating in a simulated domestic environment. The name,

RSSI and Phase of the objects are printed on the laptop screen at the bottom of each figure.

5. Conclusion

This paper presented the design and implementation of a library that integratesseveral software components, providing a seamless interface for the task of RFIDlocalisation. The successful application of the library to detect RFID tags wasdemonstrated using an experimental set-up mounted on a mobile robot. Devel-opers could benefit from using this library with its ease of access to RFID read-ers from various programming environments in order to manipulate and processRFID data. A ROS publisher is also included, allowing this information to be com-bined with other robotic sensor data for various sensor-fusion based applications.The experiments demonstrated the efficacy of the system for performing objectlocalisation with a mobile robot. They showed that the system can potentially beused in real world applications, including domestic robotics scenarios. In terms ofreal-world potential, the system is not solely limited to the specific context, butcould also be used in other scenarios, including those without a mobile robot.

From a technical point of view, the system could be extended further to takeinto account other approaches for RFID signal processing. These include exploit-ing techniques such as Minimum Required Transmission Power, which would lookat reducing the power of the antenna to the point where a particular tag is onlyjust being detected, and use this to discard locations where estimated drop inRSSI does not match that observed. Another technique involves making use ofthe signal phase information to enhance the accuracy of the system. Future workwill look at whether the limitations of phased-based techniques can be overcomeby integrating RSSI information as well. Furthermore, the sensor model used inthis work represents only a coarse approximation of the antenna. A refined, higherresolution sensor model will enable more accurate estimations of the tags in therobot’s surroundings. This preliminary work is part of a more advanced project,

ENRICHME, to assist the elderly at home. One of the key tasks in this projectis indeed to develop an RFID-based object localisation system that can help theuser finding lost items. These solutions will be eventually validated in real homesof elderly with MCI, and evaluated in terms of technical performance as well asquality of life improvement.

References

[1] Ian J Deary, Janie Corley, Alan J Gow, Sarah E Harris, Lorna M Houlihan, Riccardo E

Marioni, Lars Penke, Snorri B Rafnsson, and John M Starr. Age-associated cognitivedecline. British Medical Bulletin, 92(1):135–152, 2009.

[2] Clement T Loy, Peter R Schofield, Anne M Turner, and John BJ Kwok. Genetics of

dementia. The Lancet, 383(9919):828–840, 2014.[3] Michael Caley and Khesh Sidhu. Estimating the future healthcare costs of an aging

population in the UK: expansion of morbidity and the need for preventative care. Journal

of Public Health, 33(1):117–122, 2011.[4] Philip A Greiner, David A Snowdon, and Frederick A Schmitt. The loss of independence

in activities of daily living: the role of low normal cognitive function in elderly nuns.

American Journal of Public Health, 86(1):62–66, 1996.[5] Lionel M Ni, Yunhao Liu, Yiu Cho Lau, and Abhishek P Patil. LANDMARC: indoor

location sensing using active RFID. Wireless networks, 10(6):701–710, 2004.[6] Se-gon Roh and Hyouk Ryeol Choi. 3-D tag-based RFID system for recognition of object.

IEEE Trans. on Automation Science and Engineering, 6(1):55–65, 2009.

[7] Thierry Germa, Frederic Lerasle, Noureddine Ouadah, and Viviane Cadenat. Vision andRFID data fusion for tracking people in crowds by a mobile robot. Computer Vision and

Image Understanding, 114(6):641–651, 2010.

[8] Jue Wang and Dina Katabi. Dude, where’s my card?: RFID positioning that works withmultipath and non-line of sight. ACM SIGCOMM Computer Communication Review,

43(4):51–62, 2013.

[9] Samer S Saab and Zahi S Nakad. A standalone RFID indoor positioning system usingpassive tags. IEEE Trans. on Industrial Electronics, 58(5):1961–1970, 2011.

[10] Byoung-Suk Choi, Joon-Woo Lee, Ju-Jang Lee, and Kyoung-Taik Park. A hierarchical

algorithm for indoor mobile robot localization using RFID sensor fusion. IEEE Trans. onIndustrial Electronics, 58(6):2226–2235, 2011.

[11] Travis Deyle, Matthew S Reynolds, and Charles C Kemp. Finding and navigating tohousehold objects with UHF RFID tags by optimizing RF signal strength. In Intelligent

Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on, pages

2579–2586. IEEE, 2014.[12] Dominik Joho, Christian Plagemann, and Wolfram Burgard. Modeling RFID signal

strength and tag detection for localization and mapping. In Robotics and Automation,

2009. ICRA’09. IEEE International Conference on, pages 3160–3165. IEEE, 2009.[13] Emidio DiGiampaolo and Francesco Martinelli. Mobile robot localization using the phase

of passive UHF RFID signals. IEEE Trans. on Industrial Electronics, 61(1):365–376, 2014.

[14] Q. Gan and C.J. Harris. Comparison of Two Measurement Fusion Methods for Kalman-Filter-Based Multisensor Data Fusion. IEEE Trans. on Aerospace and Electronic Systems,

37(1):273–280, 2001.

[15] Sebastian Thrun, Wolfram Burgard, and Dieter Fox. Probabilistic robotics. MIT press,2005.

[16] Petr Stepan, Miroslav Kulich, and Libor Preucil. Robust data fusion with occupancygrid. IEEE Trans. on Systems, Man, and Cybernetics, Part C: Applications and Reviews,35(1):106–115, 2005.

[17] Sebastian Thrun. Learning occupancy grids with forward sensor models. AutonomousRobots, 15:111–127, 2002.