person-following using fuzzy inference - wseas · person-following using fuzzy inference samir...

Person-Following using Fuzzy Inference

SAMIR SHAKER

American University of Beirut

ME Department

Riad El-Solh 1107 2020, Beirut

LEBANON

[email protected]

JEAN J. SAADE


ECE Department


LEBANON

[email protected]

DANIEL ASMAR


ME Department


LEBANON

[email protected]

Abstract: For a service robot, being able to follow a person to do a required task is a must have skill. Such an ability,

however simple in concept, has some requirements to be met which makes a simplistic direct implementation

unsatisfactory. Most notably, the robot following the human has to keep at a certain safe distance from the person

and at the same time move in a smooth manner which does not appear threatening to the person. Also, such

a smooth movement would potentially prolong the lifetime of the robot’s locomotive system. In this paper, we

propose using fuzzy inference as the decision system achieving smoothness and person-following behavior while

keeping a safe distance from that person. The system was tested using a laser range finder to detect a person’s legs

giving the inference system a distance and a direction to go to. The experimental results showed that even though

the detection of legs was subject to noise and false negatives, the robot still achieved the goals of smoothness of

motion, keeping of a safe distance, and of course, following its target.

Key–Words: Person following, Fuzzy inference, Leg detection, Smooth movement

1 Introduction

Following a person is an important ability for mobile

robots which interact with people. Also, any robot

which interacts with a person needs to exhibit move-

ment and behaviour which appear unthreatening to a

person having had no interaction with robots before

[1]. A jerky movement by a mobile robot can be per-

ceived as a threat by a person having had no a priori

interaction with a robot and even by someone who has

in case the movement is totally unexpected. Take the

example of an elderly person equipped with a robot

wheelchair required to follow a nurse. Smoothness

of robot motion is especially required in this case as

the robot’s elderly passenger might get hurt from jerky

motion.

Therefore, this paper proposes a fuzzy inference

system meant to control and smoothen out the motion

of a robot while following a person and, at the same

time, keeping it at a safe distance from the person that

is being followed. This ability has been developed in

[2] for the case of a car following another car. The

developed ability, however, has been modified here to

accommodate the case of a mobile robot following a

human. Human motion type is always as smooth as

that of a car and autonomously detecting and tracking

people is relatively more difficult unless the person is

fitted with specialized tracking equipment [3] which

is not convenient and not always possible.

The problem of person following by a robot has

been addressed previously in some research. The

method developed in [4] requires a previously ac-

quired map of the environment to achieve tracking.

This limitation imposed on the method makes it un-

suitable for cases where the robot is required to fre-

quently change the place in which it operates. The

method developed in [5] uses two kinds of sensors,

a laser range finder and a camera for tracking. This

requires unneeded extra cost for its implementation

seeing that it requires two sensors instead of one to

achieve its goal.

Kobilarov et al. [6] propose two different meth-

ods for tracking people. The first method failed be-

cause of the lack of needed environmental features

which were not adequately provided by just an omni-

directional camera. This admitted failure motivated its

authors to pursue another method. The second method

compensates for the camera failure by adding a laser

to keep track of the person which adds to the overall

system cost. The method developed in [7] also uses

an omni-directional camera and requires a new train-

ing of an Artificial Neural Network (ANN) for each

person that has to be followed. Such a requirement

restricts the generality of the method and is unsuitable

for emergency situations such as an elderly sitting on

a robot wheelchair having to follow a new doctor or

nurse.

Consequently, in order to achieve the smoothness

RECENT ADVANCES IN SYSTEMS, COMMUNICATIONS & COMPUTERS,Selected Papers from the WSEAS Conferences in Hangzhou, China, April 6-8, 2008

ISBN: 978-960-6766-61-9 19 ISSN: 1790-5117

and safety objectives of the person following problem

of a mobile robot while keeping the offered problem

solution as general as possible and its implementation

cost as low as possible, a method is proposed in this

paper that makes use of a laser range finder mounted

on a wheeled mobile robot to detect a persons legs and

follow them in a smooth motion while keeping a safe

distance from that person. To achieve these goals, a

fuzzy inference system has been designed using a vi-

sualization program specially developed for that pur-

pose. This fuzzy inference system will be made to

work in conjunction with a leg detection algorithm

that uses the data provided by the laser. The remain-

der of this paper is structured as follows. Section 2

describes the method used to detect a person’s legs.

The output of this algorithm is fed into the fuzy infer-

ence system described in Section 3. Then, Section 4

presents the test platform. Finally, Section 5 shows the

results of the tests which were performed by making

the robot follow a person.

2 Leg Detection

The first step in following a person is detecting the

person’s legs. The output of that step is a position

of a person’s legs relative to the robot, the differen-

tial of which is fed into the fuzzy system described in

Section 3 as the relative velocity of the person with

respect to the robot.

The leg detection algorithm is based on the work

of Milella et al. [8]. It proceeds as follows. First, a

180o laser scan is performed with a 1

o separation be-

tween two consecutive readings [10]. The laser has

a maximum resolution of 0.5o, but this was not used

since it was not necessary for the practicality of the

considered problem and also to improve real-time per-

formance. The data is then processed as shown in the

flowchart of Figure 1(a).

The laser range readings are scanned consecu-

tively looking for a large difference defined to be

above a certain threshold, here taken to be equal to 1

m. This threshold was chosen because a moving per-

son will typically not have an object in front of him

closer than 1 m. So any laser beam passing next to

or between his legs will stop at a distance of at least

1 m above the one blocked by his legs. The bound-

aries we are looking for are shown in dots in Figure

1(b). These also include the first and last laser read-

ings. The regions between these boundaries constitute

possible legs. Then, each region’s width is compared

with a typical human leg width. If it is, the location

of the leg and its size are stored. On the next scan,

this information is used to to make sure the same set

of legs are being followed.

(a) Leg Detection Algorithm

(b) The laser readings are shown as lines and the identified bound-

aries are marked with dots.

Figure 1: Leg Detection


ISBN: 978-960-6766-61-9 20 ISSN: 1790-5117

Once one or more legs are detected, the closest

moving two legs are considered those of the person

which needs to be followed. To get the position of that

person, the two positions of each of a persons leg are

averaged to obtain the center position of that person,

and in case only one leg was detected, only that legs

center is taken. This error introduced in the detection

of legs can be compensated for by the fuzzy inference

system thereafter.

3 The Fuzzy Inference System

Fuzzy logic is called for when the problem to be dealt

with is humanistic and complex and where the use of

classical mathematical models are deemed too precise

for a problem or situation which is imprecise by na-

ture. Hence, fuzzy logic and inference is most suitable

when the input to the system is noisy, and the solution

requires a human kind of reasoning. Following a hu-

man while maintaining smoothness and safety of mo-

tion requires humanistic reasoning because first, it is a

human or intelligent behavior that is being duplicated,

and second, the data sensed by the robot to detect two

moving human legs is sometimes noisy and could be

occasionally wrong.

The fuzzy inference is based on a system for car

following developed by in [2]. To adapt that sys-

tem from the car following scenario to following hu-

mans, a visualization program was developed which

gives real time feedback in response to direct ma-

nipulation of the membership functions. It is impor-

tant to note here that such adaptation from following

cars to following humans is one of the major advan-

tages of fuzzy logic over Artificial Neural Networks

as the fuzzy solution allows redesign and modification

based on its easy-to-understand linguistic solution to

the problem.The robot should follow the legs in a smooth way.

That is, it should decelerate and accelerate smoothlyif the legs change speed. It should also keep a safedistance in case the legs stop abruptly. Since the robotshould remain at a larger safe distance if it was mov-ing faster, the safe distance should be dependant onthe current robot speed. After some experimentation,it was observed that a parabolic formula is the mostsuitable to represent the safe distance. The exact equa-tion of the parabola was derived empirically as:

Safe Dist. = 88×10−6

V2

+ 0.66336 V + Min Safe Dist.

Where V is the current robot velocity taken in mm/s.

The Minimum Safe Distance was taken here to be

equal to 1 m to take into consideration the robot’s

bounding radius.

In order to account for distances and velocities

of human beings, the ranges of the input and output

−6000 −4000 −2000 0 2000 4000 60000

0.5

1

1.5

Relative − Safe Distance (mm)

Me

mb

ers

hip

De

gre

e

Very Small

Small

Near Zero

Large

Very Large

(a) Relative minus Safe Distance

−2500−2000−1500−1000 −500 0 500 1000 1500 2000 25000

0.5

1

1.5

Relative Speed (mm/s)

Me

mb

ers

hip

De

gre

e

Robot Much Slower

Robot Slower

Robot Near

Person Faster

Person Much Faster

(b) Relative Speed

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

x 104

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Acceleration (mm/s2)

Mem

bers

hip

Degre

e

Deceleration Strong

Deceleration Normal

Deceleration Mild

Not Moving

Acceleration Mild

Acceleration Normal

(c) Acceleration

Figure 2: Membership Functions


ISBN: 978-960-6766-61-9 21 ISSN: 1790-5117

RV RD − SD Very Small Small Near Zero Large Very Large

Robot Much Slower Not Moving Acc. Mild Acc. Normal Acc. Normal Acc. Normal

Robot Slower Dec. Normal Not Moving Acc. Mild Acc. Normal Acc. Normal

Robot Near Dec. Strong Dec. Mild Not Moving Acc. Mild Acc. Normal

Person Faster Acc. Normal Acc. Normal Acc. Normal Acc. Normal Acc. Normal

Person Much Faster Dec. Normal Dec. Normal Acc. Mild Acc. Mild Not Moving

Figure 3: Fuzzy inference rules

variables of the fuzzy inference system as well as the

ranges and shapes of the assigned membership func-

tions, as in [2], were modified appropriately, and this

is shown in Figure 2. The fuzzy system uses two in-

puts, the relative velocity of the robot to the person’s

legs and the difference between the relative distance

to the legs and the safe distance. The output is the

needed acceleration (See [2]).

In addition, the amount of acceleration outputed

by the system is influenced by the fuzzy inference

rules of Figure 3. The rules in [2] used directly for

the case of person following produced a lot of oscil-

lations and very slow movement towards the person.

Therefore, the rules in [2] were drastically modified to

make the robot keep going forward towards the person

when the person is obviously far away waiting for the

robot to arrive. Also, they were modified to make the

robot arrive to a complete stop when it is within the

defined safe minimum distance, meaning, to prevent

oscillations. Moreover, the very abrupt deceleration

that was used in [2] was not applied here. While such

decelerations are justified for the case of automobiles

in order to avoid accidents, robots drive at much lower

speeds and therefore can accommodate lower accel-

erations while ensuring speedy arrivals with minimal

oscillations.

As for the defuzzification technique used, the first

of maxima was tested in the beginning but it resulted

in large oscillations. The reason for this is that the

first of maxima defuzzification gives large values of

acceleration between different acceleration levels. So,

going from Normal to Mild Acceleration for example

will result in a large change in speed. This is why this

method was replaced with the usual centroid defuzzi-

fication method which yielded a smoother change in

velocity.

For all possible inputs, the output acceleration can

be visualized in a 3D graph as seen in Figure 4. The

drawn surface shows a smooth transition between very

different acceleration plateaus which translate into an

even smoother transition in velocity given that accel-

eration is the derivative of velocity. Note here that the

flat parts surrounding the surface are simply limits put

on the boundaries of the system so it will not exceed

the maximum velocity that the used robot can attain.

Such a non-linear function would have been very diffi-

cult to model in a classical mathematical way without

the aid of fuzzy logic, and if that were to happen, it

would have been computationally expensive to com-

pute. Moreover, the fuzzy system developed is much

more intuitive to modify and adapt to other situtations.

−6000

−4000

−2000

0

2000

4000

−3000

−2000

−1000

0

1000

2000

3000

−3000

−2000

−1000

0

1000

2000

Relative − Safe Distance (mm)Relative Velocity (mm/s)

Accele

ration (

mm

/s2)

Figure 4: Acceleration in response to all possible in-

puts

(a) P3-AT (b) Hallway

Figure 5: Lab Hallway Test Run


ISBN: 978-960-6766-61-9 22 ISSN: 1790-5117

4 Test Platform

The algorithm was implemented on P3-AT by Mobile

Robots [9] shown in Figure 5(a). The P3-AT robot

is a research platform suitable for many research ar-

eas including mapping and localization. It has a Pen-

tium 3 processor on board with USB and serial ports

to mount different sensors.

To detect legs, the laser range sensor [10], shown

in Figure 5(a), was mounted on the robot and inter-

faced to the serial port of the on-board computer.

The algorithm was implemented in C++. The

robot commands where sent using the Advanced

Robotics Interface for Applications (ARIA) [11] and

then was simulated on MobileSim (Mobile Robots

Simulator) [12]. After successful simulations, the sys-

tem was implemented in real-time on the robot.

For additional visualization aid and fuzzy system

design, the result of laser scans and leg detection, as

well as the visualization of the member functions were

drawn using the Open Graphics Library (OpenGL)

[13] as seen in Figure 6. These visualization were

made interactive as their shape were modifiable at run-

time. This allowed tweaking the membership func-

tions emprically to find the shapes that gave the op-

timum results: no oscillations, a smooth ride and a

reasonable time to arrive at the target goal, i.e., the

person being followed.

5 Results

The algorithm was first tested on the MobileSim sim-

ulator [12]. As seen in Figure 7, the relative distance

minus the safe distance tends to zero smoothly com-

batting sensor noise. Also, Figure 8(b) shows the ve-

locity of the robot when the velocity of the person’s

legs vary and are preceived by the robot as shown

in Figure 8(a). Even though the velocity of the per-

son’s legs (Figure 8(a)) show high-frequency fluctua-

tions, the velocity response of the robot (Figure 8(b))

show smaller fluctuations of lower frequency than the

ones exhibited by the sensed leg velocity. In prac-

tice, when the algorithm was tested on the robot, these

fluctuations in velocity were barely noticable. There-

fore, the algorithm achieved the goal of moving the

robot smoothly towards the minimum safe distance,

here taken to be equal to 1 m, without exhibiting any

noticable oscillations.

Then, the algorithm was tested in the lab hallways

shown in Figure 5(b). First, detection of legs was

tested by letting the robot orient itself towards them

0 5 10 15 20 25 30 35−400

−200

0

200

400

600

800

1000

1200

1400

Time (s)

Re

lative

− S

afe

Dis

tan

ce

(m

m)

StandingStill

ToRobot

MovingAway

StandingStill

StandingStill

Figure 7: Relative - Safe Distance

0 5 10 15 20 25 30 35−1000

−800

−600

−400

−200

0

200

400

600

800

1000

Time (s)

Pe

rso

n V

elo

city (

mm

/s)

StandingStill

MovingAway

StandingStill

ToRobot

StandingStill

(a) Person Velocity

0 5 10 15 20 25 30 35−1000

−800

−600

−400

−200

0

200

400

600

800

1000

Time (s)

Ro

bo

t V

elo

city (

mm

/s)

StandingStill

MovingAway

StandingStill

ToRobot

StandingStill

(b) Robot Velocity

Figure 8: Velocity of the Person compared to that of

the Robot


ISBN: 978-960-6766-61-9 23 ISSN: 1790-5117

Figure 6: Algorthim Simulation and Visual Tweaking Platform. The OpenGL rendering of sensed data (Top Left),

the MobileSim simulator (Top Right) and the fuzzy membership functions of three fuzzy variables (Bottom three

windows)

while staying in its place. Then, the robot was allowed

to follow a passing person in the hallway. The robot

followed the person in front of it smoothly and moved

backwards when the person approached it at a distance

larger than the safe distance. Moreover, when another

person passed during the experiment, the robot cor-

rectly kept following the original person as it detected

that the distance to the legs of the new person would

not correspond to a typical human walking speed. Fur-

thermore, the detected legs of the person originally

followed were closer to the last known position of the

legs.

6 Conclusion

In this paper, a fuzzy inference system used by a robot

for following people in a smooth and safe manner was

developed. It consists of a fuzzy rule based inference

system which can output the acceleration needed for

a robot to follow a human detected by a range sen-

sor. Only a laser sensor was used to detect the legs

of the person which reduced the cost of the system.

Also, the system does not require any special adapta-

tion to the environment that it is supposed to work in.

These objectives combined were not met in previous

studies. This robot ability opens way for new, innova-

tive applications in various fields for aiding people in

their tasks. Future work should test the system using

different sensors such as using a camera for measur-

ing the width of a person’s legs as a way to estimate

their distance from the robot instead of a laser. Such a

way to measure distance, which is more prone to error

than using a laser, would test the system against more

noisy data and see the extents of which it can combat

it. Also, it would help develop a more general system

which can be easily adapted to any sensor from which

range data can be extracted.

References:

[1] R. Gockley, J. Forlizzi, and R. Simmons.

Natural person-following behavior for social

robots. Proceedings of the ACM/IEEE inter-

national Conference on Human-Robot interac-

tion (Arlington, Virginia, USA, March 10 - 12,

2007). HRI ’07. ACM, New York, NY, pp. 17-

24.


ISBN: 978-960-6766-61-9 24 ISSN: 1790-5117

[2] J. J. Saade, O. Houalla, H. Merhi and J.

Kabbani, Fuzzy Inference Based Car Following

Collision Prevention Controller, WSEAS Trans-

actions on Systems. Vol. 5, October 2006, pp.

2291-2298.

[3] R. Bianco, M. Caretti, and S. Nolfi. Developing

a robot able to follow a human target in a domes-

tic environment, Proceedings of the First Robo-

Care Workshop, Rome, Italy, October 2003, pp.

11-14.

[4] S. Thrun; M. Montemerlo; and W. Whittaker,

2002. Conditional Particle filters for Simulta-

neous Mobile Robot Localization and People-

Tracking, Proceedings. ICRA ’02. IEEE Interna-

tional Conference on Robotics and Automation.

Vol. 1, 2002, pp. 695-701.

[5] M. Kleinehagenbrock, S. Lang, J. Fritsch, F.

Lomker, G. A. Fink, and G. Sagerer. Person

tracking with a mobile robot based on multi-

modal anchoring. In IEEE Int, Workshop on

Robot and Human Interactive Communication

(ROMAN), 2002, pp. 423-429.

[6] M. Kobilarov, G. Sukhatme, J. Hyams, and

P. Batavia. People tracking and following with

mobile robot using an omnidirectional camera

and a laser, Proceedings of the IEEE Interna-

tional Conference on Robotics and Automation,

Orlando, Florida, May 2006, pp. 557–562.

[7] G. Cielniak, M. Miladinovic, D. Ham-

marin, L. Goransson, A. Lilienthal, and

T. Duckett. Appearance-based tracking

of persons with an omnidirectional vi-

sion sensor, Proceedings of the Fourth

IEEE Workshop on Omnidirectional Vision.

http://www.cs.wustl.edu/ pless/omnivisFinal/duckett.pdf

[8] A. Milella, C. Dimiccoli, G. Cicirelli and

A. Distante, Laser-Based People-Following

for Human-Augmented Mapping of Indoor En-

vironments, Proceedings of the 25th IASTED

International Multi-Converence Artificial Intel-

ligence and Applications. Innsbruck, Austria,

February 12-14, 2007, pp. 151 - 155.

[9] P3-AT: The High Performance

All-Terrain Robot. Homepage:

http://www.activrobots.com/ROBOTS/p2at.html

[10] Laser Mapping and Navigation Sys-

tem with ARNL Software. Homepage:

http://www.activrobots.com/ACCESSORIES/laser.html

[11] Advanced Robotics Interface for

Applications (ARIA). Homepage:

http://www.activrobots.com/SOFTWARE/aria.html

[12] MobileSim: The MobileR-

obots Simulator. Homepage:

http://robots.mobilerobots.com/MobileSim/README.html

[13] Open Graphics Library (OpenGL). Homepage:

http://www.opengl.org/


ISBN: 978-960-6766-61-9 25 ISSN: 1790-5117

person-following using fuzzy inference - wseas · person-following using fuzzy inference samir...

Documents