Abstract: Swarms of autonomous robots demand for simple, robust and flexible algorithms for navigation and communication. Biological evolution has developed behaviors in animals which are efficient and robust. Inspired by the trophallactic behavior (mouth-to-mouth feedings) of social insects , we developed a simple local-neighbor communication strategy that allows a swarm of autonomous robots to make optimal collective decisions concerning the navigation of individual robots [21, 19]. In this article we present a novel elaboration of this distributed algorithm, that allows the robot swarm to collectively avoid unsuitable terrain by forming trails of robots that circumvent such areas. We demonstrate the key features of this new algorithm, analyze its performance in several environmental situations and show some interesting solutions found by the robot swarm in complex environments. Key-Words: swarm robotics, self-organization, honeybees, trophallaxis, swarm-intelligence.

1. INTRODUCTION Swarm robotics is the latest “hype” in the field of

mobile autonomous robotics. In recent years, the progress in miniaturization of technical devices lead to a substantial reduction of the size of the autonomous robots and simultaneously to a decrease of the costs of robots. The EU-funded project I-SWARM [22] has the goal to develop a swarm of 1000 very small robots (size approx. 2mm x 2mm x 2mm). A spin-off of this project was the development of the JASMINE robot [9, 10, 11, 26], which is a cheap and reliable robot (size 3cm x 3cm x 3cm). These robots have two wheels driven by two DC motors [26] and are able to communicate with their local neighbors using 6 LED light beams and photo-receptors.

The aim of constructing such a robot swarm is to

investigate self-organization and swarm-intelligence in

a physically embodied unit, which makes results and insights more reliable compared to bodiless computer simulations. The common goal of the swarm-robotic community is to find methodologies to derive individual rules for swarm robots that allow them to achieve a common (swarm level) goal in a desired manner. It is desired that such rules are simple and that the emerging collective behavior of the robot swarm is flexible (e.g., concerning different environments) and robust (e.g., concerning fault tolerance).

One approach to derive such individual rules is bio-

inspiration. During the process of biological evolution, nature has developed lifeforms that fulfill the demands characterized above. Natural selection favours these principles (simple rules, robustness, and flexibility) inherently without the need to explicitly select for these features. Simple behaviors minimize the need of complexity in the information-processing system that allows an animal to produce its behavior and also minimize energetic expenditures. Only animals which have a behavioral program that is flexible enough to adapt to the encountered environmental changes and which is also robust enough to deal with erroneous circumstances (noisy perception, noisy actuators), are able to survive and reproduce successfully. Thus the biological world shows plentiful examples of behaviors that can be used as a source of inspiration for algorithms that determine the behavior of swarm robots.

In a robot swarm, the desired behavior is a

collective one. This means that single individuals do not act egoistical to reach a desired goal. Instead of that, the members of a swarm can draw advantage from their ability to communicate and collectively allocate tasks among them in a way that increases the global efficiency of the whole swarm. The bigger the swarm gets, the higher is the required communication

Bio-inspired Navigation of Autonomous Robots in Heterogenous Environments

Schmickl Thomas*, Möslinger Christoph, Thenius Ronald, Crailsheim Karl

*) corresponding author: Department of Biological Sciences, East Tennessee State University, Box 70703,

Johnson City, TN 37614-1710, USA, schmickl@nextra.at, thomas.schmickl@uni-graz.at

bandwidth if a centralized communication technique is used. Thus bigger swarms require decentralized communication strategies like nearest-neighbor communication. On the one hand this makes it more difficult to “steer” the swarm from outside but on the other hand, it allows parallel and distributed problem solving.

In the paper we present here, we use a

communication principle that is derived by the trophallactic behavior (mouth-to-mouth feedings) of social insects. We presented a basic algorithmic framework of this “trophallaxis-derived strategy” in recent articles [19, 21] and showed that it allows a robot swarm to perform tasks like trail formation, finding of the shortest way between two points and collective perception of size measurements of obstacles in the arena. For the experiments presented in this article, we elaborated this strategy further, so that our robot swarm is able to collectively explore the arena for target points, to scan the arena for unsuitable (rough) terrain and to communicate a collectively generated “map” of the terrain among the swarm members. This map is represented by a gradient that is constantly updated by the swarm members by communication that mimics trophallactic acts and the resulting gradient is used by the individual robots to navigate in a self-organized manner.

We investigated the properties of this elaborated

version of the “trophallaxis-derived” swarm algorithm by using a multi-agent computer simulation that resembles the sensor model and the motion model of JASMINE robots [26]. Using this simulation platform (LaRoSim, [19, 25]) we simulated a swarm of 222 robots performing the task of collective floor cleaning. We introduced different environmental situations (sizes and distribution of rough terrain as well as several levels of roughness within these areas) and investigated the global behavioral performance of the robot swarms, as well as its efficiency in performing the cleaning task.

2. PROBLEM FORMULATION For investigating our trophallaxis-derived strategy

and the emergent behavior of our robot swarm, we created several variants of arena setups. We used our robot swarm simulation platform LaRoSim V 0.59 [19, 21] to simulate a swarm of 222 (= optimal robot-density for arena of this size) robots. The robot

swarm’s task was to perform collective cleaning. In the arena there was one area of dust where the unloaded robots had to pick up dust particles and transport them to the dump area. As soon as the loaded robots arrived at that designated dump area, they had to drop the dust particle and again head towards the dust area. To make the task more challenging, we introduced additional areas which represent unsuitable terrain for the robots. These terrains will further on be called “rough” terrain and affect the robots mainly by slowing down their motion speed. In our analysis we varied the “roughness” of these terrains, which means that we tested several degrees of speed reduction in these terrain.

There are several plausible ways to assess the

overall efficiency of the robot swarm in such experiments (compare [19, 21]]): e.g., the number of delivered dust particles after a given time period or the mean carry time per dirt particle. We decided to concentrate on the transportation paths of the loaded robots and to analyze the period it took them to transport their load from the dust area to the dump area. We also investigated the behavioral aspect of our elaborated control algorithm by analyzing the paths chosen by the loaded robots. We think that this is the most critical parameter in this setup, because the transport of particles is a critical task. The longer it takes, the more likely a particle falls out of the transporting system and probably gets lost. For details, we refer to [25]. In this article we analyzed the properties of an electrostatic needle which can be used to transport small particles.

For the basic analysis of the properties of our

control algorithm, we designed a simple basic setup. One dust area was located in the lower left corner of the arena, one dump area was located in the upper right corner of the arena and one circular area of rough terrain was located in the center of the arena, so that the loaded robots had to move along the whole diameter of the rough terrain area if they chose the shortest path from dust to dump (see Fig. 1). We tested this simple symmetrical setup with different degrees of “roughness” inside the unsuitable terrain.

Fig. 1: (a) Initial arena setup with the dust area in

the lower right, the dump area in the upper right and the robots depicted as small red cubes. The large circular green area in the center represents rough terrain. (b) Cumulative paths chosen by loaded robots which transport dust particles from the dust area to the dump area. In this sub-figure we used our original algorithm which did not result in avoidance of the rough terrain. (c) In this sub-figure, we used our improved algorithm which enabled the robots to respond to the rough terrain and which lead to collective avoidance of the rough terrain.

In the second part of this article, we introduced

several areas of rough terrain in the arena and observed how the chosen paths of loaded robots were affected by varying the parameter “k” of our control algorithm (for description of “k” see chapter “Novel elaboration of the algorithm”). We also simulated a scenario where the fastest path for the robots could be blocked by a wall and observed the adaptive abilities of the swarm.

In the final third part of this article, we created a

scenario which involved a collective choice for the optimal dump area to deposit the picked-up particles. One dump area is closer to the dust area, but on the direct way to this dump an area of rough terrain increases transportation time. The other dump is further away but allows direct unhindered transportation. By varying parameters of our suggested control algorithm we investigated the collective decisions of the robot swarm.

3. PROBLEM SOLUTION 3.1 Previous status of the algorithm The “trophallaxis-derived” strategy is inspired by a

frequently found behavior in social insects: The mouth-to-mouth transfer of liquid food between animals. Beekeepers often install internal feeders in the honeybee hives to provide the bees with sugar-water (in the following called “nectar”). At these feeders, some bees fill their crops and then move away. On their way through the hive, they meet other bees and can share parts of their nectar load with them. It is assumed, that the more nectar the donor bee has and the less nectar the receiver bee has, the more nectar is transferred on average. On their way, the bees also consume a fraction of their nectar load to gain energy from it. In the robot-swarm, the nectar crop of the bee is represented by a memory place

inside of the robot. Basically each robot i starts with random movement and with a memory value m(i,t)=0. If the robot encounters a target area (= equivalent to internal feeders in a honeybee colony), it adds a defined amount of ‘virtual nectar’ to its memory aa(i,t)=ra (ra: addition-rate, aa(i,t): amount of addition). Every time step, robot i communicates (via direct robot-to-robot comunication) with its local neighbors j and exchanges an amount of ‘virtual nectar’ with them. The amount at(i,t) of this exchange is proportional to the differences in the memory values among the robots and is determined by the transfer-rate rt: at(i,t)=0.5*(m(j,t-1)-m(i,t-1))*rt/N. The variable N represents the number of local neighbors the focal robot communicates with. In case of N=0, the value of at(i,t) is set to 0. Every time-step, each robot i also decreases its memory value by an amount ac(i,t) which is defined by the consumption rate rc: ac(i,t)=m(i,t-1)*rc. After all these in-flows and out-flows of ‘virtual nectar’ are calculated by each robot, the memory-value can be updated according to the following equation: m(i,t)=m(i,t-1)+aa(i,t)+at(i,t)-ac(i,t). By the rules mentioned above, a gradient of memory values emerges within the robot swarm. Each robot i turns towards its local neighbor with the highest memory value, but only if this value is higher than its own memory value. If there is no local neighbor with a higher memory value around, the robot turns randomly. Then the robot moves in this direction. By repeating this procedure each time step, the robots find their way to the target areas.

3.2 Novel elaboration of the algorithm To enable robots to avoid unsuitable areas using the

‘trophallaxis-inspired’ strategy, we extended the existing strategy. The key idea was that a robot on an unfavorable area (e.g. an slippery area that reduces the robots motion speed) can react by increasing its consumption rate rc. For the sake of simplicity, we assumed a linear correlation of the increase of consumption rate to the decrease of the motion speed. This increase of the consumption-rate leads to the emergence of deep “valleys” in the shared gradient map which is constructed in the “trophallaxis-derived” strategy. Because the robots always move uphill in this gradient, the increased consumption rate rc in unfavorable areas forces other robots to avoid these areas and follow trails that are formed along the emerging “ridges” to their target areas. The change of the robots’ consumption rates is described by the equation 1:

max

maxminmin v

kv(i,t)vrcrcrc(i,t)

⋅−+= (1)

In this equation, rc(i,t) represents the actual consumption-rate of the robot i at time t, rc_min defines the base consumption-rate of robot i, v_max represents the maximum speed of the robot, v represents the actual motion speed of robot i at time t. k describes the steepness of the linear correlation of the robot’s speed reduction to the robot’s increase in consumption rate. A higher value of k leads to a higher increase of the consumption rate with the same speed reduction. Please note that with this extension of the algorithm the former constant consumption rate rc became an individual variable of each robot that changes over time due to the local environment of the robot. The value of k is a global constant, which has the aim to allow us to regulate the spatial resolution of the repellent effect of unfavorable terrain.

Fig. 2: The dependence of the increase of the

consumption-rate (rc(i,t)) on the decrease of robot speed with different values for the parameter k.

4. RESULTS AND CONCLUSION In this article we confronted a simulated swarm of

autonomous robots with the goal to pick up dust particles and to carry these dust particles on the fastest way to a dump area. In contrast to previous experiments [19, 21] the complexity of this task was increased by introducing unfavorable (rough) terrain into the arena (see Fig. 1a). We used the trophallaxis-inspired control algorithm that was already successfully tested in [19, 21] to control the navigation of the robots. To solve the afore mentioned problem

with the presence of the rough terrains, we had to extend the strategy by adding equation 1 to the algorithm. By doing so, we changed the former global parameter “consumption-rate” (rc) into a speed-dependent variable that is a property of each individual robot. This addition allows the robots to adapt their trophallactic behavior according to the local environmental circumstances they encounter. The most important parameter in equation 1 is the parameter k (see Fig. 2). By choosing a low value, we can modulate the swarm’s behavior in a way that the trail of loaded robots (that is heading towards the dump area) is more likely to traverse the rough terrain (see Fig 1b). By choosing high values for k, the robots tend to circumvent areas of rough terrain (see Fig. 1c). Please keep in mind, that the behavior of the swarm is still characterized by high plasticity in its reaction to different degrees of roughness of the terrain (see Fig. 3a).

We evaluated this behavioral plasticity with

different values for the parameter k and with different levels of roughness of the terrain in the central circular area (as it is depicted in Fig. 1a). This analysis revealed that there is a graduated response in the swarm’s collective reaction to different degrees of roughness. The higher the value of k and the higher the roughness of the arena was, the higher was the percentage of loaded robots that avoided the rough terrain and circumvented it (see Fig. 3a). To get a clearer picture of the behavioral change that is induced by the central area of rough terrain (Fig. 3a), we only measured the time while the robots had to pass the central, “critical” area.

To assess the efficiency of this emergent behavioral

adaptation, we investigated the mean carry times of dust particles, based on the definition that shorter carry times represent a higher efficiency of behavior. As can be seen in Fig. 3b, our elaboration of the original algorithm indeed led to a higher efficiency of the swarms behavior. With the original algorithm (k = 0) a higher roughness led to a significant increase in the mean carry time of particles due to the reduced speed of the robots in the central area of the arena (Fig. 3b, first row of bars). With higher values of k, the swarm starts to circumvent the central area as the speed reduction increases. Fig. 3b shows only slight increases of the mean carry time per particle with increasing speed reduction in the central area. Thus we can conclude that this behavioral change is efficient and adaptive to the environmental circumstances.

Fig. 3: (a) Avoidance behavior of the robots in an

arena with rough terrain; with different levels for the speed-reduction and with different values for the parameter k. Higher values of k or the speed-reduction lead to a steeper change in the avoidance behavior of the robots. (b) Mean carry periods of the loaded robots on their way to the dump area. The mean carry time increases with higher levels of speed-reduction but this increase is damped when using higher values for k. The red arrays (k = 0) represent the results using our original algorithm. The blue arrays (k > 0) show results of simulations of our improved algorithm.

To demonstrate the potential of our elaborated

algorithm, we tested the algorithm in more complex environments (see Fig. 4). In the first setup (Fig. 4a) we confronted the swarm with an arena containing several areas of rough terrain. With a moderately high value of k (k = 10), the majority of the robots chose the shortest path from the dust area to the dump area (see Fig. 4b). Please note that of the two short passages the broader one was chosen autonomously. With a higher value of k (k = 35), the swarm almost completely avoided the channels between the rough terrain and circumvented the whole area of rough patches (see Fig 4c).

Fig. 4: Paths of loaded robots in an environment

with several rough terrains and with high and low settings of the parameter k. For a detailed description, please see the text. (a) Arena with several smaller rough terrains. (b) Cumulative paths chosen by loaded robots with a moderate high value of k. (c) Cumulative paths chosen by loaded robots with a very high value of k.

In a second setup, we confronted the swarm with

two bigger areas of rough terrain (Fig. 5a) and chose a high value of k = 20. As expected, the swarm reacted

by forming an s-shaped path that writhes between those rough terrains (see Fig. 5b). After we closed that passage by introducing a wall between the two rough terrains, the swarm adapted to this change and chose the shortest way through the rough terrain (Fig. 5c).

Fig. 5: Paths of loaded robots in an environment

with two larger areas of rough terrain. (a) Spatial setup of the arena with an optional (gray) wall in the middle. (b) Cumulative paths chosen by loaded robots when no wall blocked the passage between these two areas of rough terrain. (c) Cumulative paths chosen by loaded robots when a wall blocked the passage between these two areas of rough terrain. For details see text.

In a third setup, we offered the swarm (with k = 10)

the choice of depositing the dust particles at two dump areas (Fig. 6a). The direct path to one of these dumps was impeded by an area of rough terrain and the path to the other dump was significantly longer than the path to the first one. With a high degree of roughness (speed-reduction = 70%), the robots predominantly selected the second (more distant) dump as deposition target area (see Fig. 6b). With decreased roughness (speed-reduction = 35%), the robots predominantly selected the first (closer) dump as deposition target area (see Fig. 6c). This example demonstrates that our elaborated algorithm allows the robot swarm to make decentralized collective decisions that take environmental circumstances into consideration. Without any global knowledge of the arena conditions, the swarm autonomously converges to an optimal collective decision based on the communication of local perceptions among the swarm members. This swarm behavior can be attributed by the terms “collective perception”, “self-organization”, and “swarm-intelligence”.

Fig. 6: Paths of loaded robots in an environment

with two dumps (upper and lower right side of arena) that can be chosen for deposition by the loaded robots.

(a) Spatial setup of the arena. (b) With a high degree of roughness in the terrain, the robots establish a trail to the more distant dump. (c) With a low degree of roughness in that terrain, the robots collectively approach the closer dump area. For details see text.

In recent years several algorithms have been

developed to control swarms of autonomous robots without a central unit of control [2, 8, 13, 14, 23, 24]. We (as biologists) have been inspired by several approaches that used bio-inspired algorithms to mimic the behavior of social insects [1, 15, 16, 17] or other life-forms [2, 5, 12, 18]. We (and colleagues) deeply investigated the internal organization of honeybee colonies [3, 4, 7, 6, 20], with special focus on the trophallactic behavior of honeybees [7, 20]. Our trophallaxis-derived control algorithm and its elaboration that is presented in this article, show again that natural selection has produced a rich source of efficient, flexible and robust behavioral algorithms that can be transformed into behavioral programs for technical appliances like robot swarm navigation strategies.

In our future work we will investigate how we can

adapt our algorithm to result in self organized jam avoidance or in self-optimizing movement processes. For example, we want to establish self organizing one-way-systems to minimize the time spent by a single robot for collision avoidance. Furthermore we want to use the algorithm for optimization of working processes in heterogeneous swarms, for example by enabling a faster change of needed robot-subcastes at the working site. We also want to investigate the advantages of nonlinear relations between terrain-quality and consumption rate.

In future studies, we plan to make the parameter k

an individual variable of each robot, which is changed in response to dynamics of the individual’s encounters in the environment. This way we expect the swarm to be able to explore the geometric properties of rough terrains and to respond with an even better suited collective behavior to the properties of these terrains.

ACKNOWLEDGEMENT The writing of this article was supported by the

“Fonds zur Förderung der Wissenschaftlichen Forschung (FWF)”, project no. P15961-B06, and by the EU IST-FETopen project (IP) ‘I-Swarm’, no.

507006.

REFERENCES [1] C., Anderson, F.L.W., Ratnieks, “Task partitioning in insect societies. I. Effect of colony size on queueing delay and colony ergonomic efficiency”, Am. Naturalist 154, 1999, pp. 521 –535. [2] E., Bonabeau, M., Dorigo, G., Theraulaz, “Swarm intelligence: From natural to artificial systems” Oxford University Press, New York, NY, USA, 1999. [3] S., Camazine, “The regulation of pollen foraging by honey bees: How foragers assess the colony’s need for pollen”, Behavioral Ecology and Sociobiology 32, 1993, pp. 265 – 273. [4] S., Camazine, K., Crailsheim, N., Hrassnigg, G.E., Robinson, B., Leonhard, H. Kropiunigg, “Protein trophallaxis and the regulation of pollen foraging by honey bees (Apis mellifera L.)”, Apidologie 29, 1998, pp. 113 – 126. [5] S., Camazine, J.L., Deneubourg, N., Franks, J., Sneyd, G., Theraulaz, E., Bonabeau, “Selforganization in biological systems”, Princeton University Press, NJ, USA, 2001. [6] K., Crailsheim “Trophallactic interactions in the adult honeybee (Apis mellifera L.)”, Apidologie 29, 1998, pp. 97–112. [7] K., Crailsheim, “The flow of jelly within a honeybee colony”, Journal of Comparative Physiology B 162, 1992, pp. 681 – 689. [8] J., Kennedy, R.C., „Eberhart, Swarm Intelligence“, Morgan Kaufmann Publishers, Academic Press, USA, 2001. [9] S., Kornienko, O. Kornienko, C., Constantinescu, M., Pradier, P., Levi, “Cognitive microagents: individual and collective perception in a microrobotic swarm”, In: Proceedings of the IJCAI-05 Workshop on Agents in Real-Time and Dynamic Environment, Edinburgh, Scotland, 2005, pp. 33 – 42. [10] S., Kornienko, O., Kornienko, P., Levi, “Swarm embodiment - a new way for deriving emergent behavior in artificial swarms”, Autonome Mobile Systeme, 2005, pp. 25-32. ISBN 978-3-540-30291-9. [11] S., Kornienko, O., Kornienko, P., Levi, “Minimalistic approach towards communication and perception in microrobotic swarms” In: Proceedings of IEEE/RSJ International Conference on Intelligent

Robots and Systems, Edmonton, Alberta, Canada 2005, pp. 4005 – 4011. [12] Y., Liu, K.M., Passino,”Biomimicry of Social Foraging Behavior for Distributed Optimization: Models, Principles, and Emergent Behaviors”, Journal of Optimization Theory and Applications, V 115, N 3, 2002, pp. 603 – 628. [13] L. Parker, K. Fregene, Y. Guo, R. Madhavan, “Distributed Heterogeneous Sensing for Outdoor Multi-Robot Localization, Mapping, and Path Planning”, in A. C. Schultz and L. E. Parker (eds.), Multi-Robot Systems: From Swarms to Intelligent Automata, Kluwer, 2002, pp. 21-30. [14] J.D., McLurkin, “Stupid robot tricks: a behavior-based distributed algorithm library for programming swarms of robots”. Master thesis at the MIT, 2004. [15] D., Payton, M., Daily, R., Estowski, M., Howard, C.,Lee, “Pheromone Robotics”. Autonomous Robots 11,2001, pp. 319 – 324. [16] D., Payton, R., Estkowski, M., Howard, “Compound behaviors in pheromone robotics”, Robotics and Autonomous Systems 44, 2003, pp. 229 – 240. [17]F.L.W., Ratnieks, C., Anderson, “Task partitioning in insect societies. II. Use of queueing delay information in recruitment”, Am. Naturalist 154, 1999, pp. 536 – 548. [18] T., Schmickl, K., Crailsheim, “A Navigation Algorithm for Swarm Robotics Inspired by Slime Mold Aggregation”. In: E. Şahin, W.M. Spears, and A.F.T. Winfield (Eds.): SAB 2006, LNCS 4433, pp. 1–13, 2007, Springer-Verlag, Berlin, Heidelberg. [19] T., Schmickl, K., Crailsheim, “Trophallaxis among swarm-robots: A biological inspired strategy for swarm robotics”, In: Proceedings of BioRob 2006, Biomedical Robotics and Biomechatronics, Pisa, Italy, 2006, ISBN 1-4244-0040-6. [20] T., Schmickl, K., Crailsheim “Inner nest homeostasis in a changing environment with special emphasis on honeybee brood nursing and pollen supply”, Apidologie 35, 2004, pp. 249-263. [21] T., Schmickl, C., Möslinger, K., Crailsheim, “Collective Perception in a Robot Swarm”, In: E. Şahin, W.M. Spears, and A.F.T. Winfield (Eds.): SAB 2006, LNCS 4433, pp. 144–157, 2007, Springer-Verlag, Berlin, Heidelberg 2007.

[22] J., Seyfried, M., Szymanski, N., Bender, R., Estana M., Thiel, H., Wörn, “The I-SWARM project: intelligent small world autonomous robots for micro-manipulation”. In: Sahin E. & Spears W.M., editors, Swarm Robotics, LNCS 3342, Springer Verlag, 2005, pp. 70-83. [23] K., Sugawara, T., Kazama and T., Watanabe, “Foraging Behavior of Interacting Robots with Virtual Pheromone”, In: Proceedings of 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, September 28 - October 2, Sendai, Japan, 2004. [24] V., Trianni, S., Nolfi, M., Dorigo, Hole Avoidance: “Experiments in Coordinated Motion on Rough Terrain”, In: Groen, F, Amato, N., Bonarini, A., Yoshida, E., Krose, B., (eds.), Intelligent Autonomous Systems 8, 2004, pp. 29-36. [25] P., Valdastri, P., Corradi, A., Menciassi, T., Schmickl, K., Crailsheim, J., Seyfried, P., Dario, “Micromanipulation, communication and swarm intelligence issues in a microrobotic platform”, Robotics and Automation Systems (in press). [26] http://www.swarmrobot.org/