Theoretical Distributed Computing meets

Biology: A Review

Ofer Feinerman1? and Amos Korman2??

1 The Weizmann Institute of Science, Israel

2 CNRS and University Paris Diderot, France.

Abstract. In recent years, several works have demonstrated how thestudy of biology can benefit from an algorithmic perspective. Since bi-ological systems are often distributed in nature, this approach may beparticularly useful in the context of distributed computing. As the studyof algorithms is traditionally motivated by an engineering and techno-logical point of view, the adaptation of ideas from theoretical distributedcomputing to biological systems is highly non-trivial and requires a deli-cate and careful treatment. In this review, we discuss some of the recentresearch within this framework and suggest several challenging futuredirections.

? The Weizmann Institute of Science, Rehovot, Israel. E-mail:feinermanofer@gmail.com. Supported by the Israel Science Foundation (grant1694/10).

?? CNRS and University Paris Diderot, France. E-mail:amos.korman@liafa.univ-paris-diderot.fr. Supported by the ANR projectsDISPLEXITY and PROSE, and by the INRIA project GANG.

1 Introduction

1.1 Background and motivation

Nature serves as inspiration for scientists in all disciplines and computer scienceis, certainly, no exception. The reverse direction, that of applying studies in com-puter science to improve our understanding of biological organisms, is currentlydominated by the field of bioinformatics. A natural question to be asked is thusthe following: how can we apply our knowledge in other aspects of computerscience to enhance the study of biology? This direction of research may becomeparticularly fruitful in the context of distributed computing, since indeed, bio-logical systems are distributed in nature (e.g., cells are composed of proteins,organs of cells, populations of organisms and so on).

It is important to note that distributed computing is traditionally studiedfrom an engineering and technological point of view where the focus is on the de-sign of e�cient algorithms to solve well defined problems. Analyzing algorithmsas used in the biological world requires a di↵erent point of view, since the settingis usually unknown, as are the details of the algorithm and even the problem thatit aims at solving. Hence, the adaptation of ideas from theoretical distributedcomputing to biological systems is highly non-trivial and requires a delicate andcareful treatment. In many cases, such a study would require a collaborationbetween biologists that empirically investigate the phenomena, and computerscience theoreticians who analyze it. The hope is that despite their enormouscomplexity, some aspects of biological systems can still be captured by relativelysimple abstract models which can be analyzed using distributed computing tech-niques. In this case, the potential benefit of this direction of research would behuge, not only in terms of understanding large biological systems but also inenriching the scope of theoretical distributed computing.

The synergy between distributed computing and experimental biology is be-ing tightened by methodological advances on both sides. On the distributedcomputing side, the last twenty years have been very fruitful in terms of ad-vancing our fundamental understanding on topics such as dynamic networks,mobile agents, population protocols, and network computing in general. Theseadvances may indicate that the field of distributed computing has reached thematurity level of being useful also for the context of understanding large bi-ological systems. Since distributed computing addresses the relations betweenthe single entity and the group from the theoretical perspective, experimentally,there is a need to simultaneously probe these two scales. Indeed, from the bi-ological side, the main experimental challenge lies in being able to follow largenumbers of identified individuals within behaving populations. The sheer sizeof these ensembles has, for many years, made this a formidable task. Howeverseveral technological advances, and above all the huge increase in the availabilityof computing power, have brought this goal to within our reach.

1.2 Recent technological advances in experimental biology

Examples for the observation of large population are numerous and span variousmethodologies and biological systems. Fluorescence tagging methods provide de-tailed information of the internal state of cells in terms of both protein levelsand fast phosphorylation dynamics. Coupled with microscopy or FACS measure-ments systems, fluorescence can be used to simultaneously and quantitativelymeasure ten or more such di↵erent internal variables over populations of mil-lions of cells [27]. A second example comes from the field of neuroscience wherethere has been a growing emphasis on recording from large neuron populationsof behaving animals. The relevant techniques include light-weight tetrodes andmulti-electrode arrays [53] as well as the use of activity sensitive fluorescent pro-teins [10]. Cooperating biological individuals often engage in collective motions.The availability of high-resolution cameras and strong computers together withimage processing analysis make these movements tractable. Here, too, examplesare many ranging from immune cells moving within the body [29], cells withingrowing plants [45], birds in flocks [7], fish in schools, and ants within theircolonies. Recent tagging technology has been applied to allow for long-term in-dividual tracking of all members within a group (see Figure 1 and [42]). Anotherinteresting example lies in the ability to measure and quantify population hetero-geneity and specifically map the spread of behavioral thresholds of individualsto di↵erent stimuli [42, 55].

Fig. 1. A. a photo of a tagged carpenter ant worker. B. walking trajectories obtainedfrom automatically tracking tagged ants.

Tracking all individuals within a group leads to the accumulation of hugedata sets. On the one hand, this detailed provides us with the rich and detailedinformation that is required to test theoretical hypothesis. On the other hand,enormous datasets are di�cult to manage towards the extraction of relevantinformation. This challenge has sparked the development of high-throughputautomated ethonomics [8].

Experimental manipulations o↵er a great tool towards deciphering the datawe collect and deciding between alternative hypotheses. Traditionally, such ma-nipulations were administered at the level of the group. More recently, severalmethodologies for manipulating specific individuals within the group have beendescribed. The most striking example is probability the ability to excite neuronsby directed illumination which has opened up the field of opto-genetics [10] .Automated manipulation of the trajectories of specific individuals within groupshave been demonstrated both in the context of fish and ant colonies [43].

1.3 Bringing the two disciplines closer

Many fields have contributed to biology but we believe distributed computingcan bring a new and fresh perspective. To elaborate more on that, we first discussan interesting di↵erence in the way the notion of a model is typically perceived bynatural scientists and by computer scientists. In a sense, when it comes to settingsthat involve multiple individual entities (e.g., network processors, mobile agents,sensors, robots, etc.), computer scientists distinguish the model (sometimes calledalso setting), which includes a description of the environment and the restrictedcapabilities of the individuals, from the algorithm, which includes the course ofactions which individuals follow. In a way, this corresponds to the distinctionbetween hardware and software. For example, the topological space in whichentities operate as well as their memory capacity are considered to be part ofthe model, and the particular way in which the entities utilize their memory ispart of their algorithm. In contrast, researchers in the natural sciences typicallydo not make this distinction and treat all these components together as part ofthe model. We believe that the point of view which makes a distinction betweenthe model and the algorithm can benefit the study of natural phenomena forseveral reasons.

Although life is highly plastic and evolvable one can still make distinctionsbetween factors that constrain a living system and courses of actions that maybe employed within such constrains. A simple example involves physical con-straints such as quantum shot noise that defines a lower bound for any lightperception be it biological [6] or artificial. Other examples are more subtle andinclude the distinction between the di↵erent time scales inherent to an evolution-ary process. One can clearly classify slow evolving features as constraints withinwhich fast evolving can be tuned [30]. Similarly, evolution is not a reversibleprocess so that historical evolutionary decisions may constrain organisms forprolonged periods of times, this may even lead to evolutionary traps [44] whichmay be perilous for species survival. These internal “hardware” constraints to-gether with environmental constraints are analogous to the computer scientific“model”. Faster evolving parameters as well as actual cognitive, behavioral deci-sions are analogous to an “algorithm”. We therefore believe that the distinctionbetween these two terms promises a novel and relevant perspectives which canbenefit the biological sciences. Theoretical computer science, and theoretical dis-tributed computing in particular, can contribute in this direction.

Recently, several works have utilized methods from distributed computing toimprove our understanding of biological systems . It is our belief, however, thatall current achievements are very preliminary, and that this direction of researchis still making its first steps. Indeed, one important issue to note in this respect,is that, currently, all corresponding works su↵er from an (arguably inherent) gapbetween the analyzed setting and the “real setting”, that is, the one apparent innature. In each particular case, bridging this gap (or even just slightly reducingit), is a challenging task that must be a combined e↵ort of both field biologistsand theoreticians.

In this review, we describe four paradigms that aim to incorporate distributedcomputing with biological phenomena. In the following sections, we shall discussthe paradigms in detail, list current results, explain their potential impact onbiology, and suggest future research directions. The four paradigms are dividedaccording to the extent in which the model or the algorithm are assumed tobe “known” (or “given”). That is, even though the biological setting is never,actually, known; assumptions regarding the model or algorithm can actuallyfacilitate di↵erent lines of investigation.

1.4 The four paradigms

Before dwelling into the details, let us first list the paradigms and discuss thembriefly. The paradigms are segregated using the distinction between model andalgorithm as described above. At this point we would like to stress that ourclassification of papers into the four paradigms below is, to some extent, (asmany other classifications) a matter of opinion, or interpretation.

1. “Unknown” algorithm and “unknown” model. Here, both the modeland the algorithm are, mostly, absent, and two approaches have been con-sidered to handle this complex situation.(a) Surmising. This classical approach aims at understanding some par-

ticular phenomena that has been empirically observed. The approachinvolves coming up with (somewhat informed) guesses regarding boththe model and the algorithm, together with their analysis. The assumedmodel and algorithm typically do not claim to represent reality accu-rately, but rather to reflect some aspects of the real setting. The dis-tributed computing experience would come handy for allowing the guess-ing of more involved and e�cient algorithms and for enhancing their rig-orous analysis. This type of study was conducted recently by Afek et al.[2] concerning the fly’s brain, where the phenomena observed was that avariant of the Minimum Independent Set (MIS) problem is solved duringthe nervous system development of the fly.

(b) Finding dependencies between parameters. This new paradigmaims at obtaining knowledge regarding the model by connecting it tothe output of the algorithm, which is typically more accessible exper-imentally. The paradigm is composed of three stages. The first stage

consists of finding an abstract setting that can be realizable in an exper-iment, parameterized by an unknown parameter a (e.g., the parametercan be the number of possible states that entities can possibly possess).The second stage involves analyzing the model and obtaining theoreticaltradeo↵s between the parameter a and the performance e�ciency of thealgorithm; the tradeo↵s are obtained using techniques that are typicallyassociated with distributed computing. Finally, the third stage consistsof conducting suitable experiments and measuring the actual e�ciencyof the biological system. The idea is, that using the tradeo↵s and the ex-perimental results on the e�ciency of the algorithm, one would be ableto deduce information (e.g., bounds) regarding the parameter a. A firststep to demonstrate this paradigm was recently made by the authors ofthis review [20], in the context of foraging strategies of ants.

2. “Known” algorithm and “unknown” model. This paradigm corre-sponds to the situation in which the algorithm is already fairly understood,and the challenge is to find a simple and abstract setting that, on the onehand, somehow captures some essence of the “real” setting, and, on the otherhand, complies well with the given algorithm to explain some empirically ob-served phenomena. We are not aware of any work in the framework of biologywhere this paradigm was employed. However, the work of Kleinberg [31] onsmall world phenomena can serve as an example for this paradigm withinthe context of sociology.

3. “Unknown” algorithm and “known” model. In the context of largebiological ensembles, this paradigm fixes an abstract model consisting ofmultiple processors (either mobile, passively mobile, or stationary) operatingin a given setting. Two approaches are considered.

(a) Complexity analysis. Here, the complexity of the abstract model isanalyzed, aiming at bounding the power of computation of the proces-sors as a group. In all cases studied so far, the model seems to representa very high level abstraction of reality, and the resulting computationalpower is typically very strong. Furthermore, known complexity resultsare obtained on models that seem too far from reality to be realizedin experiments. Hence, this line of research currently not only does notinvolve experimental work, but also does not seem to be related to suchexperiments in the near future. As the focus of this review is on con-nections between theoretical work in distributed computing and exper-imental work in biology, we decided to only briefly discuss this line ofresearch in this review.

(b) Guessing an algorithm. Here, again, the aim is to provide an explana-tion to some observed phenomena. The model is given to the researchers,and the goal is to come up with a simple algorithm whose analysis com-plies with the phenomena. For example, Bruckstein [9] aims at explainingthe phenomena in which ant trails seem to be relatively straight. Thepaper relies on a fairly reasonable model, in which ants can see theirnearest neighbors, all ants walk in one direction and in the same speed.

Then the paper contributes by providing an algorithm whose outcomeis straight lines.

4. “Known” algorithm and “known” model. This paradigm assumes amodel and algorithm that have been substantiated to some extent by empir-ical findings. The e�ciency of the algorithm is then theoretically analyzed,as is, with the goal of obtaining further insight into system function. Thecorresponding study is currently very limited: to the best of our knowledge,its merely contains works analyzing bird flocking algorithms.

1.5 Other related work

Computer science and biology have enjoyed a long and productive relationshipfor several decades. One aspect of this relationship concerns the field of bioin-formatics, which utilizes computer science paradigms to retrieve and analyzebiological data, such as nucleic acid and protein sequences. The field of naturalcomputing is another aspect of this relationship; this field brings together natureand computing to encompass three classes of methods: (1) simulating naturalpatterns and behaviors in computers, (2) potentially designing novel types ofcomputers, and (3) developing new problem-solving techniques. (For detailedreviews, see e.g., [11, 33, 40].)

The motivation for the latter class of methods is to provide alternative so-lutions to problems that could not be (satisfactorily) resolved by other, moretraditional, techniques. This direction of research is termed bio-inspired com-puting or biologically motivated computing [12, ?], or computing with biologicalmetaphors [50]. Swarm intelligence, for instance, refers to the design of algo-rithms or distributed problem-solving devices inspired by the collective behaviorof social insects and other animal societies. This sort of work is inspired by bi-ology but not bound by biology, that is, it doesn’t have to remain true in thebiology context. Examples of this approach can be found, e.g., in the book byDorigo and Stutzle [16] which describes computer algorithms inspired by antbehavior, and particularly, the ability to find what computer scientists wouldcall shortest paths. Another example of this approach comes in the context ofapplications to robotics. Swarm robotics, for example, refers to the coordinationof multi-robot systems consisting of large numbers of mostly simple physicalrobots [17].

The reverse direction, that of applying ideas from theoretical computer sci-ence to improve our understanding of biological phenomena, has received muchless attention in the literature, but has started to emerge in recent years, fromdi↵erent perspectives. An example for such an attempt is the work of Valiant[51] that introduced a computational model of evolution and suggested that Dar-winian evolution be studied in the framework of computational learning theory.Several other works (which are discussed in the following sections) took this di-rection by applying algorithmic ideas from the field of distributed computing tothe context of large and complex biological ensembles.

At this point, we would like to note that in several works both directionsof research co-exist. However, since the topic of this review focuses on implying

ideas from distributed computing to biology, then, when discussing particularworks, we shall focus our attention on this direction of research and typicallyignore the bio-inspired one.

2 “Unknown” model and “unknown” algorithm

This paradigm corresponds to the situation in which both model and algorithmare, to some extent, absent. Roughly speaking, two approaches have been con-sidered to handle this complex situation. The first approach is classical in thecontext of science. It concerns the understanding of a particular phenomena thathas been observed experimentally, and consists in guessing both a model and analgorithm to fit the given phenomena. The second approach aims at improvingour understanding of the connections between model and/or algorithmic param-eters, and to reduce the parameter space by finding dependencies between pa-rameters. Furthermore, when coupled with suitable experiments, this approachcan be used to obtain bounds on parameters; which may be very di�cult toobtain otherwise. Let us first describe the more classical approach.

2.1 Surmising

This approach concerns a particular phenomena that has been observed empir-ically. After some preprocessing stage, consisting of high level observations andsome data-analysis, the main goal is to come up with (informed) guesses for botha model and an algorithm to fit the given phenomena. In other words, the modeland algorithm are tailored to the particular phenomena. Note that this guessingapproach is hardly new in biology, and in fact, it is one of the more commonones. However, it is far less common to obtain such guesses using the types ofreasoning that are typically associated with distributed computing. Indeed, thedistributed computing experience would come handy here for two main reasons:(1) for allowing the guessing of more involved e�cient algorithms and (2) forenhancing their rigorous analysis. This was the case in the recent study by Afeket al. [2] concerning the fly’s brain, where the phenomena observed was that avariant of the Minimum Independent Set (MIS) problem is solved during thedevelopment of the nervous system of the fly.

MIS on the fly: Informally, the classical Maximal Independent Set (MIS) prob-lem aims at electing a set of leaders in a graph such that all other nodes in thegraph are connected to a member of the MIS and no two MIS members are con-nected to each other. This problem has been studied extensively for more thantwenty years in the distributed computing community. Very recently, Afek et al.[2] observed that a variant of the distributed MIS problem is solved during thedevelopment of the fly’s nervous system. More specifically, during this process,some cells in the pre-neural clusters become Sensory Organ Precursor (SOP)cells. The outcome of this process guarantees that each cell is either an SOP ordirectly connected to an SOP and no two SOPs are connected. This is similar

to the requirements of MIS. However, the solution used by the fly appears tobe quite di↵erent from previous algorithms suggested for this task. This maybe due to the limited computational power of cells as compared to what is as-sumed for processors in traditional computer science solutions. In particular, inthe ”fly’s solution”, the cells could not rely on long and complex messages or oninformation regarding the number of neighbors they have.

Afek at al. suggested an abstract (relatively restrictive) model of computationthat captures some essence of the setting in which flies solve the SOP selectionproblem. For this abstract model, the authors were able to develop a new MISalgorithm that does not use any knowledge about the number of neighbors anode has. Instead, with probability that increases exponentially over time, eachnode that has not already been connected to an MIS node proposes itself as anMIS node. While the original algorithm of Afek et al [2] requires that nodes knowan upper bound on the total number of nodes in the network, a new version ofthis algorithm [1] removes this requirement.

The algorithms in [1, 2] are motivated by applications to computer networks,and hence primeraly follow the bio-inspired approach. Nevertheless, it is interest-ing to note that some aspects of these algorithms are consistent with empiricalobservations. Indeed, in [2] the authors used microscopy experiments to followSOP selection in developing flies, and discovered that a stochastic feedback pro-cess, in which selection probability increases as a function of time, provides agood match to the experimental results; such a stochastic feedback process isalso evident in the corresponding algorithms. Hence, these works also follow thedirection of research which is the topic of this review.

2.2 Finding dependencies between parameters

In contrast to the previous approach, this approach does not focus on under-standing a particular phenomena, but instead aims at understanding the un-derlying connections between the model and/or algorithm ingredients. Indeed,a common problem, when studying a biological system is the complexity of thesystem and the huge number of parameters involved. Finding ways of reduc-ing the parameter space is thus of great importance. One approach is to dividethe parameter space into critical and non-critical directions where changes innon-critical parameters do not a↵ect overall system behavior [22, 26]. Anotherapproach, which is typically utilized in physics, would be to define theoreticalbounds on system performance and use them to find dependencies between dif-ferent parameters. This approach may be particularly interesting in the casewhere tradeo↵s are found between parameters that are relatively easy to mea-sure experimentally and others that are not. Indeed, in this case, using suchtradeo↵s, relatively easy measurements of the “simple parameters” would allowus to obtain non-trivial bounds on the “di�cult parameters”. Note that suchtheoretical tradeo↵s are expected to depend highly on the setting, which is byitself di�cult to understand.

Very recently, a first step in the direction of applying this approach hasbeen established by the authors of this review, based on ideas from theoretical

distributed computing [20]. That work considers the context of central placeforaging, such as performed by ants around their next. The particular theoreticalsetting, involving mobile probabilistic agents (e.g., ants) that search for fooditems about a source node (e.g., the nest) was introduced in [21] and has twoimportant advantages. On the one hand, this setting (or, perhaps, a similarone) is natural enough to be experimentally captured, and on the other hand,it is su�ciently simple to be analyzed theoretically. Indeed, in [20] we establishtradeo↵s between the time to perform the task and the amount of memory (or,alternatively, the number of internal states) used by agents to perform the task.Whereas the time to perform the task is relatively easy to measure, the numberof internal states of ants (assuming they act similarly to robots) is very di�cultto empirically evaluate directly.

As mentioned in [20], the natural candidates to test this paradigm on wouldbe desert ants of the genus Cataglyphys and the honeybees Apis mellifera. Thesespecies seem to possess many of the individual skills required for the behavioralpatterns that are utilized in the corresponding upper bounds in [20, ?], and henceare expected to be time e�cient.

It is important to note that it is not claimed that the setting proposed in[20] precisely captures the framework in which these species perform search (al-though it does constitute a good first approximation to it). Indeed, it is not un-reasonable to assume that a careful inspection of these species in nature wouldreveal a somewhat di↵erent framework and would require the formulation ofsimilar suitable theoretical memory bounds. Finding the “correct” frameworkand corresponding tradeo↵s is left for future work. Once these are established,combining the memory lower bounds with experimental measurements of searchspeed with varying numbers of searchers would then provide quantitative evi-dence regarding the number of memory bits (or, alternatively, the number ofstates) used by ants. Furthermore, these memory bits must mainly be used byants to assess their own group size prior to the search. Hence, such a result wouldprovide insight regarding the ants’ quorum sensing process inside the nest.

3 “Known” algorithm and “unknown” model

This paradigm corresponds to the situation in which the algorithm is alreadyfairly understood, and the challenge is to find a simple and abstract model that,on the one hand, somehow captures some essence of the “real” setting, and, onthe other hand, complies well with the given algorithm to explain some empiri-cally observed phenomena. Although this paradigm could be applied within theframework of biology we are not aware of any such work. Nevertheless, to clarifythe usefulness of this paradigm, we describe works on small world phenomenaand their applicability to sociology.

Small world phenomena: It is long known that most people in social networks arelinked by short chains of acquaintances. The famous “six degrees of separation”experiment by Milgram [38] implied not only that there are short chains between

individuals, but also that people are good at finding those chains. Indeed, indi-viduals operating with only local information are expected to find these chainsby using the simple greedy algorithm. Kleinberg [31] investigated this small worldphenomena from a distributed computing point of view, and came up with anextremely simple abstract model that, on the one hand, somehow captures someessence of the “real” setting, and, on the other hand, complies well with a greedyalgorithm to explain the small world phenomena.

The model consists of a two-dimensional grid topology augmented with long-range connections, where the probability Pr(x, y) of a connecting node x with anode y is some (inverse proportional) function of their lattice distance d(x, y),that is, Pr(x, y) ⇡ 1/d(x, y)↵, for some parameter ↵. This abstract model doesseem to represent some essence of social networks, where it is reasonable toassume that each person has several immediate acquaintances and fewer “longrange acquaintance”, and that it is less likely to have a long range acquaintanceif this acquaintance is “far away” (in some sense). Kleinberg [31] then studieddistributed greedy algorithms that resemble the one used by Milgram: for trans-mitting a message, at each step, the holder of the message must pass it acrossone of its (either short or long range) connections, leading to one who minimizesthe distance to the destination. Crucially, this current holder does not know thelong range connections of other nodes. The algorithm is evaluated by its ex-pected delivery time, which represents the expected number of steps needed toforward a message between a random source and target in a network generatedaccording to the model. It turns out that when the long-range connections followan inverse-square distribution, i.e., the case ↵ = 2, the expected time to delivera message is small: polylogarithmic in the number of nodes. The setting was fur-ther analyzed proving that the exponent ↵ = 2 for the long range distributionis the only exponent at which any distributed algorithm on the grid can achievea polylogarithmic time delivery.

Following [31], several other works investigated extensions of this model, inwhat has become a whole area of research (see [23] for a survey). In particular,Fraigniaud and Giakkoupis proved [24] that all networks are smallworldizable,in the sense that, for any network G, there is a natural way to augment G withlong-range links, so that (a minor variant of) greedy routing performs in 2

plog n

steps. Before, [25] proved that this bound is essentially the best that you canexpect in arbitrary networks. In addition, Chaintreau et al. [14] studied how theKleinberg’s harmonic distribution of the long-range links could emerge naturallyfrom a decentralized process. It appears that if individuals move at random andtend to forget their contact along with time, then we end up with connectionsbetween individuals that are distributed harmonically, as in Kleinberg’s paper.

4 “Unknown” algorithm and “known” model

In the context of large biological ensembles, this paradigm fixes an abstractmodel consisting of multiple processors (either mobile, passively mobile, or sta-tionary). Two approaches are considered. The first approach analyzes the com-

putational power of such models, and the second suggests simple algorithms thatcan potentially operate within model constraints and explain known phenomena.

4.1 Computational aspects

The fundamental question of what can be computed by biological systems isfascinating. One of the aspects of this question concerns, for example, the com-putational power of ants. It is quite evident that the computational abilities ofthe human brain are much more impressive than those of a typical ant colony,at least in some respects. A basic philosophical question is whether this com-putational gap is a consequence of the di↵erent physical settings (e.g., in thecase of ants, this includes the physical organization of ants and their individuallimitations), or because it was simply not developed by evolution as it wasn’tnecessary for survival. To put it more simply: is the reason that an ant colony isnot as smart as a human brain because it cannot be or because it doesn’t needto be?

While we are very far away from answering such a question, some very initialsteps have been taken in this direction. Various abstract models which looselyrepresent certain settings in nature are suggested and their computational poweris analyzed. Broadly speaking, in all such previous works, the analyzed modelappears to be very strong, and is often compared to a Turing machine. It isimportant to note, however, that, as a generalization, the motivations for someof the suggested models in the literature come only partially from biology, andare a↵ected equally by sensor networks and robotics applications. Indeed, in allcases studied so far, the model seems to represent a very high level abstraction ofreality, and in particular, seems too far from reality to be realized in experiments.Hence, this line of research currently not only does not involve experimentalwork, but also does not seem to be related to such experiments in the nearfuture. As the focus of this review is on connections between theoretical workin distributed computing and experimental work in biology, we decided to onlybriefly discuss corresponding computational results.

Population protocols: The abstract model of population protocols, introduced byAngluin et al. [3], was originally intending to capture abstract features of com-putations made by tiny processes such as sensors, but it was observed also thatit may be useful for modeling the propagation of diseases and rumors in humanpopulations as well as stochastically interacting molecules. The question of whatcan be computed by population protocols has been studied quite thoroughly.Specifically, perhaps the main result in the setting of population protocols isthat the set of computable predicates under a “fair” adversary is either exactlyequal to or closely related to the set of semilinear predicates [4]. The model wasshown to be much more powerful under a (uniform) random scheduler, as it cansimulate a register machine. For a good survey of population protocols, referto [5].

A related model was studied by Lachmann and Sella [34], which is inspiredby task switching of ants in a colony. The model consists of a system composed

of identical agents, where each agent has a finite number of internal states. Theagent’s internal state may change either by interaction with the environment orby interaction with another agent. Analyzing the model’s dynamics, the authorsprove it to be computationally complete. A result in a similar flavor was obtainedby Sole and Delgado [49].

Ant robotics: Ant robotics is a special case of swarm robotics, in which the robotscan communicate via markings [54]. This model is inspired by some species ofants that lay and follow pheromone trails. Recently, Shiloni at el. showed thata single ant robot (modeled as finite state machine) can simulate the executionof any arbitrary Turing machine [46]. This proved that a single ant robot, usingpheromones, can execute arbitrarily complex single-robot algorithms. However,the result does not hold for N robots.

Hopfield model: The celebrated Hopfield model [28] uses intuition from the fieldof statistical mechanics and neuroscience to provide intuition for associativememory in the brain. The model includes a learning stage at which a largenumber of distributed memories are imprinted onto a neuronal network. Thisis achieved by a very simple learning algorithm that fine tunes the connections(synapses) between the neurons. Once the memories are set, they can be retrievedby employing simple dynamics which is inspired by actual neuronal dynamics.Namely, neurons are modeled to be in one of two states as inspired by the all-or-none nature of action potentials; Second, neuronal dynamics are governed bylocal threshold computations as inspired by the actual physiological membranethresholds. Formed memories are associative in the sense that partial memoriesare enough to reconstruct the full ones. The model was shown to be highly robustto failures such as connection, or neuronal deletions. Further, it has been proven,that this model for neuronal networks is computationally complete [47].

The Hopfield model is not only biologically inspired but also inspires biolo-gists. It would be fair to say, that much of our intuition for distributed informa-tion storage and associative memories in the brain derives from this model. Onthe other hand, the Hopfield model is very far from being biologically accurateand as such it is not as useful as one might expect in the modeling of actualmicro-circuits in the brain.

Distributed computing on fixed networks with simple processors: Recently, Emeket al. [18] introduced a new relaxation of the Beeping model from [1, 2], wherethe computational power of each processor is extremely weak and is based onthresholds. Despite the weak restrictions of the model, the authors show thatsome of the classical distributed computing problems (e.g., MIS, coloring, max-imal matching) can still be solved somewhat e�ciently. This shows that thepower computation of such a model is high, at least by judging it from the pointof view of tasks typically associated with computer networks.

4.2 Guessing an algorithm

Here, again, the goal is to provide an explanation to some observed phenomena.The model is given to the researchers, and the goal is to come up with a simplealgorithm whose analysis complies with the phenomena. For example, Bruckstein[9] aims at explaining the phenomena in which ant trails seem to be relativelystraight. The paper relies on a fairly reasonable model, in which ants walkingfrom their nest towards a food source initially follow a random, convoluted pathlaid by the first ant to find the food. The model further includes the reasonableassumption that an ant on the trail can see other ants in its proximity. The papergoes on to suggest a simple algorithm in which each ant continuously orients herdirection toward the ant walking directly in front of her. It is shown that thisalgorithm results in a “corner-cutting” process by which the ant trail quicklyconverges to the shortest distance line connecting the nest and the food source.

5 “Known” algorithm and “known” model

This paradigm concerns the analysis of specific algorithms operating in partic-ular given models. The corresponding study currently includes works on birdflocking algorithms. In these cases, the specific algorithms and models studiedare supported by some empirical evidence, although this evidence is, unfortu-nately, quite limited. The main objective of this paradigm is to have a better,more analytical, understanding of the algorithms used in nature. Nevertheless,potentially, this paradigm can also be used to give some feedback on the valid-ity of the proposed algorithm. Indeed, on the one hand, some positive evidenceis obtained if the (typically non-trivial) analysis finds the algorithm feasibleand/or explaining a certain phenomena. On the other hand, proving its unfeasi-bility, e.g., that it requires unreasonable time to be e↵ective, serves as a negativefeedback, which may lead to disqualifying its candidature.

Bird flocking algorithms: The global behavior formed when a group of birds areforaging or in flight is called flocking. This behavior bares similarities with theswarming behavior of insects, the shoaling behavior of fish, and herd behaviorof land animals. It is commonly assumed that flocking arises from simple rulesthat are followed by individuals and does not involve any central coordination.Such simple rules are, for example: (a) alignment - steer towards average head-ing of neighbors, (b) separation - avoid crowding neighbors, and (c) cohesion -steer towards average position of neighbors (long range attraction). With thesethree simple rules, as initially proposed by Reynolds [41], computer simulationsshow that the flock moves in a “realistic” way, creating complex motion and in-teraction. The basic flocking model has been extended in several di↵erent wayssince [41]. Measurements of bird flocking have been established [19] using high-speed cameras, and a computer analysis has been made to test the simple rulesof flocking mentioned above. Evidence was found that the rules generally holdtrue and that the long range attraction rule (cohesion) applies to the nearest

5-10 neighbors of the flocking bird and is independent of the distance of theseneighbors from the bird.

Bird flocking has received considerable attention in the scientific and engi-neering literature, and was typically viewed through the lens of control theoryand physics. Computer simulations support the intuitive belief that, by repeatedaveraging, each bird eventually converges to a fixed speed and heading. This hasbeen proven theoretically, but how long it takes for the system to converge hasremained an open question until the work of Chazelle [13]. Using tools typicallyassociated with theoretical computer science, Chazelle analyzed two classicalmodels that are highly representative of the many variants considered in theliterature, namely: (a) the kinematic model, which is a variant of the classicalVicsek model [52], and (b) the dynamic model [15]. Chazelle proved an upperbound on the time to reach steady state, which is extremely high: a tower-of-twosof height linear in the number of birds. Furthermore, it turns out that this up-per bound is in fact tight. That is, Chazelle proved that with a particular initialsettings, the expected time to reach steady state is a tower-of-twos of height.

That lower bound is, of course, huge, and no (reasonably large) group ofreal birds can a↵ord itself so much time. At first glance, it may seem as if thisresult already implies that real birds do not perform the considered algorithms.However, for such an argument to be more convincing, several assumptions needto modified to suit better the setting of real birds. The first issue concerns thenotion of convergence. As defined in [13], reaching a steady state, means that thesystem no longer changes. Of course, birds are not expected to actually achievethis goal. It would be interesting in come up and investigate some relaxationto this notion of convergence, that would correspond better to reality. Second,it would be interesting to prove lower bounds assuming the “average initialsetting”, rather than the worst case one.

6 More future directions

Generally speaking, the direction of applying ideas from theoretical distributedcomputing to biology contexts is currently making its first baby steps. Hence,this direction is open to a large variety of research attempts. We have describedseveral research paradigms to proceed in. Particular suggestions for future workusing these paradigms can be found by inspecting some of the examples men-tioned in the survey by Navlakha and Bar-Joseph [40] on bio-inspired computing.In particular, as mentioned in [40], two important examples of problems com-mon to both biological and computational systems are distributed consensus[36] and synchronization [35, 48]. In biology, consensus has an important role incoordinating populations. Fish, for instance, must be able to quickly react tothe environment and make collective decisions while constantly being under thethreat of predation. Synchronization is apparent in fireflies that simultaneouslyflash [39, 56] and pacemaker cells in the heart [32]. We believe that the vastliterature and advancements concerning synchronization and consensus in the

context of theoretical distributed computing may also be used to enhance theunderstanding of corresponding biological phenomena.

References

1. Y. Afek, N. Alon, Z. Bar-Joseph, A. Cornejo, B. Haeupler, and F. Kuhn. Beeping aMaximal Independent Set. In Proc. 25th International Symposium on DistributedComputing (DISC), 32–50, 2011.

2. Y. Afek, N. Alon, O. Barad, E. Hornstein, N. Barkai, and Z. Bar-Joseph. Abiological solution to a fundamental distributed computing problem. Science,331(6014):183–185, 2011.

3. D. Angluin, J. Aspnes, Z. Diamadi, M. J. Fischer, and R. Peralta. Computationin networks of passively mobile finite-state sensors. Distributed Computing, 18(4),235–253, 2006.

4. D. Angluin, J. Aspnes, D. Eisenstat, E. Ruppert. The computational power ofpopulation protocols. Distributed Computing 20(4), 279-304, 2007.

5. J. Aspnes and E. Ruppert. An introduction to population protocols. Bulletin of theEuropean Association for Theoretical Computer Science, Distributed ComputingColumn, 93:98–117, October 2007. An updated and extended version appears inMiddleware for Network Eccentric and Mobile Applications, Benoit Garbinato,Hugo Miranda, and Luis Rodrigues, eds., Springer-Verlag, pp. 97–120, 2009.

6. W. Bialek. Physical limits to sensation and perception. Annual Review of Bio-physics and Biophysical Chemistry, 16:455–478, 1987.

7. W. Biallek, A. Cavagnab, I. Giardinab, T. Morad, E. Silvestrib, M. Vialeb, and A.M. Walczake. Statistical mechanics for natural flocks of birds. PNAS , 109(13),4786–4791, 2012.

8. K. Branson, A. A. Robie, J. Bender, P. Perona, and M. H. Dickinson. High-throughput ethomics in large groups of Drosophila. Nature Methods 6, 451–457,2009.

9. A.M. Bruckstein. Why the ant trails look so straight and nice. The MathematicalIntelligencer, Vol. 15/2, pp. 58-62, 1993.

10. J.A. Cardin, M. Carlen, K. Meletis, U. Knoblich, F. Zhang, K. Deisseroth, L.H.Tsai, and C.I. Moore. Targeted optogenetic stimulation and recording of neurons invivo using cell-type-specific expression of Channelrhodopsin-2. Nature Protocols,5, 247–254, 2010.

11. L. N. de Castro. Fundamentals of Natural Computing: Basic Concepts, Algorithms,and Applications. CRC Press, 2006.

12. L. N. de Castro and F.J. Von Zuben. Recent developments in biologically inspiredcomputing. Idea Group Publishing; 2004.

13. B. Chazelle. Natural algorithms. Proc. 19th ACM-SIAM Symposium on DiscreteAlgorithms (SODA), 422–431, 2009.

14. A. Chaintreau, P. Fraigniaud, and E. Lebhar. Networks Become Navigable as NodesMove and Forget. Proc. 35th International Colloquium on Automata, Languagesand Programming (ICALP), 133-144, 2008.

15. F. Cucker, S. Smale. Emergent behavior in ocks. IEEE Trans. Automatic Control52, 852–862, 2007.

16. M. Dorigo and T. Stutzle. Ant colony optimization. MIT Press, 2004.

17. M. Dorigo, E. Sahin. Swarm Robotics - Special Issue. Autonomous Robots, vol.17, pp. 111-113, 2004.

18. Y. Emek, J. Smula and R. Wattenhofer. Stone Age Distributed Computing. Arxiv2012.

19. T. Feder. Statistical physics is for the birds. Physics today 60 (10): 2830, 2007.20. O. Feinerman and A. Korman. Memory Lower Bounds for Randomized Collabo-

rative Search and Implications to Biology. In Proc. 26th International Symposiumon Distributed Computing (DISC), 61-75, 2012.

21. O. Feinerman, A. Korman, Z. Lotker and J.S Sereni. Collaborative Search onthe Plane without Communication. In Proc. 31st Annual ACM SIGACT-SIGOPSSymposium on Principles of Distributed Computing (PODC), 77-86, 2012.

22. O. Feinerman, J. Veiga, J.R. Dorfman, R.N. Germain and G. Altan-Bonnet. Vari-ability and robustness in T Cell activation from regulated heterogeneity in proteinlevels. Science 321(5892): 1081–1084, 2008.

23. P. Fraigniaud. Small Worlds as Navigable Augmented Networks. Model, Analysis,and Validation. Proc. 15th Annual European Symposium on Algorithms (ESA),2-11, 2007.

24. P. Fraigniaud and G. Giakkoupis. On the searchability of small-world networkswith arbitrary underlying structure. Proc. 42th ACM Symposium on Theory ofComputing (STOC), 389–398, 2010.

25. P. Fraigniaud, E. Lebhar, and Z. Lotker. A Lower Bound for Network Navigability.SIAM J. Discrete Math. 24(1): 72-81, 2010.

26. R.N. Gutenkunst, J.J. Waterfall, F.P. Casey, K.S. Brown, C.R. Myers and J.P.Sethna. Universally sloppy parameter sensitivities in systems biology models.PLOS Computational Biology 3(10): e189, 2007.

27. L.A. Herzenberg, and S.C. De Rosa. Monoclonal antibodies and the FACS: comple-mentary tools for immunobiology and medicine. Immunology Today 21(8), 383-390,2000.

28. J.J. Hopfield Neural networks and physical systems with emergent collective com-putational abilities. PNAS, 79(8), 2554-2558, 1982.

29. T. Ishii and M. Ishii Intravital two-photon imaging: a versatile tool for dissectingthe immune system. Ann Rheum Dis 70, 113-115, 2011.

30. E. Jablonka and M.J. Lamb. Evolution in four dimensions: Genetic, epigenetic,behavioral, and symbolic variation in the history of life. MIT Press, 2005.

31. J. M. Kleinberg. Navigation in a small world. Nature 406 (6798): 845, 2000.32. T. Kuramoto and H. Yamagishi. Physiological anatomy, burst formation, and burst

frequency of the cardiac ganglion of crustaceans. Physiol Zool 63: 102–116, 1990.33. S. Olarius and A.Y. Zomaya. Handbook of Bioinspired Algorithms and Applica-

tions. Chapman & Hall/CRC, 2005.34. M. Lachmann and G. Sella. The Computationally Complete Ant Colony: Global

Coordination in a System with No Hierarchy. ECAL, 784–800, 1995.35. C. Lenzen, T. Locher, and R. Wattenhofer. Tight bounds for clock synchronization.

J. ACM 57(2), 2010.36. N.A. Lynch. Distributed Algorithms. San Francisco, CA: Morgan Kaufmann Pub-

lishers Inc. 1996.37. D. Mange and M. Tomassini. Bio-inspired computing machines: towards novel

computational architecture. Presses Polytechniques et Universitaires Romandes;1998.

38. S. Milgram. The small-world problem. Psychol. Today 1, 61–67, 1967.39. R.E. Mirollo and S.H. Strogatz. Synchronization of pulse-coupled biological oscil-

lators. SIAM J. Applied Math 50: 1645–1662, 1990.

40. S. Navlakha and Z. Bar-Joseph. Algorithms in nature: the convergence of systemsbiology and computational thinking. Nature-EMBO Molecular Systems Biology,7:546, 2011.

41. C. Reynolds. Flocks, herds and schools: A distributed behavioral model. SIG-GRAPH ’87: Proceedings of the 14th annual conference on Computer graphicsand interactive techniques, 25–34, 1987.

42. E. J. H. Robinson, T. O. Richardson, A. B. Sendova-Franks, O. Feinerman, andN. R. Franks. Radio tagging reveals the roles of corpulence, experience and socialinformation in ant decision making. Behavioral Ecology and Sociobiology, 63(5),627–636, 2009.

43. E.J.H Robinson, O. Feinerman, and N.R. Franks Experience, corpulence and de-cision making in ant foraging. Journal of Experimental Biology, 215, 2653–2659,2012.

44. M.A. Schlaepfer, M.C. Runge, and P.W. Sherman. Ecological and evolutionarytraps. Trends in Ecology and Evolution, 17(10), 474–480, 2002.

45. G. Sena, Z. Frentz, K. D. Birnbaum, and S. Leibler. Quantitation of CellularDynamics in Growing Arabidopsis Roots with Light Sheet Microscopy. PLOS1,2011.

46. A. Shiloni, N. Agmon, and G. A. Kaminka. Of Robot Ants and Elephants: AComputational Comparison. In Theoretical Computer Science, 412, 5771-5788,2011.

47. H.T. Siegelmann and E.D. Sontag. On the Computational Power of Neural Nets.Journal of Computer and System Sciences 50, 132–150, 1995.

48. O. Simeone, U. Spagnolini, Y. Bar-Ness, and S. Strogatz. Distributed synchroniza-tion in wireless networks. IEEE Signal Process Mag 25: 81–97, 2008.

49. R. V. Sole and J. Delgado. Universal Computation in Fluid Neural Networks.Complexity 2(2), 49-56, 1996.

50. R. Paton. Computing with biological metaphors. Chapman & Hall; 1994.51. L. G. Valiant. Evolvability. J. ACM 56(1), 2009.52. T. Vicsek, A. Czirok, E. Ben-Jacob, I. Cohen, and O. Shochet. Novel type of

phase transition in a system of self-driven particles. Physical Review Letters 75,1226–1229, 1995.

53. J. Viventi et al. Flexible, foldable, actively multiplexed, high-density electrodearray for mapping brain activity in vivo. Nature Neuroscience 14, 1599–1605,2011.

54. I. Wagner and A. Bruckstein. Special Issue on Ant Robotics. Annals of Mathe-matics and Artificial Intelligence, 31(1-4), 2001.

55. C. Westhus, C.J. Kleineidam, F. Roces and A. Weidenmuller. Behavioural plastic-ity in the fanning response of bumblebee workers: impact of experience and rateof temperature change. Animal Behavior, in press, 2012.

56. A.T. Winfree. Biological rhythms and the behavior of populations of coupledoscillators. J. Theor. Biololgy. 16: 15–42, 1967.