Proceedings of the 2008 IEEEInternational Workshop on Safety, Security and Rescue RoboticsSendai, Japan, October 2008

Tolerati'ng, Malici'ous Moni*to-rs i*nD1r~etectingMisbehaving Robots

Antonio Bicchi and Adriano Fagiolini Gianluca Dini and Ida Maria SavinoInterdepartmental Research Center "E. Piaggio" Dipartimento dell'InformazioneFaculty of Engineering, Universit'a di Pisa, Italy Faculty of Engineering, Universita of Pisa, Italy

{bicchi, a.fagiolini}@ing.unipi.it {gianluca.dini, ida.savino}@ing.unipi.it

Abstract -This paper considers a multi-agent system and focuses upon whether an agent misbehaves or not. Every node broad-on the detection of motion misbehavior. Previous work by the authors casts its "view" of the system in order to enrich the limitedproposed a solution, where agents act as local monitors of their sensing capabilities ofthe neighbors. As malicious agents areneighbors and use locally sensed information as well as data received sensin tieste nors as maliciou antserfrom other monitors. In this work, we consider possible failure of present vi the system, every node has to vcalidate anothermonitors that may send incorrect information to their neighbors due agent's view. Validation is based on two criteria: it must notto spontaneous or even malicious malfunctioning. In this context, be in contrast with the physical dynamics of the system andwe propose a distributed software architecture that is able to tolerate it must be agreed upon by a majority of agents deemed nonsuch failures. Effectiveness of the proposed solution is shown through malicious by the agents.preliminlary simulation results.

Security and safety are crucial challenges in multi-agentKeywords: distributed detection, malicious monitors, moni- system but they have been dealt with by different research

tor failure, robust consensus communities. The advanced control community has mainlyfocused on safety issues such as detecting agents that misbe-

I. INTRODUCTION have in terms of movement [1]-[3]. In contrast, the computerWe consider a set of mobile autonomous robotic agents that and communication community mainly focuses on security by

communicate over a wireless network and share a common protecting vehicular communications and detecting maliciousenvironment to accomplish their tasks. In order to safely move vehicular traffic information [4]-[6]. As a security infringe-within that environment, agents follow a predefined common ment may translate into a safety infringement, in this paperset of decentralized motion rules. This approach allows an we attempt to fill the gap between the two communities andagent to locally decide the next maneuver according to the propose an algorithm that detects misbehaving agents in thestate of a limited set of neighboring agents in order to improve presence of malicious monitors issuing fake information.the overall scalability of the system. In such cooperative multi-agent systems, an agent that violates the motion rules may II. PROBLEM STATEMENTimpair the overall system safety and availability. Therefore We consider a set A {a,, a2,.... .an} of nmobile,detecting misbehaving agents is crucial. In a decentralized autonomous robotic agents that move in a shared environmentapproach, misbehavior detection has to be locally performed where they safely interact according to a finite set R ofby each agent using its sensing capability. Nevertheless, limi- decentralized cooperation rules. At any given instant, thetations in sensing translates into limitations in monitoring and cooperation rules allow an agent to locally decide upon its nextthus in detection [1]. In order to overtake this obstacle, agents maneuver according to the configurations of its neighboringhave to cooperate by exchanging information and achieving agents. We call i?fluence set of agent ah at time t, and denoteconsensus on it. In a previous work we have shown that it by Ih(t), the set of agents that determine the agent ah'sit is possibile to detect a misbehaving agent by exchanging next maneuver at time t. The influence set of a node dependsaggregated information with a sufficiently large subset of its on the rule set R.neighbors [2]. This solution is effective in the case agents An agent that moves according to the rule set Rh is coopera-broadcast correct information but fails in the presence of tive, misbehaving otherwise. The movements of a misbehavingmalicious agents that deliberately broadcast false information. agent may arbitrarily deviate from that dictated by the rule set

In this paper we cope with the problem of detecting misbe- 7Z. We call Safety Set of agent ah at time t, and denote ithaving agents in the presence of malicious agents. A malicious by Sh (t), the set of agents of which ah wants to determineagenlt may falsely repor its own position, invenlt a non-existing whether they are misbehaving or not in order to take adequateneighbor, omit an existing one, or pretend that it is elLsewhere safet countermeasures (naot discussed iln this paper).with the aim of impairing safet or causing denlial of service. Let us suppose that an agent ah wants to determine whetherWe propose a distributed collLaborative alLgorithm that allows another agenat ai Sh(t) is misbehaving or lnot. In order to

agenlts with limited sensing capabilities to achieve consensus do that, ah has to monlitor both the configurationl of ai anld the

978-1-4244-2032-2/08/$25.00 ©)2008 IEEE. 109

Local Monitorconfigurations of all agents in ai influence set. However, due tolimited sensing capabilities, the agent ah is able to determinethe configuration of only a limited set of neighboring agents. Commnnic ton Message MotionWe call Visibility Set of agent ah at time t, and denote it T-=--by Vh(t), the set of agents of which ah iS able to determine _

and

the configuration at time t. By definition, the Visibility Set Sensing 1 Reanctionof an agent always includes the agent itself. We assume thatthe Visibility Set of an agent contains its Influence Set, i.e.,for every agent ai, 1i(t) C Vi(t). Finally, we assume that Fig. 1. The agent architecture in terms of its constituting modules and itsvisibility is a symmetric relationship. Therefore, if an agent relationships.belongs to the visibility set on another agent, then the latterbelongs to the visibili set of the former. receiving a message the Message Validation module verifies

Iln order to cope with the lilmited visibili , we assume thatagent ooeratebyoperwiodthically exchangin

,te niurati the correctness of data in the incoming message to filter outagents cooperate by periodically exchagng the configurations invalid one, and it passes valid data to the Motion Validation

of agents itiwvand Planner module. The Motion Validation is responsibleSet of agent ah at time t, and denote it by Ch(t), the set of fiagnt wit whc acacomnatatiet. By prpel for detectiLng malfunctioning in the motion of neighboring

agmentswitwi ahe communicat e.attiept. By proery agents, whereas the Planner module decides on the agentd.imsioninte .conican set withrespec ,tonthaf motion on the basis of its securely estimated neighborhoodset,~~~~~~~~ ~itisposil tocne naeteog nomto htiformation. In case a non-cooperative agent is detected, theintegrates that locally available and allows the agent to decide MotionVaIn m odendsian alarm to tePaer soupon the behaviors of agents in its safety set. Furthermore, all that the ses aneuvers tha maynequiragents that have a given agent in their respective safety sets e.g. to divert from the planned route. In the remainder ofnot reach consensus on whether that agent is misbehaving orthis section, we detail the Message Validation, the Motionnot L2].'This approach works well under the assumption that agents Validation, and the Planner modules.

report correct information. Problems arise if there are mali- A. Message Validation Modulecious agents that instead report false information. A maliciousagent may cheat about both its configuration, e.g., by pretend- In order to guarantee the system safety an agent ah hasing being elsewhere, and the configuration of its neighbors, to montor eablehvor ofeqachagenteasubelning toe.g., by inventing a non-existing neighbor, omitting to repo no as-tooberatbe tohrun a eqatecountermeasuresi case ofthe presence of a neighbor, or reporing a wrong configuration. non-cooperative neighbors. We assume that a suitable set ofA malicious node deliberately acts against the system in order emergency maneuvers are specified in the setr itself allowing

toimai th syte saf orcuedna .fsrie each agent ah to escape frolm a non-coopelrative othelr agent intoinpairthispapermweaconsideraasystemnwhereotheeagents its safety set. To this aim, we define Sh (t) as the set of agentsareloosely synhrwonsizederand ystewhereommunication is laying at time t in the circle centered at ah position with radius

arelab eleandaheonticaed. Rabl.where communicationmas otha Ps Thus, given vm the greatest of the agents velocities andreliable~~ ~~~~~~~~~~~~1andauhniae.Rlalcmuiainmasta AT, the maximum time that agent ah needs to react to theeach message broadcast by the agent ah is received by all other the ma timetat agentah nes to ret t thagents belonging to the ah Communication Set. Authenticated snecessary that the following inequality holds Ps > vmAT,communication means that each received message can beattributed to its legitimate originator. Finally, we assume that a Hence, ah has to receive the a state and the state of allmajori of agents around a given agent is not malicious. This agents influencing ai instantaneous motion. Let us define d_

means the maximum distance between an agent and all the agentsmeanstially maicou on this as is necessar to belonging to its Influence set. It is woghwhile to notice that

makentianly fardciosprogess. This assumption is necessaryto the distance d_ strictly depends on the rule set 7Z. Hence,given Pc the radio communication range it follows that the

III. AGENT ARCHITECTURE following condition must hold:The architecture of an agent is depicted in Fig. 1. The pc > p d.

Sensing module is composed of a set of sensors that allow eachagent to measure the configuration state of all its neighbors. Each agent periodically broadcasts a message containingThen, collected configuration data are periodically broadcast the state of all agents belonging to its Visibility set. Giventhrough the Communication module. Such a module allows Dmax the maximum error of an agent position that the ruleeach agent to share informlation with other agents so as to set X? can tolLerate without compromrising the system safetyimprove the sensing capability. Upon receiving a message, each agent has to bDroadcast amessage every ATc seconds sothe Locall Monitor immrediately evalLuates the valLidit of the that A\TC . Drnecr /t'M.received colnfigurations and attempts to detect misbehavilng Let us consider an agent ay belonging to the Communicationagenlts on the basis of validated ones. VMore in detail, uponl set of ah so that ah receives the message ]Xij(t). Sinlce

110

the communication time is negligible with respect to the the current maneuver based, on its goal, the configurations ofsystem time constant, the assertions received by agent ah the car and of other neighboring cars. In this example, theare considered at the time of message arrival. In case ai actions defined by 7Z are accelerate, decelerate, and change tobelongs to aj Visibility set, llj (t) contains (qi (t)) j, that is the the next left or right lane. As an example, the event requiringconfiguration state of ai sensed by aj. Hence, the agent ah has a maneuver change that will allow a vehicle overtaking isto verify the validity of the assertion (qi(t)) contained in the represented by a slower car in the front and a free lane onmessage. In case the agent ai belongs to Vh(t), ah discards the left. Therefore, the planner is a module that decides thethe assertion (qi (t)) j contained in lIj (t) by assuming valid its trajectory of agent ai based on the cooperation rule set XZ, andown observations. Otherwise, ah has to veriny whether (qi(t)) valid configurations of agents in its influence set li(t) at thesatisfies the following cond.itions: 1) (qi(t)) is consistent with current time t.the physical dynamics of the system (physical condition), and This type of system, where agents have a physical dynamics,2) (qi(t)%) has been confirmed by a majority of non-malicious but interact according to event-based cooperation rule setsagents (logical condition). In case both the conditions are JZ, can be modeled. as hybrid systems [1]. For the readersatisfied, it follows that (qi(t))h = (qi(t>j, where (qi(t))h convenience we recall this result in the following. Let qi(t) Cthe state of agent aj accord.ing to agent ah at time t. Q be a vector describing the configuration of agent ai at time

With reference to the physical condition, let us consider t and Q be the corresponding configuration space. It should(qi(t*))h the last configuration of ai validated by agent ah at be clear that in this description qi(t) is a short-hand for thetime t* so that t* < t. Hence, the assertion (qi(t))% has to valid configuration of agent ai computed on board of agentsatisfy the following equation: ai itself, i.e. qi(t) = (qi(t))i. Furthermore, let us denote with

(qi(t)) C fp((qi(t*)) h (t -t*)) (2) vi(t) E E the maneuver that agent ai is currently executing.We showed that qi(t) has a continuous-time dynamics

where the function fp(q,AT)) retums the set of feasibleconfigurations starting from q during the interval AT and h(t) (q(t),u(t)),according to the rule set JZWccordinthorefe e touthelgcaodi u con e te where ui (t) is a control input. Moreover, ui (t) is aset7h(refere tothe allthedasset aconsid t

feedback law generated by a low-level controller g: Q x Eset 'h(ai,t) containing all the assertions about dependin on the current maneuver ait i.e.ration state received by ah in the interval [t - ATE, t]. It is gonworthwhile to notice that such set contains only assertions that ui(t) = g(qi(t), a(t)).have satisfied the physical condition.

Let mr be the number of malicious agents the system must On the contrary, ai(t) has a discrete-time dynamics d: E xbe able to tolerate. As these nodes might collude and report E E and indeed it changes value only at discrete times tkfalse information about a given configuration qi(t), then we when an event from E occurs. More precisely, we have thatassume that the Visibility Set of a any given node must containat least (Tn+ 1) correct agents. This assumption, together with ori(tk+l) = 6(oi(tk), C(tk))the condition expressed in Equation 1, guarantees that (qitsatisfies the logical propert if and, only if the following wheuere c(t)ainte eventoccrdariet n eurncondition holds: maneuver change from i(tk) to ari(tk+l). Furthermore, event

activation is detected by a static map D Q x QP x Z -* E,fL((qi(t)) j ,Ph(ai, t)) > (mr + 1) (3) where p is the maximum agent number in the influence set

where the function fL(q, P) returns the number of configura- ii(t). With reference to the example mentioned above, maption contained in P that are compatible with q according to D encodes conditions such as the presence of a slower car inthe.ru.leset R. Noticeth the value of fea.tures a securi }-the front, and a free lane on the left. The event detected atperformance trade-off. Actually, a higher value of n implies time tk isthat the system tolerant to an higher number of malicious e(t)=D(qi(t),Ni(t),(i(t))colluding agents but it requires a denser network in order toguarantee a reliable validation. where N (t) (qi1 (t).... qi, (t)) is a vector impiling all

B. Planner Module valid configurations in the influence set 1I (t), and (i (t) C Z isa parameter that is reset at any maneuver transition. Note thatAccorllAaI\JIblkdingt tthese Ulofue k, agents can perfor at .~ (t is a sh -hn.o9~~~~~~~~1i (t ii S a shaort-haand. forRqij (t ))............ AS a wh1ole, th1e cdlynamicsany instant t one of maneuvers, 1 2 s s(7~ K of ag;ent ai can be cdescribed. by the following hybrid model:and must change from a current maneuever to another one

Vlwhenever onel ofI al setI of1 V evets,tlJ £ {e-1,_ e2 . eV}J, qijt) =7(qi(t), q1l (t), , qj( t))depending on itS influence set SI(t) occurs. For the sake ofclanit, consider as an examrplLe the case of n cars moving on where X Q x QP -2 , and i,,,, . .iare the in-a multi-:lalned. highway. Such cars are supposed to have the dices of agelnts in SI(t). Under this view, we w:lL: conasidersame dynamics, and au.tomated. pilots are supposed to decid.e q l (t), ... ., q i(t) as inpu.ts of model X anld q (t) as its ou.tput.

111

the computation of such a UIO leads to find ad-hoc solutions__ for specific cases. However, we showed in [1] how this can

0obs I l6 14 b e avoided for the considered class of robotic multi-agent- -H - - - -1 -l - --------- - systems. The property that in our opinion mak(es our approach

appealing is that all components of the proposed decentralizedmotion validation module can be automatically generated oncethe agent dynamics f, and the cooperation rules 7? are given.

Fig. 2. Partition of the influence set Io(t) of agent ao wrt. agent a45s The reader may refer to our works [1], [2], [7] for a completeextended visibility V4 description of the method and can assume the existence of a

procedure to build. a UIO, AHt, such that

C. Motion Validation Module ( +(t)' (t)) X'Ht((qi(t) h(qi I(t) 'h ',(t)We consider here a generic agent ah that is monitoring the (5)

motion of one of the agents, ai, that are in its safety set Sh(t). where qj (t) for m = no + 1,... ,p are continuous setsTo this aim, suppose that a measure (q(t))h of the trajectory estimating configurations of agents in Ii4(t) that can explainof ai during successive limited time intervals, Tk = [tk, tk+1], the validated motion (qi(t)Wh of agent ai (see Fig. 6 in thefor k = 0,1, ...., is available for the observing agent ah. example).

Let us denote with 4i(t) the expected evolution of agent ai By using this mechanism, agent ah may decide on thethat is the output of the hybrid model 7H receiving as inputs cooperativeness (bh)i of agent ai according to the followingthe configurations (qi (t))h,h * *, (qi, (t) ) h of all agents that rules. If ah has complete knowledge of agent ai's influenceare in the influence set i (t) of agent a? itself. Then, agent set, it will be able to say it is cooperative or not. Other-ails motion is non-cooperative if the expected motion differs wise, by observing agent ai's motion, agent ah is able tofrom the validated one, i.e. estimate the presence or the absence of other agents in the

unknown part of ai's influence set. As long as a choice for(qi(t))h # &i(t) for some t CTk. %1 exists, agent ai can be considered as possibly coopera-

The fact that makes this monitoring task difficult for the tive or uncertain (as a matter of fact, a can not verifyobserving agent ah is its partial knowledge of the influence set the correctness of these estimates). If no values for theseli(t) of agent ai. In the example in study, some cars affecting estimates exist, agent ai is considered as noncooperative.the behavior of agent ai maybe out agent ahs visibility set Vh In brief, the cooperativeness (bh)i of agent ai according(since they remain hidden by other cars as in Fig. 2), and no to agent ah is a discrete variable taking values in the setvalid messages specifying this information have been received B = {cooperative, noncooperative, uncertain, unknown}.yet by other monitoring agents. The introduction of the value "unknown" is instrumental forWe can conveniently define an extended visibilit set V* as the purpose of communication. Indeed, if agent ah does not

the region over which agent ah has received valid information see agent a, but has to paricipate in an agreement on theby either its own sensing module or by other agents in Ch (t). value of its cooperativeness, it will initially exchange the valueThen, the influence set 1i(t) of agent ai can be partitioned unknown.w.r.t. ah into a known region Ii (t) and an unknown one Ih (t), IV. EXAMPLEi.e.

_Ti It)= >t) U 1j(t). (4) A. An Automated Highway

Then, we are interested in solving the following: Consider mmobile agents that are traveling along a highwayProblem 1:- Given agent ai's (hybrid) motion model 'H with different maximum speed. and, dlifferent final positions.

the parition of its influence set i(t) in Eq. 4 w.r.t. Agents are supposed to cooperate according to the common

agent ahs extended visibility Vh*, and n, configurations driving rules in order to avoid collisions. Informally, the rule(q 1 (t>)h, * ., Kqin (t) h 1i (t) of known neighbors of set 7? is the following:agent a,i determine, if it exists, a choice of p -T config- RI) proceed at the maximum speed along the rightmost freeurations &. +,(t),... , i,, (t) C Ii,(t) such that the expected lane when possible (fast maneuver);motion R2) if a slower vehicle proceeds in front on the same

. (qi3 T. lane, then overtake the vehicle if the next lane on the.i(t) (q(tk +h 7((i())h, (i (T))h . left is free (left maneuver), or reduce the speed (slow

(q T) )h +1±(T), . . .,,qi (T)) dT, maneuver) otherwise;equals themeasureone, i.e. i(t) K(t)\ for all t CTk R3) as soon as the next lLane on the rightbecoImes free

Solving this problem iS in general a har task dlue to the changle to that laneW (righlt manelluLver);nonlLinear and differentialL nature of the mnotion mrodelL X.1 It R4) overtaking anly vehicle on the right is forbiddenl.basicallLy requires that an unknown input observer (UIO) X-T The generic agent chooses one of these maneuvers based olnofthe hybrid model is built. Furthermore, adirect approach for events onL its nleighborhood. With reference to Fig. 3, agenlt

112

{x;= oXi;y, Oi, vi} ODeS0j~~~~~~~~~~~y i=f2}Vl

,. Lyi , -1I' '

Lyij = 2Xi Fig. 5. Simulation run where agent a3 is non-cooperative since it keeps

traveling on the second lane, even though the lane on the right is free.Fig. 3. A 2-lane automated highway with a set of common individual drivingrules. Agent Visihility set

al = aa2aVIV(t) {aa ,a3a4}a2 V2 (t) = {al, a2,a3 }

Fa aa ;a3 V3 (t) {a2,a3,a4,a}6I

F- L F°R oa4 IVt) {tal,a3,a4,a5l0 th,,,i,;, ~~~~~~a5 V5(t) = {a3,a4 a5}

\v/ eLF/f\/W eRF X TABLE I

VISIBILITY SETS( 0 tews eS F ( F-S X {o tews )

{A if,< VMAX A if v < VMAX

/s a agent laying on the border of the radio communication range.Hence, function fp defines the ring centered, at (Xh(t), Yh(t))

aA If v < VMAX /

< / hwith radii Pc and Pc + d1As to the logical condition, function fL (Eq. 3) returns the

numhler: of configulrations (qi (t')) helonging to Ph(a, t) soFig. 4. Discrete dynamics 5 of the automaton, and low-level feedback control that (qc(to))f,(q(t)ion (tt')VMk where

t P ais theg ensuring that the plant f behaves according to the rule set RT. j '

Euclidean distance. This condition takes into account configu-rations contained in the messages are sensed at different times.

ai's configuration is qi(t) = (xi(t), yi(t), 0i(t), vi (t)) and has With reference to the automated highway, consider thethe continuous-time unicycle-like dynamics f: case depicted in Fig. 5, where agent a3 is trying to damage

=t- t) cos( i(t))the system by violating rule R3 and moving on the second

r x1(t) Vi(>) cos(6i(>)), lane whereas the lane on its right is free. Fig. 6 shows theQ1Yi(t) = Vi(t) sin(01(t)), information on the influence set of agent a3 that every agentL i (t) = wi(t) is able to estimate using only local sensing as in Eq. 5. Suchb1(t) = ai(t), estimates are possibly non-convex regions where the presence

where ai (t) and wi (t) are linear acceleration and angular of agents is required (when reported in red in the figure) or isvelocities, respectively. According to the set R, the maneuver excluded (when reported in green). The figure also reveals thatcri(t) of the i-th robot may take value on the set E = none of them is individually able to detect the misbehavior.{fast, left, right, slow} and has the discrete dynamics d of the Then, agent a3 will send a false assertion that tends to justifyautomaton in Fig. 4, where the low-level feedback controller g its non-cooperative behavior. In particular, agent a3 is tryingensures that the current maneuver ori is performed. The generic to leverage on the partial visibility of the following agent a2,event o,, of Fig. 4 is described in [2]. and indeed it claims the existence of an another agent a6 in

front of it. Clearly, on the basis of agent a3's message, agentB. Tolerating malicious agents a2 will determine that agent a3's behavior is cooperative. ThisAs stated above, the Message Validation module is respon- example shows the necessity of a mechanism that passes valid

sible for validating all assertions that an agent receives. Let us messages and discards the others. Table I reports the visibilityconsider the agent ah that has to validate (qi(t))j contained sets of every agent and in particular shows the malicious datain illj(t). According to the considered cooperation rule set kz, produced by agent a3.the physical and logical conditions can be specified as follows. To overcome local sensing limitation, all agents share theirAs to the physical condition, let us consider whether ah has estimated information. Let us focus on how agent a2 updates

recently validated the configuration of ai or not. In the first its local view based on received messages. Table II showscase. let (q1 (t*))h1 be the lLast configuration of ai vali;dated the lmessage validation lmechanislm performed by agent a2by agent ah at tilme t*' so that t*~K t. Thus, function fp whenever it receives a message. In the examrplLe, given the(Eq. 2) defines the sector centered at (x?(t*) y1(*)Y?VJwith low density Of the network, we consider rnU 1. Te frstradius v1(t*)(t -t*) anad anagle 01(t*), fI:n the seconad case, and second. co:lLumn shows message receptioln order at a2, thewe assume that ai is a new member of the influ.ence set of an third. colu.mn specifies the set of validated. agenlt configu.rationls,

113

m te essage Valid agents Uncertain agentsI I1I2 {al,a2 a3} {

4 V]3 {G1,G2,G3,a4} {l ,}4A~~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~lli {ia2,a3la,5

..........................4.....T....................................UN.OF..THE...MESSAGE...VALIDATION....MECHANISM....AT..AGENT...a2

..........................proposed...solution...is..robust..to..possible...failure...of.some..moni-.

................tors.sending.incorrect.inform atin3 uuew r ilcnie..........................the..development....of..emergency...strategies....by..using...which..such4...........................robotic..systems..can..react.to..the.presence...of.malicious...agents.

..........................Authors...wish...to..thank...undergraduate.....student....Francesco..... ............................Babboni....for...his...useful....help....This....work...has..been3 partially2fa5

.......................04..contract... ..ST..224428...(2008)..."CHAT......Control....of...letero-...........................geneous...Automation....Systems:...Technologies.....for..sca..ability,...............................r..co.....gurab......t...and......curity.......by..CONET,....theA1 "Cooperatinga,a6........................Objects....Network...of..Excellence",.....funded....by..the..European..

........................Commission ....under...F....with...contract....number...FP7..2...7..2..........................224053,...by......YC.N,..the..Network....of..Excellence....on.."Hybrid................>..Control:....Taming...Heterogeneity......and...Complexity....of..Networked..........................Embedded... .Systems",....funded...b....the..European....Commission...

..........under .FP6.with.contract.number....T.2.04M5...1368, and5by..................j..Research....Project....2007...funded....by..Cassa...di..Risparmio.....diLivorno, Lucc~~a1e4ia

............................................R FE..........E ES.a

[1.A.agoln....aeni.L.Plotin,G.Dm,an...ic...Deetrlizd.ntuio.DtetonEo.ecreCopraiv.uli-getSytes.EE...rnt.na.o.ernc.o.Dciin.ndCotrl.2072.A..gi.ni.M.Pllncc....alni,G.Dm,.n.A.Bc.i"Cnsnssbaeddstibte.ntuso.dtetonfo.mli-obtsytesEig.6. Initialresultsof tbe motion validation module at agentsGiG~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~in Proceedings of the IEEE internationalConferenceonRoboticsand~~~~~~~~~~~~~~~~~~~~~~~~~~~.............04.and03ibatareactingasmonitors of ibeir neigbbor oq and are irving Automation(ICRA2008),Pasadena,CA,USA,1923May2008, pp.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.............

to explain ibe its motion bebavior, Tbe live pictures reports...tbe .monitors'. 120B127.viewsatdifferenttimes.wberered and green colors indicate regions wbere [3] F. Pasqualetti, A. Bicebi, andE.Bulb,"Distributedintrusiondetection~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.............tbe presence of an agent is required or is excluded, respectively.Agenta~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~for secure consensus computations," in proceedings of the 46thIEEE~~~~~~~~~~~~~~~~~~~~~~~~~.............sends an assertioncontaininga false agent GE ibat tends to justify its non International ConferenceonDecisionandControl,New Orleans,LA,~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.............cooperative bebavior USA, 12-14 December 2007, pp. 5594-5599.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~..........

[4. PG.....Gren,an...ado,.Deetig.n.crrcin.mlciudaa.n.aet."i.Poceins.fth.jrs.CM.okho.o.ehcuaAd.o..tors,Pildefa.Penylana.1Ocobr204,p..9-7andthefourth columnagentswhose configuration has to be [5] i-P. Hubaux, S. Capkun,andjunLuo,'Tbesecurityandprivacyofsmart~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.............

valiate. Atthestep4, vlidaes 's osiion wheeasat RvbicesF IEEEMSecurit VAnIDAPrIvac MagaANIne, vol 2, ENo T3,2p.4 5step5itdoesnotvalidatetheposition of a6. As showninthe May-June2004.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.......................

table,agent a6 is correctly not handed to the Planner module [6] M. Raya, P Papadimitratos, and J -P 1-inbaux, "Securing vebicular.......................................................................comuictins".EE.Wrees.Cmmniaton,.ol.1,.o.5,ppThesameprocedureisrunateveryagent and revealsthatthe 8-15,October2006.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~......................proposedmechanism is able to tolerate the malicious agent [7] A. Eagiolini 0 Valenti L. Pallottino, 0, Dini, and A.Bicebi,"Local~~~~~~...................................................................Monitr.Impleentatio.for.Deentralied.Intrsion.Deection.n.Secura3 Multi-AgentSystems," IEEE ConferenceonAutomation,Science,and~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.......................................................

Theproblemofdetectingmisbehavior inmulti-agentsys-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.....................................................temsisconsidered.Asolution where agentsactas local~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.....................................................

monitorsoftheirneighborsanduselocally sensedinformation~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~......................................................aswellasdatareceivedfromothermonitorsispresented.The~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~......................................................