Distributed cooperation and coordination using the Max-Sum
algorithm
Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices Using the Max-Sum Algorithm. In: Seventh International Conference on Autonomous Agents and Multi-Agent
Systems (AAMAS-08), 12-16 May 2008, Estoril, Portugal.
Waldock, A., Nicholson, D. and Rogers, A. (2008) Cooperative Control using the Max-Sum Algorithm. In: Second International Workshop on Agent
Technology for Sensor Networks, Estoril, Portugal.
What is Max-Sum?
• Max-sum is a derivative of the sum-product algorithm, formed by transforming the max-product algorithm into log-space.
What is Sum-Product?
• General form of the forward-backward algorithm
• Iterative algorithm to compute the marginals of functions
• a.k.a Belief Propagation (when applied to probabilistic graphical models)– Information Theory– Machine Learning– Vision
An abstract problem
Consider a function, F, that is dependent on N variables, x = {x1 . . . xN}, and is defined as a product of M factors such that:
where each of the factors fm(xm) is a function of a subset xm of the variables that make up x.
An abstract problem
Find the marginal function zn(xn) that describes the dependency of F(x) on variable xn:
The marginal zn(a) is defined to be the sum of F(x) over all configurations of variables where
xn = a.
Factor graph representation
• Bipartite graph consisting of two types of nodes– Variable nodes– Function nodes
Factor graph representation
• Example: F = f1(x1,x2)f2(x2,x3)
What is Sum-Product?
• Given a factor graph
• Apply a local message passing procedure
• Compute the marginals of functions
Two local message types
• From variable to function:
• From function to variable:
Canonical update procedure
• Graph is cycle-free (a tree)
• Leaf nodes send identity messages to parents
• When a node receives a message on some edge, it computes and transmits the outgoing message for all other edges
• After variable node has received a message from every neighbor, it can compute its exact marginal value:
Message Passing
Message Passing
Message Passing
“Loopy” Belief Propagation
• Sum-product algorithm is only guaranteed to converge on graphs containing up to one loop
• Could oscillate, or fail to converge on cyclic graphs
• Empirically, it often converges pretty well on many types of graphs
“Loopy” Belief Propagation
• All variable messages initialized to 1
• All messages updated at every iteration
• Normalize variable to function message:
Choose αnm such that
What is Max-Product?
• Instead of computing the marginal functions, we can calculate arg maxx F(x)
• Replace function-to-variable message:
What is Max-Product?
• Now, when we take the product of the messages:
• For arg maxx F(x), each component is given by:
• A.k.a: Viterbi algorithm
What is Max-Sum?
• Suppose we have an F(x) that is the sum of function nodes, e.g.
• Use max-product in logarithm space
Two new local message types
• From variable to function:
• From function to variable:
What is Max-Sum?
• Now, when we take the sum of the messages:
Advantages
• Existing theoretical results• Simple, scalable local algorithm
– Exponential only in # of neighbors in graph
• Already decentralized and asynchronous– Partition nodes as desired– Only exchange local messages
• Can estimate variables before convergence
Caveats
• No strong theoretical results for loopy BP– Convergence not guaranteed
• Must exchange local messages between related factor and variable nodes
• Not necessarily easy to evaluate functions and compute messages
How to use Max-Sum to solve your distributed problem:
Construct utility function
How to use Max-Sum to solve your distributed problem:
Construct utility function
Turn it into factor
graph
Application #1:Decentralized Coordination
Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices Using the Max-Sum
Algorithm. In: Seventh International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-08), 12-16 May 2008,
Estoril, Portugal.
Problem
• Solve graph coloring on sensor network
• Utility can be formulated for each agent:
Penalty for same color:
A priori color preference:
Factor-graph Formulation
• Utility function is only dependent on neighbors• Let each sensor maintain its state and utility
nodes:
• Exchange local messages as per Max-Sum
Comparison Algorithms• BR - Best Response
– Each agent chooses the “best” state for its variable (i.e. the one that minimizes the conflicts) according to the current states of its neighbors
– When a state change occurs, the agent sends the updated state to all of its neighbors
– Represents a lower bound on the performance of any algorithm since it uses the minimum computation and communication possible
• DSA - Distributed Stochastic Algorithm– Same as best response, except that an agent only actually performs the state
change according to a predefined probability (called the activation probability)– Whenever an agent’s preferred state actually changes, it sends a message to its
neighbors informing them of this fact– Set the activation probability to 0.6 (additional experiments indicate little
sensitivity to the exact value of this parameter in our setting)• DPOP - Dynamic Programming Optimization Protocol
– A complete algorithm that maintains optimality by preprocessing the constraint graph to produce a pseudo-tree, and then performs local message passing on this tree
Test Conditions
• Known colorable– 3-color random graphs – 10, 20, 30, 40, 50 nodes– Link density of 3– 50 instances
• ADOPT graph repository– Random graphs – 10, 14, 18, 25 nodes– Link density of 3– Not generally colorable with 3-colors
• Lattice graphs– 4-color colorable– 36, 49, 64, 81, 100 nodes– Nodes connected to eight neighbors
Conflicts over time
Conflicts over time
Conflicts over time
What happened here?
• Combination of small and large irregular loops
• Cycling causes poor convergence
• Modify utility function:
Summary of conflict results
Message Size
Message Loss
Hardware Implementation
• http://www.youtube.com/v/T6H1AwQ2gXw
• http://www.youtube.com/v/D6vWvs3Lsj0
Application #2:Cooperative Control
Waldock, A., Nicholson, D. and Rogers, A. (2008) Cooperative Control using the Max-Sum Algorithm. In: Second International Workshop on
Agent Technology for Sensor Networks, Estoril, Portugal.
Problem
• Sensor network doing distributed tracking• Sensors share information using
decentralized data fusion (DDF)
Problem
• Select control parameters for each sensor– Maximize sum of global info for all targets
Problem
• Define U as a summation of target factors Uj over only sensors that can observe target, e.g.
where Ub is defined as:
Problem
• Unfortunately, we can’t break into smaller factors:
≠
Factor-graph Formulation
• Each sensor maintains variable node for its own control parameter
• Define a function node for each target– Maintained by the sensor that first saw target
• Use DDF to determine edges of graph– Add/remove edges as target moves in and out of
sensor range
• Exchange local messages as per Max-Sum
Factor-graph Formulation
Sensor 3
θ2
Sensor 4
θ4
Sensor 2
θ2 UB
Sensor 1
θ1 UA
Sensor 5
θ5 UC
Test conditions
• Sensors initialized with weak priors and target responsibilities
• Performance evaluated using global information from one sensor
Global Information
Adjusting negotiation time
How to use Max-Sum to solve your distributed problem:
Construct utility function
Turn into factor graph