1

Scalable Real-Time Negotiation Toolkit

Organizational-Structured, Distributed Resource Allocation

PI: Victor R. Lesser

University of Massachusetts

3

Problem Description/Objective

Organizational-Structured Distributed Resource Allocation

• The specific technical problems we are trying to solve:– Development of Soft Real-Time, Distributed Resource

Allocation Protocols– Development of Techniques for Specification,

Implementation and Adaptation of Agent Organizations

• Relevance to DoD:– Techniques for building large-scale, soft real-time,

multi-agent applications involving complex resource allocation decisions• Distributed sensor networks, distributed command and control

5

Approach to Soft, Real-Time Distributed Coordination/Resource Allocation

• Structured as a distributed optimization problem with a range of “satisficing” solutions

• Adaptable to available time and communication bandwidth

• Responsive to dynamics of environment• Organizationally-constrained — range of

agents and issues are limited• Can be done at different levels of abstraction• Does not require all resource conflicts to be

resolved to be successful — resource manager agents able to resolve some issues locally

6

Multi-level Approach to Distributed Resource Allocation and Coordination

• Organizational Design: Determine appropriate agent roles and responsibilities

• Team-Based Negotiation: Managers negotiate solutions to allocation conflicts

• Local Autonomy: Individuals decide local unresolved allocation details

7

Capabilities

• Scaling: Support for large-scale adaptive agent sensor networks

• Efficiency: Organizationally grounded resource allocation

• Responsiveness: Dynamic, soft real-time resource allocation

• Adaptability: Organizational self-design and maintenance

8

Major Issues in Implementing this Approach

• What is an appropriate organization for agents– Scalability and Robustness

• What is the protocol for distributed resource allocation– Soft Real-Time, Graceful Degradation, Efficient

• What is the structure of an agent architecture that supports:– agents functioning in an organizational context– agents implementing complex distributed resource

protocols – agents operating under soft real-time constraints

How domain-independent and efficient can we make these approaches?

9

Our Solution at the Organizational Level

• Decompose environment to form a partitioned organization.– Each partition (sector) will contain a set of sensor nodes,

each with its own controlling agent.– Individual sectors are relatively autonomous.

• Specialize members of the agent population to dynamically take on multiple, different goals/roles.– Individual agents become “managers” of different aspects

of the problem.– Managers form high-level plans to address their goals, and

negotiate with other nodes to achieve them.

10

Sectored-Based Agent Organization

Agents multiplex among different roles

Sector Manager

Tracking Manager

Tracking Agent

Scanning Agent

Organizationally-Structured Communication among Agents

DrADrQ

DrRTBRRTDPTCRBPCDATBUES

Sector Manager

Tracking Manager

Scanning Agent

Tracking Agent

12

Managing Conflicted Resources:Sensors, Processors, Communication

• Sensors– Conflicting scanning tasks from different sector managers

• Locally resolved by agent connected to sensor — SRTA agent

– Tracking tasks wanting same sensor resources• Negotiation among track managers — SPAM protocol

• Communication– Communication degradation due to lack of locality

• Track manager migration among sectors

– Communication channel overload• Sector manager assignment of track manager roles

• Processors– Data fusion overload/knowledge locality

• Sector manager assignment of data fusion/track manager roles

– Multiplexing Roles• SRTA local agent control/scheduling

13

SRTA: Soft Real-Time Agent Architecture

• Facilitates creation of multi-resource management agents

• Basis of building complex “virtual” agent organizations

• Allows for abstract negotiation — maps abstract assignment into detailed resource allocations

• Ability to resolve conflicts locally that are not resolved through negotiation

These are key to building soft real-time distributed allocation policies

14

Soft Real-Time Control Architecture

Resource Modeler

Conflict Resolution Module

Task Merging

Problem solverPeriodic Task Controller

TÆMS Library

Cache Check

DTC-Planner

Partial Order Scheduler

Parallel Execution Module

LearningUpdateCache

CacheHit

Linear Plan

TAEMS-Plan Network/Objective

Goal Description/Objective

Parallel ScheduleScheduleFailure

Results

Update Expectations

Schedule Failure

OtherAgents

Schedule

Resource Uses

MultipleStructures

Negotiation(e.g. SPAM)

Commitments/Decommitments

Schedule failure/Abstract view

15

SPAM: Resource Adaptive Family of Anytime Negotiation Strategies

• Low bandwidth or not a lot of time:

– Single-shot — single assignment message

to the sensor agent based on

uncertain/incomplete information

• Relaxing of objectives based on local

information

• High bandwidth, a lot of time:

– Multi-step negotiation with track managers

and sensors

16

Mediator ViewInterdependency Graph

Mediation-Based Negotiation

M20M33

M0 M8

M7

M25M14

S15

S32

S53S18

S25,S20

S7

S18

S5 S8

S12, S22

S2, S14

M20M33

M8

M7

M25

1

1

11

2

1

1

1

World View- Multi-Linking of Resource Allocations

World View- Multi-Linking of Resource Allocations

17

Stage 2 - Track Manager to Track Manager Negotiation

• Originating track manager acts as mediator– Generates solution space– Recommends solution quality reductions– Chooses final solution

• Negotiation Mediator gets partial non-local information– Some/All of the sensor schedules relevant to specific track

• Used to find neighbors (other track managers) in the constraint graph

– Conflicting track managers’ information• Domain of acceptable assignments

– Current solution quality– Number of possible sensors that can be used for tracking– Sensors that are in conflict (mediator to neighbor and neighbor to

neighbor)

• Additional constraints – fuzzy notion of constraints on non-directly conflicted sensors

18

Major accomplishments/contributions of the project

• Development of SPAM heuristic resource allocation protocol– Showed importance of mediation-based negotiation (partial

centralization) with overlapping context and extended views along critical paths for search and communication efficiency

• Development of APO distributed constraint algorithm based on SPAM concepts– Better performance than best known algorithm - AWC

• Development of SRTA soft real-time architecture– Demonstrated that a sophisticated domain-independent agent

architecture that operates in soft real-time could be built

• Demonstrated importance of organizational structuring for distributed resource allocation– Showed how using negotiation organization could be dynamically

constructed and efficiently modified as the environment changed

19

Recent Accomplishments

• New Results on SPAM, APO and Opt APO

• First Results on Organizational-Structured Coalition Formation

• Performance Improvements in FARM

• Organization Design Framework

20

SPAM’s Effectiveness

• 20 randomly placed sensors.• Between 2 and 9 randomly placed fixed targets.• 160 test runs (20 runs for each number of targets)• Ran until SPAM converged.• Optimal

– utility was computed using a Branch and Bound search where the domain for each track was the possible objective levels.

– tracks were computed by using a Branch and Bound search where the domain for each track was either the minimal utility for tracking or nothing.

• Greedy– Each track manager requests 4 of the available sensors at

random for every time slot. – Commitments override each other in the sensor schedules.

21

Utility Comparison as a % of Optimal

• SPAM stays closer to the optimal value and has less variance in its utility.

22

Tracking Comparison

• SPAM tracks nearly 100% of the optimal number of targets that can be tracked.

• Greedy ignores more targets as contention increases.

23

Time to Convergence

The time to converge increases linearly with an increase in contention.

24

SPAM’s Scalability

• 100-800 agents.• Each agent was either a sensor or track

manager.• Fixed ratio of sensors to targets of 2.5

sensors per target.– Fairly overconstrained

• Sensors are randomly placed.• Targets move with a random velocity that is

uniformly distributed from 0.0 to 1.0 ft/s.• Environment size had a fixed expected

sensor density of 4 sensors per point.• Twenty 3-minute runs per data point.

25

Utility Scalability

SPAM consistently maintains a higher utility than a greedy assignment

26

Tracking Scalability

SPAM also tracks a higher percentage of the targets that are viewable by 3 or more sensors.

27

Communication Scalability

There is no apparent increase in the communications per agentas the number of agents increase.

28

Asynchronous Partial Overlay (APO)

• A new algorithm for Distributed Constraint Satisfaction (DCSP)

• Three basic principles– Mediation-based– Overlapping views and exploiting local

context– Extending views along critical paths

• Proven to be both complete and sound

29

How it Works

• Agents take the role of mediator when they have conflict

• Mediator gathers information from other agents in the session concerning value preferences and their effects

• Mediator chooses a solution that removes local constraint violations and minimizes the effect outside of its view

• Mediator then links with agents for which it caused violations (expanding context along critical paths)

30

Testing APO

• Implemented the graph coloring domain in the Farm simulator– 3 Coloring problems– nodes = 15, 30, 45, 60, 75, 90– edges

• 2.0 X nodes (Low, left of phase transition)• 2.3 X nodes (Medium, in phase transition)• 2.7 X nodes (High, right of phase transition)

• Compared APO against the Asynchronous Weak-Commitment (AWC) protocol (Yokoo ‘95)– 10 random, solvable problems, each with 10 different starting

assignments (Minton et al. ‘92)

• AWC is currently the best known method for solving DCSPs

31

Results – Low Density Graphs

32

Results – Medium Density Graphs

33

Results – High Density Graphs

34

SPAM II — Optimal APO

• Currently working on an optimization version

of APO.

• Based on the three main APO principles.

– Mediation-based

– Overlapping views and exploiting local context

– Extending views along critical paths

35

How it works

• Each agent computes the optimal value for their local sub-problem (upper bound) and the current value based on their current view.

• Mediation occurs when:– upper bound is greater than its current value– one of the links they own has a value that is not the highest

available

• Mediator gathers information from other agents in the session concerning value preferences and their effects

• Mediator chooses a solution that is optimal and minimizes the impact outside of its view

• Mediator then links with agents that had their current value lowered as a result of the mediation (expanding context along critical paths)

36

Testing Optimal APO

• Preliminary testing on partial constraint satisfaction in 3-coloring– Optimal APO appears to be sound

and optimal– It may be better than other DCOP

techniques

• More testing is needed to confirm these suspicions

• Optimality and soundness proofs are underway

37

Organizationally-Structured Distributed Coalition Formation

Formal Definition of the Task Allocation Problem:• Let R = {R1, …, Rk} be the set of resources.

• Let A = {a1, … , an} be the set of agents, where each agent a i controls a set of resources CRi = {cri,1, …, cri,k}

• Let T = {T1, …, Tm} be the set of tasks to be undertaken, where each task Tj has a utility, a set of required resources RRj = {rrj,1, …, rrj,k}, an arrival time, a duration, and possibly a deadline.

• The goal is to maximize the total utility of accomplished tasks.

• A task Tj is accomplished if it was assigned to a coalition C j of agents that collectively has enough resources to accomplish T j while satisfying its timing constraints.

How to construct for a large number of agents an organization of agents and associated allocation

policy that optimizes this allocation process over an ensemble of tasks

38

An Organization for Distributed Coalition Brokering

a0

a1 a4a3a2 a5

a0

a1

a2

a3 a4

a5

a6

• The only way to achieve scalability is to “organize” agents into a hierarchical structure.

• We can then use this structure in allocating agents (teams, coalitions, etc.) to incoming tasks.

39

Now the organization in action…

a0

a1

a3

a5

Task T2 (200,400) arrives at a5

Task T1 (100, 50) arrives at a3

40

The Need for Search

a0

a1

a3

a5

a3 successfully formed a coalition for T1.

a5 failed to form the coalition

and sending task up to a0.

a6

a7

a6’s schedule

a7’s schedule

a3’s schedule

Can we learn a policy for deciding how to search at an agent based on meta-level information on resources at children agents.

Can we learn a policy for deciding how to search at an agent based on meta-level information on resources at children agents.

41

Elements of an Organization

• Organization structure

• Decision making

• Information abstraction

• Goal decomposition

42

The Local Decision Problem

• Each manager has the following state:– For each sub-cluster, its size, average resources,

and standard deviation

• and is required to make an action:– serial: select the best candidate agent for asking

for resources– parallel: decompose the required resources over

the sub-managers

• Both versions of the decision problem can be modeled as an MDP, where RL can be used.– Q learning algorithm with neural nets as

functional approximators

43

Experiments Setup

We tested two organizations:1. A small organization, consisting of 40 individual agents,

managed by 10 managers.

2. A larger organization consisting of 90 individual agents, managed by 13 managers.

• Tasks were chosen randomly from a fixed pool (one pool for each organization).

• We compared learned policies (different exploration rates) against random and heuristic policies. We measured two quantities:

– the average utility achieved by the organization– the average number of messages exchanged by the

organization.

44

Results for Small Organization

730

740

750

760

770

780

790

800

810

820

0 20000 40000 60000 80000 100000 120000 140000 160000

episodes

learned. e=1.0learned. e=0.3

heauristicrandom

45

Results for Small Organization

37

38

39

40

41

42

43

44

45

0 20000 40000 60000 80000 100000 120000 140000 160000

episodes

learned. e=1.0learned. e=0.3

heauristicrandom

46

Results for Large Organization

5400

5600

5800

6000

6200

0 20000 40000 60000 80000 100000 120000 140000 160000

episodes

learned. e=1.0learned. e=0.3learned. e=0.8

heauristicrandom

47

Results for Large Organization

45

50

55

60

65

70

0 20000 40000 60000 80000 100000 120000 140000 160000

episodes

learned. e=1.0learned. e=0.3learned. e=0.8

heauristicrandom

48

Conclusion on Organization for Distributed Coalition Brokering

• The learned policies outperformed both random and heuristic policies for both the small and large organizations, achieving higher utility with less communication.– Less exploration seems to better due to interaction among learners

• Though using neural nets threatens policy convergence, our learned policies always converged. Abstraction and decomposition functions highly affects convergence.

• We expect more improvement in the performance of the learned policy with better abstraction and decomposition functions.

• Our next step is to study the optimization of the organizational structure and how this interacts with optimizing the decision making (different organizations have different optimal policies).

49

Farm

Distributed, generic multi-agent simulation environment

• Provides– Environmental state accessors– Controllable communication mechanism– Plug-in mechanism for adding functionality

• Agents run in “allocated” real time.– Each agent receives an amount of real CPU

time to run in

50

Analyses• State / trend analysis

GUIs• State visualization

Driver• Non-agent activity

System Architecture

Farm Core• Plug-in management

• Control flow

…

Agent Agent Agent

Meta-Agent• Thread scheduling

• Communication

… Agent Agent Agent

Meta-Agent• Thread scheduling

• Communication

… Agent Agent Agent

Meta-Agent• Thread scheduling

• Communication

…

• Each component may be run on a separate processor

• Most components are optional, and may be added dynamically

51

Global Data

• Allows global data “properties” to disseminate information– Environmental simulation

• (e.g. target location, visibility lists)

– Statistics and instrumentation• (e.g. current utility, message totals)

• Data flows among components– Readers: Analysis, agents,

visualization..– Writers: Environmental drivers, agents…

52

Global Data Bottleneck

• Central storage of these properties is impractical– Thousands of agents may cause millions of

accesses– Creates a potential bottleneck, as well as

high communication overhead

• Distribute data across components– Farm core tracks ownership of properties– Storage and access control is distributed– Data may be proactively pushed

53

Distributed Storage

• Pattern of ownership matters– Co-location of data and usage is desirable– 11 strategies were evaluated across 3 domains– Compared simulation duration and message

overhead

• Examples:– FRP: First-reader owner, subsequent readers

added to push list– S: Offline learned, max-access owner and push

list contains net gain plug-ins (reads-writes > 0)

54

Results

• ANTs/DSN domain:

– S performed most consistently over the 3 domains– FRP is (mostly) reasonable, lower overhead choice– Simulation duration improvements vs. centralized:

TypeC LW FR FW LWP FRP FWP S FRAP FWAP SAP

0

10

20

30

40

50

60

70

80

90

Comparison of Different Token Assignment Strategies

Overhead

Remote Get

Remote Set

TypeC LW FR FW LWP FRP FWP S FRAP FWAP SAP

290

300

310

320

330

340

350

Comparison of Different Token Assignment Strategies

Strategy / Domain DSN Graph Color Learning

FRP 7.5% 21.6% 0.7%

S 4.8% 28.5% 44.4%

55

Organization Design and Instantiation

• Develop automated organization design & instantiation capabilities– Domain independent approach

• evaluation includes models for EW Challenge-like domain• range from small, simple organizations (1s-10s of agents)

to large, complex organizations (1000s of agents)

– Ability to generate automatically appropriate organizational structure

• based on performance requirements & task-environment expectations

• e.g., change from simple peer-to-peer, to single-level hierarchy, to multi-level hierarchy as scale/requirements change

56

Automating Organization Design and Instantiation

Organization Designer

Instantiator Evaluator

Coordination Knowledge

CandidateOrganizations

Evaluations

DomainOrganization

Model

PerformanceRequirements Analysis

Models

Operational Simulator

ScenarioGenerator

Abstract-Task Executer

Org Design Knowledge & Search Strategies

DetailedTask-LevelEvaluation

TaskEnvironment

Resources/Agent Capabilities

Role/AgentBindingsE

nv

Mo

de

l &R

eq

uire

me

nts

Domain &Coordination

ActivityModels

57

Designer Input

• Problem-Domain Goal Tree

• Task Environment

• Performance Requirements

Monitor

Detect Track

Verify

Handle Fuse UpdateScan

coverageArea (0 0 1000 1000)maxSimultaneousTracks 60maxArrivalRate 5/minexpectedTrackDistribution :uniformmaxVelocity 20

. . .

maxDetectDelay 4trackingResolution 8tradeoffWeights: Detection .4 Tracking .6evaluationWeights: Communication .6 Computation .1 Scanning .3

58

Designer Input

• Agents/ResourcesAgent: 1 Location: (40 90) Role: Radar Scanner Focused Radar Fuser Manager Subordinate . . . Cap: radarRadius 20 scanRate .25/sector focusedScan .75 computePower 1.6 memory 1.2 storage 20 comMethod broadcast-1 comRange 600 comRate 15 . . . Agent: 2

. . .

ComMethod: broadcast-1 bandwidth 240 maxEffectiveLoad .7

. . .

• RolesRadarScanner Achieves: scan Requirements: area coverageArea rate fn(maxDetectDelay,maxArrivalRate) com fn(maxArrivalRate) DecompositionMethod fn()

LinearSweepScanner Achieves: scan Requirements: area coverageArea sweepSpeed fn(maxDetectDelay,maxArrivalRate) com fn(maxArrivalRate) DecompositionMethod fn()

. . .

59

Designer Output

• Role-Goal-Agent BindingsAgent-1 RadarScanner area (20 40 60 80) scanInterval 3 sendTo {Agent-34} recFrom {} FocusedScanner sendTo Fuser recFrom FusertradeoffWeights: RadarScanner .4 FocusedScanner .6

Agent-2RadarScanner area (60 80 100 120) scanInterval 3 sendTo {Agent-34} recFrom {} FocusedScanner sendTo Fuser recFrom FusertradeoffWeights: RadarScanner .4 FocusedScanner .6

. . .

60

Automating Organizational Design and Instantiation

Design &Instantiation

Process

Problem DomainDescription

Goal TreeRoles

Environmental ModelPerformance Requirements

Agent Set

Role-Goal-AgentBindings

Stable for a variety of contexts

Variable/Context specific

The organizationalstructure: Includesboth problem domainand coordination domainbindings

CoordinationKnowledge

61

Problem-Domain Knowledge

• Automated designer is provided:– An organizational goal tree

parameterized by an environmental model and performance requirements

– A set of roles that could be used to satisfy the organizational leaf-goals

• For each role a requirement function dependent on a goal and its parameters

– A set of agents and a capability list for each (e.g. scan rate, scan radius, communication resources, CPU resources, …)

Monitor

Detect Track

PRmax. delay,min. res.EM

area dimensionsmax # vehicles,vehicle speed,distributionetc.

max delay

min. res.

Scanner Verifier Fuser …

A1, A2, …, An

62

A1, A2, …, Ai, Aj, Ak …, An

• For each role-goal binding, bind a set of agents that together satisfy the requirements

Problem-Domain Bindings

• To each leaf goal bind a role that satisfies it

Scan

Scanner VerifierFuser

Scanner VerifierFuser

Fuse

63

Coordination Domain

Scan

Radar

Ai, …, Aj

The above results in a set of agent bindings to roles and goals. Alone this is not enough to guarantee that the organizational goals are satisfied. The agents’ behavior must be coordinated

Each set of role-goal-agent bindings generates a Coordination Goal when needed

Fuse

Fuser

Ak, …, Al

CG

CG

64

Coordination Goals

• To satisfy a coordination goal, the design and instantiation process uses domain-independent coordination knowledge

– Chooses coordination mechanisms and

roles

– Assigns coordination role-goal-agent

bindings

• Coordination bindings may themselves generate coordination goals

65

Coordination Goals (cont.)

• Which mechanism to use depends on the character of the role-goal-agent bindings. Examples:– A simple goal satisfied by a small number of

agents may require only peer-to-peer coordination– A goal such as scanning for new vehicles satisfied

by many agents requires a single-level hierarchy to assign scan schedules

– A goal of tracking multiple vehicles may generate resource contention, thus requiring a coordination mechanism such as SPAM

66

Teams

• Many goals can be satisfied with long-term organizational structures

• Other goals are more transient and may be better satisfied with teams– Temporary “organization structures” established to satisfy

particular goals and disbanded when the goals no longer hold

• e.g., Tracking a particular vehicle

– Formed in response to dynamically generated subgoals– Thus, not strictly organizational structures

• Rather than generate teams, the design and instantiation process must ensure that organizational structures and resources exist for agents to generate and participate in teams as needed

67

Implementation

• We are building an automated organizational design and instantiation system

• We frame the process of organizational design and instantiation as a search process

– Use heuristics to generate a reasonable set of complete role-goal-agent bindings

– Evaluate each organization (set of bindings)

• described shortly

• Eventually want to be able to perform early evaluation of partial bindings during generation of the candidate sets

68

Heuristics guide the search to a set {a,b,c} of reasonable organizations

• Roles multiplexed within agents• Communication costs kept low• Computational load balanced

Implementation

a c

b

a

b

c

Org.Evaluation

a highest rated organization

Role-goal bindings

Role-goal-agent bindings

Domain + coordination bindings

69

Organization Evaluation

Goals:

• Determining performance evaluation values for MAS structures

• Mainly, comparison of different structures given the same environment – no absolute evaluation (specific performance

numbers)– rather comparable performance-estimator values

• Predicting the performance of the system regarding the given circumstances

70

Evaluating an Organization

Org.Evaluation

EM PR

Agent1: Cap. List, {B1, B2, …, Bm}

Agent2: Cap. List, {B1, B2, …, Bq}

…

Agentn: Cap. List, {B1, B2, …, Br}

GoalsExpectedGoalSatisfaction

Agent Loading/Performance

ScalarEvaluationRole-Goal

Com. ToCom. FromExec. TimeExec. PeriodCom. Time

Bi=

74

• Load calculation for the agents, based on expected number of task occurrences – Goal-specific load– Total load of an agent

• Delays computed based on work- and communication load– computed from expected task occurrences

• Performance estimation based on delayed tasks – decreased task utility

• Response time estimation– Based on processor and communication load

Evaluation

76

Contributions

• Separation of problem-domain from organizational coordination

• Domain-independent coordination knowledge is part of the automated design system– Developer need not pre-specify organizational

relationships, only problem-domain-specific goals and roles

• Provide a framework for representing organizational activities and relationships

• Developing algorithms for effectively searching and evaluating organization design space

81

Lessons Learnt

• RADSIM and Debugging/Analysis Tools key to success of effort on EW Challenge Problem- would have put more manpower earlier in project to make them more flexible

and efficient tools, and accurate representations of hardware• We needed to build separate simulation systems because could not scale

RADSIM• Could not explore different communication and processing assumptions

• Extension of Oracle procedure of University of South Carolina to multiple targets– Would have allowed us to better understand the effectiveness of our

resource allocation protocols• EW problem did not stress limits on computational resources only on

communication and sensors– Dramatic limits on communication dominated design issues too much and

thus narrowed scope of issues/solutions pursued– Sensors Processing/Fusion not a time-consuming process nor were there

any realistic options about trading off time for accuracy

EW Challenge Problem was sufficiently interesting and relevant to lead us to very interesting

and important results

83

Deliverables

• SPAM: Soft real-time distributed resource allocation protocol– Transferred to Zhang and Selman

• Farm: Distributed simulator for large-scale multi-agent systems– available via web

• SRTA: Soft real-time local agent control– Transferred to Honeywell, also available via web

• APO: Mediation-based distributed constraint satisfaction protocol

84

Technology Transition/Transfer

• New Book -- Distributed Sensor Networks– Edited by V. Lesser, M. Tambe and C. Ortiz– Kluwer Academic Publishers

• Part of a book series on Multiagent Systems, Artificial Societies and Simulated Organizations

– Contains articles from ANT’s researchers on • EW Challenge Problem Testbed • Distributed Resource Allocation: Architectures and

Protocols• Formal Analysis of Protocols

: :

85

Technology Transition/Transfer

• Rockwell/Collins considering SRTA architecture components and SPAM/APO for DARPA HURT

• Boeing investigating the potential of using SPAM/APO for the new DISA GIG project and for air space deconfliction for future JDAM projects.

• Honeywell Laboratories has licensed the TÆMS/ SRTA technologies for research use.– Used in First-Responder Application

• Real-time Tornado Tracking using Network of Phase-Arrayed Radars as part of NSF ERC – SPAM type real-time resource allocation– Initial deployment in 2006 in Oklahoma

87

Project Successes

• APO -- Best performing distributed constraint satisfaction algorithm– Measured against currently best known distrtibuted

constraint satisfaction algorithm (AWC)

• SPAM -- close to optimal in small experiments, scales well to large number of sensors and vehicles, resource and time adaptive protocol– Experimentally validated in RADSIM– Large Scale Experiments in FARM– Used in EW Challenge Problem hardware

• SRTA -- demonstrated a sophisticated agent arhcitecture with planning and scheduling technologies is appropraiate for use in a soft-real time application– Extensive experimentation in RADSIM– Used in EW Challenge Problem hardware

88

Project Successes (cont.)

• Organization Structuring -- demonstrated advantages of using an organization approach for distributed resource allocation; show ed that these advantages grow with scale; demonstrated ability to create and adapt organization on the fly.– Extensive experimentation in RADSIM– Used in EW Challenge Problem hardware

• Extensive Publication– 1 book, 6 book chapters, 3 journal articles + 5 in preparation,

18 highly selective conference papers, 16 workshop papers

• Received Honorable Mention in FIPA Software Prototypes Track Demonstration Competition for Distributed Sensor Network Application

89

Cumulative List of Publications

Books/Book Chapters• Egyed, Alexander; Horling, Bryan; Becker, Raphen; and Balzer, Robert

(2003). “Visualization and Debugging Tools.” Distributed Sensor Nets: A Multiagent Perspective. V. Lesser, C. Ortiz, and M. Tambe (editors), Kluwer Academic Publishers, pp. 33–41.

• Horling, Bryan; Mailler, Roger; Shen, Jiaying; Vincent, Regis; and Lesser, Victor (2003). “Using Autonomy, Organizational Design and Negotiation in a Distributed Sensor Network.” Distributed Sensor Nets: A Multiagent Perspective. V. Lesser, C. Ortiz, and M. Tambe (editors), Kluwer Academic Publishers, pp. 139–183.

• Lesser, V.; Ortiz, C.; Tambe, M. (editors). Distributed Sensor Networks: A Multiagent Perspective. In Series: Multiagent Systems, Artificial Societies, and Simulated Organizations, Volume 9, May 2003.

• Wang, Guandong; Zhang, Weixiong; Mailler, Roger; and Lesser, Victor. (2003). “Analysis of Negotiation Protocols by Distributed Search.” Distributed Sensor Nets: A Multiagent Perspective. V. Lesser, C. Ortiz, and M. Tambe (editors), Kluwer Academic Publishers, pp. 339–361.

• Horling, B.; Mailler, R.; Lesser, V. (To appear.) “Farm: A Scalable Environment for Multi-Agent Development and Evaluation.” Advances in Software Engineering for Multi-Agent Systems.

90

Publications in Journals and Highly Refereed Conferences

• Lesser, V.; Decker, K.; Wagner, T.; Carver, N.; Garvey, A.; Horling, B.; Neiman, D.; Podorozhny, R.; NagendraPrasad, M.; Raja, A.; Vincent, R.; Xuan, P.; Zhang, X.Q. (To appear). “Evolution of the GPGP/TAEMS Domain-Independent Coordination Framework.” Autonomous Agents and Multi-Agent Systems, Kluwer Academic Publishers.

• Zhang, XiaoQin; Lesser, Victor; Podorozhny, Rodion (To appear). “Multi-Dimensional, MultiStep Negotiation for Task Allocation in a Cooperative System.” Autonomous Agents and MultiAgent Systems (conditionally accepted for publication).

• Mailler, Roger; Lesser, Victor; and Horling, Bryan (2003). “Cooperative Negotiation for Soft Real-Time Distributed Resource Allocation.” Proceedings of Second International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2003), Melbourne, Australia, ACM Press, pp. 576–583.

• Sims, Mark; Goldman, Claudia; and Lesser, Victor (2003). “Self-Organization through Bottom Coalition Formation.” Proceedings of Second International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2003), ACM Press, Melbourne, Australia, pp. 867–874.

• Xiang, Yang; Lesser, Victor. (2003). “On the Role of Multiply Sectioned Bayesian Networks for Cooperative Multiagent Systems.” IEEE Transactions on Systems, Man, and Cybernetics, Part A, Vol. 33(4): 489–501.

91

Publications in Journals and Highly Refereed Conferences, Continued

• Zhang, X.Q.; Lesser, V.R.; Wagner, T. (2003). “Integrative Negotiation in Complex Organizational Agent Systems.” In Proceedings of the 2003 IEEE/WIC International Conference on Intelligent Agent Technology (IAT 2003), pp. 140–146.

• Zhang, X.Q.; Lesser, V.R.; Wagner, T. (2003). “A Two-Level Negotiation Framework for Complex Negotiations.” In Proceedings of the 2003 IEEE/WIC International Conference on Intelligent Agent Technology (IAT 2003), pp. 311–317.

• Zhang, XQ; Lesser, V.; and Abdallah, S. (2003). “Efficient Ordering and Parameterization of Multi-Linked Negotiation.” In Proceedings of Second International Joint Conference on Autonomous Agents and Multiagent Systems. (Extended abstract), ACM Press, pp. 1170-1171. Full version available as University of Massachusetts Computer Science Technical Report #02-42.

• Zhang, XQ; Lesser, V.; and Wagner, T. (2003). “A Multi-Leveled Negotiation Framework.” In Proceedings of Second International Joint Conference on Autonomous Agents and Multiagent Systems. (Extended abstract), ACM Press, pp. 1172-1173. Full version available as University of Massachusetts Computer Science Technical Report #02-44.

• Horling, B.; Neiman, D.; Podorozhny, R.; NagendraPrasad, M.; Raja, A.; Vincent, R.; Xuan, P.; Zhang, X.Q. (2002). “Evolution of the GPGP/TAEMS Domain-Independent Coordination Framework.” (Plenary Lecture/Extended Abstract). Proceedings of the 1st International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS’02), pp. 1-2. (Also available in the full version as University of Massachusetts Computer Science Technical Report 02-03.)

92

Journals and Highly Refereed Conferences, Continued

• Lesser, V. (2002). “Evolution of the GPGP/TÆMS Domain Independent Coordination Framework,” (Plenary lecture/Extended abstract.) Proceedings of the 1st International Conference on Autonomous Agents and Multi-Agent Systems, Part 1. ACM Press, pp. 1–2.

• Xuan, P. and Lesser, V. (2002). “Multi-Agent Policies: From Centralized Ones to Decentralized Ones.” Proceedings of the 1st International Joint Conference on Autonomous Agents and Multiagent Systems, Part 3: pp. 1098-1105. ACM Press.

• Zhang, X.Q.; Lesser, V.; Wagner, T. (2002). “Integrative Negotiation in Complex Organizational Agent Systems.” (Extended abstract.) In Proceedings of the 1st International Joint Conference on Autonomous Agents and Multiagent Systems, Part 1: pp. 503–504. ACM Press.

• Zhang, X.Q. and Lesser, V. (2002). “Multi-Linked Negotiation in Multi-Agent Systems.” Proceedings of the 1st International Joint Conference on Autonomous Agents and Multiagent Systems, Part 3: pp. 1207-1214. ACM Press.

• Horling, B.; Benyo, B.; Lesser, V. (2001). “Using Self-Diagnosis to Adapt Organizational Structures.” Proceedings of the Fifth International Conference on Autonomous Agents (Agents 2001), Montreal, ACM Press, pp. 529–536.

• Horling, B.; Vincent, R.; Mailler, R.; Shen, J.; Becker, R.; Rawlins, K.; Lesser, V. (2001). “Distributed Sensor Network for Real-Time Tracking.” Proceedings of the Fifth International Conference on Autonomous Agents (Agents 2001), Montreal, ACM Press, pp. 417-424.

• Raja, A.; Wagner, T.; Lesser, V. (2001). “Reasoning about Uncertainty in Agent Control.” In Proceedings of the Fifth International Conference on Information Systems, Analysis, and Synthesis, Computer Science and Engineering: Part 1, Volume VII, pp. 156-161, Orlando, FL. Received Best Paper Award for session on Mathematical Methods & Optimization in Problem Solving Systems II.

93

Highly Refereed Conferences, continued

• Vincent, R.; Horling, B.; Lesser, V.; Wagner, T. (2001). “Implementing Soft Real-Time Agent Control.” In Proceedings of the Fifth International Conference on Autonomous Agents (Agents 2001), Montreal, ACM Press, June 2001, pp. 355–362. (Honorable Mention in FIPA Software Prototypes Track Demonstration Competition.)

• Wagner, T.; Lesser, V. (2001). “Evolving Real-Time Local Agent Control for Large-Scale Multi-Agent Systems.” (Extended abstract) In Proceedings of the Fifth International Conference on Autonomous Agents, Montreal: ACM Press, pp. 17–18.

• Xuan, P.; Lesser, V.; Zilberstein, S. (2001). “Communication Decisions in Multi-agent Cooperation: Model and Experiments.” In Proceedings of the Fifth International Conference on Autonomous Agents (Agents 2001), Montreal, ACM Press, pp. 616–623.

• Raja, A.; Lesser, V.; Wagner, T. (2000). “Toward Robust Agent Control in Open Environments.” In Proceedings of the Fourth International Conference on Autonomous Agents (AA2000), Barcelona, pp. 84–91.

• Xiang, Y; Lesser, V. (2000). “Justifying Multiply Sectioned Bayesean Networks.” In Proceedings of the Fourth International Conference on Multi-Agent Systems (ICMAS), Boston, pp. 349–356.

• Xiang, Y; Lesser, V. (2000). “A Constructive Bayesian Approach for Vehicle Monitoring.” In Proceedings of the Third International Conference on Information Fusion (Fusion 2000). Vol. 2, pp. 14–21, Paris.

• Zhang, X.Q.; Podorozhny, R.; Lesser, V. (2000). “Cooperative, MultiStep Negotiation Over a Multi-Dimensional Utility Function.” In Proceedings of the IASTED International Conference, Artificial Intelligence and Soft Computing (ASC), Banff, IASTED/ACTA Press, pp. 136–142.

94

Workshops, Symposia, Collected Volumes & Technical Reports

• Horling, Bryan; Mailler, Roger; Sims, Mark; and Lesser, Victor (2003). “Using and Maintaining Organization in a Large-Scale Distributed Sensor Network.” In Proceedings of the Workshop on Autonomy, Delegation, and Control (AAMAS03), Melbourne, Australia, July.

• Horling, Bryan; Mailler, Roger; and Lesser, Victor. (2003) “Farm: A Scalable Environment for Multi-Agent Development and Evaluation.” Proceedings of Second International Workshop on Software Engineering for Large-Scale Multi-Agent Systems (SELMAS), pp. 171–177.

• Mailler, Roger; and Lesser, Victor. (2003) “A Mediation-Based Protocol for Distributed Constraint Satisfaction.” In The Fourth International Workshop on Distributed Constraint Reasoning, Acapulco, Mexico, pp. 49-58.

• Horling, Bryan; Lesser, Victor; Vincent, Regis; Wagner, Thomas (2002). “The Soft Real-Time Agent Control Architecture.” In Proceedings of the AAAI/KDD/UAI-2002 Joint Workshop on Real-Time Decision Support and Diagnosis Systems. Technical Report WS-02-15, pp. 54-65. (Also available as University of Massachusetts Computer Science Technical Report 02-14.)

• Lesser, V.; Decker, K.; Wagner, T.; Carver, N.; Garvey, A.; Raja, A.; Lesser, V. (2002). “Meta-Level Control in Multi-Agent Systems.” In Proceedings of AAAI/KDD/UAI-2002 Joint Workshop on Real-Time Decision Support and Diagnosis Systems, Technical Report WS-02-15, pp. 47-53. (Also available as University of Massachusetts Computer Science Technical Report 01-49.)

• Mailler, R., Vincent, R., Lesser, V., Middlekoop, T., and Shen, J. (2001). “Soft-Real Time, Cooperative Negotiation for Distributed Resource Allocation.” In Proceedings of the AAAI Fall Symposium on Negotiation Methods for Autonomous Cooperative Systems, Falmouth, MA.

95

Workshops, Symposia, Collected Volumes & Technical Reports, Continued

• Raja, A.; Lesser, V. (2001). “Towards Bounded-Rationality in Multi-Agent Systems.” In University of Massachusetts Computer Science Technical Report 01-34.

• Vincent, R.; Horling, B.; Lesser, V. (2001). “An Agent Infrastructure to Build and Evaluate Multi-Agent Systems: The Java Agent Framework and Multi-Agent System Simulator.” Lecture Notes in Artificial Intelligence 1887: Infrastructure for Agents, Multi-Agent Systems, and Scalable Multi-Agent Systems. Wagner & Rana (eds.), Springer, pp. 102–127.

• Zhang, X.Q.; Lesser, V.; Podorozhny, P. (2001). “New Results on Cooperative, MultiStep Negotiation Over a Multi-Dimensional Utility Function.” Proceedings of the AAAI Fall Symposium on Negotiation Methods for Autonomous Cooperative Systems.

• Zhang, X.Q.; Lesser, V.; Wagner, T. (2001). “A Proposed Approach to Sophisticated Negotiation.” Proceedings of the AAAI Fall Symposium on Negotiation Methods for Autonomous Cooperative Systems.

• Horling, B.; Benyo, B.; Lesser, V. (2000). “Using Self-Diagnosis to Adapt Organizational Structures.” (Extended abstract) In The Fourth International Conference on MultiAgent Systems (ICMAS), Boston, MA: IEEE Computer Society, pp. 397-398. Also available as University of Massachusetts/Amherst Computer Science Technical Report #1999-64.

• Raja, A.; Wagner, T.; Lesser, V. (2000). “Reasoning about Uncertainty in Design-to-Criteria Scheduling.” Proceedings of AAAI 2000 Spring Symposium on Real-Time Autonomous Systems, pp. 76–83, Stanford, CA.

96

Workshops, Symposia, Collected Volumes & Technical Reports, Continued

• Wagner, T.; Benyo, B.; Lesser, V.; Xuan, P. (2000). “Investigating Interactions Between Agent Conversations and Agent Control Components.” Issues in Agent Communication, Vol. 1916, F. Dignum & M. Greaves (eds.), Berlin: Springer-Verlag, pp. 314–331.

• Wagner, Thomas (2000). “Toward Quantified Control for Organizationally Situated Agents.” University of Massachusetts/Amherst, Department of Computer Science, Ph.D. Thesis, February.

• Wagner, T.; Lesser, V. (2000). “Design-to-Criteria Scheduling: Real-Time Agent Control.” Proceedings of AAAI 2000 Spring Symposium on Real-Time Autonomous Systems, pp. 89–96. Also available as University of Massachusetts/ Amherst Computer Science Technical Report #1999-58.

• Wagner, T; Lesser, V. (2000). “State-based Control for Organizationally Situated Agents.” (Extended abstract) In Proceedings of the Fourth International Conference on Multi-Agent Systems (ICMAS), Boston, MA: AAAI Press, pp. 457-458. Also available as University of Massachusetts/Amherst Computer Science Technical Report #1999-68.

• Xuan, P.; Lesser, V.; Zilberstein, S. (2000). “Communication in Multi-Agent Markov Decision Processes.” (Extended abstract) In Proceedings of the Fourth International Conference on Multi-Agent Systems (ICMAS), Boston, MA: AAAI Press, pp. 467-468.