IAT 2006IAT 2006 11

Heuristics for Dealing with a Heuristics for Dealing with a Shrinking Pie in Agent Coalition Shrinking Pie in Agent Coalition

FormationFormation

Kevin Westwood – Utah State UniversityKevin Westwood – Utah State University

Vicki Allan – Utah State UniversityVicki Allan – Utah State University

IAT 2006IAT 2006

22IAT 2006IAT 2006

Multi-Agent CoalitionsMulti-Agent Coalitions

““A coalition is a set of agents that work together A coalition is a set of agents that work together to achieve a mutually beneficial goal” (Klusch to achieve a mutually beneficial goal” (Klusch and Shehory, 1996)and Shehory, 1996)

Reasons agent would join CoalitionReasons agent would join Coalition Cannot complete task aloneCannot complete task alone Complete task more quicklyComplete task more quickly

33IAT 2006IAT 2006

Skilled Request For Proposal Skilled Request For Proposal (SRFP) Environment(SRFP) Environment

Inspired by RFP (Kraus, Shehory, and Taase 2003)Inspired by RFP (Kraus, Shehory, and Taase 2003)

Provide set of tasks T = {TProvide set of tasks T = {T11…T…Tii…T…Tnn} } Divided into multiple subtasksDivided into multiple subtasks

requiring skill/levelrequiring skill/level

Has a payment value V(THas a payment value V(Tii) )

Service Agents, A = {AService Agents, A = {A11…A…Akk…A…App}} Associated cost fAssociated cost fkk

skill/levelskill/level

Manager AgentManager Agent Distributes tasks to service agentsDistributes tasks to service agents

44IAT 2006IAT 2006

Auctioning ProtocolAuctioning Protocol

Variation of a reverse auctionVariation of a reverse auction Agents compete for opportunity to perform servicesAgents compete for opportunity to perform services Efficient way of matching goods to servicesEfficient way of matching goods to services

Central ManagerCentral Manager1)1) Randomly orders AgentsRandomly orders Agents2)2) Each agent gets a turnEach agent gets a turn

Accepts previous offer or Proposes Accepts previous offer or Proposes

3)3) Coalitions are awarded taskCoalitions are awarded task

Multiple Rounds {0,…,rMultiple Rounds {0,…,rzz}} Our version is cyclic – so agents later in list are not Our version is cyclic – so agents later in list are not

disadvantageddisadvantaged

55IAT 2006IAT 2006

Cyclic Cyclic AuctionAuction

Service Agent 6

Manager Agent

Service Agent 5

Service Agent 4

Service Agent 3

Service Agent 2

Service Agent 7

Service Agent 1

Service Agent 8

1:

9: 2:

4:

5:

Acc

ep

ts (

3,

2,

5)

10:

Acc

epts

(3, 2

, 5)

3: Proposes Coalition (3, 2, 5) 11: Task Awarded to (3, 2, 5)

6:

7:

8:

66IAT 2006IAT 2006

05

10152025

3035

Agent 3 Agent 2 Agent 5

Costs

Agent Cost Profit

0

5

10

15

20

25

30

35

Agent 11 Agent 2 Agent 5

Costs

Agent Cost Profit

Agent costAgent costAgent costs deviate from base cost Agent costs deviate from base cost Base cost derived from skill and skill levelBase cost derived from skill and skill level

Agent paymentAgent paymentcost + proportional portion of net gaincost + proportional portion of net gain

77IAT 2006IAT 2006

DecisionsDecisions

How do I decide whether to How do I decide whether to accept?accept?

If I make an offer…If I make an offer…

What task should I propose doing?What task should I propose doing?

What other agents should I What other agents should I recruit?recruit?

88IAT 2006IAT 2006

Coalition Calculation AlgorithmsCoalition Calculation AlgorithmsCalculating all possible coalitionsCalculating all possible coalitions Requires exponential timeRequires exponential time Not feasible in most problems in which Not feasible in most problems in which

tasks/agents are entering/leaving the system tasks/agents are entering/leaving the system and values of tasks are shrinking over timeand values of tasks are shrinking over time

Divide into two stepsDivide into two steps1) Task Selection 1) Task Selection 2) Other Agents Selected for Team2) Other Agents Selected for Team polynomial time algorithmspolynomial time algorithms

99IAT 2006IAT 2006

Task SelectionTask Selection Individual Profit – obvious, greedy Individual Profit – obvious, greedy

approachapproach

Competitive: best for meCompetitive: best for me

Why not always be greedy?Why not always be greedy?Others may not accept – your membership is Others may not accept – your membership is questionedquestioned

Individual profit may not be your goalIndividual profit may not be your goal Global ProfitGlobal Profit Best Fit Best Fit Co-opetitiveCo-opetitive

1010IAT 2006IAT 2006

Two Step Coalition CalculationTwo Step Coalition CalculationTask SelectionTask Selection Individual Profit Individual Profit Global Profit – somebody should do this taskGlobal Profit – somebody should do this task

I’ll sacrificeI’ll sacrifice

Wouldn’t this always be a noble thing to Wouldn’t this always be a noble thing to do?do?

Task might be better done by othersTask might be better done by others

I might be more profitable elsewhereI might be more profitable elsewhere

Best Fit – uses my skills wiselyBest Fit – uses my skills wisely Co-opetitiveCo-opetitive

1111IAT 2006IAT 2006

Two Step Coalition CalculationTwo Step Coalition CalculationTask SelectionTask Selection Individual Profit Individual Profit Global Profit Global Profit Best Fit – Best Fit – Cooperative: uses skills wiselyCooperative: uses skills wisely

Perhaps no one else can do itPerhaps no one else can do it

Maybe it shouldn’t be doneMaybe it shouldn’t be done

Co-opetitiveCo-opetitive

1212IAT 2006IAT 2006

Co-opetitive AgentCo-opetitive Agent

Co-opetitionCo-opetition Phrase coined by business professors Phrase coined by business professors

Brandenburger and Nalebuff (1996),Brandenburger and Nalebuff (1996), to to emphasize the need to consider both emphasize the need to consider both competitive and cooperative strategies.competitive and cooperative strategies.

Co-opetitive Task SelectionCo-opetitive Task Selection Select the best fit task if profit is within P% of Select the best fit task if profit is within P% of

the maximum profit availablethe maximum profit available

1313IAT 2006IAT 2006

Costs by LevelCosts by Level

0

10

20

30

40

50

60

0 1 2 3 4 5 6 7 8 9 10

Skill Level

Co

sts

1414IAT 2006IAT 2006

What about accepting offers?What about accepting offers?

Compare to what you could Compare to what you could achieve with a proposalachieve with a proposalWorry about shrinking pieWorry about shrinking pie Utility gets smaller as the time Utility gets smaller as the time

to form a coalition increasesto form a coalition increases

Compare best proposal with Compare best proposal with best offerbest offerUse utility based on agent Use utility based on agent typetype

1515IAT 2006IAT 2006

When an offer is received…When an offer is received…

Compare best proposal with best offerCompare best proposal with best offer

Use utility based on agent typeUse utility based on agent type

Four acceptance policiesFour acceptance policies Expected Utility, discount awareExpected Utility, discount aware Expected Utility, discount unawareExpected Utility, discount unaware MonetaryMonetary CompromisingCompromising

1616IAT 2006IAT 2006

Probability of Probability of acceptance*utility +acceptance*utility +

Probability of rejection Probability of rejection *future utility*future utility

Other agents – must Other agents – must estimate probabilityestimate probability

Desperation Desperation Empathy Empathy Interaction HistoryInteraction History

Proposal

Proposer-me

Unknown 1

Unknown 2

Offer

Proposer

Me

Unknown

Expected UtilityExpected Utility

1717IAT 2006IAT 2006

Four acceptance policiesFour acceptance policies

1.1. Expected Utility, discount awareExpected Utility, discount aware future utility: probabilities, discount, time to close future utility: probabilities, discount, time to close

dealdeal accepts an offer it is as good as it can expectaccepts an offer it is as good as it can expect

2.2. Expected Utility, discount unawareExpected Utility, discount unaware future utility same as currentfuture utility same as current

3.3. MonetaryMonetary wants the highest profit, won’t accept lesswants the highest profit, won’t accept less

4.4. CompromisingCompromising accepts if offer is within 10% of bestaccepts if offer is within 10% of best

1818IAT 2006IAT 2006

Scenario 1 – Bargain BuyScenario 1 – Bargain Buy

Store “Bargain Buy” Store “Bargain Buy” advertises a great priceadvertises a great price

300 people show up300 people show up

5 in stock5 in stock

Everyone sees the Everyone sees the advertised price, but it just advertised price, but it just isn’t possible for all to isn’t possible for all to achieve itachieve it

1919IAT 2006IAT 2006

Scenario 2 – selecting a spouseScenario 2 – selecting a spouse

Bob knows all the Bob knows all the characteristics of the characteristics of the perfect wifeperfect wife

Bob seeks out such a Bob seeks out such a wifewife

Why would the perfect Why would the perfect woman want Bob?woman want Bob?

2020IAT 2006IAT 2006

Scenario 3 – hiring a new PhDScenario 3 – hiring a new PhD

Universities ranked 1,2,3Universities ranked 1,2,3

Students ranked a,b,cStudents ranked a,b,c

Dilemma for second tier Dilemma for second tier universityuniversity

offer to “a” studentoffer to “a” student

likely rejectedlikely rejected

delay for acceptancedelay for acceptance

““b” students are goneb” students are gone

2121IAT 2006IAT 2006

Test SetupTest Setup

40 Tasks40 Tasks 3 Subtasks each3 Subtasks each Skills, 1-10Skills, 1-10 Skill levels, 1-10Skill levels, 1-10 Payment – (100-200%) of base costPayment – (100-200%) of base cost

60 Agents60 Agents Matched to tasks or RandomMatched to tasks or Random Agent base costs (5,10,…50) based on skill levelAgent base costs (5,10,…50) based on skill level 4 agent types 4 agent types

5000 tests5000 tests

2222IAT 2006IAT 2006

Shows global profit ratio: profit achieved/system optimalShows global profit ratio: profit achieved/system optimalWhen Discount is greater than 50, there is likely no second round – When Discount is greater than 50, there is likely no second round – curve flattens as shows what is achieved in one roundcurve flattens as shows what is achieved in one roundAware and unaware similar for low discountsAware and unaware similar for low discountsMonetary worseMonetary worseCompromise 90% is the best for low discountsCompromise 90% is the best for low discounts

Profit Ratio

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Discount

Pro

fit

Rat

io

Exp Utility (Disc Aware) Exp Utility (Disc Unaware) Monetary Compromising 90%

2323IAT 2006IAT 2006

Why?Why?Monetary just too idealistic. “Bargain Buy” Monetary just too idealistic. “Bargain Buy” may not really be possible.may not really be possible.

Discount aware/unaware not much Discount aware/unaware not much different when low discounts.different when low discounts.

Compromising 90% works wellCompromising 90% works well picking a spouse. Others know my worth.picking a spouse. Others know my worth. bargain buy: bargain may not be possiblebargain buy: bargain may not be possible hiring a PhD. Shooting too high can backfire.hiring a PhD. Shooting too high can backfire. Others are as smart as you are!Others are as smart as you are!

2424IAT 2006IAT 2006

Tasks per round – Tasks per round – obvious trend: higher discount obvious trend: higher discount →→earlier acceptanceearlier acceptance

Tasks Per Round (Exp Utility, Discount Aware)

0

1

2

3

4

5

6

1 2 3 4 5 >=6

Round

Tas

ks

30% 20% 10% 5% 2.50%

2525IAT 2006IAT 2006

Big surprise – discount unawareBig surprise – discount unawaredoes not consider, but sees discountdoes not consider, but sees discount

Tasks Per Round (Exp Utility, Discount unaware)

0

1

2

3

4

5

6

1 2 3 4 5 >=6

Round

Tas

ks

30% 20% 10% 5% 2.50%

2626IAT 2006IAT 2006

How Discount affects choicesHow Discount affects choices

Offers (some from previous round)

Possible Tasks

Even though agent doesn’t compute discount, sees discount comparing choices from two rounds

2727IAT 2006IAT 2006

Monetary agentsMonetary agentslower tasks lower tasks 11stst round round

later later complete complete moremore

good deals good deals gone gone →→ more more reasonable reasonable expectationsexpectations

Tasks Per Round (Monetary)

0

1

2

3

4

5

6

1 2 3 4 5 >=6

Round

Tas

ks

30% 20% 10% 5% 2.50%

2828IAT 2006IAT 2006

ConclusionsConclusions

Situation is complicatedSituation is complicated

Shrinking occurs because of discount Shrinking occurs because of discount

Shrinking also occurs as agents and tasks Shrinking also occurs as agents and tasks form coalitions and leaveform coalitions and leave

Knowing best “possible” may be Knowing best “possible” may be misleadingmisleading

2929IAT 2006IAT 2006

Questions?Questions?

3030IAT 2006IAT 2006

Coalition SelectionCoalition Selection

Best Profit – pick other agents to maximize Best Profit – pick other agents to maximize total profit. (Also maximizes local profit, total profit. (Also maximizes local profit, because of way profits are divided)because of way profits are divided)

Best Fit – pick other agents to use their Best Fit – pick other agents to use their skills well (not pick a more qualified agent skills well (not pick a more qualified agent if it happens to be cheaper)if it happens to be cheaper)

3131IAT 2006IAT 2006

How do we pair the task and How do we pair the task and coalition selection methods?coalition selection methods?

Individual Individual Profit Profit

Global ProfitGlobal Profit

Co-opetitiveCo-opetitive

Best FitBest Fit

•Best Profit Coalition formation

•Best Fit coalition selection

3232IAT 2006IAT 2006

Global tasks completed – same Global tasks completed – same patternpattern

Tasks Completed

4

5

6

7

8

9

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Discount

Task

s

Exp Utility (Disc Aware) Exp Utility (Disc Unaware) Monetary Compromising 90%

3333IAT 2006IAT 2006

Accepting First ProposalAccepting First ProposalUtility Lost From Not Accepting First Proposal

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

2.5% 5% 7% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Discount

Uti

lity

Reject First Accept First

3434IAT 2006IAT 2006

Accept First ProposalAccept First Proposal

Mixture of agent types, but only Aware Mixture of agent types, but only Aware acceptance policy.acceptance policy.

Measure what achieved when first Measure what achieved when first proposal is accepted. proposal is accepted.

Measure what achieved when first Measure what achieved when first proposal is not acceptedproposal is not accepted

3535IAT 2006IAT 2006

What does it mean?What does it mean?

Competitive Proposal - could be accepted by Aware Competitive Proposal - could be accepted by Aware agentagent

your estimation of worth matches others’ estimation. your estimation of worth matches others’ estimation. good pricegood price demand for skilldemand for skill proposer picked that task and you over all other choicesproposer picked that task and you over all other choices

Likely get another proposal if first failsLikely get another proposal if first fails

Not about whether or not you should accept first, but Not about whether or not you should accept first, but “Agents who are competitive enough to receive a “Agents who are competitive enough to receive a strong first offer are competitive enough to do well.”strong first offer are competitive enough to do well.”