negotiation course 090525.pptannette/siks2009/material/slideskoenhindriks… · frontier =f( ,d) ....
TRANSCRIPT
Bilateral NegotiationAn overview of the theory of negotiation & negotiating software agents
Vught, SIKS Basic Course Learning & Reasoning
May 25, 2009
1
Koen Hindriks
Vermelding onderdeel organisatieMan-Machine Interaction Group, EEMCS
Outline
• A Classic Tale: Negotiating About an Orange
• Preferences and Utility Theory• Preferences and Utility Theory
• Bargaining Models and Solution Concepts
• Strategies and Concession Tactics
• Opponent Modeling & Learning
• Other Topics, or: What I Did Not Talk About
• The Future of Automated Negotiationg
• Negotiation Session
December 11, 2007 2
A Classic Tale:*Two Sisters Arguing Over an Orange*
Two sisters arguing over a single available orange:
I should have that orange. It's mine. No, I should have
it. It's mine.
I’m older, so I should have it. No, I should have it
because I’m younger.
Key Concepts:
y g
• Conflict of interests• Distributional negotiation problem• Positional bargaining
December 11, 2007 3
* The story about the sisters' conflict over the orange has been attributed to Mary Parker Follett. See Deborah M. Kolb, 1995, The Love for Three Oranges, Or: What Did We Miss about Ms. Follett in the Library? 11 Negotiation J. 339. Also see: Roger Fisher & Danny Ertel, 1995, Getting Ready to Negotiate.
A Classic Tale:Two Sisters Arguing Over an OrangeTwo sisters arguing over a single available orange:
11
t si
ster
Oldest sister gets
orange
Key Concepts: Uti
lity
olde
st
No one gets
Youngest sister gets
• Outcome space (Ω)• Status Quo• Utility 10 Utility youngest sister
gorange orange
December 11, 2007 4
A Classic Tale:Two Sisters Arguing Over an OrangeThe arguments are not decisive, but there is an alternative:
This is going nowhere.
I agree.
Let’s compromise.Let’s split the orange.
Key Concepts:Key Concepts:• Making a concession, compromise• Expand outcome space• Fair outcome
December 11, 2007 5
A Classic Tale:Two Sisters Arguing Over an OrangeThe arguments are not decisive, but there is an alternative:
11
t si
ster Split the
orange
Key Concepts: Uti
lity
olde
st
Key Concepts:• Pareto optimal outcome• Pareto frontier 10 Utility youngest sister
December 11, 2007 6
A Classic Tale:Two Sisters Arguing Over an OrangeBut why do both sisters want the orange?
I want the peel for an orange marmalade I
am making for a school cooking project
I want to have the juice for a refreshing b kf bbreakfast beverage.
You can have the juice. You can have
the peel.
K C tKey Concepts:• Underlying interests• Exchange of information
December 11, 2007 7
A Classic Tale:Two Sisters Arguing Over an OrangeBut why do both sisters want the orange?
11
t si
ster One gets peel,
other gets juice
Key Concepts: Uti
lity
olde
st
• Inefficient outcome• Pareto dominated• Utopia, or Ideal point 10 Utility youngest sister
December 11, 2007 8
Summary• A negotiation is aimed at resolving a conflict of interests.• Each negotiation is defined by its outcome space. Ideally, an
t i h d th t i P t ti lagreement is reached that is Pareto optimal.• During a negotiation, the outcome space is explored, and
sometimes, may be expanded to find creative agreements based th d l i i t ton the underlying interests.
Literature:• H. Raiffa, 1982, The Art and Science of Negotiation, Belknap Press• L.L. Thompson, 2005 (3rd ed.), The Mind and Heart of the
Negotiator, Prentice HallNegotiator, Prentice Hall
December 11, 2007 9
Preferences and Utility TheoryVon Neumann-Morgenstern Utility Theory
December 11, 2007
10
Vermelding onderdeel organisatie
Where Do Utilities Come From?
I i d h h f f• It is assumed that agent have preferences for outcomes, and thus are not completely indifferent about various outcomes.
• If preferences of an agent are both rational as well as well-behaved in a precise sense, then it can be shown that a utility function can be constructed that can be used for guiding the decision-making of that agent.g g g g
December 11, 2007 11
Predictable versusUnpredictable Preferences
Predictable preferencesM t l f t h
Unpredictable preferencesI lik bl th d• Most people prefer to have
more rather than less money.• I like blue more than red, you
like red more than blue.
utility utility
money red blue green …
Monotonic preferenceLogarithmic curve for money
Varies from person to personNot quite as well-behavedLogarithmic curve for money Not quite as well behaved
• Higher > Lower Grade• Less > More Work
• Peugeot > Ford• Swim > Cycle Ride
December 11, 2007 12
Rational Preferences
Preferences are complete:
Preferences can be modeled by a (binary) preference relation . A preference relation is called rational if it is complete and transitive.
Preferences are complete:• It is assumed that an agent can state his preferences when
confronted with two arbitrary potential outcomes.• Formally for all ω ω’∈Ω: either ω ω’ or ω’ ω or ω~ω’• Formally, for all ω,ω∈Ω: either ω ω , or ω ω, or ω~ω .
Preferences are transitive:• If an outcome ω1 is preferred over
Intransitive Preferences& Money PumpsIf an outcome ω1 is preferred over
ω2, and ω2 is preferred over ω3,then ω1 is preferred over ω3.F ll f ll Ω
A1 Euro1 Euro
• Formally, for all ω1, ω2, ω3∈Ω:ω1 ω2 and ω2 ω3 ω1 ω3.
C B1 Euro
December 11, 2007 13
Modeling Risky Outcomes: Lotteries
• A lottery associates chances or probabilities with outcomes.N t ti L [ ]
Outcomes that involve risk can be modeled by lotteries. For each lottery, it is assumed there is a certainty-equivalent outcome.
Notation: L = [p1, ω1 ; … ; pn, ωn].
Preferences are continuous:• If an outcome ω’ is between ω and ω’’ in preference, then there is p ,
a probability p such that an agent is indifferent between the lottery [p, ω; 1-p, ω’’] and getting ω’ for sure.*
• Formally, if ω ω’ ω’’, then there is a p s.t. [p, ω; 1-p, ω’’] ~ ω’.
Example: Deal or No Deal.• For a lottery involving monetary prizes, e.g.
[¼ 20; ¼ 100; ¼ 20 000; ¼ 40 000][¼, 20; ¼, 100; ¼, 20.000; ¼, 40.000]there is an amount of money, e.g. 5.000, such that:
[¼, 20; ¼, 100; ¼, 20.000; ¼, 40.000]~5.000.December 11, 2007 14
* ω’ is called the certainty-equivalent outcome for lottery [p, ω; 1-p, ω’’].
Preferences are Monotone
Preferences are monotone:
Preferences are assumed to be monotone. Intuitively, this assumption expresses that getting more of what you want is preferred over less.
Preferences are monotone:• If an outcome ω is preferred over ω’, then an agent prefers a
lottery with outcomes ω, ω’ that has a higher chance of getting ωover one with a lower chance of getting ω.over one with a lower chance of getting ω.
• Formally, [p, ω; (1-p), ω’] [q, ω; (1-q), ω’] whenever p≥q.
Exceptions to monotonicity?Exceptions to monotonicity?Most people prefer life to death, but still some people like to exercise dangerous sports, such as BASE jumping, heli-skiing, diving, mountaineering, big wave surfing and bull riding. The g, g, g g gchances of getting killed are significantly higher than staying at home listening to music, drinking a beer, or reading a good book.
December 11, 2007 15
Equally Valued Outcomes
Equally Valued Outcomes can be substituted:If t i i diff t b t d ’ th th t t i
Equally valued outcomes may be substituted for each other.
• If an agent is indifferent between ω and ω’, then that agent is indifferent between a lottery that involves outcome ω over that same lottery with ω’ substituted for ω.F ll [ A 1 C] [ B 1 C] h A B• Formally, [p, A; 1-p, C] ~ [p, B; 1-p, C] whenever A~B.
The intuition here is that indifference with respect to two outcomes is independent from preferences regarding other outcomesoutcomes.
December 11, 2007 16
No Fun in Gambling
Outcomes can be decomposed:
Lotteries that involve other lotteries can be reduced to simpler, non-composed lotteries.
Outcomes can be decomposed:• Compound lotteries can be reduced using the laws of probability.
E.g., [p, A; [q, B; 1-q, C]] ~ [p, A; (1-p)q, B; (1-p)(1-q), C]
The axiom has also been paraphrased as stating that there can be“no fun in gambling”. That is, an agent is indifferent between:g g , g• Playing a lottery in which prizes are tickets for a 2nd lottery,• Playing a single lottery that directly pays out cashprovided off course chances of winning cash are the sameprovided, off course, chances of winning cash are the same.
December 11, 2007 17
Utility Theory
Th
The axioms of utility theory guarantee the existence of a utility functionU: Ω ℜ that represents a rational, well-behaved preference relation.
Theorem.Suppose a preference relation satisfies all conditions listed above.Then there exists a utility function U:Ω ℜ such that:
U(ω)≥ U(ω’) iff ω ω
Example: Deal or No Deal:Example: Deal or No Deal:• Suppose [¼, 20; ¼, 100; ¼, 20.000; ¼, 40.000]~5.000 and
the preference relation satisfies all conditions above. Then the existence of a utility function U is guaranteed such that:the existence of a utility function U is guaranteed such that:
• ¼U(20)+¼U(100)+ ¼U(20.000)+ ¼U(40.000)= U(5.000).
December 11, 2007 18
Buying a SupercomputerIn practice, in complex negotiations, it is hard to construct a good utility function that matches the preferences of a party.
NCF/SARA negotiating about a HPC with IBMStep 2: Identifying preferences & utility function
Step 1: Identifying the main issues
Issues Value Space NCFWeights
IBMWeights
PlanningDelivery dates6.2.1 System architectureNumber of TFlops6.2.3 External and Internal Networking
NCF IBM
Step 2: Identifying preferences & utility function
gInterconnect Network6.2.4 StorageTotal storage6.2.5 SoftwareScheduler6.2.7 Installation (Operatonal Aspects Section C)Location6.2.6 Maintenance and support - Related to SARATraininggStartup: Onsite SupportStrategicUse of Solution as a showcase for IBMTCO (NCF) / Quote (IBM)Quote EuroMaintenance EuroTotal Costs Euro
Solution UtilityOverall NCF Utility
December 11, 2007 19
Summary• A preference relation is a binary relation over an outcome
space that models which outcomes are preferred over others.P f ti l if th t iti d l t• Preferences are rational if they are transitive and complete.
• A unique (Von Neumann-Morgenstern) utility function exists that models rational preferences that additionally satisfy continuity,
t i it b tit t bilit d d bilitmonotonicity, substitutability, and decomposability.
Literature:• R.D. Luce and H. Raiffa, 1957, Games and Decisions: Introduction
and a Critical Survey, Dover Publications• R.L. Keeney and H. Raiffa, 1976, Decisions with MultipleR.L. Keeney and H. Raiffa, 1976, Decisions with Multiple
Objectives, Cambridge University Press
December 11, 2007 20
Bargaining Models andS l i CSolution ConceptsAxiomatic Foundations for Negotiation
December 11, 2007
21
Vermelding onderdeel organisatie
Abstract Model of Negotiation
A b i i bl i i
Two parties that negotiate need to make a joint decision to settle on a final outcome. Can we predict the outcome that such agents will agree on?
Dictatorial Utopia, or• A bargaining problem is a pair ⟨Ω, d⟩ with Ω an outcome space, and d the disagreement point.
Dictatorialsolution
Utopia, orIdeal point
• A solution for bargaining problems of the form ⟨Ω, d⟩ is an outcome ω=F(Ω d)∈Ω of
age
nt
2 ParetoOptimalFrontierOutcome spaceω=F(Ω,d)∈Ω.
• F(Ω,d) can be interpreted as a prediction, or recommendation,
Uti
lity
o p= feasible region,
bounded byaxes and
Pareto frontier
p ed ct o , o eco e dat o ,for the problem ⟨Ω, d⟩. Utility of agent 1
Disagreement point,or conflict outcome
December 11, 2007 22
Which Solution Should We Select?
Ch i d i bl i f l i• Characterize desirable properties of solution.
• Ideally but not necessarily this would give a unique• Ideally, but not necessarily, this would give a unique solution point for each bargaining problem.
• Axiomatic approaches to define properties.• We discuss the axioms proposed by Nash (1950).
December 11, 2007 23
Axiom 1:All Gains Should Be Exhausted
P O i li• Pareto-Optimality:A solution ω=F(Ω,d) should bePareto optimal i e such thatPareto-optimal, i.e. such thatthere is no ω’∈Ω and ω’> ω.
Most solution concepts proposed satisfy this property.*
December 11, 2007 24
* The Egalitarian solution satisfies weak Pareto optimality, i.e. there is no outcome that is preferred by all negotiators to the Egalitarian solution. A natural extension of this solution concept that does satisfy Pareto optimality exists.
Axiom 2:Fully Symmetric Bargaining Problems
S• Symmetry:If negotiators have completelysymmetric roles the solutionsymmetric roles, the solutionshould yield them equal utility,i.e. the solution F(Ω,d)=⟨x1, x2⟩i.e. the solution F(Ω,d) ⟨x1, x2⟩should satisfy x1=x2.
Does not allow for external factors that are not modeled.
December 11, 2007 25
Axiom 3Scale Invariance
S l I i• Scale Invariance:The solution should be invariantunder “scale transformations” i eunder scale transformations , i.e.for all positive linear transformationsλ we have λ(F(Ω,d))=F(λ(Ω), λ(d))λ we have λ(F(Ω,d)) F(λ(Ω), λ(d))
Does not allow for interpersonal comparison of utilities.
December 11, 2007 26
Axiom 4Independence of Irrelevant Alternatives
C t ti I d d• Contraction Independence:If Ω’⊆Ω and F(Ω,d)∈Ω’, then F(Ω’,d)=F(Ω,d).then F(Ω ,d) F(Ω,d).
Does not allow to take into account that one negotiator hasDoes not allow to take into account that one negotiator hasmore or less options available to choose from in his favor.
December 11, 2007 27
The Nash Solutionto a Bargaining Problem
I b h h h i i h• It can be shown that there is a unique outcome that satisfies all axioms proposed by Nash, called the Nash solution.
• The Nash solution is that outcome ω=⟨x1, x2⟩ ∈Ωthat maximizes x1⋅x2.
December 11, 2007 28
Many Alternative Solutions Proposed
B t K S l ti M Oth S l tiBest Known Solutions:• Nash Solution:
outcome ω=⟨x1, x2⟩ that maximizes x1⋅x2
Many Other Solutions:•Dictatorial Solution•Raiffa Solution
• Kalai-Smorodinsky Solution:Pareto-optimal outcome ω on line that
t di t d id l i t
•Kalai-Rosenthal Solution•Lexicographic Egalitarian•Perles-Maschler Solutionconnects disagreement and ideal point
• Egalitarian Solution:Weak Pareto-optimal outcome ω=⟨x1 x2⟩
Perles Maschler Solution•The Equal Area Solution•The Equal Loss SolutionTh Utilit i S l tiWeak Pareto optimal outcome ω=⟨x1, x2⟩
such that x1=x2•The Utilitarian Solution•The Yu Solution
December 11, 2007 29
Summary• A bargaining problem consists of an outcome space and a
disagreement pointA l ti t b i i bl d i• A solution to a bargaining problem recommends a unique outcome as the final agreement
• Nash proposed to axiomatically characterize solution concepts• Many solution concepts proposed in literature
Literature:• R.D. Luce and H. Raiffa, 1957, Games and Decisions: Introduction
and a Critical Survey, Dover Publications• W. Thomson, 1994, Cooperative Models of Bargaining, in:W. Thomson, 1994, Cooperative Models of Bargaining, in:
Handbook of Game Theory, Volume 2, Elsevier
December 11, 2007 30
Strategies and Concession TacticsAlternating Offers Protocol with Incomplete Information
December 11, 2007
31
Vermelding onderdeel organisatie
Joint Exploration of Outcome Space
A i ti h t i ll h t i ibl t f
The Nash solution, and, more importantly, the Pareto frontier, cannot be computed when both parties only have incomplete preference information.
• Axiomatic approaches typically characterize possible outcomes of a negotiation, assuming complete information.
• In closed negotiations negotiators at best have partial• In closed negotiations, negotiators at best have partial information about their opponent’s preferences.
• For a good reason: Revealing information to an opponent may g g pp yresult in exploitation.
• Reaching an agreement requires negotiators with incomplete information to jointly explore the outcome space by exchanging offers, i.e. outcomes proposed to the other party.
December 11, 2007 32
Fundamental Problem of Negotiation
Wh t i th ti l t t t hi th f ll i
A dilemma faced by any negotiator is how to simultaneously achieve two typically conflicting goals: maximize own outcome and chance of a deal.
What is the optimal strategy to achieve the following:
• Maximize own outcome.Requires outcome with high own utility, i.e. that is individually rational. Rule of Thumb: Start negotiation and check whether offer that is best for self is acceptable.
• Maximize chance of reaching an agreement.Requires outcome with acceptable utility for opponent, i.e.
l i th fli t f i t tresolving the conflict of interest.
December 11, 2007 33
Negotiation Decision functions
Diff t f ili f
The complexity of negotiation with incomplete information does not allow for establishing game-theoretic optimal equilibrium strategy results. Instead negotiation heuristics are proposed in the literature.
Different families ofnegotiation heuristics:
Ti d d t t ti• Time-dependent tactics
• Behaviour-dependent tactics ?
A tactic is used to determine a next offer.
December 11, 2007 34
Time Dependent Tactics
A ti d d t t ti d t i th t ff t b d
Time dependent tactics take deadlines and discounting of future rewards into account.
• A time dependent tactic determines the next offer to be proposed as a function αa(t) of time:
⎪⎪⎨⎧ −+
=decreasingVifminmaxtmin
xaaaaa
tb
)()(α
Various time-dependent functions are used:
⎪⎩⎨ −−+→ increasingVifminmaxtmin
x aaaaaba )())(1( α
β1
)i ( ⎞⎛ ap• A polynomial function:
βα
max
max ),min()1()( ⎟⎟
⎠
⎞⎜⎜⎝
⎛−+= a
aaaa
ttt
kkt
akatt l))max,min(1( β
• An exponential function:
akataet
ln)max
)max,(1(
)(
β
α
−
=
December 11, 2007 35
Time Dependent Functions
b
C onceder(be ta=10)C onceder(be ta=50)
80
100m axb
L inear(be ta=1)Decision Parameters:•Reservation values: mins, maxb
40
60
B ou lw are(be ta=0.025)
B ou lw are(be ta=0.1)
buye
r offe
r
,•Initial offer level: k•Deadline (time available): tmax
•Rate of concession: β/μ/γ/δ,…
0 10 20 30 40 500
20k b(=5% )
m in b
0 10 20 30 40 50
tbm ax
tim e
December 11, 2007 36
Behavior Dependent Tactics
A b h i d d t t ti d t i th t ff t b
Behavior dependent tactics take the behavior of the opponent into account measured relative to the agent’s own utility space, giving rise to tit-for-tat strategies.
• A behavior dependent tactic determines the next offer to be proposed as a function ψ(tn) of the last proposed offer tn:
)),),(min(max(1 aan
tba maxmintx n ψ=+
→
Various functions ψ(tn) are used:
nba→
tψ( n)• Average tit-for-tat:
• Relative tit for tat:
1
2
)( −
−
→→
→= n
n
nt
batab
tab
n xxx
tγ
ψ
2nt δ• Relative tit-for-tat:1
22
2
)( −
+−
−
→→
→= n
n
nt
batab
tab
n xxx
tδ
δ
ψ
December 11, 2007 37
Monotone Concession Protocol
• Rule: Players are not allowed to make offers which have a lower
A monotone concession protocol requires agent’s to make concessions at each step, i.e. increase their opponent’s utility at each negotiation move.
utility for their opponent than their last offer.
• If the minimum concession is required to be >0, then a negotiation is guaranteed to terminatenegotiation is guaranteed to terminate.
• Agents accept offer on table if utility exceeds that of own last offer or that of a viable counter-offer.
• Example: Zeuthian Strategy (cf. literature).
Note: • In multi-issue negotiations with incomplete information about opponent preferences,
it is not always possible to make monotonic concessions (due to “compatible issues”)• Neither is it best to monotonically make concessions, as we will see later.
December 11, 2007 38
Smart Meta-Strategy
• Key idea: To better meet
Using domain knowledge it may be possible to improve the efficiency of a strategy to the benefit of the agent itself as well as of its opponent.
• Key idea: To better meetthe demands of an opponentit is not always required toconcess utility oneself.
Iso-utility curve
• Propose alternative offer that ismore similar to that of opponent’slast offer with same own utility.
?
• Only concess if not possible.
• Result: Issue trade-offs Example: Propose ⟨Medium Quality ↓ Low Price• Result: Issue trade-offs. Example: Propose ⟨Medium Quality ↓, Low Price ↑, 10 days =⟩ instead of ⟨High Quality, Medium Price, 10 days⟩
• See: Trade-Off Strategy (cf. literature)
December 11, 2007 39
Evaluating Strategies
P O i t d
A number of criteria are available to judge the quality of a strategy.
Process-Oriented:• Cost-Effective: How many steps are required to reach agreement?• Robustness: Is it possible to exploit the strategy?• Negotiation move types: Is strategy sensitive to opponent?
Outcome-Oriented:Outcome Oriented:• Successful: Is an agreement reached?• Efficiency: Is agreement Pareto optimal?
F i I t l t N h? K l i S di k ?• Fairness: Is agreement close to Nash? Kalai-Smorodinsky?
December 11, 2007 40
Evaluation:Dynamics of Negotiation Process
Unfortunate step
Inefficient
Trade-Off versus ABMP strategy on the City-vs-AMPO domain (Raiffa)
December 11, 2007 41
Evaluation:Dynamics of Negotiation Process
Nice moves
Sensitive to opponent
Trade-Off versus Random Walker strategy on the City-vs-AMPO domain
December 11, 2007 42
Summary• Due to incomplete information about opponent preferences, it is not
possible to define “optimal” strategies in any precise sense.• Various negotiation heuristics have been proposed, including time and g p p , g
behavior dependent tactics.• Efficiency can be improved by using domain knowledge and proposing an
offer on a utility iso-curve more similar to the opponent’s last offer.
Literature:• P. Faratin, C. Sierra, N.R. Jennings, 1997, Negotiation Decision Functions
for Autonomous Agents, in: Int. Journal of Robotics and Autonomous Systems 24(3-4) 159-182Systems, 24(3-4), 159-182
• P. Faratin, C. Sierra, N.R. Jennings, 2002, Using Similarity Criteria to Make Issue Trade-Offs in Automated Negotiations, in: Artificial Intelligence, 142, 205-237
• K.V. Hindriks, C.M. Jonker, D. Tykhonov, 2007, Analysis of Negotiation Dynamics, in: Cooperative Information Agents XI, LNCS 4676, 27-35
• J.S. Rosenschein, G. Zlotkin, 1994, Rules of Encounter, MIT Press
December 11, 2007 43
Opponent Modeling and LearningImproving negotiated agreements by estimating opponent preferences.
December 11, 2007
44
Vermelding onderdeel organisatie
Learning Opponent Preferences
Th t t i d t ti di d f bli d f
Various features of an opponent’s strategy can be learned, e.g. type of concession tactic, reservation value. We discuss learning opponent preferences.
• The strategies and tactics discussed so far are blind for opponent preferences.
• Various techniques have been proposed to learn• Various techniques have been proposed to learn features and construct an opponent model.
C i f ti b t t f b• Can information about opponent preferences be learned in single-instance negotiation?• Single-instance negotiation typical for e-commerce.g g yp• Hard because only bids exchanged can be used, no
previous history can be assumed.December 11, 2007 45
Learning: Structural Assumptions
Li dditi tilit f ti ∑n
bb )()(
In order to be able to learn opponent preferences we have to exploit certain structural features of negotiation problems.
• Linear additive utility functions:
• Sufficient to learn ranking of weights:
∑=
∈=i
tiiit bxewbu1
)()(
( )12=
rw
ji
iSufficient to learn ranking of weights:
• Shape of Evaluation functions (utility of issues):
( )1+nni
ei(xi) hei k 2 he
i k 1 hei k
Downhill,Uphill, and
1
ei(xi) he
i,1 hei,2 he
i,3 h i,k-2 h i,k-1 h i,k
p ,Triangular
0 xi
December 11, 2007 46
Learning: Rationality AssumptionsIn order to be able to learn opponent preferences we have to assume agents are (at least to some extent) rational.
• Opponent starts byproposing its best bid.
P(bt|hj)
proposing its best bid.
• Opponent uses a P(b0|hj)
P(b1|hj)
P(b2|hj)
|concession-based tactic
• Probability distributions are associated with possible0 u’(b1)u’(b2) u’(b0)
u(bt|hj)
• Probability distributions are associated with possible tactics.
December 11, 2007 47
Bayesian Learning
U i B ’ R l h b bili i i d i h
Each (new) bid provides new information that is used to update the model of the opponent’s preferences using Bayes’ rule.
• Using Bayes’ Rule the probabilities associated with hypotheses regarding weights and evaluation functions can be updated for every new bid bt:p y t
( ) ( ) ( )( ) ( )∑
= m
kktk
jtjtj
hbPhP
hbPhPbhP
1
• An estimate of the utility an opponent associates with a bid can be computed using the computed probabilities:
=k 1
p g p p
( ) ( ) ( )∑ ∑= =
∈=H
j
n
itiiijt bxewhPbu
1 1
December 11, 2007 48
Using an Opponent Model
An opponent model:
An opponent model can be used to choose a next offer in bilateral negotiation.
pp
• enables choosing an offer that has utility x for the negotiator and maximizes the utility of the opponent g y pp(which helps to increase the chance of acceptance)
• enables to select an offer that increases the utility of both parties if previous offers have deviated from the Pareto frontier, which is typical. I.e. it allows making a fortunate move.
December 11, 2007 49
Defining an Opponent-based Strategy
Strategy using opponent model:• Use a tit for tat tactic i e• Use a tit-for-tat tactic, i.e.
match negotiation move that opponent just made.
• Propose Pareto-efficient offers, i.e. go “straight up” (agent A) to the Pareto frontier in order to maximize own utility.
• Many possible variants.
December 11, 2007 50
Summary• It is possible to learn opponent preferences even in single-instance
negotiations by making some rationality and structural assumptions.Effi i b i d b i l t t f d• Efficiency can be improved by using learnt opponent preferences and by proposing an offer that increases utility of both parties.
Literature:Literature:• D. Zeng, K. Sycara, 1998, Bayesian Learning in Negotiation, in:
International Journal of Human–Computer Studies 48, 125–141• R Lin S Kraus J Wilkenfeld J Barry 2006 An Automated Agent• R. Lin, S. Kraus, J. Wilkenfeld, J. Barry, 2006, An Automated Agent
for Bilateral Negotiation with Bounded Rational Agents with Incomplete Information, ECAI 2006, 270-274
• K V Hindriks D Tykhonov 2007 Opponent Modelling in Automated• K.V. Hindriks, D. Tykhonov, 2007, Opponent Modelling in Automated Multi-Issue Negotiation Using Bayesian Learning, Submitted.
December 11, 2007 51
Other Topics, or:Wh I Did N T lk AbWhat I Did Not Talk AboutAutomated negotiation is a very active and broad area of research.
December 11, 2007
52
Vermelding onderdeel organisatie
Other Auctions
Topics
December 11, 2007 53
OtherTopics
December 11, 2007 54
Summary• Topics discussed include: Decentralized, Bilateral (one-to-one) &
closed versus open negotiation. Multi-issue, additive utility functions. Alternating offers protocols without time pressure. Automated negotiating software agents. Human-machine negotiation.
• Topics not discussed include: Multi-lateral negotiation. Auctions. Human negotiation, emotion, personality & culture. Argumentation-based and qualitative models of negotiation Deadlines Complexbased and qualitative models of negotiation. Deadlines. Complex, non-linear utility functions.
Literature:• V. Krishna, 2002, Auction Theory, Academic Press• I. Rahwan, S.D. Ramchurn, N.R. Jennings, P. McBurney, S. Parsons,
L. Sonenberg, 2003, Argumentation-based Negotiation, in: K l d E i i R i 18(4) 343 375Knowledge Engineering Review, 18(4), 343-375.
• L.L. Thompson, 2005 (3rd ed.), The Mind and Heart of the Negotiator, Prentice Hall
December 11, 2007 55
The Future of Automated NegotiationAutomated negotiation has potential applications in the near future in electronic commerce and marketplaces, and negotiation support systems.
December 11, 2007
56
Vermelding onderdeel organisatie
The Future of Automated Negotiation
Strategies:
Research Challenges
Strategies:• Optimal strategies for negotiation with incomplete information.
Learning:Learning:• negotiation tactics, opponent models, deadlines, …
Qualitative negotiation models:Qualitative negotiation models:• Qualitative preference models, extensions of the alternating offers
protocol that allow for additional forms of information exchange.
P f d liPreference modeling:• Elicitation techniques, utility spaces with issue dependencies.
December 11, 2007 57
The Future of Automated NegotiationApplications
Negotiation support systems:• Human-machine interaction, • Integration of emotion & personality, negotiation styles, cultureg p y, g y ,• Analyzing domains and real-world cases
Electronic Marketplaces:p• Online negotiation, shopping bots (pre-negotiation)• Recommender systems, personalized negotiation assistance,
information selection & provision.E i i l l i• Empirical analysis.
December 11, 2007 58