Outline

Intelligent Preference

Reasoning for Multi-Agent

Decision Making

Maria Silvia Pini

Department of Information Engineering

University of Padova (Italy)

Colloquia DEI, 25 October 2012

Outline Outline

Preferences

Multi-agent decision making

Social choice (voting theory)

Voting rules

Desirable properties

Impossibility results

Computational social choice

Computational concerns

Large set of candidates

Compact preference formalisms

Missing and imprecise preferences

Stable allocations

Preferences Outline

Outline Preferences

Preferences are ubiquitous in everyday decision making

Essential ingredients in every reasoning tool

Preferences are orderings over possible options

Options: computers, candidates, cars, books, movies …

Preferences can model levels of acceptance, or costs

Preferences are tolerant constraints

Constraints are strict requirements that must be satisfied

Constraints and preferences may be present in the same problem

Configuration, timetabling, etc.

Preferences

Outline Example: University timetabling

Professor Administration

I cannot teach on Wednesday

afternoon.

I prefer not to teach early in

the morning, nor on Friday

afternoon. Lab C can fit only 120 students.

Better to not leave 1-hour holes in

the day schedule.

Constraints

Preferences

Preferences University timetable

Outline Preferences for collective decision making in

multi-agent systems

Several agents

Common set of possible decisions

Each agent has its preferences over the possible decisions

Goal: to choose one of the decisions, based on the preferences

of the agents

Also a set of decisions, or a ranking over the decisions

AI scenarios add: imprecision, uncertainty, complexity, etc.

Outline Applications

Doodle

Several time slots under consideration

Participants accept or reject each time slot

Very simple way to express preferences over time slots

Very little information communicated to the system

Collective choice: a single time slot

The one with most acceptance votes from participants

Other applications

Meta-search engines

Group recommender systems

Outline How to compute a collective decision?

Let the agents vote by expressing their

preferences over the possible decisions

Aggregate the votes to get a single decision

Let’s look at voting theory

Agents = Voters

Decisions = Candidates

Preferences

Chosen decision = winner

Outline Main differences

In multi-agent AI scenarios, we usually have

Incomparability

Computational concerns

Large sets of candidates (w.r.t. number of voters)

Formalisms to compactly represent preferences

Uncertainty, vagueness

Outline Voting theory

(Social choice)

Voters

Candidates

Each voter expresses its preferences over the candidates

Goal: to choose one candidate (the winner), based on the voters’ preferences

Also many candidates, or ranking

Rules (functions) to achieve the goal

Properties of the rules

Impossibility results

Outline Some voting rules

Plurality

Voting: one most preferred decision

Selection: the decision preferred by the largest number of agents

Majority: like plurality, over 2 options

Approval (m options)

Voting: approval of between 1 and m-1 options

Selection: option with most votes

Doodle

Borda

Voting: rank over all options,

Score of an option: number of options that it dominates

Selection: option with greatest sum of scores

Outline

Unanimity (efficiency)

If all voters have the same top choice, it is selected

Non-dictatorship

There is no voter such that his top choice always wins, regardless of the votes of other voters

Non-manipulability

There is no incentive for agents to misrepresent the preferences

Some desirable properties

Outline Two classical impossibility results

Arrow’s theorem (1951)

Totally ordered preferences

it is impossible to find a voting rule with some desirable properties including

unanimity

non-dictatoriality

Gibbard-Sattherwaite’s theorem (1973)

Totally ordered preferences

it is impossible to have a reasonable voting rule that is

non-dictatorial

non-manipulable

These impossibility results holds also when we allow incomparability in preferences

Nobel prize in Economics 1972

Pini, Rossi, Venable, Walsh. Aggregating Partially Ordered Preferences. J. Logic and Computation 19(3): 475-502 (2009)

Outline Computational concerns

Given the impossibility result, we want to avoid rules which are

computationally easy to manipulate

We have studied computational complexity of manipulation/winner

determination for voting rules when some preferences are missing

Some voting rules are difficult to manipulate when we have weighted voters

and incomparable pairs

For some classes of voting rules it is computationally easy to find possible and

necessary winners and terminate preference elicitation

Pini, Rossi, Venable, Walsh: Incompleteness and incomparability in preference aggregation: Complexity results. Artificial Intelligence 175(7-8): 1272-1289 (2011)

Pini, Rossi, Venable, Walsh: Winner determination in voting trees with incomplete preferences and

weighted votes. Autonomous Agents and Multi-Agent Systems 25(1): 130-157 (2012)

Bartholdi, Tovey, Trick. The computational difficulty of manipulating an election. Social Choice and Welfare 1989

Outline Computational Social Choice

It is an interdisciplinary field at the interface of

social choice theory

computer science and AI

Main goals

1. Application of techniques of computer science, such as complexity

analysis or algorithm design, to the study of social choice mechanisms,

such as voting procedures

2. Importing concepts from social choice theory into computing. For

instance, the study of preference aggregation mechanisms is relevant to

multi-agent systems

Chevaleyre, Endriss, Lang, Maudet, A short introduction to Computational Social Choice, 2007

Outline Computational Social Choice

Between multi-agent systems and social choice

AI, economics, mathematics, political science, etc.

Social choice

Voting rules

Desirable properties

Impossibility results

Computational social choice

Incomparability

Computational concern

Compact preference formalism

Uncertainty and preference elicitation

Outline Formalisms to model preferences

compactly

Preference ordering over a large set of decisions (candidates,

outcomes, …) need to model them compactly

Otherwise too much space and time to handle such preferences

An Example: Soft constraint formalism

Preferences over partial assignments of the decision variables, from

which to generate the preference ordering over the solution space

Outline Soft Constraints

(the c-semiring framework)

Variables {X1,…,Xn}=X

Domains {D(X1),…,D(Xn)}=D

Soft constraints each constraint involves some of the variables a preference is associated with each assignment of the

variables

Set of preferences A Totally or partially ordered (induced by +) a ≤ b iff a+b=b

Combination operator (x) Top and bottom element (1, 0) Formally defined by a c-semiring <A,+,x,0,1>

Bistarelli, Montanari, Rossi: Semiring-based constraint satisfaction and optimization. J. ACM 44(2): 201-236 (1997)

Example: fuzzy constraints

Lunch time= 13 Meal = meat Wine = white Swimming time= 14

Decision A

pref(A)=min(0.3,0)=0

Lunch time = 12 Meal = fish Wine = white Swimming time = 14

Decision B

pref(B)=min(1,1)=1

{12, 13} {14, 15}

Lunch time

Swimming time

(12, 15) 1

(12, 14) 1 (13, 14) 0

(13, 15) 1

{fish, meat} {white, red}

meal wine

(fish, red) 0.8

(fish, white) 1 (meat,white) 0.3

(meat, red) 0.7

Example with fuzzy constraints Preference of a decision: minimal preference of its parts Aim: to find a decision with maximal preference Preference values: between 0 and 1

A soft constraint problem induces an

ordering over the solutions

Outline Uncertainty and vagueness

Missing preferences

Too costly to compute them

Privacy concerns

Ongoing preference elicitation process

Imprecise preferences

Preferences coming from sensor data

Too costly to compute the exact preference

Estimates

Compact preference formalisms and solving techniques

to model and solve problems with missing or imprecise

preferences

Gelain, Pini, Rossi, Venable, Wilson. Interval-valued soft constraint problem. Annals Mathematics and Artificial Intelligence 58(3-4): 261-298 (2010)

Gelain, Pini, Rossi, Venable, Walsh: Elicitation strategies for soft constraint problems with missing preferences: Properties, algorithms and experimental studies. Artificial Intelligence 174(3-4): 270-294 (2010)

Outline

Stable allocations

Matching of two sets

Men to women

Doctors to hospitals

Students to schools

Two-sided markets

Kidney donors and patients

Preferences

Each member of one group expresses a total order over all the members of the other group

Stability

not two agents who would prefer each other over their current counterparts

Outline Gale-Shapley algorithm

Gale-Shapley algorithm (1962)

If the number of doctors and hospitals is the same

The algorithms always find a stable allocation

Irrespective of agents’ preferences

It takes O(n^2) time, where n is the number of hospitals/doctors

A. Roth: Gale-Shapely algorithm is manipulable (1984)

Nobel prize in Economics 2012

A. Roth L. Shapely

Applications:

Doctors-Hospitals

USA, Scotland

Students-schools

New York, Boston, Spain, Hungary

Professors-schools

France, UK

Kidney transplants (donors-patients)

Spain, UK, USA, Australia

Outline Stable matching

In practical applications

It is useful to allow for ties and incomparable elements

Hospitals with many applicants have expressed the desire to use ties

It is more natural to express scores than a preference ordering

Score may model profits or costs

My research

New notions of stability and optimality in these scenarios

Algorithms that generalize Gale-Shapely alg. to find matchings that are stable and optimal according to new stability/optimality notions

Computational complexity of manipulating of stable matching procedures

Stable matching procedure based on voting rules difficult to manipulate

Pini, Rossi, Venable, Walsh: Manipulation complexity and gender neutrality in stable marriage procedure,

Autonomous Agents and Multi-Agent Systems 22(1): 183-199 (2011)

Pini, Rossi, Venable, Walsh: Stability in Matching Problems with Weighted Preferences. ICAART 2011

Pini, Rossi, Venable, Walsh: Weights in stable marriage problems increase manipulation opportunities. TARK 2011,

Best poster & presentation award

Outline Conclusions

Intelligent preference reasoning in multi-agent decision making

Computational social choice (CCS)

Between multi-agent systems and social choice

Preference modelling

Incomparability

Uncertainty and preference elicitation

Stable allocations

Cross-fertilization in both directions

Prabhakar Raghavan (Vice President of Strategic Technologies at Google)

DEI - Distinguished Lecturer Series, September 2012

“The academic challenge: How can we combine computer science and social science?”

Preference reasoning in CCS is a first step in this direction

Outline Future work

Voting theory and stable matching

Compact preference formalisms for expressing agents’ preferences

Influence, bribery and control

Stable matching procedures based on voting rules

Uncertainty and preference elicitation in stable matching procedures

Doodle with preferences

Voting rules to aggregate agents’ preferences

Preferences in recommender and reputation systems

Outline Joint work with …

Francesca Rossi

University of Padova

Brent Venable

Tulane University and IHMC

Toby Walsh

NICTA, Australia

Jerome Lang

LAMSADE, Paris

Ulle Endriss

ILLC, University of Amsterdam

Nicolas Maudet University Paris-Dauphine

Mirco Gelain University of Padova

Nic Wilson

4C, Ireland

Nick Mattei University of Kentucky

Outline

Intelligent Preference

Reasoning for Multi-Agent

Decision Making

Maria Silvia Pini

Department of Information Engineering

University of Padova (Italy)

Colloquia DEI, 25 October 2012

Thank you for your attention!