Modeling Reasoning in Strategic Situations

Avi PfefferMURI Review

Monday, December 17th, 2007

Strategic Situations Scenarios involving multiple agents,

all of which make decisions, and receive rewards based on their decisions

May be competitive, cooperative, or anything in between

May have interesting structure May be extended over time

Strategic Situations Abound Countering terrorist threats Robotic soccer Disaster response Diplomatic relations Auctions Trading

A Key Question

Can we model how agents reason in strategic situations?

Why this Question is Important Modeling how agents reason will

allow us to: predict their behavior develop counter-strategies develop computer systems that help

people in their strategic decision making

analyze situations to determine optimal strategies

allow people to explain their reasoning

Possible Approaches Classical game theory Opponent modeling Behavioral economics Psychological theories

Our Approach Identify the basic reasoning patterns

that can be used to justify decisions underlies sophisticated behavior such

as sacrificing, retaliation and tempting Model and learn the factors

underlying decision-making in particular games

Use the models to develop strategies that work well

Characterizing Reasoning Patterns Informally, a reasoning pattern is a

form of argument that leads to a decision

We characterize reasoning patterns as structures in a graph describing a strategic situation the reasoning patterns capture the way

information is used and manipulated

Reasoning Pattern #1: Direct Effect

An agent takes a decision because of its direct effect on its utility without being mediated by other agents’ actions

Drill

Profit

Reasoning Pattern #2: Manipulation

Child knows about parent’s action Parent does not care about reading, but wants child

to brush teeth Child dislikes brushing teeth but likes being read to Parent can manipulate child

Offer to Read

Parent

Brush Teeth

Child

Reasoning Pattern #3: Signaling

A communicates something that she knows to B, thus influencing B’s behavior

Recommendation

Alice

Choice

Bob

Better Restaurant

Reasoning Pattern #4: Revealing/Denying

Driller cares about oil Tester receives fee if driller drills Tester causes driller to find out (or not) about

information tester herself does not know

Seismic Structure

OilTest Result

Drill

Test

Tester’s Profit Driller’s Profit

A Question For each reasoning pattern, we provide

a graphical criterion to determine if the pattern holds

Intuitively, a node is motivated if the agent owning the node cares about its decision

If a node is motivated, does the graphical criterion characterizing one of the

reasoning patterns necessarily hold?

Answering the question Answer: it depends what strategies

we allow for other agents If we allow arbitrary strategies, any

directed path from a decision node to a utility causes the node to be motivated

But if we restrict attention to a “highly justifiable” class of strategies, we get a more interesting answer

Well-Distinguishing (WD) Strategies: Intuition A strategy is well-distinguishing if

all distinctions that it makes really make a difference whenever the strategy distinguishes

between two states of parents, the agent should receive different utility in the different states

Completeness Result

Theorem: If other agents are playing WD strategies, then a node is motivated only if at least one of the reasoning patterns holds i.e., the four patterns of reasoning are

sufficient to characterize all cases in which an agent cares about a decision

Relationship with Game Theory

Theorem: The set of WD strategies always includes a Nash equilibrium

We can view WD equilibrium as refinement of Nash

Completeness theorem holds for all WD strategies, not just equilibria different assumption from rationality

WD WD equilibria Nash

Learning how People Reason Reasoning patterns give us a

theoretical basis for what arguments a person might make

Can we learn what people actually do in particular games?

Can we use what we learn to develop automatic strategies that perform well?

Learning How to Negotiate Can we learn how people trade off

factors such as self-interest, altruism, etc. in negotiations?

Yes we developed a computer agent using

learned models it performed much better than game-

theoretic agents, and also better than people

Learning Reciprocal Reasoning Do people use reciprocal reasoning

in repeated interactions? retrospective reasoning prospective reasoning

Yes models that factor in reciprocal

reasoning perform better than those that don’t

but prospective may not be as important as retrospective

Reasoning Under Uncertainty When people have uncertainty

about other players, do they use models of the other players?

Yes modeling people as reasoning about

the potential actions of others leads to better performance

but recursive modeling has diminishing returns

Distinguishing Beliefs from Preferences A person’s behavior may be

influenced by both beliefs and preferences can we distinguish between them?

Yes we have created models that are

uniquely identifiable in this scenario, people have almost

correct beliefs

Next Steps Algorithms and analysis tools for

identifying relevant arguments in particular situations

Analyze arguments for key behaviors recruitment to terrorism diplomatic relations, e.g. North Korea

Questions?