ba yes i an multi agent beamer

7/29/2019 Ba Yes i an Multi Agent Beamer

1/15

Motivation and AimApplications

MAS:A Brief OverviewA Bayesian Framework

Conclusions

Bayesian Multi Agent Systems

Ahmad AsharGroup 256 Modelling and Simulation

December 17, 2012

Ahmad Ashar Group 256 Modelling and Simulation Bayesian Multi Agent Systems


2/15



Conclusions

Table of contents

1 Motivation and AimWhat is BayesianBayesian vs FrequentistAvoiding Over fitting

Personal Motivation2 Applications

3 MAS:A Brief OverviewDecision TheoryGame Theory

4 A Bayesian FrameworkMarkov Decision Process for Single Agent

Value Functions and Bellman Equations

MDP for MultiAgent System in Reinforcement learning

5

ConclusionsAhmad Ashar Group 256 Modelling and Simulation Bayesian Multi Agent Systems


3/15



Conclusions

Avoiding Over fittingPersonal Motivation

The questions

Why Bayesian ?What is Bayesian ?

Why is it an important paradigm ?

Why am I studying this?



4/15



Conclusions


The Answers..

What is Bayesian ?

p(|D) =p(D|)p(0)

== p(D|)p()d


M i i d Ai


5/15



Conclusions


Bayesian vs Frequentist

Why Bayesian ?

Figure: Human Evolution


M ti ti d Ai


6/15



Conclusions


Why use the Bayesian Paradigm

Bayesian Marginalization avoids the problem of overfitting

.. the phenomenon of over-fitting is really an unfortunateproperty of maximum likelihood and does not arise when wemarginalize over parameters in a Bayesian setting.

..It is a property of the marginal likelihood that itautomatically incorporates a trade-off between model fit and

model complexity Above quotes courtesy Bishops book on Pattern Recognition


Motivation and Aim


7/15



Conclusions


The details..

Overfitting occurs when a statistical model describes randomerror or noise instead of the underlying relationship[Wikipedia]


Motivation and Aim


8/15



Conclusions


Why am I studying this?

Interest in Statistical Machine Learning

Bayesian Non-parametrics: possibly the hottest thing inMachine Learning today

My interests in Gaussian Process Regression,Classification

Research includes tools from1 Statistical Physics : Mean Field Approximations

2 Data Analysis: Kernel Methods3 Neural Computation : ANN


Motivation and Aim


9/15



Conclusions

Applications

Data Analysis - Google,Microsoft,Yahoo!

Fraud Detection

Geo-statistical applications

Intelligent systems (Robots, Recommender sytems :Netflixprize)

Bioinformatics, Cheminformatics, NLP, Phylogenetic trees

Information Theory, Decision Theory

Practically anything which has a large data set!


Motivation and Aim


10/15



Conclusions

Decision TheoryGame Theory

MAS :Decision Theory

Decision Theory

Maximizing Utility over various policies (series of steps)

State Percept Action NewState

Found by searching various possible states s

Search problem Decision Theory problem

Reward R : S S A R

Distribution over actions p(a|s) for each s S where a A(s)


Motivation and Aim


11/15

ApplicationsMAS:A Brief OverviewA Bayesian Framework

Conclusions

Decision TheoryGame Theory

MAS :Game Theory

Game Theory

Framework for Co-operation and CommunicationTreat MAS as a game with agents as players

Use mathematical results in co-operative games withincomplete information as strategies for agents

Highly complex and advanced field of research


Motivation and Aim


12/15


Conclusions

Markov Decision Process for Single AgentMDP for MultiAgent System in Reinforcement learning

Bayesian Decision Making: Markov Decision Process

State transition properties depend on the current state andaction

State transition probabilities are multinomial distributions

When the rewards or transition probabilities are unknown theproblem reduces to that of reinforcement learning


Motivation and Aim


13/15


Conclusions


Value Functions and Bellman Equations

Reward function:N

t=0

trt+1

where > 0

State value function

Vp(s) = E[

t=0

trt+1|so = s]

Bellman Equation

aA(s)

p(a|s)

sS

p(s|s, a)[r(s, a, s) + Vp(s)]

State Action Value Function

=

p(s|s, a)[r(s, a, s) + Vp(s)]Ahmad Ashar Group 256 Modelling and Simulation Bayesian Multi Agent Systems

Motivation and Aim


14/15


Conclusions


A Bayesian Framework : MDP for MAS in RL

Acting, Coordinating should be integrated

One agents optimal policy affects the others

Solution: Put Bayesian Priors over MAS optimal policySolution: Likelihood could be factorized

Bayesian inference could be carried out, Often approximations

Have to model some agents joint behaviour for Machine

Learning of the parametersProblem with scaling :With more agents Parameters couldtend to


Motivation and AimA li i


15/15


Conclusions

Conclusions

MAS : Mixing of Decision Theory and Game theory

Making MAS an extension of probabilistic single agents withincomplete information

Using Bayesian Inference for optimal policy determination

Challenge: To fully develop a detailed Bayesian MAS

mathematically


ba yes i an multi agent beamer

Documents