Learning in environments with agents we don’t control.

Download Learning in environments with agents we don’t control.

Post on 18-Jan-2016

212 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Learning in environments with agents we dont control

    WRANE November, 2010 Geoff Gordon

    Making modelsSkill of making models that represent realitybringing in other disciplines (besides CS AI econ stats math control philosophy)How do we describe intelligent agents?What (approximate) equilibria (or other solution concepts) are relevant?

    WRANE November, 2010 Geoff Gordon

    Setting up the learning problemHow do we even measure success of learning? How do we express prior information?What is visible? (actions, outcomes, payoffsfor self, other agents)

    WRANE November, 2010 Geoff Gordon

    How do we get the data?Exploration / experimentation (vs. exploitation)problem of driving off a cliffbut more problems for games: e.g., accidentally revealing infoWant to avoid being taught and exploitedWant to present a table image

    WRANE November, 2010 Geoff Gordon

    ComplexityCan we get generalization bounds analogous to those from COLT, statistics?How do we measure complexity of a model or model class? Choose the right complexity?Are we doomed to model opponents as less complex than ourselves? Is this a problem?What if the [game, opponent set] changes: how stable are our performance metrics and generalization bounds?

    WRANE November, 2010 Geoff Gordon

    Complexity, contdEnsembleswork really well in Netflix, KDD cup; not as well in Lemonade Standis there something about our [adversarial, dynamic, non-Markovian] setting that hurts them?

    * opp modeling* doesnt help to learn if learner goes haywire and doesnt converge* comm is what glues it all together; humans do it well, we dont even have start of a theory of comm* underneath it all, were doing a big optimization, so we need good optimization algorithms and structured representations

Recommended

View more >