poki: the poker agent
DESCRIPTION
Poki: The Poker Agent. Greg Priebe Zak Knudson. Overview. Texas Hold’em poker Architecture and Opponent Modeling of Poki Improvements from past Poki Betting strategies w/ analysis of hand strengths, weighting, and probabilities. Texas Hold’em. Each player gets 2 “hole” cards - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/1.jpg)
Poki: The Poker Agent
Greg PriebeZak Knudson
![Page 2: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/2.jpg)
Overview• Texas Hold’em poker• Architecture and Opponent
Modeling of Poki• Improvements from past Poki• Betting strategies w/ analysis of
hand strengths, weighting, and probabilities.
![Page 3: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/3.jpg)
Texas Hold’em• Each player gets
2 “hole” cards• Community cards:
– 3 “flop” cards– 1 “turn” card– 1 “river” card
• Rounds of betting after each set of cards
![Page 4: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/4.jpg)
Poki Program Architecture
![Page 5: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/5.jpg)
Opponent Modeling• Weight table• No modeling• Generic opponent
modeling• Specific opponent
modeling
![Page 6: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/6.jpg)
The Old Way• Specific opponent modeling• Betting frequency table
– Keeps track of actions taken in different contexts
– Predicts median hand strength• Left out many relevant details
![Page 7: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/7.jpg)
New and Improved• Specific opponent modeling• Detailed• Dynamic learning/adaptation• Prelim. Neural net study
![Page 8: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/8.jpg)
Poki’s Neural Net
![Page 9: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/9.jpg)
Some Results• Training Data• Results from
80-90 percent• Confusion
matrix• Error prone?
![Page 10: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/10.jpg)
In Field Predictions#holdem1: 24% increase
![Page 11: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/11.jpg)
In Field Performance
![Page 12: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/12.jpg)
Small Bets Won
• Pro: .05 - .10• Old opponent
modeling: .09• New opponent
modeling: .22
![Page 13: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/13.jpg)
Betting Strategy• Pre-flop and Post-flop strategies are
significantly different.- Pre: little information available to
influence the betting decision.- Post: uses opponent models, private
hand, and game context to generate an action.
![Page 14: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/14.jpg)
Pre-flop Betting Strategy* - pocket pair hand (two cards of the same rank)s - suited hand o - offsuit hand
![Page 15: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/15.jpg)
Post-flop: Basic Betting Strategy• Compute Poki’s effective hand strength
(EHS).• Using the game context, betting rules,
and formulas to translate the EHS into a probability triple{ Pr(fold), Pr(call), Pr(raise) }.
• Generate a random number and use it to choose an action from the probability distribution.
![Page 16: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/16.jpg)
Hand Strength• Probability that a given hand is better
than that of an active opponent.
![Page 17: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/17.jpg)
Hand Potential• With 2 cards yet to be revealed, we want
to know the impact.• Positive potential (Ppot) – chance hand
improves to win.• Negative potential (Npot) – chance hand
ends up losing.• Calculated by enumerating over all
possible hole cards, and over all possible board cards.
• This is an expensive process.
![Page 18: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/18.jpg)
Hand potential example
![Page 19: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/19.jpg)
Effective Hand Strength• Combines hand strength and
potential to give Poki’s own relative strength against an opponent.
• Pr(win) = pr(ahead) * pr(opp doesn’t improve) + Pr(behind) * pr(we improve)
= HS * (1 – Npot) + (1 – HS) * Ppot
![Page 20: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/20.jpg)
Weighting the Enumerations• Probability of hands played to a
particular point will vary.• To account for this, Poki maintains a
weight table for each opponent.
![Page 21: Poki: The Poker Agent](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681678a550346895ddca363/html5/thumbnails/21.jpg)
Conclusions• Need opponent modeling • Complex system • Betting strategies and determining hand
strength• Reweight helps to determine
opponents likely action