Upload File

hidden information, teamwork, and prediction in trick-taking card...

  • Home
  • Documents
  • Hidden Information, Teamwork, and Prediction in Trick-Taking Card …mferey/publication/ai-400/... · 2020. 7. 11. · Card Play Baselines HadiElzayn, Mikhail Hayhoe, Harshat Kumar,
1
Rules of Four Hundred Results Betting References Card Play Baselines Hadi Elzayn, Mikhail Hayhoe, Harshat Kumar, Mohammad Fereydounian Hidden Information, Teamwork, and Prediction in Trick-Taking Card Games [email protected], [email protected], [email protected], [email protected] Future Work Random Greedy Heuristic Over400 Heuristic Win % 100 100 - 10.5 Over400 Win % 100 100 89.5 - • Players sit across from team members and are dealt 13 cards each • Players bet expected number of ‘tricks’ they plan to take • In each round after bets are placed • Players take turns choosing a card to play in order, starting with the player who took the previous trick • Suit of first card played determines ‘lead suit’ • Players must play ‘lead suit’ if they have it • Winner of trick is determined by highest card of ‘lead suit’ or highest card of ‘trump suit’ (Hearts) if any were played • After 13 rounds, score of each player is increased by number of tricks they bet if they met or exceeded it • Score of player is decreased by bet if they were unable to meet it • Game concludes when one team member has 41 points or more, and the other player has positive points (a) Tricks won over 1,000 games, based on bet made. No bets were higher than 6 (b) Histogram showing difference between tricks won and bet made (c) Average score showing the performance of the self play NN model (d) Loss function for the betting NN, trained in tandem with the playing NN Neural Network Architecture for Card Play Neural Network Architecture for Betting Learning Problem: Find an optimal playing policy ! ",∗ to maximize expected reward for player given each other and the play of the others: Reinforcement Learning Approach: We define the reward at the end of each round by Approach informed by Neural Fitted Q-iteration [2] in which the reward is provided as a label to the state representation. We define the label as We include the indicator as reward shaping [3] to speed up convergence The state representation: • Similar to AlphaGo [4], we encode the state by a matrix representation where the location and value indicate the card, time it was played, and by whom it was played • Initial bets and number of tricks won are also encoded in the state representation Learning Problem: Find an optimal betting policy ",∗ to maximize expected reward for player given each other and the play of the others: Supervised Learning Approach: Given some particular card playing strategy and initial hand, the play can expect to win some number of tricks. Therefore, develop a model which predicts tricks taken by the end of the round given some observed initial hand compositions Generate Data: Initial hands serve as input data, number of tricks won functions as label. Implement Neural Network for regression to minimize loss defined by square difference between score received and the best possible. Loss function is asymmetric: Penalizes more for bets which are higher than the tricks obtained Random Play-Random Bet: Random bet selects a random value from {2,3,4,5} during the betting stage, and in each round chooses a random card to play from the set of valid cards. Greedy Play-Model Bet: During play, greedy simply selects the highest card in its hand from the set of valid cards, without consideration of its partner Heuristic Play-Model Bet: A heuristic strategy was defined based on human knowledge of the game. This heuristic player takes into account available knowledge of the team member’s actions as well as opponents’ actions • Examine approach on similar games such as Spades, Hearts, and Tarneeb • Consider invariance discovery for generalizeation [1] A. J. Hoane Jr. M. Campbell and F. Hsu. Deep blue. Artificial Intelligence, 134(1-2):57-83, 2002 [2] M. Riedmiller. Neural Fitted Q iteration – First experiences with a data efficient neural reinforcement learning method. European conference on Machine Learning, 05:317-328, 2005 [3] V. Minh et al. Human-level control through deep reinforcement learning. Nature, 518:529-533, 2015 [4] D. Silver et al. Mastering the game of Go without human knowledge. Nature, 550:354-359, 2017

Upload: others

Post on 26-Jan-2021

2 views

Category:

Documents


0 download

Report

TRANSCRIPT

  • RulesofFourHundred Results

    Betting

    References

    CardPlay

    Baselines

    Hadi Elzayn,MikhailHayhoe,HarshatKumar,MohammadFereydounian

    Hidden Information, Teamwork, and Prediction in Trick-Taking Card Games

    [email protected],[email protected],[email protected],[email protected]

    FutureWork

    Random Greedy Heuristic Over400

    HeuristicWin% 100 100 - 10.5

    Over400Win% 100 100 89.5 -

    • Playerssitacrossfromteammembersandaredealt13cardseach• Playersbetexpectednumberof‘tricks’theyplantotake• Ineachroundafterbetsareplaced• Playerstaketurnschoosingacardtoplayinorder,startingwiththeplayerwhotooktheprevioustrick

    • Suitoffirstcardplayeddetermines‘leadsuit’• Playersmustplay‘leadsuit’iftheyhaveit• Winneroftrickisdeterminedbyhighestcardof‘leadsuit’orhighestcardof‘trumpsuit’(Hearts)ifanywereplayed

    • After13rounds,scoreofeachplayerisincreasedbynumberoftrickstheybetiftheymetorexceededit

    • Scoreofplayerisdecreasedbybetiftheywereunabletomeetit• Gameconcludeswhenoneteammemberhas41pointsormore,andtheotherplayerhaspositivepoints

    (a) Tricks won over 1,000 games, based on bet made. No bets were higher than 6

    (b) Histogram showing difference between tricks won and bet made

    (c) Average score showing the performance of the self play NN model

    (d) Loss function for the betting NN, trained in tandem with the playing NN

    Neural Network Architecture for Card Play

    Neural Network Architecture for Betting

    LearningProblem:Findanoptimalplayingpolicy𝜋!",∗ tomaximize

    expectedrewardforplayer𝑖 giveneachotherandtheplayoftheothers:

    ReinforcementLearningApproach:

    Wedefinetherewardattheendofeachroundby

    ApproachinformedbyNeuralFittedQ-iteration[2]inwhichtherewardisprovidedasalabeltothestaterepresentation.Wedefinethelabelas

    Weincludetheindicatorasrewardshaping[3]tospeedupconvergence

    Thestaterepresentation:• Similarto AlphaGo[4],weencodethestatebyamatrixrepresentationwherethelocationandvalueindicatethecard,timeitwasplayed,andbywhomitwasplayed

    • Initialbetsandnumberoftrickswonarealsoencodedinthestaterepresentation

    LearningProblem:Findanoptimalbettingpolicy𝛽",∗ tomaximizeexpectedrewardforplayer𝑖 giveneachotherandtheplayoftheothers:

    SupervisedLearningApproach:Givensomeparticularcard playingstrategyandinitialhand,theplaycanexpecttowinsomenumberoftricks.Therefore,developamodelwhichpredictstrickstakenbytheendoftheroundgivensomeobservedinitialhandcompositions

    GenerateData:Initialhandsserveasinputdata,numberoftrickswonfunctionsaslabel.

    ImplementNeuralNetworkforregressiontominimizelossdefinedbysquaredifferencebetweenscorereceivedandthebestpossible.

    Lossfunctionisasymmetric:Penalizesmoreforbetswhicharehigherthanthetricksobtained

    RandomPlay-RandomBet:Randombetselectsarandomvaluefrom{2,3,4,5}duringthebettingstage,andineachroundchoosesarandomcardtoplayfromthesetofvalidcards.

    GreedyPlay-ModelBet: Duringplay,greedysimplyselectsthehighestcardinitshandfromthesetofvalidcards,withoutconsiderationofitspartner

    HeuristicPlay-ModelBet: Aheuristicstrategywasdefinedbasedonhumanknowledgeofthegame.Thisheuristicplayertakesintoaccountavailableknowledgeoftheteammember’sactionsaswellasopponents’actions

    • ExamineapproachonsimilargamessuchasSpades,Hearts,andTarneeb• Considerinvariancediscovery forgeneralizeation

    [1]A.J.Hoane Jr.M.CampbellandF.Hsu.Deepblue.ArtificialIntelligence,134(1-2):57-83,2002[2]M.Riedmiller.NeuralFittedQiteration– Firstexperienceswithadataefficientneuralreinforcementlearningmethod.EuropeanconferenceonMachineLearning, 05:317-328,2005[3]V.Minhetal.Human-levelcontrolthroughdeepreinforcementlearning.Nature,518:529-533,2015[4]D.Silveretal.MasteringthegameofGowithouthumanknowledge.Nature,550:354-359,2017


THE MARITIME AREAS ACT Areas Act.pdf · MARITIME AREAS 5 6.-(1) The archipelagic baselines shall consist of straight Archipelagic baselines. baselines joining the outermost points

Baselines and Additionality

David Steven Brown_ Oyster Baselines

When Meet - National Association of Evangelicalsnae.net/wp-content/uploads/2015/06/When-God-and... · When Meet Mark A. Noll Dorothy Boorse Katharine Hayhoe and W. Douglas Hayhoe

CONTROL AND MANAGEMENT SYSTEM BASELINES FOR EUROPEAN … · CONTROL AND MANAGEMENT SYSTEM BASELINES FOR EUROPEAN UNION ... CONTROL AND MANAGEMENT SYSTEM BASELINES FOR ... Article

Metric Baselines

precRec vs. baselines

47528090 a Colour Atlas of Haematological Cytology by Hayhoe Pg

Section 3 Conducting Baselines and Collecting Datappd.cipe.org/.../2014/08/3.-Conducting-Baselines-and-Collecting-Data.… · Section 3: Conducting Baselines and Collecting data 64

LSC Tech Baselines

Australia GPS baselines

Martenson - Shifting Baselines v2

Security Baselines and Risk Assessments

Luke Hayhoe GM Commercial and Customer BA CityFlyer

Hints Outlines/Profiles/Baselines

London City Airport Luke Hayhoe GM Commercial and Customer BA CityFlyer

Digital baselines amplify marketing effectiveness

Baselines: surveying the landscape of Weereewa. · Baselines: surveying the landscape of Weereewa. Mary Appleby, Facilitator STGE CMN. The Baselines forum was an initiative of CraftACT,

Stable Baselines Documentation

A Climate for Change:' A Presentation by Katharine Hayhoe

BASELINES TRAINING CATALOG 2019

A Colour Atlas of Haematological Cytology by Hayhoe Pg

A Colour Atlas Of Haematological Cytology by Hayhoe, Pg

7 ASTM Roughness Seminar Hayhoe-FAA

Topic 3 baselines and mrv

WP2: ‘Shifting Baselines’ Early Results

Setting Agricultural Baselines in Water Quality Trading ....… · Setting Agricultural Baselines in Water Quality Trading ... the challenges of setting agricultural baselines in

802.3 Primer for Baselines

Baselines & Additionality Methodologies

STRAIGHT BASELINES: INDONESIA

Importance of Baselines Choice of Baselines Baselines for ... · Chest X-Ray Report Generation William Boag1, Tzu-Ming Harry Hsu1, Matthew ... Need to collect clinical judgments from

Lauren Child By Charlotte Hayhoe & Catherine Janson

Kyoto Architecture - Baselines

SECURITY BASELINES

STRAIGHT BASELINES: INDONESIA · 2019-11-04 · 2 STRAIGHT BASELINES: INDONESIA SUMMARY The Republic of Indonesia has established straight baselines based upon the so-called archipelago