strategy improvement for parity games: a combinatorial...
TRANSCRIPT
![Page 1: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/1.jpg)
Strategy Improvement for Parity Games:
A combinatorial perspective
Paul Hunter
InfoSys seminar
Oxford University Computing Laboratory
28 October 2008
Paul Hunter Strategy Improvement for Parity Games
![Page 2: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/2.jpg)
Overview
Theorem
The strategy improvement algorithm for parity games is abottom-antipodal sink-finding algorithm on an acyclic unique sinkorientation of the strategy hypercube.
Paul Hunter Strategy Improvement for Parity Games
![Page 3: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/3.jpg)
Overview
Theorem
The strategy improvement algorithm for parity games is abottom-antipodal sink-finding algorithm on an acyclic unique sinkorientation of the strategy hypercube.
Paul Hunter Strategy Improvement for Parity Games
![Page 4: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/4.jpg)
Parity Games
Ubiquitous in formal verification
Polynomially equivalent to modal µ-calculus model checking
Paul Hunter Strategy Improvement for Parity Games
![Page 5: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/5.jpg)
Parity Games: Example
4
1
1 5
0
3
Played by two players: Player 0(∃ve) and Player 1 (∀dam) ona (finite) directed graph withpriorities on vertices.
Players take turns moving atoken around the graph andthe winner is determined bythe parity of the largest priorityseen infinitely often.
Paul Hunter Strategy Improvement for Parity Games
![Page 6: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/6.jpg)
Parity Games: Definitions
A parity game is a tuple (V ,V0,V1,E , χ) where:
V0 and V1 partition V ,
(V ,E ) is a directed graph,
χ : V → P ⊂ ω is a priority function
A play is a (possibly infinite) path in (V ,E )
π = v1v2 · · ·
A play is winning for Player i iff lim supχ(vn) = i mod 2
Paul Hunter Strategy Improvement for Parity Games
![Page 7: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/7.jpg)
Parity Games: Strategies
A strategy for Player i is a function σ : V ∗Vi → V .
A positional (or memoryless) strategy for Player i is a functionσ : Vi → V
A strategy is winning if all plays consistent with that strategy arewinning
Paul Hunter Strategy Improvement for Parity Games
![Page 8: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/8.jpg)
Parity Games: Strategies
4
1
1 5
0
3
Paul Hunter Strategy Improvement for Parity Games
![Page 9: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/9.jpg)
Parity Games: Strategies
4
1
1 5
0
3
Paul Hunter Strategy Improvement for Parity Games
![Page 10: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/10.jpg)
Parity Games: Properties
Theorem (Memoryless determinacy)
From every vertex, one player has a memoryless winning strategy
Open Problem
Are parity games in P?
Paul Hunter Strategy Improvement for Parity Games
![Page 11: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/11.jpg)
Parity Games: Properties
Theorem (Memoryless determinacy)
From every vertex, one player has a memoryless winning strategy
Open Problem
Are parity games in P?
Paul Hunter Strategy Improvement for Parity Games
![Page 12: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/12.jpg)
Overview
Theorem
The strategy improvement algorithm for parity games is abottom-antipodal sink-finding algorithm on an acyclic unique sinkorientation of the strategy hypercube.
Paul Hunter Strategy Improvement for Parity Games
![Page 13: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/13.jpg)
Strategy Improvement: Overview
Introduced by Voge and Jurdzinski in 2000.
Basic principle:
1 Define a measure for strategies
2 Start with a random strategy
3 Compute the measure for the current strategy
4 Make improvements to the strategy based on the measure
5 Repeat steps 3 and 4 until no improvements can be made
6 Compute winning sets based on optimal strategies
Paul Hunter Strategy Improvement for Parity Games
![Page 14: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/14.jpg)
Strategy Improvement: The details
1
1 5
4
0
3Given a strategy for Player0 and a vertex, a valuationrepresents the “best” counter-strategy for Player 1 from thatvertex.
Paul Hunter Strategy Improvement for Parity Games
![Page 15: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/15.jpg)
Strategy Improvement: The details
1
1 5
0
43Given a strategy for Player0 and a vertex, a valuationrepresents the “best” counter-strategy for Player 1 from thatvertex.
Paul Hunter Strategy Improvement for Parity Games
![Page 16: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/16.jpg)
Strategy Improvement: The details
1
1 5
0
43
Player 0 then chooses “best”successor from each vertex
Paul Hunter Strategy Improvement for Parity Games
![Page 17: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/17.jpg)
Strategy Improvement: The details
1
5
0
4
1
3
Player 0 then chooses “best”successor from each vertex
Paul Hunter Strategy Improvement for Parity Games
![Page 18: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/18.jpg)
Strategy Improvement: Observations
The strategy changes are local and independent
The valuation is easy to compute (O(|V ||E |))
Problem
How many iterations does this algorithm perform?
Paul Hunter Strategy Improvement for Parity Games
![Page 19: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/19.jpg)
Strategy Improvement: Observations
The strategy changes are local and independent
The valuation is easy to compute (O(|V ||E |))
Problem
How many iterations does this algorithm perform?
Paul Hunter Strategy Improvement for Parity Games
![Page 20: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/20.jpg)
Overview
Theorem
The strategy improvement algorithm for parity games is abottom-antipodal sink-finding algorithm on an acyclic unique sinkorientation of the strategy hypercube.
Paul Hunter Strategy Improvement for Parity Games
![Page 21: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/21.jpg)
AUSOs
An orientation of a hypercube is an acyclic unique sink orientation(AUSO) if
it is acyclic
every sub-hypercube has a unique sink
AUSOs are also known as completely unimodal pseudo-booleanfunctions
Paul Hunter Strategy Improvement for Parity Games
![Page 22: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/22.jpg)
AUSOs: Examples (2 dimensions)
Paul Hunter Strategy Improvement for Parity Games
![Page 23: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/23.jpg)
AUSOs: Examples (2 dimensions)
Paul Hunter Strategy Improvement for Parity Games
![Page 24: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/24.jpg)
AUSOs: Properties
Common combinatorial structure with elegant properties
Theorem (Williamson Hoke 88)
1 An acyclic orientation is an AUSO iff every 2-dimensionalsubcube is an AUSO
2 AUSOs satisfy the Hirsch conjecture
3 The vector of improving directions of any AUSO is a bijection
Paul Hunter Strategy Improvement for Parity Games
![Page 25: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/25.jpg)
AUSOs: Polynomial Local Search
Common problem for AUSOs: Find the (global) sink by makinglocal queries
Cornerstone problem of the complexity class PLS
Open problem
Can we find the sink of an AUSO with polynomially many queries?
Paul Hunter Strategy Improvement for Parity Games
![Page 26: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/26.jpg)
AUSOs: Polynomial Local Search
Common problem for AUSOs: Find the (global) sink by makinglocal queries
Cornerstone problem of the complexity class PLS
Open problem
Can we find the sink of an AUSO with polynomially many queries?
Paul Hunter Strategy Improvement for Parity Games
![Page 27: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/27.jpg)
AUSOs: Finding the sink
Paul Hunter Strategy Improvement for Parity Games
![Page 28: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/28.jpg)
AUSOs: Finding the sink – Bottom-Antipodal
Paul Hunter Strategy Improvement for Parity Games
![Page 29: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/29.jpg)
AUSOs: Finding the sink – Bottom-Antipodal
Paul Hunter Strategy Improvement for Parity Games
![Page 30: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/30.jpg)
AUSOs: Finding the sink – Bottom-Antipodal
Paul Hunter Strategy Improvement for Parity Games
![Page 31: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/31.jpg)
AUSOs: Finding the sink – Bottom-Antipodal
Paul Hunter Strategy Improvement for Parity Games
![Page 32: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/32.jpg)
AUSOs: Finding the sink – Bottom-Antipodal
Paul Hunter Strategy Improvement for Parity Games
![Page 33: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/33.jpg)
Overview
Theorem
The strategy improvement algorithm for parity games is abottom-antipodal sink-finding algorithm on an acyclic unique sinkorientation of the strategy hypercube.
Paul Hunter Strategy Improvement for Parity Games
![Page 34: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/34.jpg)
Tying it all together: The strategy hypercube
Can assume every vertex in the arena of a parity game hasout-degree 2
Positional strategies for Player 0 are then vectors in {0, 1}n wheren = |V0|, that is, vertices of a hypercube
Edges of the hypercube connect strategies which differ at only onevertex
Paul Hunter Strategy Improvement for Parity Games
![Page 35: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/35.jpg)
Tying it all together: The orientation
The orientation is defined by the local valuation: an edge from σto σ′ if it is “better” for Player 0 to switch at the specified vertex
Theorem (Voge and Jurdzinski 2000)
The local valuation induces an AUSO on the strategy hypercube
Corollary
The strategy improvement algorithm for parity games is abottom-antipodal sink-finding algorithm on an acyclic unique sinkorientation of the strategy hypercube.
Paul Hunter Strategy Improvement for Parity Games
![Page 36: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/36.jpg)
Tying it all together: The orientation
The orientation is defined by the local valuation: an edge from σto σ′ if it is “better” for Player 0 to switch at the specified vertex
Theorem (Voge and Jurdzinski 2000)
The local valuation induces an AUSO on the strategy hypercube
Corollary
The strategy improvement algorithm for parity games is abottom-antipodal sink-finding algorithm on an acyclic unique sinkorientation of the strategy hypercube.
Paul Hunter Strategy Improvement for Parity Games
![Page 37: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/37.jpg)
What does this tell us?
Improve bounds:
Mansour and Singh (1999): Bottom-antipodal takes at most 2n/nsteps
Szabo and Welzl (2001): Fibonacci see-saw takes at mostO(1.61n) steps
Schurr and Szabo (2005): There exist AUSOs such thatbottom-antipodal takes O(2n/2) steps
Paul Hunter Strategy Improvement for Parity Games
![Page 38: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/38.jpg)
Further work
Question
Is every AUSO realizable as the oriented strategy hypercube of aparity game?
Paul Hunter Strategy Improvement for Parity Games
![Page 39: Strategy Improvement for Parity Games: A combinatorial ...pi.math.cornell.edu/...StrategyImprovement-slides.pdfStrategy Improvement: Overview Introduced by V¨oge and Jurdzin´ski](https://reader035.vdocuments.net/reader035/viewer/2022071215/60463c5f7896ae09ff7f3411/html5/thumbnails/39.jpg)
Further work
Improve bounds on the number of iterations necessary for thestrategy improvement algorithm
Classify AUSOs defined by parity games and restricted classes ofparity games
Paul Hunter Strategy Improvement for Parity Games