![Page 1: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/1.jpg)
![Page 2: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/2.jpg)
2
Symbolic Finite State Transducers:Algorithms and Applications
Margus VeanesPieter HooimeijerBenjamin LivshitsDavid Molnar Nikolaj Bjørner
![Page 3: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/3.jpg)
3
Symbolic Finite State Transducers:Algorithms and Applications
Margus VeanesPieter HooimeijerBenjamin LivshitsDavid Molnar Nikolaj Bjørner
![Page 4: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/4.jpg)
4
Formal languagesare well-studied.
![Page 5: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/5.jpg)
5
a*b+𝑞0
a𝑞1
b
b
✔abb aaaa✘
![Page 6: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/6.jpg)
6
Series10
20
40
60
80
100
120103
Num
ber o
f pap
ers
“automata”
POPL (2001–2011)
![Page 7: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/7.jpg)
7
What about
transformation?
![Page 8: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/8.jpg)
8http://en.wikipedia.org/wiki/Osborne_1
![Page 9: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/9.jpg)
9
![Page 10: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/10.jpg)
10
Compute image:
Check properties: Equivalence Composition
✔ abb{baa}
aaaa ✘𝑞0 𝑞1
a/b
b/a
b/a
![Page 11: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/11.jpg)
11
Series10
20
40
60
80
100
120103
8
Num
ber o
f pap
ers
“automata” “transducers”
POPL (2001–2011)
![Page 12: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/12.jpg)
12
Talk Outline
Background Approach Case Studies
![Page 13: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/13.jpg)
13
Background
“Fast and Precise Sanitizer Analysis with BEK”
Idea:Develop a language for commonly-used string transformations. Prove properties about those transfor-mations.
![Page 14: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/14.jpg)
14
Code
𝑞0 𝑞1
a/b
b/a
b/a
t := iter(c in s)[b := false;]{ case (!b && c in "['\"\\]"): b := false; yield('\\', c); case (c == '\\'): b := !b; yield(c); case (true): b := false; yield(c);};
FSTs
Gap
![Page 15: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/15.jpg)
15
Code
𝑞0 𝑞1
a/b
b/a
b/a
t := iter(c in s)[b := false;]{ case (!b && c in "['\"\\]"): b := false; yield('\\', c); case (c == '\\'): b := !b; yield(c); case (true): b := false; yield(c);};
FSTs
1
domain-specific languages
![Page 16: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/16.jpg)
16
Code
𝑞0 𝑞1
a/b
b/a
b/a
t := iter(c in s)[b := false;]{ case (!b && c in "['\"\\]"): b := false; yield('\\', c); case (c == '\\'): b := !b; yield(c); case (true): b := false; yield(c);};
FSTs
1
more expressive
transducers2
domain-specific languages
![Page 17: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/17.jpg)
17
domain-specific languages
Code
𝑞0 𝑞1
a/b
b/a
b/a
t := iter(c in s)[b := false;]{ case (!b && c in "['\"\\]"): b := false; yield('\\', c); case (c == '\\'): b := !b; yield(c); case (true): b := false; yield(c);};
FSTs
1
more expressive
transducers2
![Page 18: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/18.jpg)
18
Talk Outline
Background Approach Case Studies
![Page 19: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/19.jpg)
19
![Page 20: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/20.jpg)
20
Symbolic Finite State Transducers
Idea:• Equip transitions with formulae• Allow the use of any decidable
theory
![Page 21: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/21.jpg)
21
Definition
Symbolic Finite State Transducer (SFT):
![Page 22: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/22.jpg)
22
Symbolic Finite State Transducer (SFT):
- states- start state- final states
![Page 23: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/23.jpg)
23
Symbolic Finite State Transducer (SFT):
- states- start state- final states
𝑅𝑞𝜙/ 𝒇→
𝑟
![Page 24: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/24.jpg)
24
Symbolic Finite State Transducer (SFT):
- states- start state- final states
𝑅𝑞𝜙/ 𝒇→
𝑟
predicates output
![Page 25: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/25.jpg)
25
Symbolic Finite State Transducer (SFT):
- states- start state- final states- transition
Background Theory:
- predicates
- label theory
![Page 26: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/26.jpg)
26
Example𝑞0
𝑞1(𝜆 𝑥 . 𝑥=0 )/ [𝜆𝑥 .1 ]
(𝜆 𝑥 .𝐭 )/ [𝜆𝑥 .2𝑥 ]
![Page 27: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/27.jpg)
27
𝑞0
𝑞1(𝜆 𝑥 . 𝑥=0 )/ [𝜆𝑥 .1 ]
(𝜆 𝑥 .𝐭 )/ [𝜆𝑥 .2𝑥 ]guards symbolic outputs
![Page 28: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/28.jpg)
28
![Page 29: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/29.jpg)
29
Closure under composition
SFT A B
in outSFT A in outSFT B
Requirement:
![Page 30: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/30.jpg)
30
Single-valued equivalence
Definition:1𝑎 :𝜎 ∗
𝑏𝑐 :𝛾∗
𝑏∈ 𝐴(𝑎)𝑐∈𝐵(𝑎)
𝑏=𝑐
![Page 31: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/31.jpg)
31
Algorithm:• Construct 2-output
product transducer• Find conflicts (dft):– output length– output value
Complexity:
𝑂 (𝑛2 ⋅ 𝑓 (𝑚 ) )number of rules
complexity of decision procedure
![Page 32: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/32.jpg)
32
Key restriction: single-valuedness
Transducer A is single-valued if, for all inputs, A has at most one out-put.
𝐴=𝐴1
![Page 33: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/33.jpg)
33
Note: This definition permits non-determinism, e.g.:
b/[]
b/[]
......
...
Transducer A is single-valued if, for all inputs, A has at most one out-put.
𝐴=𝐴1
![Page 34: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/34.jpg)
34
![Page 35: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/35.jpg)
35
algebra
![Page 36: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/36.jpg)
36
subsumption equivalence idempotence
commutativity
...
algebra interesting properties
![Page 37: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/37.jpg)
37
Talk Outline
Background Approach Case Studies
![Page 38: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/38.jpg)
38
![Page 39: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/39.jpg)
39
Case Studies
HTMLdecode
"b"'b'
MalwareFingerprinting
ImageBlurring
LocationPrivacy
![Page 40: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/40.jpg)
40
HTMLdecode
"b"'b'
MalwareFingerprinting
ImageBlurring
LocationPrivacy
![Page 41: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/41.jpg)
41
HTMLdecode
"<""<" "<""<"
Decode
![Page 42: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/42.jpg)
42
"<""<" "<""<"
Decode
The Task: Prove that HTMLdecode is not idempotent
The Metric: Running time
![Page 43: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/43.jpg)
43
"<""<" "<""<"
Decode
The Problem: Unicode defines 1,114,112 code points.
![Page 44: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/44.jpg)
44
Three Participating Representations
C#
SFT (Eager)
C# C#
SFT+Registers(Eager)
+REG+REG
SFT+Registers(Lazy)
![Page 45: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/45.jpg)
45
1
10
100
1,000
10,000
100,000
1,000,000
10,000,000
2 3 4 5 6
Transducer size ()6.6M
maximum number of digits
![Page 46: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/46.jpg)
46
C#
SFT (Eager)
C# C#
SFT+Registers(Eager)
+REG+REG
SFT+Registers(Lazy)
![Page 47: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/47.jpg)
47
1
10
100
1,000
10,000
100,000
1,000,000
10,000,000
2 3 4 5 6
Transducer size ()6.6M
51
SFT
SFT + Symbolic State Space
maximum number of digits
![Page 48: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/48.jpg)
48
1.000
10.000
100.000
1,000.000
10,000.000
100,000.000
1,000,000.000
Tim
e (s
econ
ds; l
og s
cale
)
maximum number of digits2 3 4 5 6
Idempotence Checking: TimeSFT SFT +
REG(lazy)
SFT + REG(eager)
![Page 49: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/49.jpg)
49
Talk Outline
Background Approach Case Studies
![Page 50: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/50.jpg)
50
Conclusion
• Introduced Symbolic Finite State Transducers over any decidable background theory
• Presented decidability and complexity results
• Comes with a scalable and robust* implementation
![Page 51: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/51.jpg)
51
Thank you!Please try our…
implementation
http://research.microsoft.com/automata/
online tutorial
http://www.rise4fun.com/Bek/tutorial
![Page 52: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/52.jpg)
52
![Page 53: Symbolic Finite State Transducers: Algorithms and Applications](https://reader036.vdocuments.net/reader036/viewer/2022062520/56815b20550346895dc8dba3/html5/thumbnails/53.jpg)
53
http://www.rise4fun.com/Bek