powerpoint presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 theorem: if 𝑳has a...
TRANSCRIPT
![Page 1: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/1.jpg)
1
6.045Lecture 8: More on
Communication Complexity,Start up Turing Machines
![Page 2: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/2.jpg)
2
6.045Announcements:
- Dylan’s office hours? Vote!- Midterm: March 19
![Page 3: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/3.jpg)
3
Communication Complexity
A theoretical model of distributed computing
• Function f : {0,1}* £ {0,1}* ! {0,1}
– Two inputs, 𝑥 ∈ {0,1}* and 𝑦 ∈ {0,1}*
– We assume |𝒙|=|𝒚|=𝒏. Think of 𝒏 as HUGE
• Two computers: Alice and Bob
– Alice only knows 𝑥, Bob only knows 𝑦
• Goal: Compute f(𝒙, 𝒚) by communicating as few bits as possible between Alice and Bob
We do not count computation cost. We only care about the number of bits communicated.
![Page 4: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/4.jpg)
4
A(x,ε) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
x y
f(x,y)=0f(x,y)=0
Def. A protocol for a function f is a pair of functionsA, B : {0,1}* × {0,1}* → {0, 1, STOP} with the semantics:
On input (𝒙, 𝒚), let 𝒓 := 0, 𝒃𝟎 = ε. While (𝒃𝒓 ≠ STOP),
𝒓 + +If 𝒓 is odd, Alice sends 𝒃𝒓 = 𝑨 𝒙, 𝒃𝟏⋯𝒃𝒓−𝟏
else Bob sends 𝒃𝒓 = 𝑩 𝒚, 𝒃𝟏⋯𝒃𝒓−𝟏Output 𝒃𝒓−𝟏. Number of rounds = 𝒓 − 𝟏
![Page 5: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/5.jpg)
5
A(x,ε) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
x y
f(x,y)=0f(x,y)=0
Def. The cost of a protocol P for f on 𝒏-bit strings is
𝐦𝐚𝐱𝒙,𝒚 ∈ 𝟎,𝟏 𝒏
[number of rounds in P to compute f(𝒙, 𝒚)]
The communication complexity of f on 𝒏-bit strings, cc(f),is minimum cost over all protocols for f on 𝒏-bit strings= the minimum number of rounds used by any protocol
computing f(𝒙, 𝒚), over all 𝒏-bit 𝒙, 𝒚
![Page 6: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/6.jpg)
6
A(x,ε) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
x y
f(x,y)=0f(x,y)=0
Example. Let f : {0,1}* × {0,1}* → {0,1} be arbitrary
There is always a “trivial” protocol:Alice sends the bits of her 𝒙 in odd roundsBob sends whatever bit he wants in even roundsAfter 𝟐𝒏 − 𝟏 rounds, Bob knows 𝒙 and can send 𝒇(𝒙, 𝒚)
Proposition: For every 𝒇, cc(𝒇) ≤ 𝟐𝒏
![Page 7: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/7.jpg)
7
x y
f(x,y)=0f(x,y)=0
Example. EQUALS(𝒙, 𝒚) = 1 ⇔ 𝒙 = 𝒚
What’s a good protocol for computing EQUALS?
????
Communication complexity of EQUALS is at most 2𝒏
![Page 8: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/8.jpg)
8
x y
Examples:𝑳 = { x | x has an odd number of 1s}
⇒ 𝒇𝑳 𝒙, 𝒚 = PARITY(x,y) = σ𝒊𝒙𝒊 + σ𝒊𝒚𝒊 mod 2𝑳 = { x | x has more 1s than 0s}
⇒ 𝒇𝑳 𝒙, 𝒚 = MAJORITY(x,y)𝑳 = { xx | x ∈ {0,1}*}
⇒ 𝒇𝑳 𝒙, 𝒚 = EQUALS(x,y)
Connection to Streaming and DFAs
Let 𝑳 ⊆ {0,1}*Def. 𝒇𝑳: {0,1}*×{0,1}*→ {0,1}
for 𝒙, 𝒚 with |𝒙|=|𝒚| as:𝒇𝑳 𝒙, 𝒚 = 𝟏 ⇔ 𝒙𝒚 ∈ 𝑳
![Page 9: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/9.jpg)
9
Theorem: If 𝑳 has a streaming algorithm using ≤ 𝒔 space, then cc(𝒇𝑳) ≤ 4𝒔 + 4.
Proof Idea: Alice runs streaming algorithm A on 𝒙, reaches a memory state 𝒎. She sends 𝒎 to Bob in 4s+3rounds. Then Bob starts up A from state 𝒎, runs A on 𝒚. Gets an output bit, sends bit to Alice.
(…why 4𝒔+3 rounds?)
Connection to Streaming and DFAs
x y
Let 𝑳 ⊆ {0,1}*Def. 𝒇𝑳: {0,1}*×{0,1}*→ {0,1}
for 𝒙, 𝒚 with |𝒙|=|𝒚| as:𝒇𝑳 𝒙, 𝒚 = 𝟏 ⇔ 𝒙𝒚 ∈ 𝑳
![Page 10: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/10.jpg)
10
Theorem: If 𝑳 has a streaming algorithm using ≤ 𝒔 space, then cc(𝒇𝑳) ≤ 4𝒔 + 4.
Corollary: For every regular 𝑳, cc(𝒇𝑳) ≤ O(1).
Example: cc(PARITY) = 2
Corollary: cc(MAJORITY) ≤ O(log n),because there’s a streaming algorithm for {x : x has more 1’s than 0’s} with O(log n) space
What about the Comm. Complexity of EQUALS?
Connection to Streaming and DFAs
Let 𝑳 ⊆ {0,1}* Def. 𝒇𝑳 𝒙, 𝒚 = 𝟏 ⇔ 𝒙𝒚 ∈ 𝑳
![Page 11: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/11.jpg)
11
Theorem: cc(EQUALS) = 𝚯(𝒏).In particular, every communication protocol for EQUALS needs to send ≥ 𝒏 bits.
No communication protocol can do much better than “send your whole input”!
Corollary: 𝑳 = {xx | x in {0,1}*} is not regular.
Moreover, every streaming algorithm for 𝑳needs 𝒄 𝒏 bits of memory, for some constant 𝒄 > 0!
Communication Complexity of EQUALS
![Page 12: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/12.jpg)
12
Theorem: cc(EQUALS) = 𝚯(𝒏). In particular, everyprotocol for EQUALS needs ≥ 𝒏 bits of communication!
Idea: Consider all possible ways A & B can communicate.
Definition: The communication pattern of a protocol on inputs (𝒙, 𝒚) is the sequence of bits Alice & Bob send.
0
1
1
0
Pattern: 0110𝒙 𝒚
Communication Complexity of EQUALS
![Page 13: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/13.jpg)
13
A(x’,ε) = 0
B(y’,0) = 1
A(x’,01) = 1
B(y’,011) = 0
𝒙’ 𝒚’
A(x,ε) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
𝒙 𝒚
Key Lemma: If (𝒙, 𝒚) and (𝒙′, 𝒚′) have the same pattern 𝑷in a protocol, then (𝒙, 𝒚′) and (𝒙′, 𝒚) also have pattern 𝑷
![Page 14: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/14.jpg)
14
A(x’,ε) = 0
B(y’,0) = 1
A(x’,01) = 1
B(y’,011) = 0
𝒙’ 𝒚’
A(x,ε) = 0
B(y’,0) = 1
A(x,01) = 1
B(y’,011) = 0
𝒙 𝒚
Key Lemma: If (𝒙, 𝒚) and (𝒙′, 𝒚′) have the same pattern 𝑷in a protocol, then (𝒙, 𝒚′) and (𝒙′, 𝒚) also have pattern 𝑷
![Page 15: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/15.jpg)
15
Key Lemma: If (𝒙, 𝒚) and (𝒙′, 𝒚′) have the same pattern 𝑷in a protocol, then (𝒙, 𝒚′) and (𝒙′, 𝒚) also have pattern 𝑷
A(x,ε) = 0
B(y’,0) = 1
A(x,01) = 1
B(y’,011) = 0
𝒙’ 𝒚’
A(x,ε) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
𝒙 𝒚
![Page 16: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/16.jpg)
16
Theorem: The comm. complexity of EQUALS is 𝚯(𝒏).In particular, every protocol for EQUALS needs ≥ 𝒏 bits of communication.
Proof: By contradiction. Suppose cc(EQUALS) ≤ 𝒏 − 𝟏.Then there are ≤ 𝟐𝒏 − 𝟏 possible communication patterns of that protocol, over all pairs of inputs (𝒙, 𝒚) with n bits each.
Claim: There are 𝒙 ≠ 𝒚 such that on (𝒙, 𝒙) and on (𝒚, 𝒚), the protocol uses the same pattern 𝑷.
By the Key Lemma, (𝒙, 𝒚) and 𝒚, 𝒙 also use pattern 𝑷
So Alice & Bob output the same bit on (𝒙, 𝒚) and (𝒙, 𝒙).But EQUALS(𝒙, 𝒚) = 0 and EQUALS(𝒙, 𝒙) = 1. Contradiction!
Communication Complexity of EQUALS
![Page 17: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/17.jpg)
17
Randomized Protocols Help!
EQUALS needs ≥ 𝒏 bits of communication, but…
Theorem: For all 𝒙, 𝒚 of 𝒏 bits each, there is a randomized protocol for EQUALS 𝒙, 𝒚 using
only O(log 𝒏) bits of communication, which is correct with probability 99.9%!
![Page 18: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/18.jpg)
18
Turing Machines
![Page 19: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/19.jpg)
Turing Machine (1936)
FINITE
STATE
CONTROL
INFINITE REWRITABLE TAPE
I N P U T
q0q1
A …
19
In each step:- Reads a symbol- Writes a symbol- Changes state- Moves Left or Right
![Page 20: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/20.jpg)
Turing Machine (1936)
INFINITE REWRITABLE TAPE
I N P U TA …
20
![Page 21: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/21.jpg)
21
https://www.cs.utah.edu/~draperg/cartoons/2005/turing.html
![Page 22: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/22.jpg)
Turing Machines versus DFAs
The TM can both write to and read from the tape,and can write symbols that aren’t part of input
The “tape head” can move right and left
The input is written on an infinite tape
Accept and Reject take immediate effect
with “blank” symbols after the input
22
![Page 23: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/23.jpg)
23
Deciding the language L = { w#w | w {0,1}* }
STATE
0 1 1 # 0 1 10 #1 0 1X XX
1. If there’s no # on the tape (or more than one #), reject. 2. While there is a bit to the left of #,
Replace the first bit with X, and check if the first bit bto the right of the # is identical. (If not, reject.) Replace that bit b with an X too.
3. If there’s a bit to the right of #, then reject else accept
![Page 24: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/24.jpg)
Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, , q0, qaccept, qreject), where:
Q is a finite set of states
Γ is the tape alphabet, where Γ and Σ Γ
q0 Q is the start state
Σ is the input alphabet, where Σ
: Q Γ → Q Γ {L, R}
qaccept Q is the accept state
qreject Q is the reject state, and qreject qaccept
24
= “blank”
![Page 25: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/25.jpg)
25
0 → 0, R
read write move
→ , R
qaccept
qreject
0 → 0, R
→ , R
This Turing machine decides the language {0}
Σ = {0}
= “blank”
![Page 26: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/26.jpg)
26
0 → 0, R
read write move
→ , R
qaccept
0 → 0, R
→ , R
0 → 0, R
→ , L
This Turing machine recognizes the language {0}
Σ = {0}
= “blank”
![Page 27: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/27.jpg)
Turing Machine Configurations
q01101000110 2 (Q [ Γ)*
corresponds to the configuration:
q0
1 0 0 0 0 01 1 1 1
27
![Page 28: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/28.jpg)
Turing Machine Configurations
0q1101000110 2 (Q [ Γ)*
corresponds to the configuration:
q1
1 0 0 0 0 00 1 1 1
28
![Page 29: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/29.jpg)
Turing Machine Configurations
0000011110q7 2 (Q [ Γ)*
corresponds to the configuration:
q7
0 0 0 1 1 00 0 1 1
29
![Page 30: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/30.jpg)
Defining Acceptance and Rejection for TMs
Let C1 and C2 be configurations of a TM MDefinition. C1 yields C2 if M is in configuration C2
after running M in configuration C1 for one step
Example. Suppose (q1, b) = (q2, c, L)Then aq1bb yields q2acbSuppose (q1, a) = (q2, c, R)Then abq1a yields abcq2
Let w Σ* and M be a Turing machine.M accepts w if there are configs C0, C1, ..., Ck, s.t.
• C0 = q0w [the initial configuration]• Ci yields Ci+1 for i = 0, ..., k-1, and • Ck contains the accept state qaccept
30
accepting computation
history of M on x
![Page 31: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/31.jpg)
A TM M recognizes a language L if M accepts exactly those strings in L
A TM M decides a language L if M accepts all strings in L and rejects all strings not in L
A language L is recognizable (a.k.a. recursively enumerable)
if some TM recognizes L
A language L is decidable (a.k.a. recursive)if some TM decides L
31
![Page 32: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/32.jpg)
A Turing machine for deciding { 0 | n ≥ 0 }2n
1. Sweep from left to right, x-out every other 02. If in step 1, the tape had only one 0, accept3. If in step 1, the tape had an odd number of 0’s,
reject4. Move the head left to the first input symbol.5. Go to step 1.
Turing Machine PSEUDOCODE:
Why does this work?
Observation: Every time we return to step 1, the number of 0’s on the tape has been halved.
32
![Page 33: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/33.jpg)
even0’s
Step 1
odd 0’s
0 → , R
→ , R
qacceptqreject
0 → x, R
x → x, R→ , R
x → x, R
0 → 0, Lx → x, L
x → x, R
→ , L→ , R
0 → x, R0 → 0, R
→ , Rx → x, R
q0 q1
q2
q3
q4
{ 0 | n ≥ 0 }2n
Step 2
Step 3
Step 4
33
![Page 34: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/34.jpg)
0 → , R
→ , R
qacceptqreject
0 → x, R
x → x, R→ , R
x → x, R
0 → 0, Lx → x, L
x → x, R
→ , L→ , R
0 → x, R0 → 0, R
→ , Rx → x, R
{ 0 | n ≥ 0 }2n
q0 q1
q2
q3
q4
q00000
q1000
xq300
x0q40
x0xq3
x0q2x
xq20x
q2x0x
q2x0x
… 34
q1x0x
xq10x
![Page 35: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/35.jpg)
Multitape Turing Machines
: Q Γk → Q Γk {L,R}k
FINITE
STATE
CONTROLk
35
![Page 36: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/36.jpg)
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. . .
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
36
![Page 37: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/37.jpg)
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. . .
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
37
![Page 38: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/38.jpg)
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. . .
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
38
![Page 39: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/39.jpg)
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. .
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
39
.
![Page 40: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/40.jpg)
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. . .
40
![Page 41: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/41.jpg)
41
Theorem: Every nondeterministic Turing machine Ncan be transformed into a Turing Machine M that accepts precisely the same strings as N.
Nondeterministic Turing Machines
Proof Idea (more details in Sipser p.178-179)Pick a natural ordering on the strings in (Q [ Γ [ #)*
M(w): For all strings D 2 (Q [ Γ [ #)* in the ordering,Check if D = C0# #Ck where C0, …,Ck is an
accepting computation history for N on w. If so, accept.
Have multiple transitions for a state, symbol pair
![Page 42: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/42.jpg)
Fact: We can encode Turing Machines as bit strings
0n10m10k10s10t10r10u1 …
n states
m tape symbols
(first k are input
symbols)
start
state
accept
state
reject
state
blank
symbol
( (p, i), (q, j, L) ) = 0p10i10q10j101
( (p, i), (q, j, R) ) = 0p10i10q10j100142
![Page 43: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/43.jpg)
Similarly, we can encode DFAs and NFAs as bit strings, and w 2 Σ* as bit strings
ADFA = { (B, w) | B encodes a DFA over some Σ,and B accepts w 2 Σ* }
ANFA = { (B, w) | B encodes an NFA, B accepts w }
ATM = { (M, w) | M encodes a TM, M accepts w }
For x 2 Σ* define bΣ(x) to be its binary encodingFor x, y 2 Σ*, define the pair of x and y as
(x, y) := 0|bΣ(x)|1 bΣ(x) bΣ(y)Then we define the following languages over {0,1}:
43
![Page 44: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/44.jpg)
44
ATM = { (M, w) | M encodes a TM over some Σ,w encodes a string over Σ and M accepts w}
ATM = { z | z decodes to (M, w) and M does not accept w }
Technical Note: We’ll use an decoding of pairs, TMs, and strings so that every binary string decodes to some pair (M, w)
If z ∈ {0,1}* doesn’t decode to (M, w) in the usual way, then we define that z decodes to the pair (D, ε)
where D is a “dummy” TM that accepts nothing.
![Page 45: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/45.jpg)
Universal Turing Machines
Theorem: There is a Turing machine Uwhich takes as input:- the code of an arbitrary TM M- and an input string wsuch that U accepts (M, w)M accepts w.
This is a fundamental property of TMs:
There is a Turing Machine that can run arbitrary Turing Machine code!
Note that DFAs/NFAs do not have this property.That is, ADFA and ANFA are not regular.
45
![Page 46: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/46.jpg)
46
ADFA = { (D, w) | D is a DFA that accepts string w }
Theorem: ADFA is decidable
Proof: A DFA is a special case of a TM. Run the universal U on (D, w) and output its answer.
ATM = { (M, w) | M is a TM that accepts string w }
Theorem: ATM is recognizable
ANFA = { (N, w) | N is an NFA that accepts string w }
Theorem: ANFA is decidable. (Why?)
![Page 47: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/47.jpg)
The Church-Turing Thesis
Everyone’s Intuitive Notion
of Algorithms
= Turing Machines
This is not a theorem –it is a falsifiable scientific hypothesis.
And it has been thoroughly tested!
![Page 48: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/48.jpg)
w L ?
accept reject
TM
yes no
w Σ*
L is decidable
(recursive)
w L ?
accept reject or loop
TM
yes no
w Σ*
L is recognizable
(recursively enumerable)
Theorem: L is decidableiff both L and L are recognizable
48
![Page 49: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/49.jpg)
Recall: Given L µ Σ*, define L := Σ* \ L
How? Any ideas?M1 always accepts x, when x is in L
M2 always accepts x, when x isn’t in L
Theorem: L is decidableiff both L and L are recognizable
Given:a TM M1 that recognizes L anda TM M2 that recognizes L,
want to build a new machine M that decides L
49
![Page 50: PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If 𝑳has a streaming algorithm using Q space, then cc(𝒇𝑳Q4 + 4. Proof Idea: Alice runs streaming](https://reader030.vdocuments.net/reader030/viewer/2022040801/5e38f40627f38615b65c6211/html5/thumbnails/50.jpg)
Theorem: L is decidableiff both L and L are recognizable
Decider for L:Simulate M1(x) on one tape, while also simulating M2(x) on another tape. Exactly one will accept. If M1 accepts then accept. If M2 accepts then reject
Recall: Given L µ Σ*, define L := Σ* \ L
Given:a TM M1 that recognizes L anda TM M2 that recognizes L,
want to build a new machine M that decides L
50