1 unit 1: automata theory and formal languages readings 1, 2.2, 2.3
TRANSCRIPT
![Page 1: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/1.jpg)
1
Unit 1: Automata Theoryand Formal Languages
Readings 1, 2.2, 2.3
![Page 2: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/2.jpg)
2
What is automata theory
• Automata theory is the study of abstract computational devices
• Abstract devices are (simplified) models of real computations
• Computations happen everywhere: On your laptop, on your cell phone, in nature, …
• Why do we need abstract models?
![Page 3: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/3.jpg)
3
A simple computer
BATTERY
SWITCH
input: switch
output: light bulb
actions: flip switch
states: on, off
![Page 4: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/4.jpg)
4
A simple “computer”
BATTERY
SWITCH
off onstart
f
f
input: switch
output: light bulb
actions: f for “flip switch”
states: on, off
bulb is on if and only if there was an odd number of flips
![Page 5: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/5.jpg)
5
Another “computer”
BATTERY
off offstart
1
inputs: switches 1 and 2
actions: 1 for “flip switch 1”actions: 2 for “flip switch 2”
states: on, off
bulb is on if and only if both switches were flipped an odd number of times
1
2
1
off on
1
1
2 2 2 2
![Page 6: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/6.jpg)
6
A design problem
Can you design a circuit where the light is on if and only if all the switches were flipped exactly the same number of times?
4
BATTERY
1
2
3
5
?
![Page 7: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/7.jpg)
7
A design problem
• Such devices are difficult to reason about, because they can be designed in an infinite number of ways
• By representing them as abstract computational devices, or automata, we will learn how to answer such questions
![Page 8: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/8.jpg)
8
These devices can model many things• They can describe the operation of any
“small computer”, like the control component of an alarm clock or a microwave
• They are also used in lexical analyzers to recognize well formed expressions in programming languages:ab1 is a legal name of a variable in C
5u= is not
![Page 9: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/9.jpg)
9
Different kinds of automata
• This was only one example of a computational device, and there are others
• We will look at different devices, and look at the following questions:– What can a given type of device compute, and
what are its limitations?– Is one type of device more powerful than
another?
![Page 10: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/10.jpg)
10
Some devices we will see
finite automata Devices with a finite amount of memory.Used to model “small” computers.
push-down automata
Devices with infinite memory that can be accessed in a restricted way.
Used to model parsers, etc.
Turing Machines
Devices with infinite memory.
Used to model any computer.
time-bounded Turing Machines
Infinite memory, but bounded running time.
Used to model any computer program that runs in a “reasonable” amount of time.
![Page 11: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/11.jpg)
11
Some highlights of the course
• Finite automata– We will understand what kinds of things a
device with finite memory can do, and what it cannot do
– Introduce simulation: the ability of one device to “imitate” another device
– Introduce nondeterminism: the ability of a device to make arbitrary choices
• Push-down automata– These devices are related to grammars, which
describe the structure of programming (and natural) languages
![Page 12: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/12.jpg)
12
Some highlights of the course
• Turing Machines– This is a general model of a computer,
capturing anything we could ever hope to compute
– Surprisingly, there are many things that we cannot compute, for example:
– It seems that you should be able to tell just by looking at the program, but it is impossible to do!
Write a program that, given the code of another program in C, tells if this program ever outputs the word “hello”
![Page 13: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/13.jpg)
13
Some highlights of the course
• Time-bounded Turing Machines– Many problems are possible to solve on a
computer in principle, but take too much time in practice
– Traveling salesman: Given a list of cities, find the shortest way to visit them and come back home
– Easy in principle: Try the cities in every possible order
– Hard in practice: For 100 cities, this would take 100+ years even on the fastest computer!
Hong Kong
Beijing
ShanghaiXian
Guangzhou
Chengdu
![Page 14: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/14.jpg)
14
Preliminaries of automata theory
• How do we formalize the question
• First, we need a formal way of describing the problems that we are interested in solving
Can device A solve problem B?
![Page 15: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/15.jpg)
15
Problems
• Examples of problems we will consider– Given a word s, does it contain the subword
“fool”?– Given a number n, is it divisible by 7?– Given a pair of words s and t, are they the
same?– Given an expression with brackets, e.g. (()()),
does every left bracket match with a subsequent right bracket?
• All of these have “yes/no” answers. • There are other types of problems, that ask
“Find this” or “How many of that” but we won’t look at those.
![Page 16: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/16.jpg)
16
Alphabets and strings
• A common way to talk about words, number, pairs of words, etc. is by representing them as strings
• To define strings, we start with an alphabet
• Examples
An alphabet is a finite set of symbols.
S1 = {a, b, c, d, …, z}: the set of letters in English
S2 = {0, 1, …, 9}: the set of (base 10) digits
S3 = {a, b, …, z, #}: the set of letters plus the special symbol #
S4 = {(, )}: the set of open and closed brackets
![Page 17: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/17.jpg)
17
Strings
• The empty string will be denoted by e• Examples
A string over alphabet S is a finite sequenceof symbols in S.
abfbz is a string over S1 = {a, b, c, d, …, z}
9021 is a string over S2 = {0, 1, …, 9}
ab#bc is a string over S3 = {a, b, …, z, #}
))()(() is a string over S4 = {(, )}
![Page 18: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/18.jpg)
18
Languages
• Languages can be used to describe problems with “yes/no” answers, for example:
A language is a set of strings over an alphabet.
L1 = The set of all strings over S1 that containthe substring “fool”
L2 = The set of all strings over S2 that are divisible by 7 = {7, 14, 21, …}L3 = The set of all strings of the form s#s where s is any
string over {a, b, …, z}L4 = The set of all strings over S4 where every ( can be
matched with a subsequent )
![Page 19: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/19.jpg)
19
Finite Automata
![Page 20: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/20.jpg)
20
Example of a finite automaton
• There are states off and on, the automaton starts in off and tries to reach the “good state” on
• What sequences of fs lead to the good state?
• Answer: {f, fff, fffff, …} = {f n: n is odd}• This is an example of a deterministic finite
automaton over alphabet {f}
off on
f
f
![Page 21: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/21.jpg)
21
Deterministic finite automata
• A deterministic finite automaton (DFA) is a 5-tuple (Q, S, d, q0, F) where– Q is a finite set of states– S is an alphabet– d: Q × S → Q is a transition function– q0 Î Q is the initial state– F Í Q is a set of accepting states (or final
states).
• In diagrams, the accepting states will be denoted by double loops
![Page 22: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/22.jpg)
22
Example
q0 q1 q21 0
0 0,11
alphabet S = {0, 1}set of states Q = {q0, q1, q2}initial state q0
accepting states F = {q0, q1}
state
s
inputs
0 1q0
q1
q2
q0 q1
q2
q2q2
q1
transition function :d
![Page 23: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/23.jpg)
23
Language of a DFA
The language of a DFA (Q, S, d, q0, F) is the set of all strings over S that, starting from q0 and following the transitions as the string is read leftto right, will reach some accepting state.
• Language of M is {f, fff, fffff, …} = {f n: n is odd}
off on
f
f
M:
![Page 24: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/24.jpg)
24
q0 q1
q0 q1
q0 q1 q2
0 0
1
1
0 1
1
0
1 0
0 0,11
What are the languages of these DFAs?
Examples
![Page 25: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/25.jpg)
25
Examples
• Construct a DFA that accepts the language
L = {010, 1}
( S = {0, 1} )
![Page 26: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/26.jpg)
26
Examples
• Construct a DFA that accepts the language
• Answer
L = {010, 1}
( S = {0, 1} )
qe
q0
q1
q01 q010
qdie 0, 1
0
1 0
0, 11
0 10, 1
![Page 27: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/27.jpg)
27
Examples
• Construct a DFA over alphabet {0, 1} that accepts all strings that end in 101
![Page 28: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/28.jpg)
28
Examples
• Construct a DFA over alphabet {0, 1} that accepts all strings that end in 101
• Hint: The DFA must “remember” the last 3 bits of the string it is reading
![Page 29: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/29.jpg)
29
Examples
0
1
…
…
……
qe
q0
q1
q00
q10
q01
q11
q000
q001
q101
q111
0
1
0
1
0
1
1
1
1
1
0
• Construct a DFA over alphabet {0, 1} that accepts all strings that end in 101
• Sketch of answer:
![Page 30: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/30.jpg)
30
Would be easier if…
• Suppose we could guess when the string we are reading has only 3 symbols left
• Then we could simply look for the sequence 101and accept if we see it
qdie
1 0 13 symbols left
This is not a DFA!
![Page 31: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/31.jpg)
31
Nondeterminism
• Nondeterminism is the ability to make guesses, which we can later verify
• Informal nondeterministic algorithm for language of strings that end in 101:
1. Guess if you are approaching end of input2. If guess is yes, look for 101 and accept if you see it3. If guess is no, read one more symbol and go to step 1
![Page 32: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/32.jpg)
32
Nondeterministic finite automaton
• This is a kind of automaton that allows you to make guesses
• Each state can have zero, one, or more transitions out labeled by the same symbol
1 0 1
0, 1
q0 q1 q2 q3
![Page 33: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/33.jpg)
33
Semantics of guessing
1 0 1
0, 1
q0 q1 q2 q3
• State q0 has two transitions labeled 1
• Upon reading 1, we have the choice of staying in q0 or moving to q1
![Page 34: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/34.jpg)
34
Semantics of guessing
1 0 1
0, 1
q0 q1 q2 q3
• State q1 has no transition labeled 1
• Upon reading 1 in q1, we die; upon reading 0, we continue to q2
![Page 35: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/35.jpg)
35
Semantics of guessing
1 0 1
0, 1
q0 q1 q2 q3
• State q3 has no transition going out
• Upon reading anything in q3, we die
![Page 36: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/36.jpg)
36
Meaning of automaton
1 0 1
0, 1
q0 q1 q2 q3
Guess if you are 3 symbols away from end of input
If so, guess you will see the pattern 101
Check that you are at the end of input
![Page 37: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/37.jpg)
37
Formal definition
• A nondeterministic finite automaton (NFA) is a 5-tuple (Q, S, d, q0, F) where– Q is a finite set of states– S is an alphabet– d: Q × S → subsets of Q is a transition function– q0 Î Q is the initial state– F Í Q is a set of accepting states (or final
states).
• Only difference from DFA is that output of d is a set of states
![Page 38: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/38.jpg)
38
Example
1 0 1
0, 1
q0 q1 q2 q3
alphabet S = {0, 1}states Q = {q0, q1, q2, q3}initial state q0
accepting states F = {q3}
stat
es
inputs0 1
q0
q1
q2
{q0, q1}
transition function :d
q3
{q0}
{q2}
Æ
Æ
Æ
{q3}
Æ
![Page 39: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/39.jpg)
39
Language of an NFA
The language of an NFA is the set of all strings forwhich there is some path that, starting from the initial state, leads to an accepting state as the string is read left to right.
1 0 1
0, 1
q0 q1 q2 q3
• Example
– 1101 is accepted, but 0110 is not
![Page 40: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/40.jpg)
40
NFAs are as powerful as DFAs
• Obviously, an NFA can do everything a DFA can do
• But can it do more?
![Page 41: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/41.jpg)
41
NFAs are as powerful as DFAs
• Obviously, an NFA can do everything a DFA can do
• But can it do more?
• TheoremA language L is accepted by some DFA if andonly if it is accepted by some NFA.
NO!
![Page 42: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/42.jpg)
42
Proof of theorem
• To prove the theorem, we have to show that for every NFA there is a DFA that accepts the same language
• We will give a general method for simulating any NFA by a DFA
• Let’s do an example first
![Page 43: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/43.jpg)
43
Simulation example
1 0
0, 1
q0 q1 q2NFA:
DFA: 1q0 q0 or q1
1
q0 or q2
1
0 0
0
![Page 44: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/44.jpg)
44
General method
NFA DFAstates q0, q1, …,
qn
, {Æ q0}, {q1}, {q0,q1}, …, {q0,…,qn}
one for each subset of states in the NFA
initial state
q0 {q0}
transitions
d d’({qi1,…,qik}, a) =
d(qi1, a) ∪…∪ d(qik, a)
accepting states
F Í Q F’ = {S: S contains some state in F}
![Page 45: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/45.jpg)
45
Proof of correctness
• Lemma
• Proof by induction on n• At the end, the DFA accepts iff it is in a
state that contains some accepting state of NFA
• By lemma, this is true iff the NFA can reach an accepting state
After reading n symbols, the DFA is in state {qi1,…,qik} if and only if the NFA is in one of the states qi1,…,qik
![Page 46: 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3](https://reader031.vdocuments.net/reader031/viewer/2022020320/56649e9e5503460f94ba04b1/html5/thumbnails/46.jpg)
46
Exercises
• Construct NFAs for the following languages over the alphabet {a, b, …, z}:– All strings that contain eat or sea or easy– All strings that contain both sea and tea– All strings that do not contain fool