1 csse 350 automata, formal languages, and computability
TRANSCRIPT
1
CSSE 350Automata, Formal Languages, and
Computability
2
Mathematical Preliminaries and Background
• Sets
• Relations
• Functions
• Logic
• Graphs
• Trees
• Proof Techniques
• Languages and Strings
• Automaton
3
Sets
4
}3,2,1{A
A set is a collection of elements, having a property that
characterizes those elements.
SETS
},,,{ airplanebicyclebustrainB
We write Belongs To and Does not Belong To as:
A1
Bship
5
Set Representations• One way is to enumerate the elements completely.
All the elements belonging to the set are explicitly given. – For example, A = {1,2,3,4,5}
• Alternate way is to give the properties that characterize the elements of the set. – For example, B = {x | x is a positive integer less than
or equal to 5} – Or B = {x : x is a positive integer less than or equal to 5}
• Ellipses may be used in the set specification if the meaning is clear. For example, C = { a, b, …, z}, which stands for all the lower-case letters of the English alphabet
6
Set Representations
C = { a, b, c, d, e, f, g, h, i, j, k }
C = { a, b, …, k }
S = { 2, 4, 6, … }
S = { j : j > 0, and j = 2k for some k>0 }
S = { j : j is nonnegative and even }
finite set
infinite set
7
A = { 1, 2, 3, 4, 5 }
Universal Set: all possible elements U = { 1 , … , 10 }
1 2 3
4 5
A
U
6
7
8
910
8
Set Operations
A = { 1, 2, 3 } B = { 2, 3, 4, 5}
• Union
A U B = { 1, 2, 3, 4, 5 }
• Intersection
A B = { 2, 3 }
• Difference
A - B = { 1 }
B - A = { 4, 5 }
U
A B2
31
4
5
2
3
1
Venn diagrams
9
A
• Complement
Universal set = {1, …, 7}
A = { 1, 2, 3 } A = { 4, 5, 6, 7}
12
3
4
5
6
7
A
A = A
10
02
4
6
1
3
5
7
even
{ even integers } = { odd integers }
odd
Integers
11
DeMorgan’s Laws
A U B = A B
U
A B = A U BU
12
Empty, Null Set:= { }
S U = S
S =
S - = S
- S =
U= Universal Set
13
Subset
A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }
A B
U
Proper Subset: A B
UA
B
14
Disjoint Sets
A = { 1, 2, 3 } B = { 5, 6}
A B =
UA B
15
Set Cardinality
• For finite sets
A = { 2, 5, 7 }
|A| = 3
(set size)
16
Powersets
A powerset is a set of sets
Powerset of S = the set of all the subsets of S
S = { a, b, c }
2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }
Observation: | 2S | = 2|S| ( 8 = 23 )
17
Cartesian ProductA = { 2, 4 } B = { 2, 3, 5 }
A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }
|A X B| = |A| |B|
Generalizes to more than two sets
A X B X … X Z
18
Set Identities A, B, C represent arbitrary sets and ø is the empty set
and U is the Universal Set.
The Commutative laws: A B = B A
A B = B A The Associative laws:
A (B C) = (A B) C A (B C) = (A B) C
The Distributive laws: A (B C) = (A B) (A C)
A (B C) = (A B) (A C)
19
The Idempotent laws: A A = A A A = A
The Absorptive laws: A (A B) = A A (A B) = A
The De Morgan laws: (A B)' = A' B' (A B)' = A' B'
Other laws involving Complements: ( A' )' = A A A' = ø A A' = U
20
Other laws involving the empty set A ø = A A ø = ø
Other laws involving the Universal Set: A U = U A U = A
21
RELATIONS
22
Relations
Let A and B be sets. A binary relation from A into B is any subset of the Cartesian product A B.
For example: let A = {2, 3, 5, 6}, and B = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}, and a relation R from A into B by (a, b) R if and only if 2a = b
So, R = {(2, 4), (3, 6), (5, 10), (6, 12)}.
23
A typical element in R is an ordered pair (x, y). In some cases R can be described by actually listing the pairs which are in R, as in the previous example. This may not be convenient if R is relatively large.
Other notations are used depending on the past practice. Consider the following relation on real numbers. R = {(x, y) | y is the square of x} and S = {(x, y) | x <= y}.
R could be more naturally expressed as R(x) = x2 or R(x) =y where y = x2.
24
RELATIONS R = {(x1, y1), (x2, y2), (x3, y3), …}
xi R yi
e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1
25
Relation on a Set
A relation from a set A into itself is called a relation on A.
For example, let A = {1, 2, 3, 4, 5, 6}, and a relation R from A to A itself by (a, b) R if and only if a is a divisor of b
So, R = {(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (2, 2), (2, 4), (2, 6), (3, 3), (3, 6), (4, 4), (5, 5), (6, 6)}.
Let A be a set of people and let P = {(a, b) | a A b A a is a child of b}. Then P is a relation on A which we might call a parent-child relation.
26
Composition
Let R be a relation from a set A into set B, and S be a relation from set B into set C. The composition of R and S, written as RS, is the set of pairs of the form (a, c) A C, where (a, c) RS if and only if there exists b B such that (a, b) R and (b, c) S.
For example PP, where P is the parent-child relation given, is the composition of P with itself and it is a relation which we know as grandparent-grandchild relation.
27
Properties of Relations Assume R is a relation on set A; in other words, R A
A. Let us write a R b to denote (a, b) R. 1. Reflexive: R is reflexive if for every a A, a R a.2. Symmetric: R is symmetric if for every a and b in A, if aRb, then bRa. 3. Transitive: R is transitive if for every a, b and c in A, if aRb and bRc, then aRc. 4. Equivalence: R is an equivalence relation on A if R is reflexive, symmetric and transitive.
For example, the equality relation (=) on a set of numbers such as {1, 2, 3} is an equivalence relation.
28
Equivalence Relations
• Reflexive: x R x
• Symmetric: x R y y R x
• Transitive: x R y and y R z x R z
Example: R = ‘=‘
• x = x
• x = y y = x
• x = y and y = z x = z
29
FUNCTIONS
30
FunctionsA function is something that associates each element of
a set with an element of another set (which may or may not be the same as the first set).
Or a function is a rule that assigns to elements of one set a unique element of another set.
For example, a social security number uniquely identifies a person, in this case, to each member of the set of social security number, some member of the set of person is assigned.
31
FUNCTIONSdomain
12
3
a
bc
range
f : A -> B
A B
If A = domain
then f is a total function
otherwise f is a partial function
f(1) = a4
5
32
Growth Rates of FunctionsLet f(n) and g(n) to be two functions whose domain is a
subset of the positive integersIf there exists a positive constant c such that for all
sufficiently large nf(n) ≤ cg(n)
f has the order at most gf(n) = O(g(n))
cg(n)
f(n)
33
If there exists a positive constant c such that for all sufficiently large n
f(n) cg(n)f has the order at least g
f(n) = (g(n))
cg(n)
f(n)
34
If there exist positive constant c1 and c2, such that for all sufficiently large n
c1g(n) ≤ f(n) ≤ c2g(n)
f and g have the same order of magnitudef(n) = (g(n))
c1g(n)
f(n)
c2g(n)
35
ExamplesExample 1: 2n is Θ(n)
f(n) = 2n, g(n) = n For all integers n ≥ 0, 2n ≤ 2n, which is f(n) ≤ 2g(n);
so constant c ≥ 2 suffices for the definition of O.For all integers n ≥ 0, 2n ≥ n, which is f(n) ≥ 1g(n); so any 0 < c ≤ 2 suffices for the definition of Ω.Since 2n is both O(n) and Ω(n), it is Θ(n).
36
Example 2: n is O(n2) but not Ω(n2)f(n) = n, g(n) = n2
For all integers n ≥ 0, n ≤ n2; which is f(n) ≤ 1g(n); so constant c ≥ 1 suffices for the definition of O.However, n cannot be Ω(n2) since no matter how small c>0 is chosen, for all n > 1/c, n = c(1/c)n < cnn = cn2.
37
Logic
Propositional Logic & Predicate Logic
38
Propositional Logic
Proposition: a declarative statement having a specific truth-value, true or false. For example:8 is an even number. True4 is a negative number. False
39
Two or more propositions can be combined together to make compound propositions with the help of logical connectives. For example: above two propositions can be used to make a compound proposition using any of the logical connectives. 8 is an even number AND 4 is a negative number.
False8 is an even number OR 4 is a negative number.
True
For the first compound proposition to be true both the propositions have to be true as the connective is AND. As OR is the connective for the second one, if either of the propositions is true, the truth value of the compound proposition is true.
40
logical Connectives Conjunction: The compound proposition truth-value is
true if and only if all the constituent propositions hold true. It is represented as " ^ ". Truth table for two individual propositions p and q with conjunction is given below.
p q p q
T T T
T F F
F T F
F F F
41
Disjunction: This is logical "or" read as either true value of the individual propositions. Truth table is given below.
p q p V q
T T T
T F T
F T T
F F F
42
Negation: This is the logical "negation" and it is expressed by as ¬p for "not p". Truth table is given below.
p ¬ pT F
F T
43
Conditional: This is used to define as "a proposition holds true if another proposition is true" i.e. p q is read as "if p, then q". Truth table is given below.
p q p q
T T T
T F F
F T T
F F T
44
Biconditional: A proposition (p q) ^ (q p) can be abbreviated using biconditional conjunction as p q and is read as "if p then q, and if q then p".
Tautology: A compound proposition, which is true in every case. E.g., p V ¬ p
Contradiction: This is the opposite of tautology, which is false in every case. E.g., p ¬ p
45
Identities ¬ (P Q) (¬ P ¬Q) ----- DeMorgan's Law ¬ (P Q) (¬ P ¬ Q) ----- DeMorgan's Law (P Q) (¬ P Q) ----- implication [(P Q) R] [P (Q R)] ----- exportation (P Q) (¬ Q ¬P) ----- contrapositive
46
Implications
[(P Q) ¬Q] P ----- modus tollens [(P Q) (R S)] [(P R) (Q S)] [(P Q) (Q R)] (P R)
47
Predicate LogicThe propositional logic is not powerful enough to represent all
types of assertions. For example, the statement: x is a non-zero integer, where x is a variable, is not a proposition because you can not tell whether it is true or false unless you know the value of x. Thus the propositional logic can not deal with such sentences.
The predicate logic is one of the extensions of propositional logic and it is fundamental to most other types of logic. Central to the predicate logic are the concepts of predicate and quantifier.
A predicate is a feature of language which you can use to make a statement about something, e.g. to attribute a property to that thing. If you say "Peter is tall", then you have applied to Peter the predicate "is tall". We also might say that you have predicated tallness of Peter or attributed tallness to Peter.
48
A predicate is a template involving a verb that describes a property of objects, or a relationship among objects represented by the variables. For example, the sentences:The flower is red.The sweater is red. The frame is red.
All these sentences come from the template "is red" by placing an appropriate noun/noun phrase in front of it. The phrase "is red" is a predicate and it describes the property of being red. Predicates are often given a name.
For example any of "is_red", "Red" or "R" can be used to represent the predicate "is red" among others. We can adopt Red as the name for the predicate “is_red". Sentences that assert an object is red can be represented as "Red(x)", where x represents an arbitrary object. Red(x) reads as "x is red".
49
A predicate with variables, called atomic formula, can be made a proposition by applying one of the following two operations to each of its variables: assign a value to the variable quantify the variable using a quantifier
For example, x > 1 becomes 3 > 1 if 3 is assigned to x, and it becomes a true statement, and it becomes a proposition.
In general, a quantification is performed on formulas of predicate logic (called wff), such as x > 1 or P(x), by using quantifiers on variables.
50
There are two types of quantifiers:universal quantifierexistential quantifier
The universal quantifier turns, for example, the statement x > 1 to "for every object x in the universe, x > 1", which is expressed as " x x > 1". This new statement is true or false in the universe of discourse. Hence it is a proposition once the universe is specified.
Similarly the existential quantifier turns, for example, the statement x > 1 to "for some object x in the universe, x > 1", which is expressed as " x x > 1." Again, it is true or false in the universe of discourse, and hence it is a proposition once the universe is specified.
51
The universe of discourse, also called universe, is the set of objects of interest. The propositions in the predicate logic are statements on objects of a universe.
The universe is thus the domain of the (individual) variables. It can be:the set of real numbersthe set of integersthe set of all cars on a parking lotthe set of all students in a classroom etc.
The universe is often left implicit in practice. But it should be obvious from the context
52
Predicate logic is more powerful than propositional logic. It allows one to reason about properties and relationships of individual objects. In predicate logic, one can use some additional inference rules, as well as those for propositional logic such as the equivalences, implications and inference rules.
Predicate logic permits reasoning about the propositional connectives and also about quantification ("all" or "some"). A classic, if elementary, example of what can be done with the predicate logic is the inference from the premises: All men are mortal.Socrates is a man.
to the conclusion Socrates is mortal
53
Important Inference Rules of Predicate Logic
First there is the following rule concerning the negation of quantified statement which is very useful:
¬x P(x) x ¬P(x)
Next there is the following set of rules on quantifiers
and connectives: x [P(x) Q(x)] [x P(x) x Q(x)] [x P(x) x Q(x)] x [P(x) Q(x)]
x [P(x) Q(x)] [x P(x) x Q(x)]
x [P(x) Q(x)] [x P(x) x Q(x)]
54
GRAPHS
55
GRAPHSA directed graph
• Nodes (Vertices)
V = { a, b, c, d, e }
• Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }
node
edge
a
b
c
d
e
56
Labeled Graph
a
b
c
d
e
1 3
56
26
2
57
Walk
a
b
c
d
e
Walk is a sequence of adjacent edges
(e, d), (d, c), (c, a)
58
Path
a
b
c
d
e
Path is a walk where no edge is repeated
Simple path: no node is repeated
59
Cycle
a
b
c
d
e
12
3
Cycle: a walk from a node (base) to itself
Simple cycle: only the base node is repeated
base
60
Euler Tour
a
b
c
d
e1
23
45
6
7
8 base
A cycle that contains each edge once
61
Hamiltonian Cycle
a
b
c
d
e1
23
4
5 base
A simple cycle that contains all nodes
62
Finding All Simple Paths
a
b
c
d
e
origin
63
(c, a)
(c, e)
Step 1
a
b
c
d
e
origin
64
(c, a)
(c, a), (a, b)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
Step 2
a
b
c
d
e
origin
65
Step 3
a
b
c
d
e
origin(c, a)
(c, a), (a, b)
(c, a), (a, b), (b, e)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
66
Step 4
a
b
c
d
e
origin
(c, a)
(c, a), (a, b)
(c, a), (a, b), (b, e)
(c, a), (a, b), (b, e), (e,d)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
67
Trees
68
Treesroot
leaf
parent
child
Trees have no cycles
69
root
leaf
Level 0
Level 1
Level 2
Level 3
Height 3
70
Binary Trees
71
PROOF TECHNIQUES
72
PROOF TECHNIQUES
• Proof by induction
• Proof by contradiction
73
Induction
We have statements P1, P2, P3, …
If we know
• for some b that P1, P2, …, Pb are true
• for any k >= b that
P1, P2, …, Pk imply Pk+1
Then
Every Pi is true
74
Proof by Induction• Inductive basis
Find P1, P2, …, Pb which are true
• Inductive hypothesis
Let’s assume P1, P2, …, Pk are true,
for any k >= b
• Inductive step
Show that Pk+1 is true
75
Example
Theorem: A binary tree of height n
has at most 2n leaves.
Proof by induction:
let L(i) be the maximum number of
leaves of any subtree at height i
76
We want to show: L(i) <= 2i
• Inductive basis
L(0) = 1 (the root node)
• Inductive hypothesis
Let’s assume L(i) <= 2i for all i = 0, 1, …, k
• Induction step
we need to show that L(k + 1) <= 2k+1
77
Induction Step
From Inductive hypothesis: L(k) <= 2k
height
k
k+1
78
L(k) <= 2k
L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1
Induction Step
height
k
k+1
(we add at most two nodes for every leaf of level k)
79
Remark
Recursion is another thing
Example of recursive function:
f(n) = f(n-1) + f(n-2)
f(0) = 1, f(1) = 1
80
Proof by Contradiction
We want to prove that a statement P is true
• we assume that P is false
• then we arrive at an incorrect conclusion
• therefore, statement P must be true
81
Example
Theorem: is not rational
Proof:
Assume by contradiction that it is rational
= n/m
n and m have no common factors
We will show that this is impossible
2
2
82
= n/m 2 m2 = n2
Therefore, n2 is evenn is even
n = 2 k
2 m2 = 4k2 m2 = 2k2m is even
m = 2 p
Thus, m and n have common factor 2
Contradiction!
2
83
Languages and Strings
84
A language is a set of strings
String: A sequence of letters
Examples: “cat”, “dog”, “house”, …
Defined over an alphabet: zcba ,,,,
85
Alphabets and StringsWe will use small alphabets:
Strings
abbaw
bbbaaav
abu
ba,
baaabbbaaba
baba
abba
ab
a
86
String Operations
m
n
bbbv
aaaw
21
21
bbbaaa
abba
mn bbbaaawv 2121
Concatenation
abbabbbaaa
87
12aaaw nR
naaaw 21 ababaaabbb
Reverse
bbbaaababa
88
Concatenation and Reverse of Strings
Theorem: If w and x are strings, then (w x)R = xR wR.
Example:
(nametag)R = (tag)R (name)R = gateman
89
Concatenation and Reverse of Strings Proof: By induction on |x|:
|x| = 0: Then x = , and (wx)R = (w )R = (w)R = wR = R wR = xR wR.
n 0 (((|x| = n) ((w x)R = xR wR)) ((|x| = n + 1) ((w x)R = xR wR))):
Consider any string x, where |x| = n + 1. Then x = u a for some character a and |u| = n. So:
(w x)R = (w (u a))R rewrite x as ua= ((w u) a)R associativity of concatenation= a (w u)R definition of reversal= a (uR wR) induction hypothesis= (a uR) wR associativity of concatenation= (ua)R wR definition of reversal= xR wR rewrite ua as x
90
String Length
Length:
Examples:
naaaw 21
nw
1
2
4
a
aa
abba
91
Length of Concatenation
Example:
vuuv
853
8
5,
3,
vuuv
aababaabuv
vabaabv
uaabu
92
Empty StringA string with no letters:
Observations:
abbaabbaabba
www
0
93
SubstringSubstring of string:
a subsequence of consecutive characters
String Substring
bbab
b
abba
ab
abbab
abbab
abbab
abbab
94
Prefix and Suffix
Prefixes Suffixesabbab
abbab
abba
abb
ab
a
b
ab
bab
bbab
abbab uvw
prefix
suffix
95
Another Operation (Replication)
Example:
Definition:
n
n wwww
abbaabbaabba 2
0w
0abba
96
The * Operation : the set of all possible strings from alphabet
*
,,,,,,,,,*
,
aabaaabbbaabaaba
ba
97
The + Operation : the set of all possible strings from alphabet except
,,,,,,,,,*
,
aabaaabbbaabaaba
ba
*
,,,,,,,, aabaaabbbaabaaba
98
LanguagesA language is any subset of
Example:
Languages:
*
,,,,,,,,*
,
aaabbbaabaaba
ba
},,,,,{
,,
aaaaaaabaababaabba
aabaaa
99
Note that:
}{}{
0}{
1}{
0
Sets
Set size
Set size
String length
100
Another Example
An infinite language }0:{ nbaL nn
aaaaabbbbb
aabb
ab
L Labb
101
Operations on LanguagesThe usual set operations
Complement:
aaaaaabbbaaaaaba
ababbbaaaaaba
aaaabbabaabbbaaaaaba
,,,,
}{,,,
},,,{,,,
LL *
,,,,,,, aaabbabaabbaa
102
Reverse
Definition:
Examples:
}:{ LwwL RR
ababbaabababaaabab R ,,,,
}0:{
}0:{
nabL
nbaL
nnR
nn
103
Concatenation
Definition:
Example:
2121 ,: LyLxxyLL
baaabababaaabbaaaab
aabbaaba
,,,,,
,,,
104
Another OperationDefinition:
Special case:
n
n LLLL
bbbbbababbaaabbabaaabaaa
babababa
,,,,,,,
,,,, 3
0
0
,, aaabbaa
L
105
More Examples
}0:{ nbaL nn
}0,:{2 mnbabaL mmnn
2Laabbaaabbb
106
Star-Closure (Kleene *)
Definition:
Example:
210* LLLL
,,,,
,,,,
,,
,
*,
abbbbabbaaabbaaa
bbbbbbaabbaa
bbabba
107
Positive Closure
Definition:
*
21
L
LLL
,,,,
,,,,
,,
,
abbbbabbaaabbaaa
bbbbbbaabbaa
bba
bba
108
Grammar
109
GrammarMathematically describe language
English: informal, inadequateSet notations: limited
GrammarEnglish grammar describes whether a sentence is well-formed
or not<sentence> <noun_phrase><predicate>
Then, what is noun_phrase, what is predicate?<noun_phrase> <article><noun><predicate> <verb>
Basic idea of formal grammars: Start from the top level conceptReduce it to the irreducible building blocks
110
Grammar DefinitionA grammar G is defined as a quadruple
G = (V, T, S, P)V is a finite set of objects called variablesT is a finite set of objects called terminal
symbolsS V is a special symbol called the start
variableP is a finite set of productionsv and T are non-empty and disjoint
111
Production RulesHeart of a grammarDescribe how the grammar transforms one string into anotherDefine a language associated with grammarProduction rules are of the form
x y, where x (V T)+, y (V T)* For example, given w = uxv, x y,
then z = uyv, z is a new stringwritten as: w z, means that w derives z or
z is derived from w w1 w2 … wn is the same as w1 wn *
112
DefinitionLet G = (V, T, S, P) be a grammar. Then the set
L(G) = { w T* : S w}is the language generated by G.
If w L(G),
S w1 w2 … wn w
is a derivation of the sentence w.
Strings S, w1, w2, …, wn are called sentential forms of the derivation. All these strings may contain variables and terminals.
*
113
Automaton
114
Automaton
Input
“Accept” or“Reject”
OutputS1 S2 S3
S4 S5
Storage
ControlUnit
115
AutomataAn abstract model of a digital computer• Input mechanism
– Can read but not change an input file • Input file
– Contains a string over a given alphabet– Divided into cells, each cell holds one symbol
– The reading is from left to right, one symbol at a time– Can sense the end of the input string
• Storage device– Consist of an unlimited number of cells– Each cell holds a symbol– Can be read and changed
• Control unit– Internal states– Transition function (or next-state): gives the next state of the control
unit • Current state• Current input symbol• Current information in the temporary storage
116
To Illustrate “State”
State: status, A state stores information about the past, i.e. it reflects the input changes from the system start to
the present moment Example: a robot
117
• Configuration– A particular state of the control unit, input file, and temporary
storage• Move
– The transition of the automaton from one configuration to the next
• Deterministic automata– Each move is uniquely determined by the current
configuration– If the internal state, the input, and the contents of the
temporary storage are known, the future behavior of the automata can be predicted
• Nondeterministic automata– At some point, the automata may have several moves– A set of possible actions can be predicted
• Accepter– An automata whose output response is either “yes” or “no”