dfas and regular languages nfas and their equivalence to dfas regular...

What we have done so far

DFAs and regular languages

NFAs and their equivalence to DFAs

Regular expressions.

Regular expressions capture exactly regular languages:Construct a NFA from a regular expression.Left to do: Construct a regular expression from anNFA

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 1

Computational Models - Lecture 3

Constructing a regular expression from an NFA

What can’t DFAs do? Non Regular Languages: TwoApproaches1. Pumping Lemma2. Myhill-Nerode Theorem

Algorithmic questions for NFAs

Context Free Grammars (time permitting)

Sipser’s book, 1.4, 2.1, 2.2

Hopcroft and Ullman, 3.4


Regular Expression from an NFA (⇐=)

We now define generalized non-deterministic finiteautomata (GNFA).An NFA:

Each transition labeled with a symbol or ε,

reads zero or one symbols,

takes matching transition, if any.

A GNFA:

Each transition labeled with a regular expression,

reads zero or more symbols,

takes transition whose regular expression matchesstring, if any.

GNFAs are natural generalization of NFAs.


A Special Form of GNFA

Start state has outgoing arrows to every other state, butno incoming arrows.

Unique accept state has incoming arrows from everyother state, but no outgoing arrows.

Except for start and accept states, an arrow goes fromevery state to every other state, including itself.

Easy to transform any GNFA into special form.

Really? How? . . .


Converting DFA to Regular Expression (⇐=)

Strategy – sequence of equivalent transformations

given a k-state DFA

transform into (k + 2)-state GNFA

while GNFA has more than 2 states, transform it intoequivalent GNFA with one fewer state

eventually reach 2-state GNFA (states are just start andaccept).

label on single transition is the desired regularexpression.


Converting Strategy (⇐=)

3-stateDFA

5-stateGNFA

4-stateGNFA

3-stateGNFA

2-stateGNFA

regularexpression


Removing One State

We remove one state qr, and then repair the machine byaltering regular expression of other transitions.

qi jq

qr

R1 3R

4R

qi jq

2R

2R*R1 3R U 4R


Formal Treatment – GNFA Definition

qs is start state.

qa is accept state.

R is collection of regular expressions over Σ.

The transition function is

δ : (Q − {qa}) × (Q − {qs}) → RIf δ(qi, qj) = R, then arrow from qi to qj has label R.

Arrows connect every state to every other state except:

no arrow from qa

no arrow to qs


Formal Definition

A generalized deterministic finite automaton (GNFA) is(Q,Σ, δ, qs, qa), where

Q is a finite set of states,

Σ is the alphabet,

δ : (Q− {qa}) × (Q − {qs}) → R is the transition function.

qs ∈ Q is the start state, and

qa ∈ Q is the unique accept state.


A Formal Model of GNFA Computation

A GNFA accepts a string w ∈ Σ∗ if there exists a parsing ofw, w = w1w2 · · ·wk, where each wi ∈ Σ∗, and there exists asequence of states q0, . . . , qk such that

q0 = qs, the start state,

qk = qa, the accept state, and

for each i, wi ∈ L(Ri), where Ri = δ(qi−1, qi).

(namely wi is an element of the language described bythe regular expression Ri.)


The CONVERT Algorithm

Given GNFA G, convert it to equivalent GNFA G′.

let k be the number of states of G.

If k = 2, return the regular expression labeling the onlyarrow.

If k > 2, select any qr distinct from qs and qa.

Let Q′ = Q − {qr} .

For any qi ∈ Q′ − {qa} and qj ∈ Q′ − {qs}, let

R1 = δ(qi, qr), R2 = δ(qr, qr),

R3 = δ(qr, qj), and R4 = δ(qi, qj).

Define δ′(qi, qj) = (R1)(R2)∗(R3) ∪ (R4).

Denote the resulting k − 1 states GNFA by G′.


The CONVERT Procedure

We define the recursive procedure CONVERT(·):

Given GNFA G.

Let k be the number of states of G.

If k = 2, return the regular expression labeling the onlyarrow of G.

If k > 2, let G′ be the k − 1 states GNFA produced bythe algorithm.

Return CONVERT(G′).


Correctness Proof of Construction

Theorem: G and CONVERT(G) accept the same language.

Proof: By induction on number of states of G

Basis: When there are only 2 states, there is a single label,which characterizes the strings accepted by G.

Induction Step: Assume claim for k − 1 states, prove for k.

Let G′ be the k − 1 states GNFA produced from G by thealgorithm.


G and G′ accept the same language

By the induction hypothesis, G′ and CONVERT(G′) acceptthe same language.

On input G, the procedure returns CONVERT(G′).

So to complete the proof, it suffices to show that G and G′

accept the same language.

Three steps:

1. If G accepts the string w, then so does G′.

2. If G′ accepts the string w, then so does G.

3. Therefore G and G′ are equivalent.


Step One

Claim: If G accepts w, then so does G′:

If G accepts w, then there exists a “path of states”qs, q1, q2, . . . , qa traversed by G on w, leading to theaccept state qa.

If qr does not appear on path, then G′ clearly accepts w

(recall that each new regular expression on each edgeof G′ contains the old regular expression in the “unionpart”).

If qr does appear, consider the regular expressioncorresponding to . . . qi, qr, . . . , qr, qj . . . .

The new regular expression (Ri,r)(Rr,r)∗(Rr,j) linking qi

and qj encompasses any such string.

In both cases, the claim holds.


Steps Two and Three

Claim: If G′ accepts w, then so does G.

Proof: Each transition from qi to qj in G′ corresponds to atransition in G, either directly or through qr. Thus if G′

accepts w, then so does G.

This completes the proof of the claim thatL(G) = L(G′).

Combined with the induction hypothesis, this shows thatG and the regular expression CONVERT(G) accept thesame language.

This, in turn, proves our remarkable claim:A language, L, is described by a regular expression, R,if and only if L is regular. ♣


What We Just Completed

Thm.: A language, L, is described by a regularexpression, R, if and only if L is regular.

=⇒ construct an NFA accepting R.

⇐= Given a regular language, L, construct anequivalent regular expression


Negative Results

We have made quite some progress understanding whatfinite automata can do. But what can’t they do?Is there a DFA over {0, 1} that accepts

B = {0n1n|n ≥ 0}C = {w|w has an equal number of 0’s and 1’s}D = {w|w has an equal number of occurrences

of 01 and 10 substrings}Consider B:

DFA must “remember” how many 0’s it has seen

impossible with finite state.

The others are exactly the same.


Negative Results

Is there a DFA over {0, 1} that accepts

B = {0n1n|n ≥ 0}C = {w|w has an equal number of 0’s and 1’s}D = {w|w is binary and has an equal number of

occurrences of 01 and 10 substrings}Consider B:

DFA must “remember” how many 0’s it has seen

impossible with finite state.

The others are exactly the same...

Question: This is sound intuition. But is this a proof?Answer: No. D is regular!???(See problem 7 in hw1...)


Pumping Lemma

We will show that all regular languages have a specialproperty.

Suppose L is regular.

If a string in L is longer than a certain critical length ℓ

(the pumping length),

then it can be “pumped” to a longer string by repeatingan internal substring any number of times.

The longer string must be in L too.


Pumping Lemma

Theorem: If L is a regular language, then there is an ℓ > 0(the pumping length), where if s is any string in L of length|s| ≥ ℓ, then s may be divided into three pieces s = xyz suchthat

for every i ≥ 0, xyiz ∈ L,

|y| > 0, and

|xy| ≤ ℓ.

Remarks: Without the second condition, the theorem wouldbe trivial.The third condition is technical and sometimes useful.


Pumping Lemma – Proof

Let M = (Q,Σ, δ, q1, F ) be a DFA that accepts L.

Let ℓ = |Q|, the number of states of M .

If s ∈ L has length at least ℓ, consider the sequence ofstates M goes through as it reads s:

s1 s2 s3 s4 s5 s6 . . . sn

↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑

q1 q20 q9 q17 q12 q13 q9 q2 q5 ∈ F

Since the sequence of states is of length |s| + 1 > ℓ, andthere are only ℓ different states in Q, at least one state isrepeated (by the pigeonhole principle).


Pumping Lemma – Proof (cont.)

Write down s = xyz

q1

q9

q5

x

y

z

By inspection, M accepts xykz for every k ≥ 0.

|y| > 0 because the state (q9 in figure) is repeated (thisis a DFA, no ǫ-transitions...).

To ensure that |xy| ≤ ℓ, pick first state repetition, whichmust occure no later than ℓ + 1 states in sequence.


An Application

Theorem: The language B = {0n1n|n > 0} is not regular.

Proof: By contradiction. Suppose B is regular, accepted byDFA M . Let ℓ be the pumping length.

Consider the string s = 0ℓ1ℓ.

By pumping lemma s = xyz, where xykz ∈ B for every k.

If y is all 0, then xykz has too many 0’s.


If y is mixed, then xykz is not of right form. ♣


Another Application - first attempt

Theorem: The languageC = {w ∈ {0, 1}∗|w has an equal number of 0’s and 1’s} isnot regular.Proof: By contradiction. Suppose C is regular, accepted byDFA M . Let ℓ be the pumping length.

The string to be pumped needs to be chosen carefully!

Let’s try the string s = (01)ℓ.

By pumping lemma s = xyz, where xykz ∈ C for every k.

But unfortunately we cannot reach a contradiction, asthe chosen string can be pumped as follows:

x = ǫ, y = 01, z = (01)ℓ−1

Then indeed xyiz ∈ L(M) for every i ≥ 0...Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 25

Another Application - second attempt

Theorem: The languageC = {w ∈ {0, 1}∗|w has an equal number of 0’s and 1’s} isnot regular.Proof: By contradiction. Suppose C is regular, accepted byDFA M . Let ℓ be the pumping length.

Consider the string s = 0ℓ1ℓ.

By pumping lemma s = xyz, where xykz ∈ C for every k.



If y is mixed, then since |xy| ≤ ℓ, y must be all 0’s,contradiction. ♣


Yet Another Application

Theorem: The language L3 ⊂ {1}∗, which contains allstrings whose length is a perfect square, is not regular.

L3 = {1n2 | n ≥ 0}

Proof:

Assume for contradiction that L3 is regular. Let ℓ be thepumping length.

Take the string s = 1ℓ2 . By the pumping lemma, s = xyz,where for any i ≥ 0, xyiz ∈ L3.

Note the growing gap between successive members ofthe sequence of perfect squares:

0, 1, 4, 9, 16, 25, 36, 49, ...,Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 27

Yet Another Application - cont.

We notice: large members of the sequence cannot benear each other!

Now consider the strings xyiz and xyi+1z. Their lengthdiffers only in the length of y. If we choose a largeenough i, these two strings cannot be both in L3, asthey are too close!

If m = n2, then the difference between it and the nextperfect square is:

(n + 1)2 − n2 = n2 + 2n + 1 − n2 = 2n + 1 = 2√

m + 1

By the pumping lemma, both strings xyiz and xyi+1z areperfect squares. But if we take |xyiz| = m, it means thatthey cannot be both perfect squares if |y| < 2

√

|xyiz|+1.


Yet Another Application - cont.

We can now calculate the value for i that leads to acontradiction:

Recall that |y| ≤ |s| = l2

Take i = l4, then:

|y| ≤ l2 =√

l4 < 2√

l4 + 1 ≤ 2√

|xyiz| + 1

Thus xyl4z and xyl4+1z cannot be both perfect squares -a contradiction!


And now: an alternative technique for showing a languageis not regular.


The Equivalence Relation ∼L

Let L ⊆ Σ∗ be a language.

Define an equivalence relation ∼L on pairs of strings:

Let x, y ∈ Σ∗. We say that x∼L y if for every string z ∈ Σ∗,xz ∈ L if and only if yz ∈ L.

It is easy to see that ∼L is indeed an equivalence relation(reflexive, symmetric, transitive) on Σ∗.

In addition, if x∼L y then for every string z ∈ Σ∗, xz∼L yz aswell (this is called right invariance).


The Equivalence Relation ∼L

Like every equivalence relation, ∼L partitions Σ∗ to (disjoint)equivalence classes. For every string x, let [x] ⊆ Σ∗ denoteits equivalence class w.r.t. ∼L (if x ∼L y then [x] = [y] –equality of sets).

Question is, how many equivalence classes does ∼L

induce?

In particular, is the number of equivalence classes of ∼L

finite or infinite?

Well, it could be either finite or infinite. This depends on thelanguage L.


Example: L = (ab ∪ ba)∗

∼L has the following four equivalence classes:

[ǫ] = L

[a] = La

[b] = Lb

[aa] = L(aa ∪ bb)Σ∗


Classes of ∼L: More Examples

Let L1 ⊂ {0, 1}∗ contain all strings where the number of1s is divisible by 4. Then ∼L1

has finitely manyequivalence classes.

Let L2 = L(0∗10∗). Then ∼L2has finitely many

equivalence classes.

Let L3 ⊂ {0, 1}∗ contain all strings of the form 0n1n.Then ∼L3

has infinitely many equivalence classes.


Myhill-Nerode Theorem

Theorem: Let L ⊆ Σ∗ be a language. Then

L is regular ⇐⇒ ∼L has finitely many equivalenceclasses.

Three specific consequences:

L1 ⊂ {0, 1}∗ contains all strings where the number of 1sis divisible by 4. Then L1 is regular.

L2 = L(0∗10∗) is regular.

L3 ⊂ {0, 1}∗ contains all strings of the form 0n1n. ThenL2 is not regular.


Myhill-Nerode Theorem: Proof

=⇒Suppose L is regular. Let M = (Q,Σ, δ, q0, F ) be a DFAaccepting it. For every x ∈ Σ∗, let δ(q0, x) ∈ Q be the statewhere the computation of M on input x ends.

The relation ∼M on pairs of strings is defined as follows:x ∼M y if δ(q0, x) = δ(q0, y).Clearly, ∼M is an equivalence relation.

Furthermore, if x ∼M y, then for every z ∈ Σ∗, alsoxz ∼M yz. Therefore, xz ∈ L if and only if yz ∈ L.

This means that x ∼M y =⇒ x ∼L y.


Myhill-Nerode Theorem: Proof (cont.)

=⇒

The equivalence relation ∼M has finitely manyequivalence classes (at most the number of states inM).

We saw that x ∼M y =⇒ x ∼L y.

This means that the equivalence classes of ∼M refinethose of ∼L.

Thus, the number of equivalence classes of ∼M isgreater or equal than the number of equivalenceclasses of ∼L.

Therefore, ∼L has finitely many equivalence classes, asdesired. ♠


Myhill-Nerode Theorem: Proof (cont.)

⇐=

Suppose ∼L has finitely many equivalence classes.We’ll construct a DFA M that accepts L.

Let x1, . . . , xn ∈ Σ∗ be representatives for the finitelymany equivalence classes of ∼L.

The states of M are the equivalence classes[x1], . . . , [xn].

The transition function δ is defined as follows: For alla ∈ Σ, δ([xi], a) = [xia] (the equivalence class of xia).

Why is δ well-defined?(Hint: right invariance: if x ∼L y, then xz ∼L yz).


Myhill-Nerode Theorem: Proof (conc.)

⇐=

The initial state is [ε].

The accept states are F = {[xi] |xi ∈ L}.

Easy: On input x ∈ Σ∗, M ends at state [x] (why?).

Therefore M accepts x iff x ∈ L (why?).

So L is accepted by DFA, hence L is regular. ♠


Context Switch


Algorithmic Questions for NFAs

Q.: Given an NFA, N , and a string s, is s ∈ L(N)?

Answer: Construct the DFA equivalent to N and run it on w.

Q.: Is L(N) = ∅?Answer: This is a reachability question in graphs: Is there apath in the states’ graph of N from the start state to someaccepting state. There are simple, efficient algorithms forthis task.


More Algorithmic Questions for NFAs

Q.: Is L(N) = Σ∗?

Answer: Check if L(N) = ∅.

Q.: Given N1 and N2, is L(N1) ⊆ L(N2)?

Answer: Check if L(N2) ∩ L(N1) = ∅.

Q.: Given N1 and N2, is L(N1) = L(N2)?

Answer: Check if L(N1) ⊆ L(N2) and L(N2) ⊆ L(N1).

In the future, we will see that for stronger models ofcomputations, many of these problems cannot be solved byany algorithm.


Another, More Radical Context Switch

So far we sawfinite automata,regular languages,regular expressions,Myhill-Nerode theorem and pumping lemma forregular languages.

We now introduce stronger machines andlanguages with more expressive power:

pushdown automata,context-free languages,context-free grammars,pumping lemma for context-free languages.


Context-Free Grammars

An example of a context free grammar, G1:

A → 0A1

A → B

B → #

Terminology:

Each line is a substitution rule or production.

Each rule has the form: symbol → string.The left-hand symbol is a variable(usually upper-case).

A string consists of variables and terminals.

One variable is the start variable.


Rules for Generating Strings

Write down the start variable (lhs of top rule).

Pick a variable written down in current string and aderivation that starts with that variable.

Replace that variable with right-hand side of thatderivation.

Repeat until no variables remain.

Return final string (concatenation of terminals).

Process is inherently non deterministic.


Example

Grammar G1:

A → 0A1

A → B

B → #

Derivation with G1:

A ⇒ 0A1

⇒ 00A11

⇒ 000A111

⇒ 000B111

⇒ 000#111


A Parse Tree

A

A

A

B

#0 10 0 1 1

Question: What strings can be generated in this way fromthe grammar G1?Answer: Exactly those of the form 0n#1n (n ≥ 0).


Context-Free Languages

The language generated in this way is called the languageof the grammar.

For example, L(G1) is {0n#1n|n ≥ 0}.

Any language generated by a context-free grammar iscalled a context-free language.


A Useful Abbreviation

Rules with same variable on left hand side

A → 0A1

A → B

are written as:

A → 0A1 | B


Deriving English-like Sentences

A specific derivation in G2:

< SENTENCE > ⇒ < NOUN-PHRASE >< VERB >

⇒ < ARTICLE >< NOUN >< VERB >

⇒ a < NOUN >< VERB >

⇒ a boy < VERB >

⇒ a boy sees

More strings generated by G2:

a flower seesthe girl touches


Derivation and Parse Tree

< SENTENCE > ⇒ < NOUN-PHRASE >< VERB >

⇒ < ARTICLE >< NOUN >< VERB >

⇒ a < NOUN >< VERB >

⇒ a boy < VERB >

⇒ a boy sees

SENTENCE

NOUN-PHRASE VERB

ARTICLE NOUN

a boy seesBased on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 52

dfas and regular languages nfas and their equivalence to dfas regular...

Documents