cits2211 discrete structures non-regular languages and push … · 2018-10-17 · 7/43. non-regular...

Non-regular languages Pumping lemma Context-Free Languages Context Free Grammars Pushdown Automata

CITS2211 Discrete StructuresNon-regular languages and push-down automata

Unit coordinator: Arran Stewart

1 / 43


Highlights

We have seen that FSMs are surprisingly powerful

But we also saw some languages FSMs can not recognise

Now we will learn a useful theorem for testing whether a language isregular or not: the pumping lemma for regular languages

We will study a new type of automata: pushdown automata

And a new class of languages: context-free languages

2 / 43


Reading

Introduction to the Theory of Computation by Michael Sipser

Chapter 1: Regular LanguagesSection 1.4 The pumping lemma for regular languages

Chapter 2: Context-free languagesSection 2.2 Pushdown automata

3 / 43


Non-regular languages

4 / 43


Lecture Outline

1. Non-regular languages

2. The pumping lemma for regular languages

3. Push-down automata

4. Context-free languages

5 / 43


Non-regular languagesQ: Are all languages regular?A: No

We can use a diagonalization argument to show that there arenon-regular languages. Remember we used a diagonalizationargument to show that there are uncountable sets. Now we willshow that there must be some non-regular languages over thealphabet {0, 1}.

Firstly, notice that the set of regular expressions is countable— wecan arrange them in lexicographical order. Therefore we can form acomplete list R1, R2, R3, . . ., of regular expressions.

Let L1, L2, . . ., be the corresponding list of regular languagesdescribed by the above regular expressions.

6 / 43


A diagonalization argumentNow we will form a new language L and show that it is not in thelist above.

Recall that the earlier diagonalization arguments formed a newobject that was definitely different from every object on thesupposedly complete list.

We will copy this strategy as follows:

Consider the set of strings 1, 11, 111, 1111, and form the newlanguage L as follows:

If 1 /∈ L1, then add it to L (else do nothing)If 11 /∈ L2, then add it to LIf 111 /∈ L3, then add it to LIf 1111 /∈ L4, then add it to L and so on...

7 / 43


A diagonalization argument (cont.)

Then clearly L is different from L1, L2, . . . and thus L is not regular.

This is an example of a non-constructive proof. We have shown theexistence of a non-regular language, but have not actually specifiedone.

8 / 43


Pumping lemma

9 / 43


The pumping lemma

We now turn to the problem of determining when a language isnon-regular, and hence cannot be recognized by a finite statemachine.

The main tool for this is a result called the pumping lemma whichcan be proved using the Pigeonhole Principle we studied a fewweeks ago.

The pumping lemma for regular languages was first stated byY. Bar-Hillel, Micha A. Perles, and Eli Shamir in 1961.

10 / 43


Theorem: Pumping Lemma (for regular languages)

Theorem: If L is a regular language then∃ an integer p called the pumping length of L such that∀ words w ∈ L where |w | > p∃ an expression w = xyz where

1. ∀ i ≥ 0. xy iz ∈ L

2. |y | ≥ 1

3. |xy | ≤ p

That is, if L is regular then any sufficiently long word in thelanguage contains a non-empty substring that can be repeated anarbitrary number of times (0,1 or more).

11 / 43


Finite languages

Finite languages satisfy the pumping lemma by setting p to thelength of the longest string in the language.

Then it is vacuously true that – for all strings longer than p (ofwhich there are none), the remainder of the proposition follows.

12 / 43


How to use the pumping lemma

The pumping lemma is of the form R → P. That is IF a language isregular (R) THEN certain properties must hold (P).

Usage 1: Assume R. From the lemma we also have P. Nowderive a contradiction. The contradiction tells us that theoriginal assumption R must be false. This way we can provethat a language is NOT regular.

Usage 2: R → P ↔ ¬P → ¬R. So if we can show ¬P then bythe pumping lemma we can deduce that the language NOTregular.

Note, that we can not say that if the pumping properties holdthen the language is regular, because we do not have P → R

13 / 43


Use of pumping lemmaThe trickiness of applying pumping lemma is using the ∀ and ∃conditions correctly. You need to practice!

To prove that a language is not regular, find some sufficiently longstring that is in the language, but which cannot be pumped.

The existence of such a string shows that the given language is notregular.

We use an “adversary" game argument [Hopcroft and Ullman]

Your choices in the game correspond to the ∀ quantifiers in thestatement of the Pumping Lemma (see above) and adversarychoices correspond to ∃ lines.

14 / 43


Proof of pumping lemmaRTP.

∀L. ∃p. ∀w . w ∈ L →∃xyz . w = xyz ∧ |xy | ≤ p ∧ |y | ≥ 1 ∧ ∀i > 0. xy iz ∈ L

Proof. Suppose L is a regular language. Then it can be recognizedby a finite state machine M with p states.

Now suppose that w = c1c2 . . . cn is a word of length n ≥ p. Thenconsider the states that occur when M is run with input string w .

s0 −→c1 s1 −→c2 s2 · · · −→cn sn

By the pigeonhole principle, at least two of these states must be thesame so let si and sj be the first two occurrences of the firstrepeated state.

15 / 43


Pumping Lemma (cont)

Now setx = c1c2 . . . ci

y = ci+1ci+2 . . . cj

z = cj+1 . . . cn

Now we can see that y is a string that takes the finite statemachine “in a circle” from a state back to itself.

This means that we can now “pump” the input string by repeatingthis portion as often as possible, and still get a string that isrecognized by the machine.

Hence xz , xyyz , xyyyz and in general xy iz are all recognized by Mand hence in the language L. QED

16 / 43


Adversary Argument using the Pumping Lemma

1. Select the language L to be proven non-regular

2. The adversary (she) picks the pumping constant p

3. You select some string w ∈ L (based on your knowledge of p)

4. She breaks w into any x , y , z she wants subject to theconstraints |xy | ≤ p ∧ |y | ≥ 1

5. You achieve a contradiction to the Pumping Lemma byshowing that for any x , y , z chosen by the adversary, ∃i so thatxy iz /∈ L. Your choice of i may depend on p, x , y , and w .

6. From this it can be concluded that L is not regular.

17 / 43


Adversary Argument Example

Example Show that the language L is not regular where,

L = {w | w has an equal number of 0s and 1s}

Proof

Suppose L is regular. Let the pumping length be p. Choosew = 0p1p. Clearly w ∈ L. We will see this is a useful example of wfor the proof (not all words in L are useful choices). For anyadversary choice of xyz = w , both x and y can only contain 0ssince |xy | ≤ p (constraint 3). Say x = 0m, y = 0n for somem + n ≤ p. Now, the pumping lemma states that xyyz ∈ L but weknow xyyz /∈ L because xyyz has more 0s than 1s. We have deriveda contradiction. Therefore L is not regular. QED

18 / 43


Take Care: Pumping Lemma uses → not ↔

Note that while the pumping lemma states that all regularlanguages do satisfy the conditions described above, the converse ofthis statement is not true. A language that satisfies the pumpingconditions may still be non-regular.

Example: L = {aibjck | i , j , k ≥ 0 ∧ (i = 1→ j = k)}

a) show that L is not regularb) show that w = aibjck satisfies the pumping lemma conditions(for some i,j,k)c) explain why parts a) and b) do not contradict the pumping lemma

This question is an exercise in this week’s tutorial.

19 / 43


Context-Free Languages

20 / 43


Context-Free Languages

Context-free languages can describes certain features with arecursive structure.

They are more powerful than FSMs because they have some limitedmemory in the form of a stack.

21 / 43


What are non-regular languages like?

Context-free languages were first studied for understanding humanlanguages.

For English we have the following loose rulesentence → noun-phrase verb-phrase

which we interpret as saying

“A valid sentence consists of a noun-phrase followed by averb-phrase”

To complete the description, we then need to define noun-phrase,verb-phrase and so on, which are defined in the same way

noun-phrase → article nounverb-phrase → verb adverb

22 / 43


Context Free languages for Computer Science

An important use of CF languages in Computer Science is thespecification and compilation of languages such as Java or SQL.

23 / 43


Context Free languages and Automata

Context-free languages include all the regular languages and manymore.

For example, we saw that 0n1n was not a regular language, buttoday we will learn that it is a context-free language.

Context-free languages are precisely the class of languages thatcan be recognised by pushdown automata, which are finite statemachines that use a stack as a memory device.

The most important formal languages for Computer Science, areprobably context-free languages, because many computerlanguages are context free languages (at least in part if not all).

24 / 43


Context Free Grammars

25 / 43


Grammars

Context-free languages can be specified using grammars.

For example, the language {0n1n|n ≥ 0} is generated by thegrammar

A→ 0A1 | ε

Example construction sequence: A, 0A1, 00A11, 000A111, 000111

26 / 43


Grammars: the mechanics

A grammar grows all the strings of a language.

A grammar is a collection of substitution rules called productions.

Each rule has a left hand side symbol, an arrow and a right hand side.

Variable symbols are called non-terminals and usually representedby a capital letter.

Other symbols are from the alphabet of the language calledterminals and usually represented by a lower case letter.

One symbol is designated the start variable usually written S.

Strings in the language are grown by starting with the start symboland then replacing non-terminals according to the production rules.

27 / 43


Grammar Example 1

The language of all expressions with balanced brackets is generatedby the grammar

S → SS | (S) | ε

Example construction sequence:

S,SS, SSS, (S)SS, ((S))SS, (())(S)S, (())()S, (())()(S), (())()()

28 / 43


Grammar Example 2

A→ 0A1A→ BB → x

Example derivation: A, 0A1, 00A11, 00B11, 00x11

Rules can be written on separate lines (Example 2), or using | todenote a list of rules for the same non-terminal (Example 1).

29 / 43


Context-free grammar definition

Definition: A context-free grammar is a 4-tuple (V ,Σ,R, S) where

1. V is a finite set called the variables (usually denoted by capitalletters)

2. Σ is a finite set, Σ ∩ V = ∅, called the terminals (usuallydenoted by lower case letters or symbols as the alphabet of thelanguage)

3. R is a finite set of rules, with each rule of the form V → Xwhere X is a string of variables and terminals.

4. S ∈ V is the start variableSipser Definition 2.2, page 104 in the 3rd edition

30 / 43


Pushdown Automata

31 / 43


Idea of Push-Down Automata (PDA)

Context-free languages can be recognised by automata called PDAs

PDAs are similar to non-deterministic FSMs (NFSMs)

but they have an extra component called a stack

The stack provides extra memory, in addition to states

This memory allows PDA to do “counting" that an NFSM can not

32 / 43


Formal defn of PDAs

Definition: A pushdown automata (PDA) is defined to be a6-tuple (Q,Σ, Γ,F , δ, q0,F ) where

Q is a finite set of states

Σ is a finite alphabet of input symbols

Γ is the finite stack alphabet

δ : Q × Σε × Γε → P(Q × Γε) is the transition function

q0 ∈ Q is the start state

F ⊆ Q is a set of accepting states (F may be the empty set)

33 / 43


PDA transitions

Note this definition is nearly the same as for NFSMs with theexception of the transition function and the addition of a stackalphabet Γ.

As well as changing state for a given input, a PDA may

read and pop a symbol from the stack

push a symbol onto the top of the stack.

34 / 43


Writing PDA transitions

Transitions are written a, b → c where a is an input symbol

If the machine sees input a then it may replace b on top of thestack with c

In other words, b is the symbol popped off the stack and c is thesymbol pushed onto the stack

If b is ε (the empty symbol) then make the transition without anypop (read) operation

If c is ε then make the transition without any push (write) operation

$ is a special symbol used to denote the bottom of the stack: itmeans the stack is empty

35 / 43


PDA Example for 0n1n

q1 q2

q3q4

ε, ε → $

1, 0 → ε

ε, $ → ε

0, ε → 0

1, 0 → ε

36 / 43


Reading PDA transitions

ε, ε → $ Given no input and nothing to pop, put the empty stacksymbol onto the stack. All PDAs start with this transition

0, ε → 0 On seeing an input 0, push a 0 onto the stack. Do notpop anything from the stack

1, 0 → ε On seeing input 1 and a 0 on the stack, pop the 0 fromthe stack and do not push anything on. This step pairs off all the 1swith the previously stored 0s.

ε, $ → ε On seeing no input and an empty stack, accept the stringsince we must now have seen the same number of 1s as 0s

37 / 43


Palindrome language

Design a PDA to recognize the language { wwR | w ∈ {0, 1}∗ }. w isany binary string, wR means w written backwards. So for example,001110011100 is in the language.

Here we will use non-determinism to “guess" when the middle of the stringhas been reached.

Approach:

1. Push all the symbols read onto the stack

2. Guess you have reached the middle of the word

3. Then pop elements off the stack if they match the next input symbol

4. Accept the string if every popped symbol matches the input, and thestack empties at the same time the end of the input is reached.

5. Reject otherwise

38 / 43


Palindrome language PDA

q1 q2

q3q4

ε, ε → $

ε, ε → ε

ε, $ → ε

0, ε → 0 1, ε → 1

0, 0 → ε 1, 1 → ε

39 / 43


Designing PDAs

See Sipser Lemma 2.21 for details (p117 in 3rd ed)

Idea: PDA accepts input w if grammar G generates it, by followingthe derivations

Use non-determinism to allow for choice of productions.

Push the start symbol S onto the stack ε, ε→ S

If top of stack is a non-terminal (S) then non-deterministically chooseany production and substitute S by the rule.

Since the rules generate more symbols we need a string of pushes.No inputs are consumed at this stage.

If top of stack is a terminal symbol (0 or 1) then check whether itmatches the next symbol in the input string

If they don’t match then go to a non-accept state, if they do matchthen continue

40 / 43


Designing PDAs (cont)

(See the tutorial questions for examples)

41 / 43


How to show a language is context-free

Theorem: PDAs are equivalent in power to context-free grammars.

This is a useful result because it gives 2 options for proving that alanguage is CF

1. specify a PDA for the language

2. specify a CF grammar for language

42 / 43


Backus-Naur form (for information)

Context-free grammars related to computer languages are oftengiven in a special shorthand notation known as Backus-Naur form.

<identifier> :: = <letter> | <identifer> <letter> |<identifier> <digit>

<letter> :: = a | b | c | ... | z<digit> :: = 0 | 1 | ... | 9

In BNF, the non-terminals are identifed by the angle brackets, andproductions with the same left-hand side are combined into a singlestatement with the OR symbol.

43 / 43

cits2211 discrete structures non-regular languages and push … · 2018-10-17 · 7/43. non-regular...

Documents