AutomAtA theoryA Step-by-Step Approach

Lab/Practice Work with Solution

For B.tech., m.tech., BCA, mCA, m.Sc. (CS),

DrDo, Net and GAte (CS)

manish Kumar JhaAssistant Professor

GD Goenka World Institute • Lancaster University

S. CHAND & COMPANY PVT. LTD.(An ISO 9001: 2008 COmpAny)

RAM NAGAR, NEW DELHI-110 055

S. CHAND & COMPANY PVT. LTD.(An ISO 9001 : 2008 Company)Head Office: 7361, RAM NAGAR, NEW DELHI - 110 055Phone: 23672080-81-82, 9899107446, 9911310888 Fax: 91-11-23677446Shop at: schandgroup.com; e-mail: info@schandgroup.com

Branches :AhmedAbAd : 1st Floor, Heritage, Near Gujarat Vidhyapeeth, Ashram Road, Ahmedabad - 380 014,

Ph: 27541965, 27542369, ahmedabad@schandgroup.combengAluru : No. 6, Ahuja Chambers, 1st Cross, Kumara Krupa Road, bengaluru - 560 001,

Ph: 22268048, 22354008, bangalore@schandgroup.combhOpAl : Bajaj Tower, Plot No. 2&3, Lala Lajpat Rai Colony, Raisen Road, bhopal - 462 011,

Ph: 4274723, 4209587. bhopal@schandgroup.comChAndIgArh : S.C.O. 2419-20, First Floor, Sector - 22-C (Near Aroma Hotel), Chandigarh -160 022,

Ph: 2725443, 2725446, chandigarh@schandgroup.comChennAI : No.1, Whites Road, Opposite Express Avenue, Royapettah, Chennai - 600014

Ph. 28410027, 28410058, chennai@schandgroup.comCOImbAtOre : 1790, Trichy Road, LGB Colony, Ramanathapuram, Coimbatore -6410045,

Ph: 2323620, 4217136 coimbatore@schandgroup.com (Marketing Office)CuttACk : 1st Floor, Bhartia Tower, Badambadi, Cuttack - 753 009, Ph: 2332580; 2332581, cuttack@schandgroup.comdehrAdun : 1st Floor, 20, New Road, Near Dwarka Store, dehradun - 248 001,

Ph: 2711101, 2710861, dehradun@schandgroup.comguwAhAtI : DilipCommercial(Istfloor),M.N.Road,PanBazar,guwahati - 781 001,

Ph: 2738811, 2735640 guwahati@schandgroup.comhAldwAnI : BhattColony,TalliBamori,Mukhani,haldwani -263139 (Marketing Office)Mob.09452294584hyderAbAd : PadmaPlaza,H.No.3-4-630,Opp.RatnaCollege,Narayanaguda,hyderabad - 500 029,

Ph: 27550194, 27550195, hyderabad@schandgroup.comJAIpur : 1stFloor,NandPlaza,HawaSadak,AjmerRoad,Jaipur - 302 006,

Ph: 2219175, 2219176, jaipur@schandgroup.comJAlAndhAr : MaiHiranGate,Jalandhar - 144 008, Ph: 2401630, 5000630, jalandhar@schandgroup.comkOChI : KachapillySquare,MullasseryCanalRoad,Ernakulam,Kochi - 682 011,

Ph: 2378740, 2378207-08, cochin@schandgroup.comkOlkAtA : 285/J, Bipin Bihari Ganguli Street, kolkata - 700 012, Ph: 22367459, 22373914, kolkata@schandgroup.comluCknOw : 1stFloor,AryaPratinidhiSabha,5-MirabaiMarg,NearNarhiSabziMandi,Hazratganj,

Lucknow - 226 001, Ph: 4065646, 4026791, 4027188, 4022948 lucknow@schandgroup.commumbAI : BlackieHouse,IIndFloor,103/5,WalchandHirachandMarg,Opp.G.P.O.,mumbai - 400 001,

Ph: 22690881, 22610885, mumbai@schandgroup.comnAgpur : KarnalBagh,NearModelMillChowk, nagpur - 440 032, Ph: 2720523, 2777666 nagpur@schandgroup.compAtnA : 104,CiticentreAshok,MahimaPalace,GovindMitraRoad,patna - 800 004, Ph: 2300489, 2302100,

patna@schandgroup.compune : SadguruEnclave,Groundfloor,SurveyNo.114/3,Plotno.8AlandiRoad,

Vishrantwadi pune – 411015 Ph: 64017298 pune@schandgroup.comrAIpur : Kailash Residency, Plot No. 4B, Bottle House Road, Shankar Nagar, raipur - 492 007,

Ph:2443142,Mb.:09981200834,raipur@schandgroup.com(Marketing Office)rAnChI : ShantiDeepTower,Opp.HotelMaharaja,RadiumRoad, Ranchi-834001

Mob.09430246440ranchi@schandgroup.com SIlIgurI : 122,RajaRamMohanRoyRoad,EastVivekanandapally,P.O.,Siliguri, Siliguri-734001,

Dist., Jalpaiguri, (W.B.) Ph. 0353-2520750 (Marketing Office) siliguri@schandgroup.comVISAkhApAtnAm : No. 49-54-15/53/8, Plot No. 7, 1st Floor, Opp. Radhakrishna Towers, Seethammadhara North Extn.,

Visakhapatnam-530013,Ph-2782609(M)09440100555, visakhapatnam@schandgroup.com (Marketing Office)

© 2015, Manish Kumar JhaAll rights reserved. No part of this publication may be reproduced or copied in any material form (including photocopying or storing it in any medium in form of graphics, electronic or mechanical means and whether or not transient or incidental to some other use of this publication) without written permission of the copyright owner. Any breach of this will entail legal action and prosecution without further notice.Jurisdiction : All disputes with respect to this publication shall be subject to the jurisdiction of the Courts, Tribunals and Forums of New Delhi, India only.

First Edition 2015

ISbn : 978-93-848-5792-9 Code : 1010 0657

This book is for Sale in India & other SAARC Countries only.

printed in india

By Nirja Publishers & Printers Pvt. Ltd., 54/3/2, Jindal Paddy Compound, Kashipur Road, Rudrapur-263153, Uttarakhand and published by S. Chand & Company Pvt. Ltd., 7361, Ram Nagar, New Delhi -110 055.

Dedicated to My Father Shri Rajdhar Jha

and My Mother Smt. Nutan Jha

v

This book is an outcome of my interest and teaching experience of Automata Theory. The aim of this book is to present the concepts and techniques of Automata Theory and to show the relevance of it to the students who are often skeptical. It is written in a simple and easy-to-understand language and will give the students a solid foundation in the subject.

For the instructors, my purpose is to design a flexible, comprehensive teaching tool using various easy techniques. I have provided instructors with a package of material that they could use to teach Automata Theory effectively and efficiently in the most appropriate manner.

This book is designed for undergraduate and postgraduate students of computer science and information technology. Aspirants of various competitive examinations such as NET, GATE, DRDO, etc. will also find this book useful.

I hope the book will cater to the needs of the students, teachers and professionals. Any comments and suggestions for the improvement of the book will be gratefully acknowledged.

Manish Kumar Jhamanishauthor@gmail.com

Disclaimer : While the author of this book has made every effort to avoid any mistakes or omission and has used his skill, expertise and knowledge to the best of his capacity to provide accurate and updated information, the author and S. Chand do not give any representation or warranty with respect to the accuracy or completeness of the contents of this publication and are selling this publication on the condition and understanding that they shall not be made liable in any manner whatsoever. S.Chand and the author expressly disclaim all and any liability/responsibility to any person, whether a purchaser or reader of this publication or not, in respect of anything and everything forming part of the contents of this publication. S. Chand shall not be responsible for any errors, omissions or damages arising out of the use of the information contained in this publication.Further, the appearance of the personal name, location, place and incidence, if any; in the illustrations used herein is purely coincidental and work of imagination. Thus the same should in no manner be termed as defamatory to any individual.

Preface

vi

Acknowledgements

I would like to express my gratitude to many people who saw me through this book; to all those who provided support, talked things over, read, wrote, offered comments, allowed me to quote their remarks and assisted in the editing, proofreading and design. In particular, I am grateful to Mr. B. Singh [(Ex. IES), Chairman and Managing Director, Made Easy Group, New Delhi]; Mr. S. Sudhakar Reddy (General Secretary, CPI); Dr. (Prof.) Prabal Chakraborty (Director, VGI Group, Dadri, U.P.); Mr. Sanjay Tiwari (Sr. Journalist, NDTV); Mr. Nituj Gupta (Executive Director, Sunstone Business School); Mr. Sitaram Yadav (Ex. MP); Ms. Sanju Sudan (APS Healthcare, USA); Dr. S. N. Saran (Director, Manipal University, Jaipur) and Mr. Manohar Deshkar (AITUC, CPI).

I am thankful to the management and the editorial team of S.Chand & Company Pvt. Ltd., New Delhi for their help and support in the publication of this book.

Last but not least, I beg forgiveness of all those who have been with me over the course of the years and whose names I have missed to mention.

Manish Kumar Jha

Contents

1. MatheMatical Foundations 1 – 22 1.1 Computer Scientist versus Computer Science Engineering 1 1.2 Logic 2 1.3 Sets 10 1.4 Functions and Function Notation 15 1.5 Exercises 21

2. theory oF autoMata : alphabet, strings and languages 23 – 42 2.1 Formal Languages 23 2.2 Formal Language Theory 23 2.3 Tree Order Of * 34 2.4 Countable and Uncountable Sets 38 2.5 Exercises 40 2.6 Solutions to Exercises 41

3. regular expression 43 – 55 3.1 Regular Languages and Regular Expressions 43 3.2 Design of Regular Expression 45 3.3 Algebraic Properties of Regular Expression 54 3.4 Exercises 54 3.5 Solutions to Exercises 55

4. Finite autoMata 56 – 190 4.1 Automata 56 4.2 Equivalence of NFA and DFA 65 4.3 Removing e-Cycles from NFAAn Example-based Algorithm 79

4.4 Removing e-Transition from NDFA with e-CyclesAn Example-based Algorithm 81 4.5 Removing Non-determinism from an NFA without e-Transition

An Example-based Algorithm (Conversion of NFA without e-Transition to Equivalent DFA Using Table-based Method) 82

vii

4.6 Minimization of Finite State Automata 90 4.7 Myhill-Nerode Theorem 101 4.8 Design of DFA 111 4.9 Design of NFA 133 4.10 Kleene’s Theorem 152 4.11 State Elimination Method for Conversion of DFA to Regular Expression 163 4.12 Algebraic Method by Using Arden’s Theorem for Conversion of

DFA to Regular Expression 176 4.13 Applications of Finite State Automata 177 4.14 Exercises 178 4.15 Solutions to Exercises 180

5. pushdown autoMata 191 – 219 5.1 Introduction 191 5.2 Pushdown Automaton — Formal Definition 192 5.3 Pushdown Automata — Pointwise Description 193 5.4 Design of Pushdown Automata—Directed Graph Method 194 5.5 Design of Pushdown Automata — Transition-based Method 201 5.6 Design of Pushdown Automata — Flowchart-based Method 208 5.7 Acceptance of String by Pushdown Automata 212 5.8 Closure Properties of Pushdown Automata 213 5.9 Exercises 216 5.10 Solutions to Exercises 216

6. Finite state autoMata with output-Moore and Mealy Machine 220 – 228

6.1 Introduction 220 6.2 Moore Machine 220 6.3 Mealy Machine 222 6.4 Constructing the Incrementing Machine 223 6.5 Equivalent Machines (Moore to Mealy Conversion and Vice Versa) 224 6.6 Exercises 227

7. introduction to context-Free graMMars 229 – 282 7.1 Introduction 229 7.2 Definition 230 7.3 Example 231 7.4 Derivation and Language Generation 231

viii

7.5 Parse Tree 232 7.6 Leftmost and Rightmost Derivation 234 7.7 Ambiguity 235 7.8 Designing CFG 240 7.9 Simplifications of CFGs 242 7.10 Chomsky Normal Form (CNF) 252 7.11 Greibach Normal Form (GNF) 255 7.12 Closure Properties of Context-Free Languages 258 7.13 Regular Grammar 260 7.14 Regular Language ↔ Regular Grammar 261 7.15 Conversions on Regular Grammars 262 7.16 Left-Linear and Right-Linear Grammar 268 7.17 Proving Properties of Regular Language 269 7.18 Closure Properties of Regular Languages 273 7.19 Chomsky Hierarchy of Grammars 274 7.20 Exercises 276 7.21 Solutions to Exercises 277

8. equivalence oF pda's and cFg's 283 – 307 8.1 Introduction 283 8.2 Conversion of a Context Free Grammar to Empty-Stack Pushdown Automata 283 8.3 Conversion of a Pushdown Automata to Context-Free Grammar 289 8.4 Exercises 306

9. puMping leMMa 308 – 322 9.1 Introduction 308 9.2 Pumping Lemma for Regular Languages 308 9.3 Pumping Lemma for Context-Free Languages 315 9.4 Exercises 320 9.5 Solution to Exercises 320

10. turning Machines 323 – 365 10.1 Introduction 323 10.2 Computation 323 10.3 Formal Definition of a Turing Machine 324 10.4 Acceptance, Halting, Rejection 326 10.5 Instantaneous Descriptions 326 10.6 Examples 327

ix

10.7 Working of TM 327 10.8 Design of TM 328 10.9 Modifications of TM/Simulation 360 10.10 The Church-Turing Thesis 363 10.11 Encoding TM as Binary String 363 10.12 Exercises 364

11. decidability, undecidability and recursive Function theory 366 – 383

11.1 Introduction to Language Accepted by Turing Machine 366 11.2 Programming Techniques for Turing Machines (TM and Computers) 370 11.3 Closure Properties 371 11.4 The Halting Problem: An Undecidable Problem 372 11.5 Other Undecidable Problems 374 11.6 Counter Machine 377 11.7 Primitive Recursive: Basic Functions and Operations on It 377 11.8 Unbounded Minimization 381 11.9 Exercises 381 11.10 Solutions to Exercises 381

12. lab/practice worK 384 – 458

index 459 – 462

x

1

Welcome to Computer Science Engineering. It is an education of using principles of engineering to model, analyze, and construct computing systems. People are trusting on computing systems. They works correctly within automobile control systems, medical electronic devices, telephone networks, mobile robots, farms, nuclear power plants, etc. With so much of trust on computers, how can we ensure that all of these computers are being built and operated in a responsible manner? How do we avoid potential disasters – such as mass attacks due to malevolent “viruses,” or an avionic computer crashing and rebooting mid-night? Mean to say, what exactly is computation? Can we mathematically characterize those tasks that computers are capable of performing, and those they are incapable of performing? The subject-matter of this book is about seeking answers to these questions using automata theory.

1.1 Computer SCientiSt verSuS Computer SCienCe engineering

We distinguish a computer scientist from a computer science engineer. We define those people who seek an in-depth understanding of the phenomenon of computation to be a computer scientist. We define those people who seek to efficiently apply computer systems to solve real world problems to be computer science engineers. The distinction is, in a sense, similar to that between a chemist and a chemical engineer. This book will expose us to computer science through automata theory. It will show us how computer science engineers use automata theory as the engineering mathematics, much like ‘traditional engineers’ apply differential and integral calculus to build better-engineered products such as bridges, and automobiles.

░ 1.1.1 Computation

One possible answer is: “if it involves a computer, a program running on a computer, and numbers going in and out, then computation is likely happening.”Computer science has been able to capture the essence of computing in terms of a single formal device known as Turing machine. A Turing machine is a simple device that has finite-state control that interacts with an unbounded storage tape (or, equivalently, a finite-state control that interacts with two unbounded stacks). Many effectively computable functions can be understood in terms of Turing Machine.

░ 1.1.2 Computational models

Models are created with the smallest possible set of mechanisms. In the case of computational models, namely Turing machines was the most powerful. However, with the increasing usage of computers, two other models born out of practical necessity were proposed, roughly two decades after Turing machines were proposed: finite automata in the late 1950’s, and push-down automata

Mathematical Foundations

Chapter

1

░ 2 ░ Automata Theory : A Step-by-Step Approach

shortly thereafter. We will discuss finite automata first, pushdown automata next, and finally Turing machines. All these types of machines are meant to carry out computational procedures (“procedures” for short,) consisting of instructions.

░ 1.1.3 power of Computers

In comparing computational machines (“computers,” loosely), we cannot go by any subjective measure such as their absolute speed, as such “speed records”. Instead, we go by the problem solving ability of computational devices: we say two machines M1 and M2 to be equivalent if they can solve the same class of problems, ignoring the actual amount of time taken. Problem solving can be modeled in terms of algorithmically deciding membership in languages.

░ 1.1.4 Complexity theory

In studying and comparing computing devices, we cannot ignore time or resources entirely. So, “how do we measure quantities such as time, space, and energy?” With advances in technology, computations that took hours a few years ago can nowadays be performed in seconds. Again, Turing machines come into picture.We define a unit of time to be one step of a Turing machine that is running the particular program or computation. We then define time in terms of the asymptotic worst-case complexity notation “O ( )”. However, such a characterization is often not possible: There are many problems (called NP-complete problems) whose best known solutions are exponential time algorithms. It is unknown whether these problems have polynomial time algorithms.

There are problems for which algorithms are known not to exist; for others, it is not known whether algorithms exist. However, it is true that researchers engaged in studying even these problems using Turing machines as one of their important formal models. This is because any result obtained for Turing machines can be translated into corresponding results for real computers.

░ 1.1.5 What is Automata theory?

In this book, we approach the above ideas through automata theory. In this book, we take a much broader meaning for the term ‘automata theory’.

Automata theory is a living branch of theory knowledge. It finds day-to-day applications in numerous walks of life, in the analysis of computing system behavior. Automata theory can be applied in practice. It describes many real life finite-state systems, including: games, puzzles, mathematical logic statements, programming language syntax, error-correcting codes, and combinational and sequential digital circuits. We also illustrate methods to describe, design, simplify, and transform automata.

1.2 LogiC

Logic is the study of reasoning. It is applied to prove things, whether mathematical, philosophical, or scientific. It is the basis for a branch of computer programming used in artificial intelligence. It can be applied to perform software verification. Basically, logic is a systematic method for clearly expressing and demonstrating truths.We focus on symbolic logic, that is, a logic where the statements expressed are symbols that represent concepts. We start with propositional logic where our statements are either true or false. This is a traditional two-valued logic. There are other logics, such as fuzzy logic (which deals with statements

Mathematical Foundations ░ 3 ░

that contain a degree of truthfulness, evaluated as a real number between 0 and 1).In logic, we may write a statement in English, in propositional calculus, in predicate calculus,

or through some other means. In this chapter, we will limit ourselves to these three “languages”. A statement must be well-formed. We refer to such statements as well-formed formulas (wff). There are rules for forming wffs which will be introduced later.In English, the following are legal logical statements:

• It is sunny out. • You are a computer science student. • The number twelve is larger than the number 10. • A tree is a living entity.

In English, the following are not logical statements: • How did you do on the exam? (this does not evaluate to true or false) • He went to the movie. (we do not know who “He” is so we cannot prove whether this is true

or false) • God created the Universe. (we cannot prove this) • Read the next chapter. (this is a command, not a statement)

Statements can include connectives. In logic, our connectives are AND, OR, NOT, Implication (written as →) and equivalence (written as ↔ or ≡, but we will use ↔ in this chapter). Below are the truth tables for these three connectives (also known as Boolean operators).

X Y X AND Y X OR Y NOT XT T T T FT F F T FF T F T TF F F F T

As we should expect, the result of AND is true only if all of the arguments are true, and the result of OR is true if any of the arguments are true. AND and OR can be applied to two or more Boolean values. NOT only applies to a single argument and “flips it”, that is, true becomes false and false becomes true.

Note 1.1: When building a truth table, specify the combinations of T and F values for the arguments (X and Y above) in their proper order. Above, that order is TT, TF, FT, FF. We could also order these as FF, FT, TF, TT. With 2 arguments, there are 22 combinations, if we had 3 arguments, there would be 23 combinations.

Implication, also called the conditional connective, can be thought of, in some ways, like an if-then statement, so “X → Y” can also be thought of as “If X then Y”. X is called the antecedent and in programming, we think of it as our condition. Y is the consequent (the conclusion or what happens if the antecedent is true). However, implication is more far reaching than an if statement. We often use implication to denote a truth preserving rule, which we find when defining classes and their attributes. For instance, since all humans die (eventually), we can write a rule that says “if you are a man, then you will die”, or man → mortal. We can also use implication to represent a causal situation (cause and effect) such as “if it rains, then the grass will be wet”, or rain → wet grass. Note that in logic, implications must be truth preserving. By saying rain → wet grass, we are

░ 4 ░ Automata Theory : A Step-by-Step Approach

actually violating this because there are possibilities where it could rain and yet the grass remain dry (someone who has covered their lawn with a tarp or tent, or if the rain did not fall on that part of the city, etc. We will not concern ourselves with non-truth-preserving knowledge at this point.

As with AND, OR and NOT, we want to derive a truth table for implication. What does the truthfulness of an if statement mean? Consider as an example, “if you are over seven feet in height, then you are considered tall.” The truth table of implications is perhaps less easy to understand:

X Y X → YT T TT F FF T TF F T

Here, the implication is true if X is false, or if both X and Y are true. It seems reasonable to say that the implication is true when both the antecedent and the consequent are true, but why is the implication true when the antecedent is false? Let’s examine our example again: “if you are over seven feet, you would be considered as tall” again. If you are over seven feet in height, you will be considered as tall, so if X is true, Y is true. There are three other possibilities as shown in the truth table. What if you are not over seven feet in height? In this case, X is false. Some people may still consider you to be tall anyway but some people may not. So Y may be true and Y may be false depending on the circumstance. Since, if X is false and Y is true, this is still a reasonable implication to make, then row 3 is correct. And, if X is false and Y is false, this is again a reasonable implication to make, row 4 is correct. Now consider the situation where X is true and Y is false. This means that you are over seven feet but you are not considered to be tall. This violates our implication. So row 2 is incorrect. In effect, the implication is saying that if X is true Y must be true and if X is false, Y may be true or false but if X is true, Y cannot (must not) be false.

One problem with the above example is that the conclusion is based on human perspective. Let’s instead consider an example from mathematics. “if an integer is 4, then its square is 16”. Our first row is true, if x = 4, then x2 = 16. Let’s consider the fourth row. Assume the integer is 3. Then x = 4 is false, and x2 = 16 is false and our statement then remains true. Another case is if x = – 4. Our antecedent is still false because x = 4 is false, but x2 = 16 is true. Our statement is still true then. The last possibility is row 2. Since row 2 requires that the antecedent is true, we know x = 4. However, since 42 = 16, there is no way for the consequent to be false. Therefore, the second row in the truth table, the antecedent is true and the consequent is false, is false.

Equivalence (↔) between two statements means that the two statements mean the same thing. This is the same as saying if statement1 then statement2 AND if statement2 then statement1, or statement1 is equivalent to statement2 means statement1 → statement2 AND statement2 → statement1. Here is the truth table for equivalence, worked through by applying → and AND.

X Y X → Y Y → X (X → Y) AND (Y → X)T T T T TT F F T FF T T F FF F T T T

What we see here is that X and Y are equivalent if they coincide (have the same truth values). This is the NOT of XOR or it is the COINCIDENCE Boolean operator.

Automata Theory

Publisher : SChand Publications ISBN : 9789384857929 Author : Manish Kumar Jha

Type the URL : http://www.kopykitab.com/product/11705

Get this eBook

20%OFF