FLACRewrite Systems

Klaus Sutner

Carnegie Mellon University

Fall 2019

1 Rewrite Systems

� Thue Systems

� Post Systems

Where Are We? 3

We have seen a number of major success stories in math/CS:

computability

pattern matching (FSM)

model checking (FSM)

Here are more items for this list:

rewrite systems

parsing (grammars)

Recognition versus Generation 4

So far, we have focused on the Recognition Problem for various languages.

Problem: Recognition LInstance: A language L ⊆ Σ?, a string x ∈ Σ?.Question: Is x ∈ L?

How about trying to systematically enumerate the elements of L instead?

Problem: Generate LInstance: A language L ⊆ Σ?.Solution: Enumerate all x ∈ L

A solution to this problem would be some device/algorithm that runs foreverand produces all the elements of L—but nothing else.

Deja Vu All Over 5

This idea comes straight from the old theorem that a set is semidecidable if,and only if, it is recursively enumerable.

We are now interested in (fairly easily) decidable sets, and we want theenumeration to be of a fairly simple kind, not just some arbitrary Turingmachine grinding away. Specifically, we will use rewrite systems to generatelanguages.

But the main idea is nothing new.

Axel Thue (1863–1922) 6

Thanks to CS, rewrite systemsare now an active research area,a century after their invention.

Two Problems 7

Note that there really are two issues:

Given a decision algorithm, extract an enumeration algorithm.

Given an enumeration algorithm, convert it into a decision algorithm.

The second problem is highly important in the RealWorld: the enumerationalgorithm is the specification of a programming language, and we want toconvert it into a parser.

Rewrite systems 8

DefinitionA rewrite system (r/w system) is a pair

G = 〈Σ,P〉

where Σ is an alphabet, and P ⊆ Σ? × Σ? is a finite set of productions or r/wrules.

We often write productions as

π : α � β

Rewrite systems are also called semi-Thue systems or reduction systems.

For simplicity, we well also write P for the whole system.

How Does It Work? 9

The main idea is very simple: a string x = uαv can be rewritten to y = uβvwhenever α � β is in P. The α in x is called a handle. Note: there may bemultiple handles in x, and they may overlap.

More precisely, define a notion of derivation on Σ?. First, one step:

x→P y ⇐⇒ ∃α � β in P, u, v (x = uαv and y = uβv).

Let∗→P be the reflexive transitive closure →P .

Claim∗→P is always semidecidable.

And Generating? 10

Given a r/w system 〈Σ,P〉 fix some word u ∈ Σ?, sometimes called the axiom.

Then we can produce a set of strings by collectiong all strings derivable from u:

Lu = { v ∈ Σ? | u ∗→P v }

Lu is semidecidable and thus also recursively enumerable: we can build amachine that systematically constructs all elements of Lu.

Two-Way Rewrite 11

Rewrite rules are naturally directed: u can be replaced by v, but not necessarilythe other way round. Sometimes (in particular in algebra) one wants to thinkof the rules as equations:

u→ v versus u = v

Define the symmetric closure of P to be

Ps = P ∪ Pop

We can think of Ps as a system of equations between words.

Analogously, one defines x↔P y and x∗↔P y by using the rules in Ps rather

than just P. In this case one speaks of a Thue system.

Losing Direction 12

If you prefer, you can think of this as a compact description of (infinite) graphsover the vertex set Σ?.

The step from P to Ps is then very similar to turning a digraph into a ugraphby ignoring the direction of the edges.

We can think of P as defining a digraph on Σ?: the edges are given byx→P y.

Using Ps instead will produce a ugraph and the equivalence relation x∗↔P y

corresponds to the connected components of this graph.

Example: Erasing Things 13

Consider alphabet {a, b} and P: ab � ε, ba � ε.

Recall our old counting function f(x) = |x|a − |x|b.

Claimx∗→ ε iff f(x) = 0.

Proof.

Every application of→P preserves f , so the implication left to right is obvious.

For the other direction suppose x 6= ε. Since f(x) = 0 we must have x = uabvor x = ubav. Either way, x→P uv, done by induction.

2

Mystery System 14

P: abc � ab, bbc � cb

Here are the first few steps applying Ps to abb.

abb

abcb

abb, abbbc, abccb

abcb, abcbbc, abcccb

abb, abbbc, abccb, abbbcbc, abccbbc, abccccb

abcb, abcbbc, abcccb, abbbbbcc, abcbbcbc, abcccbbc, abcccccb

abb, abbbc, abccb, abbbcbc, abccbbc, abccccb, abbbcbcbc,

abcbbbbcc, abccbbcbc, abccccbbc, abccccccb

Come Again? 15

This seems fairly chaotic, but if one focuses on strings of the form a?b?c? thereis a pattern. Write L for all the strings derivable from abb. Then

L ∩ a?b?c? = { ab2n+1cn | n ≥ 0 }

In a sense, this r/w system computes the function n 7→ 2n + 1.

Thus, very simple rewrite systems can easily produce fairly complicatedlanguages.

150 16

The first 150 words generated by the system, abb is the one on top.

Undecidability 17

Derivability is semidecidable, and in general, that is all we can say: it may wellbe undecidable.

To see why, consider configurations of a Turing machine M be written as usualas strings of the form

C = bmbm−1 . . . b1 p a1a2 . . . an ∈ Γ?QΓ?

Clearly, the one-step relation of M can be described as a rewrite system P, so∗→P corresponds to computations of the TM.

Thus, a decison algorithm for∗→P y would solve the Halting problem for M.

But, for less general rewrite systems there are lots of interesting results.

Rewriting and Computer Algebra 18

Mathematica has a large collection of built-in rewrite rules that can simplifymathematical expressions:

D[ 2 Cos[3 x - Pi], x ] - 6 Sin[3 x] ==> 0

A (rather lame) trace of the corresponding evaluation:

(((3x− π,−π + 3x), cos(−π + 3x),− cos(3x)), 2(− cos(3x)), 2(−1) cos(3x),

− 2 cos(3x),−2 cos(3x)),∂(−2 cos(3x))

∂x, 6 sin(3x),−(6 sin(3x)),

− 6 sin(3x),−6 sin(3x), 6 sin(3x)− 6 sin(3x),−6 sin(3x) + 6 sin(3x), 0

Rewriting and Functional Languages 19

There are even whole programming languages based entirely on the notion ofrewriting, see for example

http://code.google.com/p/pure-lang/

Pure is a functional programming language based on term rewriting.This means that all your programs are essentially just collections ofsymbolic equations which the interpreter uses to reduce expressions totheir simplest (“normal”) form.

let X = [x,xˆ2,xˆ3];reduce X with x = u+v end; // yields [u+v,(u+v)ˆ2,(u+v)ˆ3]

Requirements 20

To use rewrite rules as simplifiers in the real world one typically needs a fewspecial properties:

The precise order in which the rules are applied should not matter.Otherwise it gets tricky to determine the right rewrite order.

Whatever rules we apply, after finitely many steps we should come to afixed point. Otherwise we don’t get termination.

So the ultimate result of rewriting should be uniquely determined by the input;any term must reduce to a unique expression, independent of any choices alongthe way.

Irreducibility and Normal Forms 21

Definitionx ∈ Σ? is irreducible if there is no y such that x→P y.A normal form for x is an irreducible word x′ such that x

∗→P x′.

Normal forms are particularly useful if they are unique: we can use x′ as “thename” for the class of x with respect to

∗↔P .

In the erasure example above, x such that f(x) ≥ 0 has normal form af(x),and b−f(x), otherwise.

Hence the equivalence classes under∗↔P behave much like the integers. We

have a weird description of Z.

Irreducible is Regular 22

LemmaLet 〈Σ,P〉 be a finite rewrite system. Then the set of irreducible words isregular.

Proof.

Irreducibility only depends on the left hand sides of rules (`, v) ∈ P.

Suppose `1, `2, . . . , `n is a listing of all LHS.

We can clean up a bit by removing all `i that have some other `j as properfactor.

Then a word x is irreducible iff x /∈ Σ?{`1, . . . , `n}Σ? and it is not hard toconstruct a FSM for this (even a deterministic one, using, say, theAho-Korasick algorithm).

2

Another Example 23

Again let Σ = {a, b} and

P: aaaa � ε, bb � ε, ba � aaab

Burning Question: Is there a normal form? Is it unique?

One can see that repeated application of the third rule pushes all a’s in front ofall b’s, so we wind up with an expression

aαbβ

Applying rules 1 and 2 makes sure that 0 ≤ α < 4 and 0 ≤ β < 2.

Dihedral Group 24

We are really dealing with the dihedral group D4: rigid motions of a square inthe plane.

In this interpretation, a corresponds to a rotation by π/2 and b corresponds toa reflection (horizontal, vertical or diagonal, it does not matter).

As a geometric operation, ba = aaab.

Check it out.

Confluence 25

Rewrite systems are almost always nondeterministic: typically, there are manydifferent handles in a string, and thus many different ways it can be rewritten.Note that the handles may even overlap, so order of application mattersgreatly. In order for a unique normal form to exist one would like to be in asituation where the order of rule applications does not matter.

DefinitionA rewrite system is confluent if for any w, x, y ∈ Σ? such that w

∗→ x andw∗→ y then there is a z ∈ Σ? such that x

∗→ z and y∗→ z.

The system is Church-Rosser if for any x, y ∈ Σ? such that x∗↔ y there is a

z ∈ Σ? such that x∗→ z and y

∗→ z.

LemmaA rewrite system is Church-Rosser iff the system is confluent.

Termination 26

Confluence is not enough to guarantee the existence of a unique normal form:the chain of reductions might fail to terminate.

DefinitionA rewrite system is terminating if there is no infinite sequence (xi)i<ω of wordssuch that xi →P xi+1.

TheoremIn a confluent and terminating rewrite system unique normal forms exist.

Proof.

Rewrite w until no further reductions are possible (by termination), call theresult w′, a irreducible string.

Proof Cont’d. 27

Now suppose there is some other normal form w′′. If w′ 6= w′′ then at leastone of them can be rewritten by confluence, contradicting irreducibility.

2

In fact, each rewrite step can be performed by a finite state machine: bycleaning up LHS as in the lemma, for each reducible w there is a unique handle` such that w = x`y, |x| minimal, which can be found by a FSM and replacedby v such that ` � v is in P. This can easily be handled by a transducer, so weare just iterating a rational relation.

But note that there is no general bound on the number of rewrite steps.

Insuring Termination 28

A standard way to guarantee termination is to construct some well-order >, aso-called reduction order, such that

(u, v) ∈ P implies u > v

Length-lex order is often helpful: x <`` y if |x| < |y| or |x| = |y| and x < y inlexicographic order.

Note that length-lex order is a well-order by brute force. It is often more usefulin induction arguments than lexicographic order.

Termination versus Algebra 29

Note that in algebraic manipulations one often would like to use associativityand commutativity and rewrite terms according to

a ∗ (b ∗ c)→ (a ∗ b) ∗ ca ∗ b→ b ∗ a

Alas, just as often one would like to rewrite in the opposite direction: equationsin algebra really do not carry any direction.

That causes huge problems with termination: a simple-minded algorithm couldjust go back and forth between a ∗ b and b ∗ a forever.

And restricting rules in any obvious way is also problematic since we mightthen miss out on important simplifications. There are special A/C rewritesytems to deal with this.

� Rewrite Systems

2 Thue Systems

� Post Systems

The Word Problem 31

Early in the 20th century Axel Thue was interested in the following question.

Given a set of objects, and a collection of “rules” (transforma-tions that turn objects into objects). Can a given object x betransformed into object y? Is there some third object z suchthat both x and y can be transformed into z?

Thue’s questions make sense for lots of combinatorial structures such as terms,trees or graphs, but we will only consider the special case when the objects areall strings.

The Word Problem 32

There is a nice algebraic interpretation:

Problem: Word Problem for MonoidsInstance: A monoid M and elements x and y of M .Question: Is x = y in M?

Of course, for this to make any computational sense the monoid M must begiven in some useful form: its elements must have a good finitary description.

Also, we must be able to handle the algebra in M effectively: at the very leastwe must be able to perform multiplication and recognize the unit element.

Words and Equations 33

A good way to produce such effective descriptions is to start with the freemonoid Σ? over a (not necessarily finite) set Σ: this is just the collection of allfinite words over Σ.

The operation is concatenation and the unit element is the empty word ε.

For example, Σ = {a} produces a monoid isomorphic to 〈N,+, 0〉.

For better examples, we need to be able to “identify” certain words as denotingthe same element in the monoid.

For example, let Σ = {a, b} and let’s insist that

ab = ba

Then we get a monoid isomorphic to the standard free Abelian monoid〈N2,+, 0〉.

Free Monoids and Quotients 34

LemmaThe Thue equivalence

∗↔P is a congruence on Σ?.

Proof.

It is clear that∗↔P is in fact an equivalence relation.

So assume x∗↔P y and u

∗↔P v. Then xu∗↔P yv since the substitutions can

be applied to the two parts of the word separately.

2

But then we can form the quotient monoid

MP = Σ?/∗↔P

Multiplying Quotients 35

More precisely, since∗↔P is an equivalence relation, we can consider the

equivalence classes [x] of x ∈ Σ?.

Since the relation is a congruence, we can define a multiplication on theseequivalence classes in the obvious way:

[x] · [y] = [xy]

The result is a monoid MP = Σ?/∗↔P .

Finite Representations 36

In this context Σ is called a set of generators for MP .

It is easy to see that every monoid M can be written in the form MP (just useM as Σ and let P be all valid equations in M).

Of course, this is not particularly exciting. The interesting case is when Σ isfinite and P is finite, in which case we speak of a finite representation.

Given a finite representation we can easily check x↔P y by table lookup.

But how about the word problem in MP , which comes down to checkingx∗↔P y?

For chains of substitutions of bounded length the problem is primitive recursive,but what if there is no bound?

Undecidability 37

Theorem (E. Post, A.A. Markov 1947)

The word problem for finitely presented monoids is undecidable.

This theorem is often considered to be the first undecidability result in puremathematics (as opposed to inside logic).

A simple example due to G. Makanin looks like this:

P: ccbb � bbcc, bcccbb � cbbbcc, accbb � bba,

abcccbb � cbba, bbccbbbbcc � bbccbbbbcca

There is a corresponding result for groups (Novikov-Boone 1955), but that issignificantly harder to prove. For commutative monoids the word problem isdecidable.

� Rewrite Systems

� Thue Systems

3 Post Systems

1920s 39

E. Post analyzed Russell and Whitehead’s Principia Mathematica with a viewtowards building simpler systems that would still have the same proof power.

Post managed to interpret Principia as an exercise in word processing:everything comes down to a purely mechanical manipulation of strings,semantics being entirely optional.

This is very similar to the foundational approach advocated by Hilbert (pushingaround symbols on paper).

Post Canonical System 40

Let Σ be a finite alphabet and X a finite collection of variables.

A Post production rule is an expression

W1,W2, . . . ,Wk ⇒ U

where Wi, U ∈ (Σ ∪ X )?.

A Post canonical system (PCS) consists of axioms (a finite set of words overΣ) and a finite collection of production rules.

Derivations 41

The language L(Π) ⊆ Σ? of a PCS Π is define inductively:

All axioms are in L(Π).

Let σ be a binding from X to Σ? and W1,W2, . . . ,Wk ⇒ U a production.If all Wi[σ] are in L(Π), then so is U [σ].

It is straightforward to define a corresponding notion of derivation, either as alinear sequence, or as a tree.

In light of Post’s “analyze Principia” project, the language of a PCS is alsocalled its set of theorems.

Examples 42

In the following, Σ = {a, b} and the only axiom is ε.

Production rules for “balanced parentheses”:

X ⇒ aXb

X, Y ⇒ XY

Production rules for “same number of as and bs”:

XY Z ⇒ XaY bZ

XY Z ⇒ XbY aZ

A Derivation 43

Here is a sample derivation of abbaab in the second PCS.

X Y Z ruleε − − − ax.ab ε ε ε 1abab a ε b 2abbaab ab ε ab 2

ExerciseShow that this PCS works as advertised: it’s language consists of all stringsover Σ with the same number of as and bs.

Semidecidability 44

It is clear that there is an algorithm to test whether an alleged derivation isactually correct according to the rules of a particular PCS (even if we are onlygiven the first column).

But beware: it may well be undecidable whether w ∈ L: a priori, the only wayto tackle this question is to search systematically for a correspondingderivation.

Alas, this approach is really not very helpful:

If w ∈ L, our search will terminate with answer YES.

If w /∈ L, our search will go on forever.

Hence the theorems of a PCS are semidecidable, but may well fail to bedecidable.

Post Normal Form 45

Post managed to prove a surprising normal form theorem for his systems.

Theorem (Post 1943)

All the productions of a PCS can be written in the form

uX ⇒ Xv

More precisely, given a PCS Π one can effectively construct a new system Π′,possibly over a larger alphabet, using only this type of productions, so thatL(Π) = L(Π′) ∩ Σ?.

Note that derivations in normal form system are less complicated, it is easier tofind the right binding for the variable.

Post Tag System 46

Here is a rather useful way of expressing Post normal form.

DefinitionA Post d-tag system consists of an alphabet Σ, a deletion number d and a mapP : Σ→ Σ? .

The system defines the following rewrite rules

a1 a2 . . . ad x→ xP (a1)

Thus, the rewriting here only occurs at the end of the string:

First remove the first d letters, then

append the suffix P (a) where a was the first letter.

So this is much like standard queue operations.

Example 2-Tag 47

The alphabet is Σ = {a, b, c} and the rules are

a→ bc b→ a c→ aaa

For example, we have the following, rather circuitous, derivation from aaa to a:

aaa, abc, cbc, caaa, aaaaa, aaabc, abcbc, cbcbc, cbcaaa, caaaaaa, aaaaaaaa,

aaaaaabc, aaaabcbc, aabcbcbc, bcbcbcbc, bcbcbca, bcbcaa, bcaaa, aaaa, aabc,

bcbc, bca, aa, bc, a

At this point the derivation ends since we have no 2 letters left to erase.

In this system, all initial words produce a terminating derivation.

Orbit Lengths 48

50 100 150 200

20

40

60

80

100

120

The transients (leading to a fixed point), for all 243 words of length 5. Wordsare sorted in lexicographic order.

The Orbit of a30 49

Here are the lengths of all words of the form a? in the orbit of a30:

30, 15, 23, 35, 53, 80, 40, 20, 10, 5, 8, 4, 2, 1

Yup, it’s true . . .

Basic Behavior 50

Given any word w the tag system defines a sequence of words (wi), the orbit ofw.

Clearly there are the 3 basic possibilities:

Halting:for some i, |wi| < d and the rewrite process stops.

Periodic:|wi| remains bounded and the orbit is ultimately periodic.

Unbounded:|wi| grows unboundedly.

Note that it is semidecidable whether w halts or becomes periodic.Unboundedness is more complicated.

Undecidability 51

Theorem (Minsky 1961)

It is undecidable whether a word has an infinite orbit in a Post tag system.

The proof is quite hard and involves a reduction of ordinary Turing machines to“two-tape non-writing Turing machines” (two counters).

With more effort these can then be simulated by tag system.

These undecidability results are genuinely difficult, this is nothing like Halting.

Undecidability in the Real World 52

Post seems to have constructed a complete classification of all binary 2-tagsystems (alas, he did not publish).

But for the d = 3 he was stumped by the following binary system:

a � aa b � bbab

This may seem too simpleminded to cause any serious problems: forcryinoudloud, there are just two tiny rules; with a bit of effort we should beable to completely understand this system. Right?

Alas . . .

Transient Lengths 53

5 10 15 20 25 30 35

500

1000

1500

2000

2500

The length of all transients of words bk, k ≤ 40. The periods are 1, 2 and 6.

The Orbit of b14 54

Lots of structure, but just enough chaos to stump everybody.

ECA 110 55

The proof for the computational universality of elementary cellular automaton110 is based on a simulation of a type of tag system.

Major Bummer 56

Post had originally hoped to obtain a good theory that would cover all formalsystems (including e.g. Russel and Whitehead’s Principia Mathematica).

His initial success was to show that they all could be construed as tag systems.But then he realized that even for d = 3 these systems are too difficult to dealwith, a very unhappy experience.

. . . the best I can say that I would have proved Godel’s theoremin 1921—had I been Godel.

E. Post, 1938