Chapter 6

Reducibility

We demonstrated in the last chapter a direct

process of showing that ATM is not decidable,

thus unsolvable.

We will now use this result to dig out more

unsolvable problems, and particularly, we will

introduce reducibility, the primary technique of

showing a problem is not solvable. /

Reducibility always involves two problems, say

A and B. If A reduces to B, A ≤m B, then a

solution for B can be used to solve A. ,

Since B → A ≡ ¬A → ¬B, if A is not solvable,

nor will be B. ,

It says nothing about the actual solving of

problems A and B, but only deals with the

solvability of problem A, given a solution of

problem B.

1

Examples

If I made the statement that “If I have five

bucks, I will buy you lunch”, and, if you know

that I have five bucks, then you know, logically

speaking, your lunch is on me. ,

But, what do you know if I don’t buy you

lunch? ,

Leaves are on your yard.... To clean it up, you

only need to have a rake or a blower. Hence,

the problem of cleaning the yard reduces to the

problem of finding a tool.

If you can solve the latter problem, i.e., if you

have found a rake or a blower, then the original

problem, i.e., cleaning the yard can also be

solved.

What if you know for sure that you cannot

clean your yard. Can you find a rake or a

blower? ,

2

Back to the business

Reduction is a way of converting one problem

to another so that a solution to the latter can

be used to solve the former.

The insight is that if we know that the former

problem is unsolvable, so must be the latter.

The unsolvable problem we will use most of

the times is ATM, which is the set of 〈M, w〉

such that M accepts w.

We have proved in the last chapter that it is

not decidable, or solvable.

Let HALTTM be the problem that, for any

given M and w, if M halts on w.

We will prove that HALTTM is unsolvable, by

reducing ATM to HALTTM.

3

Informally speaking...

By the reduction idea, if HALTTM has a solu-

tion, then we can also solve the ATM problem,

which is a contradiction, since we know that

ATM is unsolvable.

The key is thus how to reduce ATM to HALTTM.

Assume that R is the TM that decides HALTTM,

i.e., for any given M and w, R will tell us if M

halts on w. We will construct another TM S,

that decides if M accepts w : For any given M

and w, we run R first, if R says M does not

halt on w, S rejects 〈M, w〉, since then there is

no way M will accept s. On the other hand, if

R says that M does halt on w, we then simply

run M on w, and accepts 〈M, w〉, if M accepts

w; rejects otherwise.

Therefore, any solution to the HALTTM prob-

lem indeed provides a solution for the ATM.

Hence, ATM reduces to HALTTM.

4

HALTTM is undecidable

Assume that R decides HALTTM. We construct

the following S that decides ATM.

S = “On input 〈M, w〉,1. Run R on input 〈M, w〉.2. If R rejects 〈M, w〉, reject 〈M, w〉.3. If R accepts 〈M, w〉, then we know that M

does halt on w. We thus run M on w until it

halts.

4. If M accepts w, accept 〈M, w〉; otherwise,

reject.”

But, as ATM is unsolvable, we reach a contra-

diction. Therefore, there could not exist such

a TM R, and HALTTM is not solvable. /

5

Another problem

Let ETM be {〈M〉 : M is a TM and L(M) = ∅}.

We saw earlier that both EFA and ECFL are de-

cidable (Cf. Pages 4 and 11 in the Decidability

notes).

Theorem ETM is undecidable.

Again, we are trying to reduce ATM to ETM.

Let R be the TM that decides ETM, we are try-

ing to construct another TM, S, that decides

ATM, with the help of R.

Given a TM M, R either accepts or rejects M .

One thing is easy, for any given M and w, if R

accepts M, then L(M) = ∅, namely, M accepts

nothing. In particular, M definitely will not

accept w. /

6

The other side...

If R rejects M, the only thing we know is thatL(M) is not empty, so, it accepts something.

Question: Does it accept w?

We have no clue. /

What we will do is to construct another TM,M1, such that the only string it might acceptis w,, i.e., L(M1) ⊆ {w}, depending on whetherM accepts w. We then run R on M1, instead.

If R accepts M1, L(M1) = ∅, thus, M doesn’taccept w. On the other hand, if R rejects M1,

then M accepts something, which must be w,

since this is the only string that could be ac-cepted. ,

We can thus solve ATM, which is a contra-diction. As a result, ETM cannot be solved,either.

Question: What does M1 look like?

7

The accessory

We construct M1 as follows: M1 = “On input

x

1. If x 6= w, reject;

2. otherwise, run M on w and accept w if M

does.

Thus, L(M1) ⊆ {w}, and the equality holds iff

M accepts w.

We are now ready for our final kick.

8

ETM cannot be solved...

When R rejects M1, L(M1) 6= ∅, then L(M1) =

{w}, i.e., M accepts w. On the other hand, if

R accepts M1 then L(M1) = ∅, which happens

when M does not accept, maybe rejects, w.

We thus can construct a TM, S, that decides

ATM, to accomplish a mission impossible. ,

S = “On input 〈M, w〉.1. Construct M1.

2. Run R on input 〈M1〉.3. If R accepts 〈M1〉, i.e., L(M1) = ∅, implying

L(M) = ∅; reject 〈M, w〉;4. otherwise, L(M1) = {w}, i.e., M accepts w;

thus accept 〈M, w〉.”

9

Yet another problem

Let RegularTM be {〈M〉 : M is a TM and L(M)

is regular}, where M is an FA, i.e., all the TMs

that accept a regular language.

Theorem RegularTM is undecidable.

Although an FA is a TM, there is no way we

can tell if a given TM is a FA or not. /

Assume that R decides RegularTM, we con-

struct a TM S that decides ATM.

The trick is to design an M2 so that L(M2) =

{0n1n|n ≥ 0} if M does not accept w and

L(M2) = Σ∗ if M accepts w.

Notice that {0n1n|n ≥ 0} is not regular, while

Σ∗ is. Then, for any given M and w, L(M2) is

a regular language if and only if M accepts w.

Hence, ATM reduces to RegularTM. ,

10

The details...

Let R decide RegularTM, and construct S that

decides ATM..

S = “On input 〈M, w〉 :

1. Construct the following M2

M2 = “On input x:

1.1. If x has the format 0n1n, accept.

1.2. Otherwise, run M on w and accept x if

M accepts w.

2. Run R on input 〈M2〉.3. If R accepts 〈M2〉, accept 〈M, w〉; otherwise,

reject 〈M, w〉.”

If M rejects w, then M2 will accept only those

strings in the form of 0n1n; i.e., L(M2) =

{0n1n|n ≥ 0}; which is not regular (Cf. Page 101

of the RL notes). But, if M does accept w,

M2 accepts everything, i.e., Σ∗, which is regu-

lar (Cf. Page 77 of the RL notes).

Therefore, S accepts 〈M, w〉 iff R accepts 〈M2〉iff L(M2) is regular iff M accepts w. ,

11

Want to have a look?

Let M2 be defined as discussed, the following

chart shows the construction of the machine S

that decides ATM.

Again, since ATM is not decidable, there is no

way to construct S, nor is R.

Hence, RegularTM is not decidable, either.

12

Not always ATM

So far, we always reduce ATM to the problem

that we are interested in. This is certainly not

necessary. ,

Question: Can we tell if two machines speak

the same language?

We saw that EQDFA is decidable (Cf. Page 6

of the decidability chapter), and it is stated on

Page 12 of the decidability chapter that EQCFL

is not decidable. For a detailed discussion of

the latter problem, check out the link on the

course page.

Let EQTM be {〈M1, M2〉 : L(M1) = L(M2)},

we reduce ETM to EQTM to show that EQTM

is not decidable, either. /

13

The reduction process

Assume R decide EQTM, we construct the fol-

lowing S to decide ETM.

S = “ On input 〈M〉 :

1. Run R on inputs 〈M, M1〉, where M1 rejects

everything, i.e., L(M1) = ∅.

2. If R accepts 〈M, M1〉, accepts 〈M〉; other-

wise rejects 〈M〉. ”

Thus, S decides ETM. ,

14

So what?

Hence S accepts M if and only if R accepts

〈M, M1〉 if and only if L(M) = L(M1) = ∅, i.e.,

S decides ETM, which contradicts the undecid-

ability of the emptiness problem of a Turing

machine. (Cf. Page 6)

Theorem EQTM is undecidable.

So, the whole structure looks like a tree, rooted

at the ATM problem.

This structure certainly can grow to include

more problems that are not decidable, starting

with the very beginning of ATM.

15

Computation history

The computation history method is another

important technique of showing that ATM re-

duces to other languages. It is often used to

test if something exists, when showing a prob-

lem is unsolvable.

Definition: Let M be a Turing machine and w

an input string. An accepting computation his-

tory for M on w is a sequence of configurations,

C1, . . . , Cl, where C1 is the start configuration

of M on w, Cl is an accepting configuration

of M, and each Ci legally follows from Ci−1

according to the rules of M.

Similarly, we can define rejecting computation

history.

Question: Did we talk about this stuff before?

Answer: Yes, we did on Page 13 of the Com-

putability notes, when addressing the yield re-

lation. ,

16

Linear bounded automata

A linear bounded automaton (LBA) is a re-

stricted type of Turing machine, in which the

tape head is not allowed to move off the por-

tion of the tape containing the input. If it tries

to, it will stay where it is.

A LBA comes with a quantitatively limited,

but qualitatively unrestricted, memory. Using

a tape alphabet larger than the input alpha-

bet allows the available space increased up to

a constant factor. Hence, for a given input of

length n, the amount of memory available is

linear in n.

As we discussed on Page 69 of the CFL notes,

CSL, the collection of context sensitive lan-

guages, corresponds to LBA.

17

LBA is powerful

Despite its constrained memory, LBA is quite

powerful. For example, the deciders for ADFA,

ACFG, EDFA, and ECFG are all LBAs.

Every CFL can also be decided by an LBA.

Given an CFG, Γ, we convert it to a Chomsky

normal form grammar, Γ′, first, as we discussed

on Page 18 of the CFL notes.

Then, for a word w, it only need to check all

the derivations in Γ′, with their length being

exactly 2|w| − 1, setting a limit as how many

cells we need to check.

We will accept w iff at least one of such deriva-

tions accepts w.

This again is the basis of a compiler of any

programming language (Cf. Page 10 of the De-

cidiability notes).

18

Let’s work on LBA a little

We have the following simple result.

Lemma: Let M (= (Q, Σ,Γ, δ, qaccept, qreject))

be an LBA with q states, i.e., |Q| = q, and

g symbols in its tape alphabet, i.e., |Γ| = g.

There are exactly qngn configurations of M for

a tape of length n.

Proof: Consider a configuration upiv. The state

pi can be any of q states, the R/W head points

to any of the n symbol, each of which can be

any of the g symbols.

Each such combination can be a configuration,

thus there are a total of qngn configurations.

We will now check the decidability of several

LBA related problems.

19

ALBA is decidable

Let ALBA be {〈M, w〉|M is an LBA that accepts

w.} We have the following

Theorem: ALBA is decidable.

If M goes into an infinite loop, it will get locked

up in that loop. On the other hand, the maxi-

mum length of an accepting computation his-

tory for M on w, by the lemma, is q|w|g|w|.

Hence, the following decider is to decide L.

L = “On input 〈M, w〉,

1. Simulate M on w for q|w|g|w| many steps or

until it halts.

2. If M has halted, accept if M accepts, and

reject otherwise. If M has not halted, reject.

The key here is the finiteness of the configu-

ration, which does not hold for a general TM.

20

ELBA is undecidable

Let ELBA be {〈M〉 : M is an LBA where L(M) =

∅.} We have the following

Theorem: ELBA is not decidable.

The proof is by reduction from ATM. For a

Turing machine M and w, we will construct an

LBA B that accepts all the accepting compu-

tation history of M on w, then testing if L(B)

is empty, using the assumed decider for ELBA.

If L(B) is empty, M does not accept w, other-

wise, it does. This leads to the contradiction

we need.

Below shows a possible tape content of B, the

length of which is proportional to M and |w|,

thus it is an LBA.

21

How does B work?

When it receives an input x, B is supposed to

accept x if x is an accepting computation for

M on w.

For a given 〈M〉 and w, B will designate C1 ≡

q0w as the initial configuration, where q0 is the

starting state of M, and it will accept a con-

figuration uqv if q is the accepting state of M.

For any given configuration Ci, it will gener-

ate a next configuration Ci+1 with the rules as

given in M.

For the specific construction and operation de-

tails, check out the proof of Theorem 5.10 in

the book.

The language of B, L(B), is the collection of

all the accepting computations of M on w.

22

Now what?

Clearly, L(B) 6= ∅ iff there is at least one ac-

cepting computation of M on w iff M accepts

w.

Now let R decide ELBA. Then the following S

decides ATM.

S = “On input 〈M, w〉,

1. Construct the LBA B, as described

2. Run R on 〈B〉.

3. If R rejects 〈B〉, accept (There exists at

least one accepting computation, thus M ac-

cepts w.); reject otherwise (There does not

exist any accepting computation, thus M does

not accept w /).

23

A simple problem?

We will show that some rather natural prob-

lem is also undecidable. Particularly, we will

show that the so-called Post Correspondence

Problem (PCP) is not decidable.

Given a collection of dominos, each of which

contains two strings, one on each side, an in-

stance of the PCP problem is a collection, P,

of dominos:{[

t1

b1

]

,

[

t2

b2

]

, . . . ,

[

tk

bk

]}

,

and a match is a sequence, i1, i2, . . . , il, of such

dominos in P, such that ti1ti2 · · · til = bi1bi2 · · · bil.

For example, given the following collection:{[

b

ca

]

,

[a

ab

]

,

[ca

a

]

,

[abc

c

]}

.

We could have a match:{[

a

ab

]

,

[b

ca

]

,

[ca

a

]

,

[a

ab

]

,

[abc

c

]}

.

24

Guess what?

Let PCP = {〈P 〉 : P is an instance of the Post

correspondence problem with a match.}.

Theorem: PCP is undecidable.

The proof is actually quite conceptually simple

but detailed with six parts. It basically shows

that, for any given Turing machine, M, and

string, w, we can construct an instance P of

PCP such that P has a match iff M accepts

w.

That is to say, we simply reduce ATM to PCP.

For the details of this proof, check out the

proof of Theorem 5.15 in the book.

25

What is really going on?

To solve a problem A, we reduce it to an al-

ready solved problem B.

On the other hand, to prove that a problem

B is unsolvable, we reduce an already proved

unsolvable problem A to B.

We have been showing how to apply the re-

ducibility technique to prove the undecidability

of several problems, by reducing, e.g., ATM, to

such a problem.

Now, we formalize this handy notion of reduc-

tion so that we can understand it better (or

worse /).

Definition: A function f : Σ∗ 7→ Σ∗ is a com-

putable function if some Turing machine M,

on every input w, halts with f(w) on its tape.

26

An example

If we represent any number n with n + 1 ‘1’s,

i.e., 0 is ‘1’, 1 is ‘11’, etc. Then, we have the

following Turing machine:

On input

m+1︷ ︸︸ ︷

1 · · ·1B

n+1︷ ︸︸ ︷

1 · · ·1,

1. Scan all the symbols. If the second segment

contains only one ‘1’, change it to ‘B’

2. Otherwise, go over the first segment of ‘1’s.

3. Fill the blank ‘B’ with 1.

4. Go over to the end of the second segment

of ‘1’s, and change the last two ‘1’s with ‘B’.

Thus, the addition function is unsurprisingly

computable. ,

27

Formalize reduction

We used to say that, when a problem A reduces

to another one B, we mean a solution to B can

be used to solve A.

Technically, we have the following definition.

Let A and B be languages. By saying that A

is mapping reducible to B, (A ≤m B), we mean

that, for a computable function, f : Σ∗ 7→ Σ∗,

for every w ∈ Σ∗,

w ∈ A iff f(w) ∈ B.

The function f is called the reduction of A to

B.

In other words, to find out if w ∈ A, we just use

the computable function f to figure out f(w),

then test if f(w) ∈ B.

28

What does it mean?

A mapping reduction of A to B provides a

method to convert questions about member-

ship testing in A to membership testing in B.

To test w ∈ A, we can use the reduction to

effectively compute f(w) and test if f(w) ∈

B. By definition, we can provide a definitive

answer for the former question on A based on

the result obtained from the latter question on

B : w ∈ A iff f(w) ∈ B.

In particular, if one problem is mapping re-

ducible to another, previously solved, problem,

we can obtain a solution to the original one.

On the other hand, if we can map reducible a

known unsolvable problem to a problem, then,

the latter problem must be unsolvable, as well.

29

Theorem: If A ≤m B and B is decidable, so is

A.

Proof: Let M decide B, and let f be the re-

duction of A to B. We construct the following

decider, N, for A.

N = “On input w

1. Compute f(w). //That’s why f has to be

computable.

2. Run M on f(w).

3. Accept, if M accepts; reject otherwise.”

While we use the above one most of the time,

we use the following one in a negative sense,

again based on the idea that

A → B ≡ ¬B → ¬A.

Corollary: If A ≤m B and A is undecidable, so

is B.

30

An example

We once informally showed that the HALTTM

is undecidable by reducing ATM to it. Now, we

look into this result from this new perspective

of mapping reduction.

First, we have to present a computable func-

tion, f, which, for any input 〈M, w〉, will provide

an output 〈M ′, w〉 such that

〈M, w〉 ∈ ATM iff 〈M ′, w〉 ∈ HALTTM.

Below shows F, a Turing machine that com-

putes f :

F =“ On input 〈M, w〉

1. Construct the following machine M ′

M ′ =“On input x

1.1. Run M on x.

1.2. If M accepts x, accept 〈M, w〉.

1.3. If M rejects x, enter a loop.

2. Output 〈M ′, w〉.”

31

So what?

Now let’s show that, with this function, ATM

mapping reduces to HALTTM, i.e.,

〈M, w〉 ∈ ATM iff 〈M ′, w〉 ∈ HALTTM.

Assume 〈M, w〉 ∈ ATM, i.e., M accepts w, then

by the construction, M ′ also accepts w, thus

halts with w, i.e., 〈M ′, w〉 ∈ HALTTM.

On the other hand, if 〈M, w〉 6∈ ATM, i.e., either

M rejects w or gets into an infinite loop, M ′

would get into an infinite loop in both cases,

thus, 〈M ′, w〉 6∈ HALTTM.

This shows that ATM ≤m HALTTM.

Finally, by the corollary, HALTTM is not solv-

able.

Guess we need to stop here. ,

32