Discrete Structures for Computer Science

Adam J. Lee adamlee@cs.pitt.edu 6111 Sennott Square

Lecture #27: Closures of Relations

December 3, 2013

Announcements

No more homework! Thursday: Course wrap-up and review

l  Same format as midterm review…

Today

Types of closures l  Symmetric closure l  Reflexive closure l  Transitive closure

Transitive closures l  Repeated relational composition l  Warshall’s algorithm

Examples

Motivating Example A computer network has data centers in Boston, Chicago, Denver, Detroit, New York, and San Diego. There are direct, one-way telephone lines from Boston to Chicago, Boston to Detroit, Chicago to Detroit, Detroit to Denver, and New York to San Diego. Let R be the relation containing (a, b) if there is a line from the data center in a to the data center in b. Note: The relation R is not transitive! How can we tell who can communicate with whom?

B D2

D1

S

C N

What is a closure?

Definition: Let R be some relation on a set A. R may or may not have some property P (e.g., reflexivity, symmetry, or transitivity). If there is a relation S ⊇ R with the property P such that S is a subset of every relation containing R with the property P, then S is called the closure of R with respect to P. Informally: The closure of a relation R with respect to a property P is the smallest extension to R that does have the property P.

So, how can we compute closures?

Reflexive closures are easy to compute

Definition: Let R be a relation on a set A, and let ∆A be the diagonal relation containing (a, a) for each a ∈ A. The reflexive closure of R is R ∪ ∆A. Example: Let A = {1, 2, 3} and R = {(1,1), (1,2), (2,1), (3,2)}. Given this information, ∆A = {(1,1), (2,2), (3,3)}.

1

2 3

Clearly, R is not reflexive…

1

2 3

… but R ∪ ∆A is reflexive

Symmetric closures are also easy to compute

Definition: Let R be a relation on the set A. R-1 is the inverse of R; i.e., R-1 = {(b,a) | (a,b) ∈ R}. The relation R ∪ R-1 is the symmetric closure of R. Example: Let A = {1, 2, 3} and R = {(1,1), (1,2), (2,1), (3,2)}. Given this information, R-1 = {(1,1), (2,1), (1,2), (2,3)}

1

2 3

Clearly, R is not symmetric…

1

2 3

… but R ∪ R-1 is symmetric

What is the symmetric closure of our “can call” graph from earlier in lecture?

A computer network has data centers in Boston, Chicago, Denver, Detroit, New York, and San Diego. There are direct, one-way telephone lines from Boston to Chicago, Boston to Detroit, Chicago to Detroit, Detroit to Denver, and New York to San Diego. Let R be the relation containing (a, b) if there is a line from the data center in a to the data center in b. Note:

l  R-1 = {(C, B), (D2, B), (D2, C), (D1, D2), (S, N)} l  The symmetric closure is R ∪ R-1

l  This is a “can talk” graph J

B D2

D1

S

C N

Transitive closures are related to paths in relational graphs

Definition: A path from a to b in a directed graph G is a sequence of edges (x0, x1), …, (xn-1, xn) in G, where n is a non-negative integer, x0 = a, xn = b. The path is denoted by x0, x1, …, xn-1, xn and has length n. Example: Consider the graph below. Is (a, b, e, d) a path? What about (a, e, c, d, b)? Or (b, a, c, b, a, a, b)?

a b c

d e

Yes! No! Yes!

Paths are related to relational composition

Theorem: Let R be a relation on a set A. There is a path of length n from a to b if and only if (a, b) ∈ Rn. Recall: Rn is the relation R composed with itself n times. In other words, Rn = R ° … ° R. Example:

l  R = {(1,3), (2,3), (3,4)} l  R2 = {(1,3), (2,3), (3,4), (1,4), (2, 4)}

n times

1

3 4

2

1

3 4

2

Connectivity relations help us understand the meaning of transitive closures

Definition: Let R be a relation on a set A. The connectivity relation R* consists of the pairs (a, b) such that there is a path of length at least one from a to b in R. In other words: (Why?)

Example: Let R be the relation on the set of all subway stops in New York City that contains (a, b) if it is possible to travel from stop a to stop b without changing trains. What is Rn? What is R*?

l  Rn = {(a, b) | It is possible to get from a to b within n changes}

l  R* = {(a, b) | It is possible to get from a to b}

R� =⇥�

n=1

Rn

Relating connectivity to the transitive closure…

Theorem: The transitive closure of a relation R equals the connectivity relation R* Proof intuition: We need to show that (i) R* is transitive and (ii) is contained in all other transitive extensions of R To show transitivity:

l  Given (a,b) ∈ R* and (b, c) ∈ R*, need (a, c) ∈ R* l  This means we have paths from a to b and b to c in R l  As such, we can easily construct a path from a to c in R (How?) l  It follows that (a, c) must be in R*

To show containment: l  Suppose S is a transitive relation containing R l  By a theorem we didn’t prove, S* ⊆ S l  Since R ⊆ S, then R* ⊆ S*, since all paths in R exists in S l  Hence R* ⊆ S* ⊆ S

Do we really need to consider an infinite composition to compute transitive closures?!?

Lemma: Let A be a set with n elements, and let R be a relation on A. If there is a path in R* from a to b, then there is such a path with length not exceeding n. Why? The pigeonhole principle!

l  There are n elements in A l  The longest path through A without repeats is n!

So who cares? What does this really buy us?

R� =⇥�

i=1

Ri =n�

i=1

Ri

This is now finite, and thus computable!

This result makes our lives a lot simpler…

Theorem: Let MR be the zero-one matrix of the relation R on a set with n elements. Then the zero-one matrix of the transitive closure R* is MR* = MR ∨ MR

[2] ∨ … ∨ MR[n].

Note: MR

[i] is the ith Boolean power of the matrix MR

Example: Compute MR* for the following matrix MR

MR =

�

⇤1 0 10 1 01 1 0

⇥

⌅ M [2]R =

�

⇤1 1 10 1 01 1 1

⇥

⌅ M [3]R =

�

⇤1 1 10 1 01 1 1

⇥

⌅

MR� = M [1]R �M [2]

R �M [3]R =

�

⇤1 1 10 1 01 1 1

⇥

⌅

This gives us a simple* algorithm for computing transitive closures

Procedure: transitive-closure(MR : zero-one nxn matrix) A := MR

B := A for i := 2 to n

A := A ⊙ MR

B := B ∨ A end Note: After this procedure, B contains the zero-one matrix for the

relation R*

Start with A and B both representing the relation R

Compute the matrix representing Ri

Compute R ∪…∪ Ri

What is the complexity of this algorithm?

Procedure: transitive-closure(MR : zero-one nxn matrix) A := MR

B := A for i := 2 to n

A := A ⊙ MR

B := B ∨ A end Since the for loop runs n-1 times, we have that

l  n2(2n-1)(n-1) binary operations to compute versions of A l  n2(n-1) binary operations to compute versions of B

In total: This is approximately n4 binary operations

•  n2 entries in A •  To compute each

•  n binary ANDs •  n-1 binary ORs

•  Total: n2(2n-1) operations

n2 boolean ORs

Warshall’s algorithm is a more efficient way to compute transitive closures

This algorithm is based on the idea of building a sequence of zero-one matrices describing connectivity in a relation R via paths over an increasingly large set of interior vertices Example: Consider the path p = a, x1, x2, …, xn, b Main idea:

l  Compute paths through only x1

l  Compute paths through x1 or x2

l  Compute paths through x1, x2, or x3

l  … l  Compute paths through any set of interior nodes

Interior vertices of the path p

Warshall’s algorithm: Intuition

Slightly more formally: l  Let W0 = MR

l  Compute W0, W1, …, Wn such that Wk = [wij(k)] where wij

(k) = 1 if ➣ There is a path from vi to vj

➣ All interior nodes on this path are in the set {v1, v2, …, vk}

l  Note: Wn = MR* (Why?)

Q1: Why should this work? l  Wn = MR*

Q2: Who cares? We already have a different algorithm, right?

l  It turns out to be easy* to compute Wi from Wi-1

Let’s try this technique out…

1 2

4 3

W0 = MR =

�

⇧⇧⇤

0 0 0 11 0 1 01 0 0 10 0 1 0

⇥

⌃⌃⌅

W1 =

�

⇧⇧⇤

0 0 0 11 0 1 11 0 0 10 0 1 0

⇥

⌃⌃⌅

W1 allows paths through 1 • 2, 1, 4 is now valid

W2 allows paths through {1, 2} •  Nothing new…

W3 allows paths through {1, 2, 3} • 4, 3, 1 is now valid • 4, 3, 4 is now valid

W4 allows paths through {1, 2, 3, 4} •  1, 4, 3 is now valid •  1, 4, 3, 1 is now valid •  3, 4, 3 is now valid

W3 =

�

⇧⇧⇤

0 0 0 11 0 1 11 0 0 11 0 1 1

⇥

⌃⌃⌅

W4 =

�

⇧⇧⇤

1 0 1 11 0 1 11 0 1 11 0 1 1

⇥

⌃⌃⌅

This is the transitive closure of R

But wait. There’s a shortcut!

1 2

4 3

W0 = MR =

�

⇧⇧⇤

0 0 0 11 0 1 01 0 0 10 0 1 0

⇥

⌃⌃⌅ W1 =

�

⇧⇧⇤

0 0 0 11 0 1 11 0 0 10 0 1 0

⇥

⌃⌃⌅

W3 =

�

⇧⇧⇤

0 0 0 11 0 1 11 0 0 11 0 1 1

⇥

⌃⌃⌅ W4 =

�

⇧⇧⇤

1 0 1 11 0 1 11 0 1 11 0 1 1

⇥

⌃⌃⌅

Observation: The bit wij is set in the matrix Wk if and only if there is a path from i to j that traverses a subset of {1, 2, …, k}. Better observation: This happens if and only if

l  wij was set in Wk-1 --OR-- l  wik and wkj were set in Wk-1

Path already existed

The inclusion of k connects two existing paths

Warshall’s algorithm

Procedure: warshall(MR : n x n zero-one matrix) W := MR

for k := 1 to n for i := 1 to n for j := 1 to n wij := wij ∨ (wik ∧ wkj)

end Note: After this procedure, W = [wij] contains the zero-one matrix

for the relation R*

What is the cost of using Warhsall’s algorithm?

Procedure: warshall(MR : zero-one nxn matrix) W := MR

for k := 1 to n for i := 1 to n for j := 1 to n wij := wij ∨ (wik ∧ wkj)

end Total: 2 ops * n times * n times * n times = 2n3 binary operations This is much better than our other algorithm (> n4 operations)

2 binary operations

n times n times

n times

Group work!

Let A = {0, 1, 2, 3} and let R = {(0,1), (1,1), (1,2), (2,0), (2,2), (3,0)} Problem 1: Find the reflexive closure of R Problem 2: Find the symmetric closure of R Let A = {1, 2, 3, 4} and let R = {(2,1), (2,3), (3,1), (3,4), (4,1), (4,3)} Problem 3: Draw R as a graph Problem 4: Use our first algorithm to find the transitive closure of R Problem 5: Use Warshall’s algorithm to find the transitive closure of R

Final Thoughts

Closures of relations have many uses in computer science Computing reflexive and symmetric closures is easy! Transitive closures take more work to do right

l  Intuitive algorithm: 2n3(n-1) bit operations l  Warshall’s algorithm: 2n3 bit operations