comparison networks sorting sorting binary values sorting arbitrary numbers implementing symmetric...

Comparison Networks

Sorting• Sorting binary values• Sorting arbitrary numbers• Implementing symmetric functions

Sorting Algorithms

)log ( nnO comparisons to sort n elements

)(mO comparisons required to merge two arrays of size m/2

Order of comparisons not fixed in advance.

Mergesort(array[1,…,n] of Integers):begin Mergesort(array[1,…,n/2]); Mergesort(array[n/2+1,…,n]); Merge(array[1,…,n/2], array[n/2+1,…,n]);end

Not readily implementable in hardware.

Example

Sorting Networks

Order of comparisons fixed in advance.Readily implementable in hardware.

)log ( scomparison ofnumber ,)(logdepth 22 nnOnO

CDBA

ABCD

SortingNetwork

Sorting Networks (binary values)

SortingNetwork

10010011

00001111

inputs outputs

sorted

Comparator (2-sorter)

x

y

min(x, y)

max(x, y)

inputs outputs

C

Comparator (2-sorter)

x

y

min(x, y)

max(x, y)

inputs outputs

Comparison Network

0

0

1

1

0

0

1

1

0

0

1

1

1

1

0

0

depth d

width n

Comparison Network

0

0

1

1

0

0

1

1

0

0

1

1

1

1

0

0

n / 2comparisonsper stage

d stages

Sorting NetworkAny ideas?

Sorting Network

inputs outputs

n 1Sorting

Network...... n

...

Insertion Sort Network

inputs outputs

depth 2n 3

Batcher Sorting NetworkNext Lecture

)(logdepth 2 nO

Sorting Arbitrary Numbers

x, y can be values from any linearly ordered set,e.g., integers, reals, etc.

inputs outputs

x

y

min(x, y)

max(x, y)

Comparison function:

C(X,Y) =1 if X > Y,

0 otherwise.

Idea: use C(X,Y) to select the min and the max of X and Y.

X, Y: integers represented as m-bit binary strings.

Integer Comparator


9

6

2

2

6

9

2

9

6

sorted6

9

2


1

4

5

1

4

5

1

5

4

sorted4

5

1


3

0

7

3

0

7

3

7

0

0

7

3

not sorted

How can we verify if a network sorts all possible input sequences?


inputs outputs

Try all possible 0/1 sequences.


0

0

0

0

0

0

0

0

0

0

0

0 000 000

inputs outputs



0

1

0

0

0

1

0

0

1

0

1

0

inputs outputs

000 000


001 001


0

0

1

0

0

1

0

1

0

0

1

0

inputs outputs

000 000


001 001010 001


0

1

1

0

1

1

0

1

1

1

1

0

inputs outputs

000 000


001 001010 001011 011


1

0

0

0

0

1

0

1

0

0

1

0

inputs outputs

000 000


001 001010 001011 011100 001


1

1

0

0

1

1

0

1

1

1

1

0

inputs outputs

000 000


001 001010 001011 011100 001101 011


1

0

1

1

0

1

1

1

0

0

1

1

inputs outputs

000 000


001 001010 001011 011100 001101 011110 101

not sorted!


1

1

1

1

1

1

1

1

1

1

1

1

inputs outputs

000 000


001 001010 001011 011100 001101 011110 101111 111

not sorted!


inputs outputs



0

0

0

0

0

0

0

0

0

0

0

0 000 000

inputs outputs



0

1

0

0

0

1

0

0

1

0

1

0

inputs outputs

000 000


001 001


0

0

1

0

0

1

0

1

0

1

0

0

inputs outputs

000 000


001 001010 001


0

1

1

0

1

1

0

1

1

1

1

0

inputs outputs

000 000


001 001010 001011 011


1

0

0

0

0

1

0

1

0

1

0

0

inputs outputs

000 000


001 001010 001011 011100 001


1

1

0

0

1

1

0

1

1

1

1

0

inputs outputs

000 000


001 001010 001011 011100 001101 011


1

0

1

0

1

1

1

1

0

1

1

0

inputs outputs

000 000


001 001010 001011 011100 001101 011110 011


1

1

1

1

1

1

1

1

1

1

1

1

inputs outputs

000 000


001 001010 001011 011100 001101 011110 011111 111


inputs outputs

000 000


001 001010 001011 011100 001101 011110 011111 111

all sorted!

Zero-One Principle

If a comparison network sorts all possible sequencesof 0’s and 1’s correctly, then it sorts all sequencesof arbitrary numbers correctly.

Lemma

0a

1a

Given

0b

1b

For a monotonically increasing function f,

Lemma

0a

1a

Given

0b

1b


)( 0af

)( 1af

)( 0bf

)( 1bf

Proof: Lemma

),min( 10 aa

),max( 10 aa

0a

1a

Proof: Lemma

))(),(min( 10 afaf

))(),(max( 10 afaf

)( 0af

)( 1af

Proof: Lemma

f is monotonically increasing:

yx )()( yfxf

)( 0af

)( 1af

))(),(min( 10 afaf

))(),(max( 10 afaf

Proof: Lemma


yx )()( yfxf

)),(min( 10 aaf

)),(max( 10 aaf

)( 0af

)( 1af

Proof: Lemma


)( 0bf

)( 1bf

)( 0af

)( 1af

yx )()( yfxf

Generalization

0a

1a

2a

0b

1b

2b

0c

1c

2c

0d

1d

2d

Given


)( 0af

)( 1af

)( 2af

)( 0bf

)( 1bf

)( 2bf

)( 0cf

)( 1cf

)( 2cf

)( 0df

)( 1df

)( 2df

Generalization

(by induction)

Proof: Zero-One Principle

Suppose

b) there exists a sequence that it doesn’t sort, i.e., 110 ,...,, naaa

ji aa , such that ji aa but is placed before in the output.ja ia

a) the network sorts all sequences of 0’s and 1’s,

Define

f (x) =0 if

1 otherwiseiax


0a

1a

1na

0a

1a

1na

ja

ia

SortingNetwork

......

...


SortingNetwork

)( 0af

)( 1af

)( 1naf

)( 0af )( 1af

)( 1naf

)( jaf)( iaf

......

...


SortingNetwork

)( 0af

)( 1af

)( 1naf

)( 0af )( 1af

)( 1naf

......

...

10

contradiction!

Batcher Sorting Network, n = 4

n = 4

n = 4


Lemma 1

Any subsequence of a sorted sequence is a sorted sequence.

00011111

00111

sorted

sorted

Lemma 2

For a sorted sequence, the number of 0’s in the even subsequence is either equal to, or one greater than, the number of 0’s in the odd subsequence.

00000111

0

0

0

1

0

0

1

1

sorted

even odd

denotes the the number of 0’s in y y

For two sorted sequences and : x x

11 OEOE xxxx

denotes the even subsequence of Ey y

denotes the odd subsequence of Oy y

Lemma 3

Lemma 3

00000111x

0

0

0

1

Ex

0

0

1

1

Ox

00011111

x¢

0

0

1

1

Ex¢

0

1

1

1

Ox¢

Lemma 3

1 OEO xxx

1 OEO xxx

For two sorted sequences and : x x

11 OEOE xxxx

(by Lemma 2)

(by Lemma 2)

Merge Network

0x

1x

2x

3x

0x

1x

2x

3x

sorted

sorted

Merge[4]

Merge[4]

sorted

0y

1y

2y

3y

4y

5y

6y

7y

Merge Network (pf.)

0x

1x

2x

3x

0x

1x

2x

3x

sorted

sorted

Merge[4]

Merge[4]

sorted

sorted

(by Lemma 1)

(by Lemma 1)

Merge Network (pf.)

Merge[4]

Merge[4]

sorted

sorted

Ex

Ox

Ex

Ox

OE xx OE xx and

differ by at most 1

By Lemma 3

Merge Network (pf.)

Merge[4]

Merge[4]

sorted

0y

1y

2y

3y

4y

5y

6y

7y

Ex

Ox

Ex

Ox

OE xx OE xx and

differ by at most 1

By Lemma 3

Merge Network (pf.)

Merge[4]

Merge[4]

Ex

Ox

Ex

Ox

OE xx OE xx and

differ by at most 1

By Lemma 3

0

0

1

1

0

0

0

1

0

0

0

0

0

1

1

1

Batcher Sorting Network

0x

1x

2x

3x

4x

Sort[4]

Sort[4]

0y

1y

2y

3y

4y

5y

6y

7y

5x

6x

7x

Merge[8] sorted

Merge[4]


Sort[2]

Sort[2]

Sort[4]

Sort[4]


Merge[8]

Sorting Networks

Batcher

n2log 1830depth

AKS (Chvátal)

)log1(log 2

1depth 22 nn

AKS (Ajtai, Komlós, Szemerédi) Network:),(logdepth nO based on expander graphs.

AKS better for 36592n

Balancer

Asynchronous token routing device

inputs outputs

1 bit of memory

Balancer

Asynchronous token routing device

inputs outputs

1 bit of memory

balancedtoken counts

Balancer

Snapshot

inputs outputs

1 bit of memory

x

y

2

yx

2

yx

be

a a

a

ab b

b

c

c c

cd d

e

e

e

d

d

f

g f

g

f

f

g g

Data structure for multiprocessor coordination

step sequence

Counting Network

3

1

3

0

2

2

2

1

2

2

2

1

2

2

2

1

step sequence

Counting Network

Execution trace: token counts on all wires

Counting NetworkTokens are assigned value based on the output wirenumber.

1, 5, 9, 13, . . .

2, 6, 10, 14, . . .

3, 7, 11, 15, . . .

4, 8, 12, 16, . . .

CountingNetwork

• High throughput• Low contention

Advantages

Comparator

x

y

min(x, y)

max(x, y)

inputs outputs

Balancer

x

y

inputs outputs

2

yx

2

yx

Balancer

7

2

inputs outputs

4

5

Balancing Network

3

1

3

0

2

2

2

1

2

2

2

1

2

2

2

1

depth d

width n

Smooth Sequences

BalancingNetwork

......

0x

1x

1nx

for anysmooth property: 1 ji xx ji,

Smooth Sequences

BalancingNetwork

4

3

3

3

4

3

3

3

...

for anysmooth property: 1 ji xx ji,

Step Sequences

0x

1x

1nx

BalancingNetwork

......

for anystep property: 10 ji xx ji

Step Sequences

BalancingNetwork

3

3

3

3

4

4

4

4

...

for anystep property: 10 ji xx ji

Counting Network

CountingNetwork

...

Balancing network with step output sequences:

0x

1x

1nx

...for all inputs

for any

10 ji xx

ji

Sorting vs. Counting

Counts

BalancingNetwork

......

ComparisonNetwork

......

isomorphic

Sorts?


BalancingNetwork

...

Counts

...Comparison

Network...

...

isomorphic

Sorts?


If a balancing network counts, then its isomorphiccomparison network sorts, but not vice-versa.

Theorem


• By 0/1 principle, we need only consider 0/1 inputs.

Counts Sorts

x

y

2

yx

2

yx

balancer

x

y

min(x, y)

max(x, y)

comparator


• By 0/1 principle, we need only consider 0/1 inputs.• A step sequence of 0’s and 1’s is a sorted sequence of 0’s and 1’s.

Counts Sorts

comparator

1

0

0

1

1

0

0

1

balancer

1

2

0

3

4

1

2

0

3

4

1

2

0

3

4

1

2

0

3

4

1

2

0

3

4

1

2

0

3

4

1

2

0

3

4

1

2

0

3

4

Insertion Sort: a network which sorts but doesn’t count.

Sorts Counts


16

0

16

0

0

0

0

32

0

0

8

8

16

0

0

12

4

12

4

0

8

8

12

2

2

10

5

10

5

2

7

8

10

5

2

9

8

8

5

2

Sorts Counts


be

a a

a

ab b

b

c

c c

cd d

e

e

e

d

d

f

g f

g

f

f

g g

Data structure for multiprocessor coordination

step sequence

Counting Network

3

1

3

0

2

2

2

1

2

2

2

1

2

2

2

1

step sequence

Counting Network

Execution trace: token counts on all wires

Counting NetworkTokens are assigned value based on the output wirenumber.

1, 5, 9, 13, . . .

2, 6, 10, 14, . . .

3, 7, 11, 15, . . .

4, 8, 12, 16, . . .

CountingNetwork

• High throughput• Low contention

Advantages

Batcher Counting Network

n = 4inputs

Batcher sorting network also works as a counting network!

3

1

3

0

2

2

2

1

2

2

2

1

2

2

2

1

Lemma 1

Any subsequence of a step sequence is a step sequence.

33344444

33444

step

step

Lemma 2

33344444

3

3

4

4

even

3

4

4

4odd

step

For a step sequence, the sum of the even subsequence is either equal to, or one less than, the sum of the odd subsequence.

Sum of even subseq.: 3+3+4+4=14

Sum of odd subseq.: 3+4+4+4=15

Lemma 3

1 OEO xxx

1 OEO xxx

denotes the sum of y y

For two step sequences and : x x 11 OEOE xxxx

denotes the even subsequence of Ey y

denotes the odd subsequence of Oy y

Follows from:

(by Lemma 2)

(by Lemma 2)

33344444

x

3

3

4

4

Ex

3

4

4

4

Ox

99999999

x¢

9

9

9

9

Ex¢

9

9

9

9

Ox¢

14Ex

15Ox

36Ex

36Ox

Lemma 3

Merge Network

0x

1x

2x

3x

0x

1x

2x

3x

step

step

Merge[4]

Merge[4]

step

0y

1y

2y

3y

4y

5y

6y

7y

Merge Network (pf.)

0x

1x

2x

3x

0x

1x

2x

3x

Merge[4]

Merge[4]

step

step

step

step

Merge Network (pf.)

Merge[4]

Merge[4]

step

step

Ex

Ox

Ex

Ox

OE xx OE xx and

differ by at most 1

By Lemma 3

33344444

step

Merge Network (pf.)

Merging two step sequences whose sums differ by at most one:

3

3

4

4

OE xx

3

4

4

4

OE xx

33434444

not quite step

Merge Network (pf.)

Merging two step sequences whose sums differ by at most one:

3

4

4

4

OE xx

3

3

4

4

OE xx

Merge Network (pf.)

Merge[4]

Merge[4]

step

0y

1y

2y

3y

4y

5y

6y

7y

Ex

Ox

Ex

Ox

Batcher Counting Network

0x

1x

2x

3x

4x

Count[4]

Count[4]

0y

1y

2y

3y

4y

5y

6y

7y

5x

6x

7x

Merge[8] step

Verifying that a Network Counts

A balancing network with m balancers is a counting network if and only if it satisfies the step property for all input sequences with sum .

Theorem

m2

comparison networks sorting sorting binary values sorting arbitrary numbers implementing symmetric...

Documents