advanced counting techniques csc-2259 discrete structures konstantin busch - lsu1

Advanced Counting Techniques

CSC-2259 Discrete Structures

Konstantin Busch - LSU 1

Recurrence Relations


nn aaaa ,,,:}{ 10 Sequence

),,,( 110 nn aaafa Recurrence relation:

For any 0nn


Example:

212 nnn aaa 2n

nan 3Solutions to recurrence relation:

5na

Recurrence relation


$10,000 bank deposit

%11 interest

nP :amount after yearsn

111 11.111.0 nnnn PPPP

000,100 P

022

1 )11.1()11.1(11.1 PPPP nnnn

97.922,228$000,10)11.1( 3030 P

Example:


Fibonacci sequence

21 nnn fff

1,0 10 ff

Example:


Towers of HanoiExample:

bar1 bar2 bar3

Goal: move all discs to bar3

Rule: not allowed to put larger discs on top of smaller discs

ndiscs


bar1 bar2 bar3

1n

move recursively discs to bar21nStep 1:


bar1 bar2 bar3

1n

move largest disc to bar3Step 2:


bar1 bar2 bar3

move recursively discs to bar31nStep 3:

n


11 H

12 1 nn HH

nH :total disc moves

2 recursive callswith discs(steps 1&3)

1nmovement oflargest disc(step 2)

one move for one disc


11 H

12

1222

1222

122212122

1221)12(2

12

21

21

1

23

33

2

22

2

1

n

nn-

nn-

nn

nn

nn

H

H)H(

HH

HH

12 1 nn HH

Solving Linear Recurrence Relations


Linear homogeneous recurrence relationof degree :

knknnn acacaca 2211

0kc

k

Rci Constant coefficients:


A sequence (solution) satisfying the relationis uniquely determined by the initial values:

11

11

00

kk Ca

Ca

Ca

k

(these are different constants than the coefficients)


nn ra

if an only if

knknnn acacaca 2211

knk

nnn rcrcrcr 22

11

Solution to recurrence relation:

012

21

1

kkkkk crcrcrcr

divide both sides withknr

characteristic equation


012

21

1

kkkkk crcrcrcr

characteristic equation:

factorize with roots

0)())(( 21 krrrrrr

characteristic roots: krrr ,,, 21

nn ra 1

Multiple possible solutions:n

n ra 2 nkn ra


nn ra 1

The solutionsn

n ra 2 nkn ra

may not satisfy the initial conditions

11

11

00

kk Ca

Ca

Ca


Theorem: Recurrence relation of degree 2

2211 nnn acaca

has unique solutionnn

n rra 2211

where are solutions to the characteristic equation,and are constants thatdepend on initial conditions

21, rr

21,


0212 crcr

Characteristic Equation

21, rrRoots:

02112

1 crcr 2112

1 crcr

02212

2 crcr 2212

2 crcr

Proof:


2211122

1111 rrrra

First compute from initial conditions

210

220

110 rra

21,10 ,aa


22111 rra

210 a 201 a

221201 )( rraa

21

1102 rr

ara

21

201201 rr

raaa


Prove by induction that nnn rra 2211

Basis cases: 210 a

22111 rra

true for the specific choices of 21,


Inductive hypothesis:kk

k rra 2211

for all nk 0

kkk rra 2211

Inductive step:

for nk

prove that

assume that


By inductive hypothesis:

122

1111

nnn rra

222

2112

nnn rra


)()(

)()(

2212

222112

11

222

2112

122

1111

2211

crcrcrcr

rrcrrc

acaca

nn

nnnn

nnn

By recurrence relation definition

Inductive hypothesis

2112

1 crcr 2212

2 crcr

nn

nnn

rr

rrrra

2211

22

222

21

211 )()(

End of Proof


Fibonacci sequence

21 nnn fff 1,0 10 ff

Example:

nnn rrf 2211 Has solution:

2

511

r

2

512

rCharacteristic roots:


5

1

21

1102

rr

frf

5

1

21

2011

rr

rff


nn

nnn rrf

2

51

5

1

2

51

5

1

2211


Degree recurrence relation:

has unique solution

nkk

nnn rrra 2211

where are solutions to the characteristic equation,and are constants thatdepend on initial conditions

krrr ,,, 21

k ,,, 21

knknnn acacaca 2211

k


Example: 321 6116 nnnn aaaa

15,5,2 210 aaa

Solution:

Characteristic equation: 06116 23 rrr

Roots: 3,2,1 321 rrr

nnnnnnn rrra 321 321332211


nnnna 321 321

23

22

212

13

12

111

03

02

010

32115

3215

3212

a

a

a

2,1,1 321

nnnnnna 3221322111

Final solution:

Recurrence Relations for Divide and Conquer Algorithms


Typical divide and conquer algorithm:

•Input of size n

•Divide into sub-problems each of size bn /

a

)(ng•Combine sub-problems with cost


)()/()( ngbnfanf

Divide an conquer recurrence relation:

Cost of subproblemof size bn /


Examples:

Binary search: 2)2/()( nfnf

Merge Sort: nnfnf )2/(2)(

Fast Matrix Multiplication (Stassen’s Alg.):

4/15)2/(7)( 2nnfnf


)()/()( ngbnfanf

Zkbnkbbn k ,,,0,1,

1

0

)/()1()(k

j

jjk bngafanf

Theorem: if

then


Proof:

)()/()/(

)())/()/((

)()/()(

22

2

ngbngabnfa

ngbngbnfaa

ngbnfanf


)()/()/()/(

)()/())/()/((

)()/()/(

)())/()/((

)()/()(

2233

232

22

2

ngbngabngabnfa

ngbngabngbnfaa

ngbngabnfa

ngbngbnfaa

ngbnfanf


1

0

1

0

2233

232

22

2

)/()1(

)/()/(

)()/()/()/(

)()/())/()/((

)()/()/(

)())/()/((

)()/()(

k

j

jjk

k

j

jjkk

bngafa

bngabnfa

ngbngabngabnfa

ngbngabngbnfaa

ngbngabnfa

ngbngbnfaa

ngbnfanf

End of Proof


cbnfanf )/()(Theorem: if

Rcaca

Zkbnkbbn k

,,0,1

,,,0,1,

)(nf)( log abnO

)(log nO b

1a

1athen


Proof:

1

0

1

0

)1(

)1(

)/()(

k

j

jk

k

j

jk

acfa

cafa

cbnfanf

From previous theorem


)(log

log)1(

)1(

)1()(1

0

nO

ncfa

ckfa

acfanf

b

bk

k

k

j

jk

1aCase:

kbn nk blog


)(

11)1(

1

1)1(

)1()(

log

2log

1

1

0

a

a

k

kk

k

j

jk

b

b

nO

CnC

a

c

a

cfa

a

acfa

acfanf

1aCase:

End of Proof


Example:

Binary search: 2)2/()( nfnf

2,2,1 cba

)(log)(log)(log)( 2 nOnOnOnf b


dcnbnfanf )/()(Master Theorem:

if

Rdcadca

Zkbnkbbn k

,,,0,0,1

,,,1,1,

)(nf )log( nnO d

)( dnO

dba

dba

then

)( log abnO dba


Example:

Merge Sort: nnfnf )2/(2)(

1,1,2,2 dcba

dba 122

)log()log()( nnOnnOnf d

Generating Functions


17321 eee

Find number of solutions for:

3

20

3

1317

0,, 321 eee

Answer:


Alternative solution

17321 eee

choices for 1e choices for 2e choices for 3e)1)(1)(1( 173217321732 xxxxxxxxxxxx



17321 eee

5151

1717

23

2210 xaxaxaxaxaa

17a is the total numberof solutions to equation

)1)(1)(1( 173217321732 xxxxxxxxxxxx


Another problem: 17321 eee

52 1 e

63 2 e

Find total number of solutions which satisfy:

74 3 e



17321 eee

choices for 1e choices for 2e choices for 3e

))()(( 765465435432 xxxxxxxxxxxx

52 1 e 63 2 e 74 3 e



17321 eee

))()(( 765465435432 xxxxxxxxxxxx

52 1 e 63 2 e 74 3 e

1818

1717

1010

99 xaxaxaxa

17a is the total numberof solutions to equation


Generating function:

0

33

2210

)(

k

kk

kk

xa

xaxaxaxaaxG

generating function for sequence

,,,,,, 3210 kaaaaa


Solve recurrence relation

13 kk aa 20 a

Generating functions can also be used tosolve recurrence relations

Example:


13 kk aa

Let be the generating function for sequence ,,,,, 3210 kaaaaa

)(xG

20 a

0

)(k

kk xaxG


11

0

1

0

)(

k

kk

k

kk

k

kk

xa

xa

xaxxGx


2

)3(

)3(

3

3)(3)(

0

110

110

11

10

11

0

a

xaaa

xaxaa

xaxaa

xaxaxxGxG

k

kkk

k

kk

kk

k

kk

k

kk

k

kk

k

kk

13 kk aa 20 a

0


2)(3)( xxGxG

xxG

31

2)(


xk

k

xx

1

1

0

xxG

31

12)(

0

32)(k

kk xxG

0

32)(k

kk xxG


0

32)(k

kk xxG

0

)(k

kk xaxG

kka 32

13 kk aa 20 a

Solution to recurrence relation

Inclusion-Exclusion


A BBA

|||||||| BABABA


ABA

||

||||||

||||||||

CBA

CBCABA

CBACBA

B

C

CA CB

CBA


||)1(

||

||

||||

211

1

1

121

nn

nkjikji

njiji

niin

AAA

AAA

AA

AAAA

Principle of Inclusion-Exclusion:


Proof:

We want to prove that:

an arbitrary element is counted exactly one time in the expression of the theorem

x


Suppose is a member of exactly sets: rx

riii AAAx 21

rjjjjj

r

nkjikji

njiji

nii

r

rAAA

AAA

AA

A

1

211

1

1

1

1

||)1(

||

||

||

Then is counted in the terms:x

is counted times


ni

iA1

||

)1,(rCr

In sum:

x

(since belongs exactly to sets)x r

is counted times


)2,(rC

In sum:

x


nji

ji AA1

||

is counted times


)3,(rC

In sum:

x


nkji

kji AAA1

||

is counted times


),(1 rrC

In sum:

x


rjj

jjj

r

rAAA

1

211

||


),()1()3,()2,()1,( 1 rrCrCrCrC r

Thus, in the expression of the theorem is counted so many times:x

ni

iA1

||

nji

ji AA1

||

nkji

kji AAA1

||

rjj

jjj

r

rAAA

1

211

||


),()1()2,()1,()0,()11(0 rrCrCrCrC rr

),()1()2,()1,()0,(1 1 rrCrCrCrC r

End of Proof

From binomial expansion we have that:

Thus, is counted exactly one timex


Example: Find the number of primes between 1…100

If a number is composite and between1…100 then it must be divided by a primewhich is at most :10010

2, 3, 5, 7


:the set of primes between 1…100P

2N :composites between 1…100 divided by 2


:the set of composites between 1…100N




From the principle of inclusion-exclusion:

||

||||||||

||||||||||||

||||||||||

7532

753752732532

757353725232

5532

NNNN

NNNNNNNNNNNN

NNNNNNNNNNNN

NNNNN


78

001232467101614203350

7532

100

753

100

752

100

732

100

532

100

75

100

73

100

53

100

72

100

52

100

32

100

7

100

5

100

3

100

2

100||

N

||

||||||||

||||||||||||

||||||||||

7532

753752732532

757353725232

5532

NNNN

NNNNNNNNNNNN

NNNNNNNNNNNN

NNNNN


Number of primes between 1…100:

21

7899

||99||

NP

advanced counting techniques csc-2259 discrete structures konstantin busch - lsu1

Documents

recurrence relation

degree recurrence relation

sequencerecurrence relation

characteristic equationroots

initial values

constants thatdepend

final solution

sequence solution