chapter 14 gaussian elimination (iv) bunch-kaufman diagonal pivoting (partial pivoting)

43
Chapter 14 Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting) Reference: [1] James R. Bunch and Linda Kaufman, Some Stable Met hods for Calculating Inertia and Solving Symmetr ic Linear Systems, Mathematics of Computation, volume 31, number 137, January 1977, page 163-179 [2] Cleve Ashcraft, Roger G. Grimes, and Joh n G. Lewis, Accurate Speaker: Lung-Sheng Chien

Upload: elke

Post on 13-Jan-2016

67 views

Category:

Documents


4 download

DESCRIPTION

Chapter 14 Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting). Speaker: Lung-Sheng Chien. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Chapter 14 Gaussian Elimination (IV)Bunch-Kaufman diagonal pivoting

(partial pivoting)

Reference: [1] James R. Bunch and Linda Kaufman, Some Stable Methods for Calculating Inertia and Solving Symmetric Linear Systems, Mathematics of Computation, volume 31, number 137, January 1977, page 163-179

[2] Cleve Ashcraft, Roger G. Grimes, and John G. Lewis, Accurate Symmetric Indefinite Linear Equation Solvers, SIAM J. MATRIX ANAL. APPL. Vol. 20, No. 2, 1998, pp. 513-561

Speaker: Lung-Sheng Chien

Page 2: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

OutLine

• Review Bunch-Parlett diagonal pivoting- expensive pivot-selection strategy- pivot induces swapping row/column, not efficient

• Partial pivoting: basic idea

• Implementation of partial pivoting

• Example

Page 3: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Case 1: 1 0

01 ,max ij rqi j n

A A

11max iii nA A

11

21 22

1 2

T

n n nn

a

a aA A

a a a

, 2 : 1,1, 1:P n Define permutation matrix and do symmetric permutation

Recall Criterion for pivot strategy (complete pivoting) [1]

expensive

11a

1a

a

1

2

3

2 22

, 1

a

a a

a

,1 2,1 1,1

1,

,n

a a a

a

a

11

1,1

,1n

a

a

a

A A

1

Page 4: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Then do TLDL on A with 1 1 pivot, 1 2

1TT

EE cA L

cE Ic B A

,2 2,

, 1 1,

,1 1,,1

1, 1,

, ,

, 2 : 1

1: ,

n n

a a

a aA

a ac a

a aA n

a a

2 12 : , 2 : , TB A n n A B cE c

, E a

Recall Criterion for pivot strategy (complete pivoting) [2]

1,

2,

1,

,

1,

,n

a

a

a

a

a

a

1 1 01 max maxj

jj j

accE cE a

a a

1 0 0max 1, max 1,L cE

Therefore

where det E a

Observation: if we define ,max jj

A

, then 0max 1,L

maximum of off-diagonal elements in column

Page 5: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

2 1 2 : , 2 : 2 : ,1 2 : ,1TTA B cE c A n n L n A n

Recall Criterion for pivot strategy (complete pivoting) [3]

21, 1 1,1 1,1ij i j i jA A L A 2

0A L

Question: To compute 01 ,max ij rqi j n

A A

is expensive, if we don’t want to compute it,

Can we have other choice such that controllability of

L and 2A

Observation: It suffices to change pivoting condition 1 0 to 1 , then

1 1max 1, max 1,L

20 0

1 11A

and

The same as that we do by using pivoting criterion (complete pivoting)

1 0

However computing

,max jj

A

is cheaper than

0

1 ,max iji j n

A

since

,max jj

A O n

but 2

01 ,max / 2iji j n

A O n

holds ?

Page 6: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Recall Criterion for pivot strategy (complete pivoting) [4]

Technical problem: computing ,max jj

A

does not attain

optimal.

11

2,1

1,1

a

a

a

,1 ,2 , 1a a a

1,

,n

a

a

a

,max maxjj

A

,1 , 1 ,a a

1,

,n

a

a

11a

21a

1na

22a

32a

2na

1, 1n na

, 1n na

nna

column-major based

1col 2col col n

NOT contiguous memory block, slowly.

How to improve this ?

Contiguous memory block, fast

Page 7: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

1

Recall Criterion for pivot strategy (complete pivoting) [5]

Second, if

, we need to interchange row/column

1

11a

1a

a

1

2

3

3 relates to read/write on row vector

which is NOT contiguous, slowly.

Question: why we persist on computing 11max iii nA A

, why not choose first column?

say

11E a and

,11

max jj

A

maximum of off-diagonal elements in column 1

Again, if , then we also have

1 1max 1, max 1,L

20 0

1 11A

and

Page 8: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

OutLine

• Review Bunch-Parlett diagonal pivoting• Partial pivoting: basic idea

- select pivot by at most computation of two columns- potential risk of growth rate of lower triangle matrix

• Implementation of partial pivoting

• Example

Page 9: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [1] TE c

Ac B

, choose 11E a and

1 max c maximum of off-diagonal elements in column 1

11 1a If

is determined later , then

1 2

1 TE

A LcE I A

1 1max 1,L

2 10 0

11A

where 01 ,max iji j n

A

Pros (優點 ): compute

01 ,max iji j n

A

and

11max iii nA

Cons (缺點 ): It is difficult to satisfy

11 1a , we need to deal with the case 11a

carefully such that controllability of L and 2A

holds.

Comparison with PA LU

we CANNOT move 1rA to 11A

: suppose 11 1a for 1 1ra

due to symmetric permutation,

It is impossible to keep stability by just computing one column as we do in PA LU

Page 10: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [2]

Question: how to deal with the case 11 1a

<ans> from procedure of complete pivoting, we need 2x2 pivot.

for 1 1ra

11

21 22

31 32

1,1 1,2 1, 1r r r r

a

a a

a a

a a a

,1 ,2 , 1

1,2

,1 ,2 , 1

r r r r

r

n n n r

a a a

a

a a a

1,

,

rr

r r

n r

a

a

a

1

2

43

11

1

31 3 33

1,1 1, 1, 1

r rr

r

r r r r r

a

a a

a a a

a a a

2,1 2, 2, 1

1,

,1 , , 1

r r

r r

n n r n r

a a a

a

a a a

22

1,2

,2

r

n

a

a

a

2r

ATA PAP

define permutation

1, ,3 : 1,2, 1:P r r r n , change

1ra to

21a

Page 11: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [3]

3

TT

EE cA L L

c B A

3

TT

EE cA L L

c B A

11 21

21 22

,a a

Ea a

11 1

1

,r

r rr

a aE

a a

31 32

1,1 1,2

,1 ,2

1,1 1,2

,1 ,2

r r

r r

r r

n n

a a

a a

a ac

a a

a a

31 3

1,1 1,

2,1 2,

1,1 1,

,1 ,

r

r r r

r

r r r

n n r

a a

a a

a ac

a a

a a

11 2|

IIL

l l IcE I

22 211 2 ,1 ,2

21 11

1

deti i i i

a al l a a

a aE

3 i n

11 2|

IIL

l l IcE I

3 : 1,2, 1:i r r n

11 ,1 ,

1 11

1

detrr r

i ir i i rr

a al l a a

a aE

2r

we may say

1,i r

Page 12: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [4]

33

3 4

1,3 1,

,3 ,

r r rr

r r r

n n r nn

a

a a aB

a a

a a a

33

23 24 22

1,3 1,2

,3 ,2

r r

n n nn

a

a a aB

a a

a a a

3 1 TA B cE c

,13 1,1 ,2

,2

2,122 212, 2 2,1 2,2

2,221 11

1

det

jij ij i i

j

j

i j i ij

cA B c c E

c

aa aa a a

aa aE

2,13 12, 2 2,1 2,

2,1 11

1

detjrr r

ij i j i i rj rr

aa aA a a a

aa aE

2 1, and 2 1,i r j r

3 1 TA B cE c 2r

, 2, 2i j i jB a

Page 13: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [5]

Under the case 11 1a , we want to find a condition for rra such that

L

is bounded.

11 ,1 ,

1 11

1

detrr r

i ir i i rr

a al l a a

a aE

1,i r 1 ,1 , 1

1

deti i rr i r rl a a a aE

,1 1 , 11

1

detir i r i rl a a a aE

,1 , 1 1 , 1

1det det

i rr i r r rr i r

i

a a a a a al

E E

,1 1 , 11 1 1 , 11

det det

i r i r i r

ir

a a a a a al

E E

Lower bound of det E

, ,, i r r ra aupper bound of

?L

,maxr j rj ra

maximum of off-diagonal elements in column rdefine

1 , 1, 1maxr j r rj ra a

2 If rr ra , then , ,, i r r ra aupper bound of holds.

Question: how to achieve lower bound of det E

Page 14: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

2 211 1 11 12 2

11 1 11 1 2 21 11 11 1

if det

if

rr rrrr r rr

rr rr

a a a aE a a a a a

a a a a

Partial pivoting: basic idea [6]

Since if we add assumption

rr ra

From assumption 11 1a , to keep inequality, the natural choice is 2

1 11 rra a

, then 11 11rr ra a a

If we add third assumption

3 211 1ra , then 2 2

11 11 1rr ra a a and then 2 2 21 11 1det 1rrE a a

Lower bound of det E holds.

To sum up, in order to control L

under 2x2 pivoting, we need three conditions

11 1a rr ra 211 1ra (1) (2) (3)

2 2 2

1 1 , 11 1 11 1 12 2 2 2

1 1

1

11 1det

i r rir

a a al

E

1 , 1 1 11 2 2

11

1

11det

rr i r r r ri

a al

E

1

1

1rL

1r

and moreover

Page 15: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [7]

Question: does the three condition keep controllability of 3A

2 1, and 2 1,i r j r

2,13 12, 2 2,1 2,

2,1 11

2,12, 2 2,1 2, 1 2,1 ,1 2, 11

2,

2,1 2, 1 2,1 2,12, 2

1

det

1

det

jrr rij i j i i r

j rr

ji j i rr i r r i r i r

j r

i rr i r r j i ri j

aa aA a a a

aa aE

aa a a a a a a a a

aE

a a a a a a aa

,1 2, 11 2,

deti r j ra a a

E

1 1 1 1 1 1130

det

rr r r ra aA

E

11 1a

rr ra

211 1ra

(1)

(2)

(3)

21 1 1 1 113

0det

r r r raA

E

2 2 21 1 13

0

1

det

r rA

E

23 1

0 0 02 21

2 1 2 21

1 11r rA

Page 16: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [8]

11 1a 11 1a

1x1 pivot

?

rr ra rr ra

?2

11 1ra

211 1ra

2x2 pivot

11 1a 11 1a

1x1 pivot

?

rr ra

rr ra

?

211 1ra 2

11 1ra

2x2 pivot

So far, we deal with two cases in the following decision-making tree

Question: how to deal with the other two cases and what is the order of decision-making, briefly speaking which tree is correct, left or right?

4

1

2

3

Page 17: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [9]

1

,TE c

Ac B

11E a and

1 1 maxra c maximum of off-diagonal elements in column 1

11 1a

21

1

1

2 : ,1 ,r

n

a

c A n a

a

11 1 11

1 111 11 11 11

1 1 max maxj

jj j

accE cE a

a a a a

1 1

max 1,L cE

1 2

1,T

EA L

cE I A

2 1 2 : , 2 : 2 : ,1 2 : ,1TTA B cE c A n n L n A n

21, 1 1,1 1,1ij i j i jA A L A

22

31 22

1

2 : , 2 :

n nn

a

a aB A n n

a a

20 1 0

11A L

01 ,max iji j n

A

maximum elements of A , we do not compute it explicitly.

1 1 pivot

Page 18: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [10]

,maxr j rj ra

maximum of off-diagonal elements in column rdefine

11

2,1

1,1r

a

a

a

,1 ,2 , 1r r r ra a a

1,

,

rr

r r

n r

a

a

a

,max maxr j rj r

A

,1 , 1 ,r r ra a 1,

,

r r

n r

a

a

1 1r ra

NOT contiguous memory block, slowly.

Contiguous memory block, fast

Remark: when case 1 is not satisfied, we must compute r for remaining decision-making.

Remark: In general, r may be more larger than

1 , that is why case 2 holds

2 211 1ra

Page 19: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [11]

2 211 1ra under 11 1a 1 1 pivot

21

1

1

2 : ,1 ,r

n

a

c A n a

a

11 1 1 1

111 11 11 11 1

1 max j r r

jr

accE cE

a a a a

1

1

1max 1, rL cE

1 2

1,T

EA L

cE I A

2 1 2 : , 2 : 2 : ,1 2 : ,1TTA B cE c A n n L n A n

21, 1 1,1 1,1ij i j i jA A L A

22

31 22

1

2 : , 2 :

n nn

a

a aB A n n

a a

20 1 0 1 0

1

1 11rA L

Remark: bound on lower triangle matrix is NOT good since it is proportional to1

r

however, the bound of reduced matrix 2A is good.

Page 20: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [12]

3 rr ra 1 1 pivotunder 211 1 11 1, ra a

Clearly, this is the same as case 1 if we interchange row/column 1 r

define permutation

, 2 : 1, 2, 1:P r r r n , change

rra to

11a

11a

1ra

rra

1

3

2

2 22

, 1

rr

r

r r

a

a a

a

,1 2,1 1,1

1,

,

r r

r r

n r

a a a

a

a

11

1,1

,1

r

n

a

a

a

TA PAP

1 r

A

Page 21: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [13]

Then do TLDL on A with 1 1 pivot, 1 2

1TT

EE cA L

cE Ic B A

,2 2,

, 1 1,

,1 1,,1

1, 1,

, ,

, 2 : 1

1: ,

r r

r r r r

r rr

r r r r

n r n r

a a

a aA r r

a ac a

a aA r n r

a a

2 12 : , 2 : , TB A n n A B cE c

, rrE a

1,

2,

1,

,

1,

,

r

r

r r

r r

r r

n r

a

a

a

a

a

a

1 1 1 1 max maxjr r

jrj r j r

rr rr rr rr

accE cE a

a a a a

2 1 2 : , 2 : 2 : ,1 2 : ,1TTA B cE c A n n L n A n

20 0 0

1 11rA L c

1 1max 1,L cE

Page 22: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [14]

4 rr ra 2 2 pivotunder 211 1 11 1, ra a

11

21 22

31 32

1,1 1,2 1, 1r r r r

a

a a

a a

a a a

,1 ,2 , 1

1,2

,1 ,2 , 1

r r r r

r

n n n r

a a a

a

a a a

1,

,

rr

r r

n r

a

a

a

11

1

31 3 33

1,1 1, 1, 1

r rr

r

r r r r r

a

a a

a a a

a a a

2,1 2, 2, 1

1,

,1 , , 1

r r

r r

n n r n r

a a a

a

a a a

22

1,2

,2

r

n

a

a

a

2r

ATA PAP

define permutation

1, ,3 : 1,2, 1:P r r r n , change

1ra to

21a

1

2

43

Page 23: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Then do TLDL on A with 2 2 pivot, 1 3

1TT

EE cA L

cE Ic B A

Partial pivoting: basic idea [15]

11 1

1

,r

r rr

a aE

a a

2 2 2

11 1 1det 1rr rE a a a

1,i r

1 ,1 , 1

1

deti i rr i r rl a a a aE

,1 1 , 11

1

detir i r i rl a a a aE

1

1irl

11

1

1r

il

1

1

1rL

2 1, and 2 1,i r j r 2,1 2, 1 2,1 2,1 ,1 2, 11 2,32, 2 det

i rr i r r j i r i r j rij i j

a a a a a a a a a aA a

E

30

21

1A

Remark: bound on lower triangle matrix is NOT good since it is proportional to1

r

however, the bound of reduced matrix 3A is good.

2

1 2| r n

IL

l l I

Page 24: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Partial pivoting: basic idea [16]

To sum up,

1 11 1a

2 211 1ra 1 1 pivot

1 1 pivot

3 rr ra 1 1 pivot

4 rr ra 2 2 pivot

1L

2

0

11A

1

1rL

20

11A

1L

2

0

11A

1

1

1rL

30

21

1A

1, ,3 : 1,2, 1:P r r r n

, 2 : 1, 2, 1:P r r r n

1 2,3

1TT

EE cA L

cE Ic B A

1 1 maxra c maximum of off-diagonal elements in column 1

,maxr j rj ra

maximum of off-diagonal elements in column r

Page 25: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

worst case analysis : choose 0 1 such that reduced matrix

growth rate of 1 1 pivot + 1 1 pivot growth rate of 2 2 pivot

or equivalently growth rate of 1 1 pivot growth rate of 2 2 pivot

Define 1 2max 1 , 1

1B

00 1

1 17min 2.5616 2.57

2B B

0

1 170.6404

8

where satisfies 0 0

1 21 1

1

Partial pivoting: basic idea [17]

satisfies 2 3,A A

20

11A

30

21

1A

Remark: this optimal value is the same as we have in complete pivoting.

however, the bound of lower triangle matrix is NOT good.

Page 26: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Complete pivoting versus partial pivoting

complete pivoting

0 0

1 1max , max 1.56,2.78 2.78

1L

1 10 0 02.57n nD B

partial pivoting

T TPAP LDL

0 01 1

1 1max max , 2.78 max

1

k kr rk kk k

L

1 10 0 02.57n nD B

, growth rate of reduce matrix kA contributes to final block diagonal matrix D

Reference: [1] James R. Bunch and Linda Kaufman, Some Stable Methods for Calculating Inertia and Solving Symmetric Linear Systems, Mathematics of Computation, volume 31, number 137, January 1977, page 163-179

[2] Cleve Ashcraft, Roger G. Grimes, and John G. Lewis, Accurate Symmetric Indefinite Linear Equation Solvers, SIAM J. MATRIX ANAL. APPL. Vol. 20, No. 2, 1998, pp. 513-561

Remark: Bunch in [1] only deal with controllability of reduced matrix kA

However Ashcraft in [2] point out the importance of growth rate of L

Page 27: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

OutLine

• Review Bunch-Parlett diagonal pivoting

• Partial pivoting: basic idea

• Implementation of partial pivoting

• Example

Page 28: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

1k n

we have compute 1 1 1 1T T

k k k k kP A P L A L

1

, ,

,

0 0 0

0

0

0

0 0

skk kk k n k

k kn k nn

D

DA

a a

a a

update original matrix A

1k W OL

M I

1k

1k combines all lower triangle matrix and store in L

Algorithm ( PAP’ = LDL’, partial pivot ) [1]

Given symmetric indefinite matrix n nA R , construct initial lower triangle matrix L I

while

use permutation vector P to record permutation matrix kP

let 1 :A A , 0 ,L I 0 1,2,3, ,P n and

1k

:1 1 or 2 2iD , where

,pivot zero n1 17

0.64048

Page 29: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Algorithm ( PAP’ = LDL’, partial pivot ) [2]

compute1 1 1 11

max , j rk j n

A A r k

Case 1: 1kka 1 1 pivot no interchange

1 1

1

/ /

k kT

k k k kTkk kk

k kTkk kk

D DI

A a c a L

c B c a I B cc a

do 1x1 pivot :

1: , / kkkL k n k c a

1: , 1: / kTkkA k n k n cc a

2

then 1 1 1T

k k k k kA L L A L L

3 1k k 1pivot k and

When case 1 is not satisfied, we compute

,maxr j rj ra

maximum of off-diagonal elements in column r

under lower triangular part of matrix A is used.

Page 30: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Algorithm ( PAP’ = LDL’, partial pivot ) [3]

2,

1,

kk

k

r k

a

a

a

, , 2 , 1r k r k r ra a a

1,

,

rr

r r

n r

a

a

a

max max , : 1 ,max 1: ,r A r k r A r n r 4

Case 2: 2

1kk ra 1 1 pivot no interchange

1 1

1

/ /

k kT

k k k kTkk kk

k kTkk kk

D DI

A a c a L

c B c a I B cc a

do 1x1 pivot :

1: , / kkkL k n k c a

1: , 1: / kTkkA k n k n cc a

5

then 1 1 1T T

k k k k kA L L A L L

6 1k k 1pivot k and The same as code in case 1

1 0?r

Page 31: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Algorithm ( PAP’ = LDL’, partial pivot ) [4]

Case 3: rr ra 1 1 pivot do interchange

define permutation

1: 1, , 1: 1, , 1:P k r k r k r n to do symmetric permutation

kka

rka

rra

k k Tk kA P A PTo compute , we only update lower triangle of kA

7 P k P r

1

2

3

1 , ,A k k A r r

2 1: , 1: ,A r n k A r n r

3 1: 1, , 1: 1A k r k A r k r

8

1k

k k Tkk

D

A a c

c B

then , :kkk rra a

Page 32: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Algorithm ( PAP’ = LDL’, partial pivot ) [5]

1 1

1

/ /

k kT

k k k kTkk kk

k kTkk kk

D DI

A a c a L

c B c a I B cc a

do 1x1 pivot :

1

1

1

,1 ,2 , 1

,1 ,2 , 1

k k k k

r r r k

l l l

l l l

1

1

1

1kL 1 1k k Tk kL P L P To compute

,1: 1 ,1: 1L k k L r k

We only update lower triangle matrix L9

then

1 1T T

k k k k kP A P L A L

1: , / kkkL k n k c a

1: , 1: / kTkkA k n k n cc a

10

then 1 1 1T T T

k k k k k k kP A P L L A L L

11 1k k 1pivot k and

Page 33: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Algorithm ( PAP’ = LDL’, partial pivot ) [6]

Case 4: rr ra 2 2 pivot do interchange

1, 1, 1

2, 2, 1

1, 1, 1 1, 1

kk

k k k k

k k k k

r k r k r r

a

a a

a a

a a a

, , 1 , 1

1, 1

, , 1 , 1

r k r k r r

r k

n k n k n r

a a a

a

a a a

1,

,

rr

r r

n r

a

a

a

define permutation

1: , , 2 : 1, 1, 1:P k r k r k r n , change

rka to

1,k ka

1kD

0

0

0

0

0

0

3

1

2

4

1 1, 1 ,A k k A r r

2 1: , 1 1: ,A r n k A r n r

3 1, ,A k k A r k

2 : 1: 1 , 2 : 1A k r k A r k r 4

1

,k

k T

D

A E c

c B

then

,

,

k kkk r k

k kr k rr

a aE

a a

12 1P k P r

k k Tk kA P A PTo compute , we only update lower triangle of kA

do interchange row/col 1k r 13

Page 34: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Algorithm ( PAP’ = LDL’, partial pivot ) [7]

then

1 1T T

k k k k kP A P L A L

1,1: 1 ,1: 1L k k L r k

14 do interchange row 1k r

,1 ,2

1

1

1k kl l

1,1 1, 1

,1 , 1

0

0

0

k k k

r r k

l l

l l

1

1

1

1 1

1 1

k k

Tk kT

T

D I D

A E c I E L

c B cE I B cE c

do 2x2 pivot :

12 : , : 1L k n k k cE

12 : , 2 : TA k n k n cE c

15

then 1 2 1 T T T

k k k k k k kP A P L L A L L

16 2k k 2pivot k and

endwhile

Page 35: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Question: recursion implementation

• Initialization- check algorithm holds for k=1

• Recursion formula- check algorithm holds for k or k+1, if k-1 is true

• Termination condition- check algorithm holds for k=n-1

• Breakdown of algorithm- check which condition PAP’=LDL’ fails

• No extension: algorithm works only for square, symmetric indefinite matrix.

normal

abnormal

Extension of algorithm

Page 36: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

OutLine

• Review Bunch-Parlett diagonal pivoting

• Partial pivoting: basic idea

• Implementation of partial pivoting

• Example

Page 37: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

6 12 3

12 -8 -13

3 -13 -7

-6

4

1

-6 4 1 6

1A A 1 21max 2 : 4,1 12A A max 12, 13,4 13r

0.6404

,since k +1 = r , we don’t need permutation

Example (partial pivoting) [1]

11 16 7.6847a 1

2 211 178 92.2159ra

3 8 8.325rr ra

invalid

invalid

invalid

4 rr ra 2 2 pivot

2r

12 1P k P r

1 1, 1 ,A k k A r r 2 1: , 1 1: ,A r n k A r n r

3 1, ,A k k A r k 2 : 1: 1 , 2 : 1A k r k A r k r 4

do interchange row/col 1k r 13 , since k +1 = r , we don’t need permutation

iteration 1: 1k

Page 38: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Example (partial pivoting) [2]

1,1: 1 ,1: 1L k k L r k

14 do interchange row 1k r , since k +1 = r , we don’t need permutation

1 1

1 1

k k

Tk kT

T

D I D

A E c I E L

c B cE I B cE c

do 2x2 pivot :15

6 12 3

12 -8 -13

3 -13 -7

-6

4

1

-6 4 1 6

1A

1

1

-0.6875 0.5938 1

0 -0.5 1

6 12

12 -8

2.7813 -5.5

-5.5 8

1L 3A

1T

L

16 2k k 2pivot k and

3k pivot 2 0 0 0

Page 39: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Example (partial pivoting) [3]

iteration 2 : 3k

6 12

12 -8

2.7813 -5.5

-5.5 8

3A 1 43max 3: 4,3 5.5A A max 5.5 5.5r

33 12.7813 3.5221a 1

2 211 115.2971 19.3717ra

3 8 3.5221rr ra

invalid

invalid

1 1 pivot

4r

7 P k P r 3 41,2,3,4 1,2,4,3P P

1 , ,A k k A r r

2 1: , 1: ,A r n k A r n r

3 1: 1, , 1: 1A k r k A r k r

8

k k Tk kA P A PCompute

6 12

12 -8

8 -5.5

-5.5 2.7813

3A

Page 40: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Example (partial pivoting) [4]

1 1k k Tk kL P L P Compute ,1: 1 ,1: 1L k k L r k 9

1

1

-0.6875 0.5938 1

0 -0.5 1

1L

1

1

0 -0.5 1

-0.6875 0.5938 1

1L

3 4

1 1

1

/ /

k kT

k k k kTkk kk

k kTkk kk

D DI

A a c a L

c B c a I B cc a

do 1x1 pivot :10

6 12

12 -8

8 -5.5

-5.5 2.7813

3A

6 12

12 -8

8

-1

4A

1

1

1

-0.6875 1

3L

3T

L

Page 41: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Example (partial pivoting) [5]

T TPAP LDL

6 12

12 -8

8

-1

4D A

1

1

0 -0.5 1

-0.6875 0.5938 -0.6875 1

1 3L L L

pivot 2 0 1 1

11 1k k 1pivot k and

4k pivot 2 0 1 0

iteration 3 : 4k issue proper termination condition.

To sum up

P 1 2 4 3

Page 42: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Example: complete pivot versus partial pivot

Complete pivot

Partial pivot6 12 3

12 -8 -13

3 -13 -7

-6

4

1

-6 4 1 6

A

1 17

8

pivot 2 0 1 1

P 2 3 4 1

L

1

0.1327

0.3982

1

-0.3984

-1.1681

1

-1.0967 1

D

-8

-13

-7

5.8584

-2.3202

6

12 -8

8

-1

D

1

1

0 -0.5 1

-0.6875 0.5938 -0.6875 1

L

pivot 2 0 1 1P 1 2 4 3

Page 43: Chapter 14  Gaussian Elimination (IV) Bunch-Kaufman diagonal pivoting (partial pivoting)

Exercise

11 1a 11 1a

1x1 pivot

?

rr ra rr ra

?2

11 1ra

211 1ra

2x2 pivot

11 1a 11 1a

1x1 pivot

?

rr ra

rr ra

?

211 1ra 2

11 1ra

2x2 pivot

How about flow-chart of left figure?

4

1

2

3

Bunch-Kaufman proposed flow chart