CS 312: Algorithm Analysis

CS 312: Algorithm AnalysisLecture #13: Strassens Algorithm for Matrix Multiplication; Convolution

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick1AnnouncementsHomework #8.5Due Now

Project #3You sent Dr. Ringger your chosen theorem, right?ObjectivesApply divide and conquer to matrix multiplicationAnalyze using the Master TheoremIntroduce you to the convolution operation.See an algorithm for convolution, inspired by polynomial multiplicationPart 1: Matrix MultiplicationMatrix MultiplicationWho cares?

In this setting,Scalar operations +, , and x are elementary opsAssume fixed precisionMatrix operations +, -, and x are NOT elementary opsWhat about the commutative property of matrix multiplication?

Nave Algorithm1 23 45 67 8x=Nave Algorithm1 23 45 67 8x(1x5 + 2x7) (1x6 + 2x8)(3x5 + 4x7) (3x6 + 4x8)=19 2243 50=Better solution:obtained in a manner similar to the Karatsuba Divide and Conquer algorithm for scalar multiplicationStrassens Algorithma bc de fg hx=m1 = (c + d - a) x (h f + e)m2 = (a x e)m3 = (b x g)m4 = (a - c) x (h - f)m5 = (c + d) x (f - e)m6 = (b - c + a - d) x hm7 = d x (e + h f - g)m2+m3 m1+m2+m5+m6m1+m2+m4-m7 m1+m2+m4+m5Note: this solution was obtained in a manner similar to the Karatsuba Divide and Conquer algorithm for scalar multiplication

Strassens on our Example1 23 45 67 8x=m1 = (3+4-1) x (8-6+5) = 6 x 7 = 42m2 = (1 x 5)= 5m3 = (2 x 7)= 14m4 = (1-3) x (8-6) = -2 x 2 = -4m5 = (3+4) x (6-5) = 7 x 1 = 7m6 = (2-3 + 1-4) x 8 = -4 x 8 = -32m7 = 4 x (5+8-6-7) = 4x0 = 0m2+m3 m1+m2+m5+m6m1+m2+m4-m7 m1+m2+m4+m55 + 14 42+5+7+(-32)42+5+(-4)-0 42+5+(-4)+7=19 2243 50=9Recursive ApplicationxRecursive ApplicationxRecursive ApplicationA BC DE FG HxAgain: Strassens Algorithmx=M1 = (C + D - A) x (H F + E)M2 = (A x E)M3 = (B x G)M4 = (A - C) x (H - F)M5 = (C + D) x (F - E)M6 = (B - C + A - D) x HM7 = D x (E + H F - G)M2+M3 M1+M2+M5+M6M1+M2+M4-M7 M1+M2+M4+M5A BC DE FG HWhats the efficiency? Efficiency Analysis

a= number of subinstancesn= original instance sizen/b= size of subinstancesd= polynomial order of g(n) where g(n)is cost of doing the divide (& combine)Efficiency Analysis

15Can we do better?NoteIs it correct?What does it mean for Strassens to be correct?

How can you prove it in general?Correctnessa bc de fg hx=m1 = (c + d - a) x (h f + e)m2 = (a x e)m3 = (b x g)m4 = (a - c) x (h - f)m5 = (c + d) x (f - e)m6 = (b - c + a - d) x hm7 = d x (e + h f - g)m2+m3 m1+m2+m5+m6m1+m2+m4-m7 m1+m2+m4+m5Correctnessa bc de fg hx=m1 = (c + d - a) x (h f + e)m2 = (a x e)m3 = (b x g)m4 = (a - c) x (h - f)m5 = (c + d) x (f - e)m6 = (b - c + a - d) x hm7 = d x (e + h f - g)m2+m3 m1+m2+m5+m6m1+m2+m4-m7 m1+m2+m4+m5Simplification and D/CSimplification(fewer than all branches)D/CBinary SearchQuicksortMergesortStrassensKaratsuba(int mult.)SelectionSimplification and D/CSimplificationD/CBinary SearchQuicksortMergesortStrassensKaratsuba(int mult.)SelectionPart 2: ConvolutionPolynomial Multiplication

is a polynomial of degree 2d.24Polynomial Multiplication

is a polynomial of degree 2d.25Polynomial Multiplication is Convolution

QuestionsIs it Correct?How long does it take (assuming multiplication and additional are elementary ops)?Can we do better?More QuestionsWhy is convolution interesting?Where else is it used?Can we generalize the idea of a sequence of coefficients on a polynomial?Signals

Signal: An impulse or a fluctuating quantity, such as voltage, current, or EM field strength, whose variations represent some sort of information.

SignalsNaturally Occurring:Light / Electromagnetic radiationSound (pressure) waves (e.g., in air or water)

Man-made: Encoded informationIn coax, CAT-5, etc.AM/FM Radio broadcastRadar pulses

Mathematically: Represented by a function of one or more independent variables.

Signals

tf(t)

Speech: acoustic pressure as a function of time,digitized as sequence of samples1-D Convolution

=Noise filtering!SignalsImage: brightness as a function of 2 spatial variables

xyf(x,y)2-D ConvolutionA 2-D signal (an Image) is convolved with a second Image (the filter, or convolution Kernel).-1 0 1-1 0 1-1 0 1

=AssignmentHomework: HW #9Do the upper right part of the proof of correctness for Strassens algorithmSome more probability exercises

Read Section 2.6 on the FFT!Probably the most famous Divide and Conquer algorithm