Slide 1

Algorithm Design Using Spectral Graph Theory

Richard PengJoint Work with Guy Blelloch, HuiHan Chin, Anupam Gupta, Jon Kelner, Yiannis Koutis, Aleksander Mdry, Gary Miller and Kanat Tangwongsan11OutlineMotivating problem: image denoisingFast solvers for SDD linear systemsUsing solver for L1 minimization and graph problems.2

Image DenoisingGiven image + noise, recover image.

CMUs space programRocket takes picture of moon surfaceNoisy picture, want to finding landing site from it.3Image Denoising: the Modeloriginal noiseless image.noise from some distribution added.input: original + noise, s.goal: recover original, x.Denoised Image:Noise:Input:s-xsxNoise is s-x in this setting4Explicit vs. Implicit ApproachesExplicitImplicitGoalRecover x directlyDefine conditions on x and s, solve for xBasic OperationAveraging a set of pixelsFilteringMinimize objective functionRuntimeO(n)O(n2) or higherQualityReasonableHighn > 106 for most imagesFirst give a simplified objective that can be optimized fast5Solution recovered has quality issues, will come back to this later.Simple Objective FunctionGradient: 2Ax 2sOptimal: 0 = 2Ax 2sAx = sEqual to xTAx-2sTx where x, s are length n vectors, A is n-by-n matrix

x = A-1sminimizei(xi-si)2 + i~j(xi-xj)2Same as minimizing quadratics in a single variable6Special Structure of AA is Symmetric Diagonally Dominant (SDD) if: Its symmetric In each row, diagonal entry at least sum of absolute values of all off diagonal entries

7OutlineMotivating problem: image denoisingFast solvers for SDD linear systemsUsing solver for L1 minimization and graph problems.8Fundamental Problem:Solving Linear SystemsGiven matrix A, vector bFind vector x such that Ax=b

Size of A:n-by-nm non-zero entriesFormally the running time can be characterized using n and m.9Solving Linear Systems:Explicit and ImplicitDirect (explicit)Iterative (implicit)Unit OperationModifying entryMatrix-vector multiplyMain goalOperations applied on matrix are reversibleExplored large portion of rank spaceCost per stepO(1)O(m)Numer of StepsO(n)O(n)Total RuntimeO(n)O(nm)10Explicit Algorithms[1st century CE] Gaussian Elimination: O(n3)[Strassen `69] O(n2.8)[Coppersmith-Winograd `90] O(n2.3755)[Stothers `10] O(n2.3737)[Vassilevska Williams`11] O(n2.3727)

Chinese math text, also studied by Newton11SDD Linear SystemsDirect (explicit)Iterative (implicit)Unit OperationModifying entryMatrix-vector multiplyMain ideaOperations applied on matrix are reversibleExplored large portion of rank spaceCost per stepO(1)O(m)Numer of StepsO(n)O(n)Total RuntimeO(n)O(nm)[Vaidya `91]: Hybrid methods12Nearly Linear Time Solvers[Spielman-Teng 04]

Input: n by n SDD matrix A with m non-zerosvector bWhere: b = Ax for some xOutput: Approximate solution x s.t.|x-x|A 106 Key issue: exponent on logAlgorithm isnt much faster than a O(n2) algorithm on reasonably sized problems16Practical Nearly Linear Time Solvers[Koutis-Miller-P `10, `11]

Input: n by n SDD matrix A with m non-zerosvector bWhere: b = Ax for some xOutput: Approximate solution x s.t.|x-x|A