graph-regularized generalized low rank...
TRANSCRIPT
![Page 1: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/1.jpg)
Graph-Regularized Generalized Low Rank
ModelsMihir Paradkar & Dr. Madeleine Udell
Cornell University
![Page 2: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/2.jpg)
Properties of Images- High Dimensionality
![Page 3: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/3.jpg)
Properties of Images- High Dimensionality
- Noise and Occlusions
![Page 4: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/4.jpg)
Noise and Occlusions
![Page 5: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/5.jpg)
Properties of Images- High Dimensionality
- Noise and Occlusions
- Graph Structure
![Page 6: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/6.jpg)
Graph Structure
![Page 7: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/7.jpg)
Previous Work- Generalized Low Rank Models (GLRMs)
- Spectral Embedding
![Page 8: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/8.jpg)
Low Rank Models
- Approximate a data matrix as the product of two low-rank factors
Y X’
W
(“Narrow” factor)
(“Wide” factor)
![Page 9: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/9.jpg)
Low Rank Models
- Objective Function:
![Page 10: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/10.jpg)
Low Rank Models
- Using squared error recovers truncated SVD (PCA if centered and scaled)
![Page 11: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/11.jpg)
Spectral Embedding
- Maximizes similarity along a graph using Laplacian matrix L
![Page 12: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/12.jpg)
Spectral Embedding - Laplacian Matrix
(Path Graph)
1 -1
-1 2 -1
-1 2 -1
-1 1
![Page 13: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/13.jpg)
Spectral Embedding - Laplacian Matrix(Complete Graph)
3 -1 -1 -1
-1 3 -1 -1
-1 -1 3 -1
-1 -1 -1 3
![Page 14: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/14.jpg)
Spectral Embedding
- Maximizes similarity along a graph using Laplacian matrix L
![Page 15: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/15.jpg)
Graph GLRM
- Objective Function:
![Page 16: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/16.jpg)
FittingGraphGLRM uses modified Proximal Alternating Linearized Minimization (PALM) to fit factors; it takes two alternating proximal gradient steps per iteration, one per factor
![Page 17: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/17.jpg)
Software Implementation- Implementation in Julia language available at
https://github.com/mihirparadkar/GraphGLRM.jl
![Page 18: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/18.jpg)
Software Implementation- User specifies the data table, loss function, regularizers on factors, and rank,
along with optional parameters like a list of indices of known values
gm = GGLRM(Amissing, loss, rx, ry, k, obs=obs)
![Page 19: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/19.jpg)
Experiments- Imputation of block occlusions
- Classification of faces into male/female with occluded images
![Page 20: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/20.jpg)
Results - Classification Experiment
Embedding Method Precision Recall F1-Score
None 0.727 0.4 0.516
PCA 0.381 0.4 0.390
Spectral Embedding 1 0 0
Vanilla GLRM 0.714 0.25 0.370
Graph GLRM 1 0.5 0.667
![Page 21: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/21.jpg)
Results - Imputation Experiment
Method MSE
PCA 15032
Spectral Embedding 3415.4
Vanilla GLRM 634.63
Graph GLRM 554.48
![Page 22: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/22.jpg)
Results - Imputation Experiment
![Page 23: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/23.jpg)
Results - Imputation Experiment
![Page 24: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/24.jpg)
Results - Imputation Experiment
![Page 25: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/25.jpg)
Conclusions
- Combines linear embedding of GLRM with non-linear dimensionality reduction through graph Laplacian
- Freely-available and performant software implementation for fitting
- Can improve performance in reconstructing missing values and in classification with noisy data
![Page 26: Graph-Regularized Generalized Low Rank Modelsusers.cecs.anu.edu.au/~koniusz/tensors-cvpr17/present/paradkar_mi… · Graph-Regularized Generalized Low Rank Models Mihir Paradkar &](https://reader030.vdocuments.net/reader030/viewer/2022041015/5ec60f072510452df052ec26/html5/thumbnails/26.jpg)
Thank You!