non-isometric manifold learning analysis and an algorithm
DESCRIPTION
Non-Isometric Manifold Learning Analysis and an Algorithm. Piotr Doll á r, Vincent Rabaud, Serge Belongie. University of California, San Diego. I. Motivation. Extend manifold learning to applications other than embedding - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/1.jpg)
Non-Isometric Manifold Learning Analysis and an Algorithm
Piotr Dollár, Vincent Rabaud, Serge Belongie
University of California, San Diego
![Page 2: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/2.jpg)
I. Motivation
1. Extend manifold learning to applications other than embedding
2. Establish notion of test error and generalization for manifold learning
![Page 3: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/3.jpg)
Linear Manifolds (subspaces)
Typical operations: project onto subspace distance to subspace distance between points generalize to unseen regions
![Page 4: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/4.jpg)
Non-Linear Manifolds
Desired operations: project onto manifold distance to manifold geodesic distance generalize to unseen regions
![Page 5: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/5.jpg)
II. Locally Smooth Manifold Learning (LSML)
Represent manifold by its tangent space
Non-local Manifold Tangent Learning [Bengio et al. NIPS05]
Learning to Traverse Image Manifolds [Dollar et al. NIPS06]
![Page 6: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/6.jpg)
Learning the tangent space
Data on d dim. manifold in D dim. space
![Page 7: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/7.jpg)
Learning the tangent space
Learn function from point to tangent basis
![Page 8: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/8.jpg)
Loss function
![Page 9: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/9.jpg)
Optimization procedure
![Page 10: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/10.jpg)
Need evaluation methodology to:objectively compare methodscontrol model complexityextend to non-isometric manifolds
III. Analyzing Manifold Learning Methods
![Page 11: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/11.jpg)
Evaluation metric
Applicable for manifolds that can be densely sampled
![Page 12: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/12.jpg)
Finite Sample Performance
Performance: LSML > ISOMAP > MVU >> LLE
Applicability: LSML >> LLE > MVU > ISOMAP
![Page 13: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/13.jpg)
Model Complexity
All methods have at least one parameter: k
Bias-Variance tradeoff
![Page 14: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/14.jpg)
LSML test error
![Page 15: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/15.jpg)
LSML test error
![Page 16: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/16.jpg)
IV. Using the Tangent Space
projectionmanifold de-noisinggeodesic distancegeneralization
![Page 17: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/17.jpg)
Projection
x' is the projection of x onto a manifold if it satisfies:
gradient descent is performed after initializing the projection x' to some point on the manifold:
tangents at
projection matrix
![Page 18: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/18.jpg)
Manifold de-noising
- original
- de-noised
Goal: recover points on manifold ( ) from points corrupted by noise ( )
![Page 19: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/19.jpg)
Geodesic distance
Shortest path:
gradient descent
On manifold:
![Page 20: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/20.jpg)
embedding of sparse and structured data
can apply to non-isometric manifolds
Geodesic distance
![Page 21: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/21.jpg)
Generalization
Generalization beyond support of training data
Reconstruction within the support of training data
![Page 22: Non-Isometric Manifold Learning Analysis and an Algorithm](https://reader036.vdocuments.net/reader036/viewer/2022081520/56814c4f550346895db95c8d/html5/thumbnails/22.jpg)
Thank you!