manifold alignment for multitemporal hyperspectral image classification
DESCRIPTION
Manifold Alignment for Multitemporal Hyperspectral Image Classification. H. Lexie Yang 1 , Melba M. Crawford 2 School of Civil Engineering, Purdue University and Laboratory for Applications of Remote Sensing Email: {hhyang 1 , mcrawford 2 }@ purdue.edu July 29, 2011 - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/1.jpg)
1
Manifold Alignment for Multitemporal Hyperspectral Image Classification
H. Lexie Yang1, Melba M. Crawford2
School of Civil Engineering, Purdue Universityand
Laboratory for Applications of Remote Sensing
Email: {hhyang1, mcrawford2}@purdue.eduJuly 29, 2011
IEEE International Geoscience and Remote Sensing Symposium
![Page 2: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/2.jpg)
2
Outline
• Introduction• Research Motivation
− Effective exploitation of information for multitemporal classification in nonstationary environments
− Goal: Learn “representative” data manifold• Proposed Approach
− Manifold alignment via given features− Manifold alignment via correspondences− Manifold alignment with spectral and spatial information
• Experimental Results• Summary and Future Directions
![Page 3: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/3.jpg)
3
Introduction
• Challenges for classification of hyperspectral data− temporally nonstationary spectra− high dimensionality
2001 2003 2004 2005 200620022001
June July May May May May June
N na
rrow sp
ectra
l ban
ds
123
N>>30
![Page 4: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/4.jpg)
4
• Nonstationarities in sequence of images − Spectra of same class may evolve or drift
over time
• Potential approaches− Semi-supervised methods− Adaptive schemes− Exploit similar data geometries
Explore data manifolds
Research Motivation
Good initial conditions required
![Page 5: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/5.jpg)
5
Manifold Learning for Hyperspectral Data
• Characterize data geometry with manifold learning − To capture nonlinear structures − To recover intrinsic space (preserve spectral neighbors) − To reduce data dimensionality
• Classification performed in low dimensional space
Spectr
al ba
nds
Spatial dimension
Spa
tial d
imen
sion
1234
2nd dim 1st dim
3rd dim
n
56
Original space Manifold space
![Page 6: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/6.jpg)
6
Challenges: Modeling Multitemporal Data
• Unfaithful joint manifold due to spectra shift
• Often difficult to model the inter-image correspondences
Data manifold at T1 Data manifold at T2 Data manifolds at T1 and T2
![Page 7: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/7.jpg)
7
Proposed Approach: Exploit Local Structure
Assumption: local geometric structures are similar Approach: Extract and optimally align local geometry
to minimize overall differences
Locality
Spectral space at T1 Spectral space at T2
![Page 8: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/8.jpg)
8
Proposed Approach: Conceptual Idea
(Ham, 2005)
![Page 9: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/9.jpg)
9
Proposed Approach: Manifold Alignment
• Exploit labeled data for classification of multitemporal data sets
Samples with no class labels
Joint manifold
Samples with class labels
![Page 10: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/10.jpg)
10
Manifold Alignment: Introduction
• and are 2 multitemporal hyperspectral images− Predict labels of using labeled
• Explore local geometries using graph Laplacian and some form of prior information
• Define Graph Laplacian
− Two potential forms of prior information: given features and pairwise correspondences [Ham et al. 2005]
1I 2I
L
where0 , otherwise 1 , neighbors of ij
i j
ii ijj
L D W
Wx x
D W
2I 1I
![Page 11: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/11.jpg)
11
Manifold Alignment via Given Features
Given Features is Joint Manifold *F
Minimize ( )C F
1
1 2
{ ,..., }: Given features of labeled samples : Graph Laplacian of and : Relative weighting coefficient
i ns s s nL I I
1 1 1 21 2
2 1 2 2
, ,,
, ,
I I I II I
I I I I
L LL
L L
n
i
IITiiFF
FLFsfFCF 21 ,* minarg)(minarg
![Page 12: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/12.jpg)
12
Manifold Alignment via Pairwise Correspondences
Correspondences between and 1I 2I
Minimize ( , )C F G1IL
2IL
Joint Manifold * *[ ; ]F G
1
2
1 1 1 2
1 2
1
{ ,..., } ;{ ,..., }( , ) : Pairwise correspondences in [ ; ] where index corresponds to pair ( , ) extracted from and
: Graph Laplacian of
: Graph Laplacian
N M
i i
i i
I
I
x x I y y If g F G
i x y I IL I
L
2of
: Relative weighting coefficent
I
k
iI
TI
TiiGFGF
GLGFLFgfGFCGF21,,
** minarg),(minarg];[
![Page 13: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/13.jpg)
13
MA with spectral and spatial information
• Combine spatial locations with spectral signatures− To improve local geometries (spectral) quality− Idea: Increase similarity measure when two samples are
close togetherWeight matrix for graph Laplacian:
where spatial location of each pixel is represented as
2 2spa spe
Spatial Distance ( , ) Spectral Distance ( , )exp expi j i j
ij
z z x xW a
ix2Riz
![Page 14: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/14.jpg)
14
Experimental Results: Data Three Hyperion images collected in
May, June and July 2001 May - June pair: Adjacent
geographical area June - July pair: Targeted the same
area
May June July
Class
Water
Floodplain
Riparian
Firescar
Island interior
Woodlands
Savanna
Short mopane
Exposed soils
![Page 15: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/15.jpg)
15
Experimental Results: Framework
Joint manifoldGraph
LaplacianPrior information
Given features
Correspondences
Develop Data Manifold of
Pooled Data
GF
CF
PF
Data sets Labels
Pair 1 Pair 2
May June Training data For KNN classifier
June July Testing data For overall accuracy evaluation
Classificationwith KNN
I1, I2L
I1L I2L
![Page 16: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/16.jpg)
16
Manifold Learning for Feature Extraction
• Global methods consider geodesic distance − Isometric feature mapping
(ISOMAP)
• Local methods consider pairwise Euclidian distance− Locally Linear Embedding (LLE): (Saul and Roweis, 2000)− Local Tangent Space Alignment (LTSA): (Zhang and Zha,
2004)− Laplacian Eigenmaps (LE): (Belkin and Niyogi, 2004)
(Tenenbaum, 2000)
![Page 17: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/17.jpg)
17
MA with Given Features
ISOMAP
LTSA
LLE
LE
75.69
70.18
47.65
62.38
0.620000000000006
7.69999999999999
29.64
16.63
Pooled DataAccuracy increment (Δ) with MA using extracted features
Overall Accuracy
• Baseline: Joint manifold developed by pooled data
(May, June pair)
79.21
77.29
77.88
76.31
![Page 18: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/18.jpg)
18
MA Results – Classification Accuracy
• Evaluate results by overall accuracies
MethodsOverall Accuracy
May, June June, July
Manifold learning from pooled data 62.38% 83.00%
Manifold alignment(MA)
Given features (LE) 79.21% 86.16%Correspondences 81.22% 84.27%
MethodsOverall Accuracy
May , June June, July
Given features (LE)
Spectral 79.21% 86.16%Spectral + spatial 84.21% 90.30%
CorrespondencesSpectral 81.22% 84.27%Spectral + spatial 84.74% 90.11%
![Page 19: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/19.jpg)
19
Results – Class Accuracy
May/June Pair
Typical class(Island Interior) Critical class
(Woodlands)Critical class
(Riparian)
MA: Given Features: SpectralMA: Correspondences: Spectral
MA: Given Features: Spectral / SpatialMA: Correspondences: Spectral / Spatial
Pooled data
![Page 20: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/20.jpg)
20
Summary and Future Directions
• Multitemporal spectral changes result in failure to provide a faithful data manifold
• Manifold alignment framework demonstrates potential for nonstationary environment by utilizing similar local geometries and prior information
• Spatial proximity contributes to stabilization of local geometries for manifold alignment approaches
• Future directions− Investigate alternative spatial and spectral integration
strategy− Address issue of longer sequences of images
![Page 21: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/21.jpg)
21
Thank you.Questions?
![Page 22: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/22.jpg)
22
References
• J. Ham, D. D. Lee, and L. K. Saul, “Semisupervised alignment of manifolds,” in International Workshop on Artificial Intelligence and Statistics, August 2005.
![Page 23: Manifold Alignment for Multitemporal Hyperspectral Image Classification](https://reader036.vdocuments.net/reader036/viewer/2022081515/568160f4550346895dd02e8e/html5/thumbnails/23.jpg)
23
Backup Slides