local multidimensional scaling [1em] for nonlinear ......for measuring local continuity examples...
Post on 29-May-2020
9 Views
Preview:
TRANSCRIPT
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Local Multidimensional Scaling
For Nonlinear Dimension Reduction, GraphLayout and Proximity Analysis
Lisha Chen (Statistics, Yale)Joint work with Andreas Buja (Upenn)
Feb 6th, 2007
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Machine Learning Facial Images
I Facial images vary from moment to moment.
I The mystery of perception:How does brain perceive constancy in flux?
I Goal: Construct machines to capture the variability of facialimages.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Machine Learning Facial Images
I Facial images vary from moment to moment.
I The mystery of perception:How does brain perceive constancy in flux?
I Goal: Construct machines to capture the variability of facialimages.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Characterizing Image Variability
High-dimensional data . . .
I An image is regarded as a collection of numbers.
I Each number is the light intensity at an image pixel.
I An Example: 64 by 64 images ⇒ 4096-D data.
Low-dimensional manifold . . .
Degree of freedom
I Left-right pose
I Up-down pose
I Lighting-direction
I Expression change(not shown)
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Characterizing Image Variability
High-dimensional data . . .
I An image is regarded as a collection of numbers.
I Each number is the light intensity at an image pixel.
I An Example: 64 by 64 images ⇒ 4096-D data.
Low-dimensional manifold . . .
Degree of freedom
I Left-right pose
I Up-down pose
I Lighting-direction
I Expression change(not shown)
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Classical Method 1: PCA
I Principal Components: Fit the best ellipsoid to multivariatedata x1, ..., xN ∈ IRp.
I Linear method: Provide a sequence of best linearapproximation to the data.
I Uses: Plot the wide dimensions; interpret their directions.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Classical Method 2: Multidimensional Scaling
Multidimensional Scaling(MDS) for proximity data . . .
I Make a map from given pairwise dissimilarities.
I A Toy Example:
D =
0 3 43 0 54 5 0
MDS for dimension reduction . . .
I Calculate dissimilarities(Dij) in high-dimensional space.
I Match Dij in low-dimensional space.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Nonlinear Structure in High Dimensional Data
I Classical methods fail to find nonlinear structure.
−200 0 200 400
−200
0
200
400
Swiss Roll
−300 −200 −100 0 100 200 300
−300
−200
−100
0
100
200
300
2D PCA
0 500 1000 1500
0
500
1000
1500
True 2D Manifold
I Nonlinear dimension reduction methods:I Isomap(Tenenbaum et. al., 2000)I LLE(Roweis & Saul, 2000)I Hessian Eigenmaps(Donoho & Carrie, 2003)I Laplacian Eigenmaps (Belkin & Niyogi, 2003)I Diffusion Maps(Coifman et. al., 2005)
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Isometric Feature Mapping (Isomap)
The idea . . .
I Characterize the intrinsic geometry by geodesicdistances(shortest path).
I Match geodesic distances by Euclidean distances inlow-dimensional space.
The algorithm . . .
1 : Construct neighborhoodgraph.
2 : Compute shortest path.
3 : Construct embedding by
MDS.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Local Linear Embedding(LLE)
The idea . . .
I Nearby points lie on a locally linear patch.
I Characterize the local geometry by linear coefficientsthat reconstruct each data point from its neighbors.
The algorithm . . .
1 : Select neighbors.
2 : Construct weights.
ε(W ) =∑
i
|Xi−∑
j∈N(i)
WijXj |2
3 : Embedding.
Φ(Y ) =∑
i
|Yi−∑
j∈N(i)
WijYj |2
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Commonality of The Proposals: Localization
I Discard the information of large distances.
I Use local information:I Fixed-radius metric neighborhoods.I K nearest neighborhoods(KNN).
I Find a way to reassemble local structure and flatten themanifold.
I Our goal: Apply localization to MDS.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Revisit MDS
I Data:Objects i = 1, 2, ...,N (subjects, stimuli, graph nodes,...)Dissimilarities Dij for all pairs of objects i and j
I Goal: Visualization of objectsFind configuration {xi ∈ IRk |i = 1..N} such that:
‖xi − xj‖ ≈ Dij
k: embedding dimension
I Stress function: Goodness of fit criterionThe simplest version:
MDSD(x1, ..., xN) =∑
i,j=1..N
(Dij − ||xi − xj ||)2
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Localization Problem in MDS
A simple proposal . . .
I Localization: Drop the large dissimilarities.
I Stress function becomes:
MDSD(x1, ..., xN) =∑
(i,j)∈E
(Dij − ||xi − xj ||)2
E = edge set of local links
Unstable system: Graef and Spence (1979)
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Localization Problem in MDS
A simple proposal . . .
I Localization: Drop the large dissimilarities.
I Stress function becomes:
MDSD(x1, ..., xN) =∑
(i,j)∈E
(Dij − ||xi − xj ||)2
E = edge set of local links
Unstable system: Graef and Spence (1979)
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
New Proposal: Local MDS(LMDS)
The idea . . .
I Approximate the small dissimilarities.
I Give a proper amount of repulsive force amongthe points to avoid instability.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
New Proposal: Local MDS(LMDS)
I Step 1: Localization: E = edge set of local links; EC =edge set of non-local links.
I Step 2: Replace large distances (EC ) with ∞, but withinfinitesimal weight.
SD(x1, ..., xN) =∑
(ij)∈E
(Dij − ‖xi − xj‖)2
+ w ·∑
(ij)∈EC
(D∞ − ‖xi − xj‖)2
I Step 3: Take limit w → 0 and D∞ →∞ subject to2 w D∞ = t.
LMDSD(x1, ..., xN) =∑
(ij)∈E
(Dij − ‖xi − xj‖)2
− t ·∑
(ij)∈EC
‖xi − xj‖
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
New Proposal: Local MDS(LMDS)
LMDSD(x1, ..., xN) =∑
(ij)∈E
(Dij − ‖xi − xj‖)2
− t ·∑
(ij)∈EC
‖xi − xj‖
I The first term: Local stress to match small dissimilarities.
I The second term: Repulsive force for spreading out.
I The choice of t: t = τ · (K/N) ·medianE (Dij)
I τ repulsion tuning parameter, usually 0.001 ∼ 1.I K : the number of neighbors for each point .
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
The LC Meta-criterion: Motivation andDefinition
I Motivation
I Notation:I ND
K (i): Data point i ’s K nearest neighbors with regard tothe target dissimilarities Dij .
I N XK (i): Data point i ’s K nearest neighbors with regard to
the configuration distances ‖xi − xj‖.I | S |: Cardinality of a set S.
I The local version of the LC meta-criterion:
NK (i) = | NDK (i) ∩ NX
K (i) |
I The global version of the LC meta-criterion:
NK =1
N
N∑i=1
NK (i)
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
The LC meta-criterion: Normalization andAdjustment for Baselines
I Compare LC meta-criteria for different values of K.
I Normalization:
MK =1
KNK
I The problem of meta-criterion for large K : MN−1 = 1.
I Adjusted LC Meta-criterion:
MadjK = MK −
K
N − 1
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 1: Sculpture Face
−50 0 50
−40
−30
−20
−10
0
10
20
30
3D LMDS
Left−right Pose
Up−
dow
n P
ose
−50 0 50
−40
−30
−20
−10
0
10
20
30
Left−right Pose
Ligh
ting
Dire
ctio
n
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 1: Sculpture Face
Adjust the repulsion tuning parameter τ in LMDS:
5e−04 5e−03 5e−02 5e−01
2.5
3.0
3.5
4.0
4.5
5.0
K=6
τ
KM
K
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 1: Sculpture FaceMethods PCA MDS Isomap LLE LMDS
N6 2.6 3.1 4.5 2.8 5.2
−15 −5 0 5 10
−15
−5
515
−10 0 5 10
−10
05
15
−10 0 5 10
−10
05
15
−2 −1 0 1 2
−2
−1
01
2
−2 −1 0 1 2
−2
−1
01
2
−2 −1 0 1 2
−2
−1
01
2
−2 −1 0 1 2−
20
12
−1.0 0.0 1.0
−1.
00.
01.
0
−1.0 0.0 1.0
−1.
00.
01.
0−0.25 −0.15 −0.05 0.05−
0.25
−0.
100.
00−0.25 −0.15 −0.05 0.05
−0.
25−
0.10
0.00
−0.25 −0.15 −0.05 0.05
−0.
25−
0.10
0.00
−50 0 50
−50
050
−40 −20 0 20
−40
−20
020
−40 −20 0 20−
40−
200
20
Left−
right
Pos
eU
p−do
wn
Pos
eLi
ghtin
g D
irect
ion
PCA MDS ISOMAP LLE LMDS
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 1: Sculpture Face
−15 −10 −5 0 5 10 15
−15
−10
−5
05
1015
3D PCA (M6=2.6)
−2 −1 0 1 2
−2
−1
01
2
3D ISOMAP (M6=4.5)
−0.05 0.00 0.05
−0.
050.
000.
05
3D LLE (M6=2.8)
−50 0 50
−50
050
3D LMDS (M6=5.2)
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 1: Sculpture Face
5 10 20 50 100 200 500
0.2
0.4
0.6
0.8
1.0
K
MK
MK
AdjustedMK
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 1: Sculpture Face
5 10 20 50 100 200 500
0.0
0.2
0.4
0.6
0.8
1.0
K
MK
Ko=4Ko=6Ko=8Ko=10Ko=25Ko=125Ko=650
5 10 20 50 100 200 500
0.0
0.2
0.4
0.6
0.8
1.0
K
Adj
uste
d M
K
Ko=4Ko=6Ko=8Ko=10Ko=25Ko=125Ko=650
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 2: Frey Face
−600 −400 −200 0 200 400 600
−60
0−
400
−20
00
200
400
600 3D PCA
Left−right Pose
Ser
ious
−S
mili
ng
−2 −1 0 1 2
−2
−1
01
2
3D MDS
Left−right Pose
Ser
ious
−S
mili
ng
−2 −1 0 1 2
−2
−1
01
2
3D ISOMAP
Left−right Pose
Ser
ious
−S
mili
ng
−0.10 −0.05 0.00 0.05
−0.
10−
0.05
0.00
0.05
3D LLE
Left−right Pose
Ser
ious
−S
mili
ng
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 2: Frey Face
−10000 −5000 0 5000 10000
−10
000
−50
000
5000
1000
0
3D LMDS K0=12
Left−right Pose
Ser
ious
−S
mili
ng
−3e+06 −1e+06 1e+06 2e+06 3e+06
−3e
+06
−1e
+06
1e+
063e
+06
3D LMDS K0=4
Left−right Pose
Ser
ious
−S
mili
ng
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 2: Frey Face3D PCA
Left−right Pose
Ser
ious
−S
mili
ng
3D MDS
Left−right Pose
Ser
ious
−S
mili
ng
3D ISOMAP
Left−right Pose
Ser
ious
−S
mili
ng
3D LLE
Left−right Pose
Ser
ious
−S
mili
ng
3D LMDS K0=12
Left−right Pose
Ser
ious
−S
mili
ng
3D LMDS K0=4
Left−right Pose
Ser
ious
−S
mili
ng
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 2: Frey Face3D PCA (M12=3.6)
Left−right Pose
Ser
ious
−S
mili
ng
3D MDS (M12=4.8)
Left−right Pose
Ser
ious
−S
mili
ng
3D ISOMAP (M12=4.2)
Left−right Pose
Ser
ious
−S
mili
ng
3D LLE (M12=3.2)
Left−right Pose
Ser
ious
−S
mili
ng
3D LMDS K0=12 (M12=4.6)
Left−right Pose
Ser
ious
−S
mili
ng
3D LMDS K0=4 (M12=5.1)
Left−right Pose
Ser
ious
−S
mili
ng
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 1: Frey Face
5 10 20 50 100
0.3
0.4
0.5
0.6
K
MK
Ko=4Ko=8Ko=12Ko=16Ko=20Ko=50Ko=100Ko=150Ko=1964
5 10 20 50 100
−0.
10−
0.05
0.00
0.05
K
Adj
uste
d M
K w
ith M
DS
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Outline for the Rest of the Talk
I Graphs and energy-based models.
I Stress in LMDS and energy in graph layout.
I Clustering postulate.
I Generalized LMDS: A Box-Cox family of energy functions.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Graph-layout
I A graph G = (V ,E ):I V ↔ Entities: Telephone numbers, carbon atoms . . .I E ↔ Relations: Phone calls, chemical bonds . . .
I Goal: Two-dimensional layout ↔ Visualize the relationalinformation.
I Algorithm: Force-directedPlacement(Eades, 1987)
I A physical system:Vertices → Steel rings;Edges → Springs.
I Best layout is achieved atthe minimal energy state.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Energy-based Models
I Notation:I E = edge set; EC = non-edge set; V 2 = E ∪ EC .I Denote dij = ‖xi − xj‖ = drawn distances.I U: Energy of the system.I f (·): energy corresponding to attraction
g(·):energy corresponding to repulsion
I Common in graph layout: attraction-repulsion forces
U =∑E
f (dij) −∑V 2
g(dij)
I LMDS and energy-based models.
LMDSD =∑E
(Dij − dij)2 − t ·
∑EC
dij
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Energy-based Models
Some energy functions . . .
I Eades (1984)
U =∑
E
dij ·(log(dij)− 1− dij
−2)
+∑V 2
dij−1
I Fruchterman & Reingold (1991)
U =∑
E
dij3/3 −
∑V 2
log(dij)
I Davidson & Harel (1996)
U =∑
E
dij2 +
∑V 2
dij−2
I Noack (2003), LinLog model
U =∑
E
dij −∑V 2
log(dij)
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Clustering Postulate
I A graph with two clusters:
n1ne
n2
I Problem: What should be the inter-cluster distance d?
I Energy:U(d) = ne f (d)− n1n2g(d)
I Clustering Postulate(Noack 2004):I U(d) should be minimized at d = n1n2/ne = 1/C .I C = ne/n1n2 is called coupling strength, C < 1.
I Stationary condition yields:
f ′(d) = d · g ′(d)
I Weak Clustering Postulate: More strongly coupled clustersshould be placed more closely to each other.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Distance-weighted Graphs
I Assume target distances Dij for (i , j) ∈ E .
I Extended clustering condition: In the 2-cluster situation,require the minimum at
d = D · ( 1C)λ
I D: the target distanceI ( 1
C)λ: a generalized clustering factor.
I λ > 0: clustering power; C < 1: coupling strength.
I New stationarity condition:
f ′(d) =
(d
D
) 1λ
g ′(d)
I f (d) and g(d) satisfy the extended coupling condition:
f (d) =dµ+ 1
λ − 1
µ + 1λ
, g(d) =dµ − 1
µD
1λ
with logarithmic fill-in for zero exponents.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
A Box-Cox Family of Energy Functions
I Energy:
U ∼∑
E
(dij
µ+ 1λ −1
µ+ 1λ
− Dij1λ
dijµ−1µ
)− t
∑E c
dijµ−1µ
I Special cases:I LMDS: λ = µ = 1I Energy for unweighted graph(Dij = 1):
I Noack(LinLog): λ = 1, µ = 0I Fruchterman&Reingold: λ = 1
3, µ = 0
I Davison&Harel: λ = 14, µ = −2
Advantages . . .
I Flexibility for achieving desirable configurations.
I Clustering power λ: visualize the clusters.
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 3: Olivetti Faces
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Example 3: Olivetti FacesMethods λ = 0.25 λ = 0.5 λ = 1.0 λ = 1.5
N4 0.64 1.01 1.44 1.57
−6000 −2000 0 2000 6000
−60
00−
2000
020
0060
00
λ=0.25
−15000 −5000 0 5000 15000−15
000
−50
000
5000
1500
0 λ=0.5
−20000 0 10000
−20
000
010
000
λ=1
−40000 −20000 0 20000 40000
−40
000
020
000
4000
0 λ=1.5
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Conclusion and Discussion
I Nonlinear dimension reduction, proximity analysis andgraph-layout
I ContiguityI DistancesI Localization
I Graph Layout: Two Fundamental ApproachesI Complete the graph (Isomap, Krustal and Seery(1980))I Apply repulsive force
I B-C Energy Functions
I The Selection Problem: LC Meta-Criteria
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Diagnostic Plot: Frey Face
500 1000 1500
05000
10000
15000
20000
Diagnostic Plot
D_ij
d_ij
Local MultidimensionalScaling
For Nonlinear DimensionReduction, Graph Layoutand Proximity Analysis
L. Chen
Motivation
Some Existing Methods
New proposal: LMDS
The LC Meta-criterionfor Measuring LocalContinuity
Examples
Graph-layout and Energyfunctions
Generalized LMDS
Conclusion andDiscussion
Diagnostic Plot: Frey Face
top related