shape classification using the inner-distance haibin ling david w. jacobs ieee transaction on...
Post on 22-Dec-2015
222 views
TRANSCRIPT
Shape Classification Using the Inner-Distance
Haibin Ling
David W. Jacobs
IEEE TRANSACTION ON PATTERN ANAYSIS AND MACHINE INTELLIGENCE
FEBRUARY 2007
Outline
Introduction Related work Inner-Distance Articulation invariant signatures Inner-distance shape context Shortest path texture context Experiments
Introduction
We use the inner-distance to build shape descriptors that are robust to articulation and capture part structure.
Inner-distance is defined as the length of the shortest path between landmark points within the shape boundary.
Related work
Three categories to handle parts classification: Statistical methods to describe the articulation between
parts and often require a learning process to find the model parameters.
To measures the similarity between shapes between shapes via part-to-part matching and junction parameter distribution.
To capture the part structure by considering the interior of shape boundary. Our method belongs to. Skeleton-based approaches
The inner-distance: The definition Define a shape as a connected and closed
subset of R2. Given a shape and two points ,the inner-distance between denoted as ,is defined as the length of the shortest path connecting and within .
);,( OyxdOyx ,
OO
,, yx
O
Note:(1) In rare case where there are multiple shortest paths, we arbitrarily choose one.(2) Shapes are defined by their boundary, hence only boundary points are used as landmark points.
x y
The inner-distance: Computation Shortest path algorithms:
(1) Build a graph with the sample points. For each pair of sample points p1 and p2, if the line segment connecting p1 and p2 fall entirely within the object, Let an edge between p1 and p2 is added to the graph with its weight equal to the Euclidean distance || p1 - p2 ||.
Note:(1) Neighboring boundary points ar
e always connected.(2) (2) The inner-distance reflects t
he existence of holes without using samples points from hole boundary.
The inner-distance: Computation
(2) find the inner-distance between all pairs of points according to the graph.
The whole computation takes . It takes time to checi whether a line segment
between two points is inside the given shape. The complixity of graph construction is of .
)( 3nO
)(no
)( 3nO
The inner-distance:A model of articulate objects
}{}{1
ijji
i
n
iJOO
Articulated objects. (a) An articulated shape. (b) Overlapping junstions. (c) Ideal articulation.
The inner-distance:A model of articulate objects
is constant and very small compared to the size of the articulated parts.
An articulated to another articulated object is one-to-one continuous mapping .
)};,({max)( where,)(,
PyxdPdiamJdiamPyx
ij
fO 'O
The inner-distance: articulation insensitivity Changes of the inner-distance are due to
junction deformations. That means change is very small compared to the size of parts.
niOyxOyxdOyxd iii 1,, )';','();,(
ij
ij
Jnjiji
JyxOyxdOyxd
,,1,
,,)';','();,(
The inner-distance: articulation insensitivity Theorem:
Proof:
Is decomposed into segments.
εm,m'x',y';O'dx,y;Od
O'x',y';O'Γ
Ox,y;OΓ
Ox,yO
O
}max{|)(-)(|
then.in junctionsdifferent m' through goes )(
and in junctionsdifferent m through goes )(
path shortest thesuppose , above. defined as of
on articulatian be f andobject darticulatean be Let
);,( Oyx l
)];,(),...,;,(),;,([);,( 1221110 lll RppRppRppOyx
The inner-distance: articulation insensitivity Example:
li
iii RppdOyxd1
1 );,();,(
)]';','(),...,';','(),';','([)';','(~
1221110 lll RppRppRppOyxC
mmOyxdOyxd 1)';','(~
);,(
Note:
is not the shortest path.
)';','(~
OyxC
The inner-distance: ability to capture structures It is hard to prove because no clear part
decomposition.
Show how the inner-distance capture part structure with examples:
The inner-distance: ability to capture structures
With about the same number of sample points, the four shapes are virtually indistinguishable using distribution of Euclidean distance. However, their distributions of the inner-distance are quite different except for the first two shapes.
Note: more sample points will not affect the above statement.
Articulation Invariant Signatures The inner-distance is used to build articulation
invariant signatures for 2D shapes using multidimensional scaling (MDS).
Given sample points on the shape O.the inner-distance .MDS finds the transformed points such that the Euclidean distance minimize the stress S(Q)
niipP 1}{
njiijd 1,}{
niiqQ 1}{
njijiij qqQe 1,})({
ji ij
ji ijijij
d
QedwQS
2
2))(()(
Articulation Invariant Signatures Example:
MDS+SC+DP Use MDS to get articulation invariance signatures. Build the shape context on the signatures. Use dynamic programming for shape context matching
(a) and (c) show two shapes related by articulation. (b) and (d) show their signatures.
Related work:Shape Contexts for 2D shape The shape context was introduced by Belongi
e et al. Due to its simplicity and discriminability, the s
hape context has become quite popular recently in shape matching tasks.
It describes the relative spatial distribution.
Related work:Shape Contexts for 2D shape Given n sample on shape.
The shape context at points is defined as a histogram of the relative coordinates of the remaining n-1 points.
Where the bins uniformly divide the log-polar space. The shape context uses the Euclidean distance to m
easure the spatial relation between landmark points. This causes less discriminability for complex shapes with articulations.
ix
nxxx ,...,, 21
ixih
)}(,:{#)( kbinxxijxkh iiji
Inner-Distance Shape Context (IDSC) To extend the shape context, Euclidean
distance is directly replaced by the inner-distance.
Inner-Distance Shape Context (IDSC) The angle between the contour tangent at p
and the direction of at p is insensitive to articulation, called inner-angle, denoted .
Inner-angle is used for the orientation bins. Noise may reduces the stability of the inner-
angle, smoothing contour before computing it.
);,( Oqp
);,( Oqp
Inner-Distance Shape Context (IDSC) Example:
In the histogram, the x axis denotes the orientation bins and the y axis denotes log distance bins.
Shape matching through Dynamic programming Given two shapes A and B, points sequences
on their contour , say, for A and for B, assume .
A matching from A to B is a mapping. is matched to if , and otherwise
left unmatched. should minimize the match cost.
ni
iicC1
))(,()(
nppp 11 mqqq 11
mn
ip
)(iqip 0)( i
is the penalty for leaving unmatched, and for , is the cost of matching to .
and are the shape context histogram of and . K is the number of histogram bins.
Kk jBiA
jBiA
khkh
khkhjic
1 ,,
2,,
)()(
)]()([
2
1),(
)0,(ic ip
mj 1 ),( jicip iq
iAh , jBh ,
ip iq
Shape matching through Dynamic programming
Shape matching through Dynamic programming
DP is used to solve the matching problem since it uses the ordering information provided by shape contours.
By default, assumes the two contours are already aligned at their start and end points.
Without this assumption, one simple solution is to try different alignments at all points on the first contour and choose the best one.
)()(:complexity the 32 nOnO
Shape matching through Dynamic programming
Because shapes can be first rotated according to their moments, it is sufficient to try aligning a fixed number of points, say k points.
Usually, k is much smaller than m and n.
)()( still is complexity the 22 nOknO
Shape distance
The matching cost is used to measured the similarity between shapes.
IDSC+DP is better than SC+DP Better performance Only two parameters to tune
The penalty for a point with no matching, usually set 0.3.
The number of start points k for different alignments, usually set 4-8.
Easy to implement since it does not require the appearance and transformation model.
)(C
Shortest path texture context
The combination of texture and shape information, because Shapes from different classes sometimes are
more similar than those from the same class. Shapes are often damaged due to occlusion and
self-overlapping.
Shortest path texture context
The texture information along these paths provides a natural articulation insensitive texture description.
The angles between intensity gradient directions and shortest path directions are used, called relative orientations.
Shortest path texture context
The SPTC for each is a three dimensional histogram . The inner-distance The inner-angle The (weighted) relative orientation
The relative orientations are weighted by gradient magnitudes.
ix
ih
Experiment
the number of inner-distance bins: or the number of inner-angle bins:the number of relative orientation bins:
The number of different starting points for alignment:
The penalty for one occlusion:
5dn12n
8rn
8dn
84 k
3.0
Experiment
MPEG7 CE-Shape-1 shape database is widely tested, which consists of 1400 silhouette images from 70 classes. Each class has 20 different shapes.
Bullseye test: for every image in the database, it is matched with all other images and the top 40 most similar candidates are counted.
Experiment
The score of the test is the ratio of the number of correct hits of all images to the highest possible number of hits (which is 20x1400).
Experiment
The ETH-80 image set:
This data set contains 80 objects from eight classes, with 41 images of each object obtained from different viewpoints.