a game-theoretic perspective on registration and recognition of 3d shapes
DESCRIPTION
A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes. Emanuele Rodolà . Surface registration. The aim is to rigidly align (“register”) two or more 3D surfaces so as to attain a utomatic assemblage of range data (demo). Surface registration. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/1.jpg)
EMANUELE RODOLÀ<RODOLA@DSI .UNIVE. IT>
A Game-Theoretic Perspective on Registration and Recognition of
3D Shapes
![Page 2: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/2.jpg)
2
Surface registration
The aim is to rigidly align (“register”) two or more 3D surfaces so as to attain automatic assemblage of range data (demo)
![Page 3: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/3.jpg)
3
Surface registration
Typically a 2-step process: Coarse motion estimation Refinement
![Page 4: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/4.jpg)
4
Coarse alignment
Knowledge of the acquisition processMarker-basedRANSAC-based DARCESPROSAC variantsPCA / 4PCS / Genetic
![Page 5: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/5.jpg)
5
Refinement
Given a “good enough” initial alignment, it is possible to refine the registration iteratively.
This is usually done by establishing pointwise correspondences among the two surfaces, and using them to estimate the rigid transformation.
(demo chef)
![Page 6: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/6.jpg)
6
A Game-Theoretic approach
We cast the registration problem to an inlier selection scenario:
We are given a set of candidate correspondences (strategies)
Then we look for a robust set of inlier correspondences wrt some notion of “rigidity”
Finally we can estimate the rigid transformation between the two surfaces
![Page 7: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/7.jpg)
7
Enforcing rigidity
Given a model mesh M, a data mesh D, and a set of candidate correspondences (or strategies):
We define a rigidity-enforcing payoff function giving a measure of compatibility among strategies:
We wish to bring global information into the matching process by favoring sets of point-associations that are mutually compatible with a single rigid transformation. We do this by operating at a local level.
222112121
babaλe,bb,,aaπ,Sπ:S
DMSSba ,,
![Page 8: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/8.jpg)
8
The evolutionary process
The search for a solution is performed by simulating the evolution of a natural selection process. The choice of an actual selection process is not crucial and can be driven mostly by considerations of efficiency and simplicity.
![Page 9: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/9.jpg)
9
Building the strategies set
It is not practical to deal with all the surface points from both the surfaces, i.e. we restrict to
Moreover, the isolation of interest points can help to avoid false correspondences
DMS
![Page 10: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/10.jpg)
10
Surface descriptors
Surface vertices can be described using information at the vertex and of a local patch around it.
o Spin Images (ref. axis)o Integral Invariants (no ref.)o Point Signatures (ref. frame)o Signatures of Histograms (stable ref. frame)o And many more
![Page 11: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/11.jpg)
11
Surface Hashes
To fully exploit our inlier selection method, we need descriptors with the following properties:
o high repeatabilityo weak distinctiveness
We introduce the Surface Hashes (demo):
o Normal Hasho Integral Hasho Mixed
![Page 12: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/12.jpg)
12
Interest point detection
Relevance-based samplingClustering (via a Matching Game)Simple threshold on the Integral Hash
![Page 13: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/13.jpg)
13
Enhancing the framework
The set of strategies can now be greatly reduced:
The descriptor prior gives better candidatesThe least likely correspondences are prunedThe selection process converges more rapidlyMuch lower memory requirements
Memory is the bottleneck!
![Page 14: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/14.jpg)
14
Object-in-clutter
Focus is on recognition rather than alignment
We now have a known, usually complete model to match against an incomplete and cluttered scene. (demo)
![Page 15: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/15.jpg)
15
Object-in-clutter
A good point selection strategyRobust descriptors wrt to occlusion and clutter
![Page 16: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/16.jpg)
16
Object-in-clutter
![Page 17: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/17.jpg)
17
Future directions
Scale invariance could be attained by taking into account the geodesic path between strategies
Non-rigid registration
![Page 18: A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes](https://reader036.vdocuments.net/reader036/viewer/2022062520/568165d0550346895dd8deca/html5/thumbnails/18.jpg)
18
Questions?