object modelling by registration of multiple range images

Institut für Robotik und Kognitive Systeme | Jirapong Manit

Chen Y. and Medioni G., Object Modeling by Registration of Multiple Range Images

Object Modelling by Registration of Multiple Range Images

[Yang Chen and Gérard Medioni, 1991]

Jirapong ManitInstitute for Robotics and Cognitive Systems

University of Lübeck



Outline

• Introduction• Range Image Registration

– Choosing the Evaluation Function– Line-Surface Intersection– Registration Algorithm

• Integration of Multiple Range Images• Results and Discussion• Conclusion



Introduction

• Model of physical object is necessary component machine of biological vision modules.

• CAD models are hardly accessed in practices.

• Object from multiple views was proposed, but not always enough.



Introduction

Object Modelling Procedures1. Data acquisition2. Registration between views3. Integration of views



Introduction

http://carlos-hernandez.org/research.html



Introduction

http://carlos-hernandez.org/research.html

Transformation

Correspondence problem



Introduction

The range images of a Mozart bust



Range Image Registration

http://blogs.rediff.com/alafearrea1983/2015/04/13/download-point-cloud-registration/

Point cloud registration




• Rigid transformation

∀𝑝𝑖∈𝑃 ,∃𝑞 𝑗∈𝑄∨‖𝑇 𝑝𝑖−𝑞𝑖‖=0

D (𝑃 ,𝑄 )=∬Ω

❑

‖𝑇𝑝(𝑢 ,𝑣 )−𝑞( 𝑓 (𝑢 ,𝑣 ) ,𝑔(𝑢 ,𝑣 ))‖2𝑑𝑢𝑑𝑣=0

or

where and and are correspondence mapping functions




• Transformation Matrix

T=[𝑐𝛼 𝑐 𝛽𝑠𝛼𝑐 𝛽−𝑠 𝛽0

𝑐 𝛼𝑠 𝛽 𝑠𝛾−𝑠𝛼𝑐𝛾𝑠𝛼 𝑠 𝛽𝑠𝛾+𝑐𝛼 𝑐𝛾

𝑐 𝛽𝑠𝛾0

𝑐𝛼𝑠 𝛽𝑐𝛾+𝑠𝛼𝑠 𝛾𝑠𝛼𝑠 𝛽𝑐𝛾−𝑐𝛼 𝑠𝛾

𝑐 𝛽𝑐 𝛾0

𝑡𝑥𝑡𝑦𝑡 𝑧1

]Consist of 6 DOF:



Choosing the Evaluation Function

Minimise : correspondence points

𝑒=∑𝑖=1

𝑁

‖𝑇 𝑝𝑖−𝑞𝑖‖2

Where and ,

𝑒=∑𝑖=1

𝑁

‖𝑇 𝑝𝑖−𝑞 𝑗‖2,with𝑞 𝑗=𝑞|min

𝑞𝜖𝑄‖𝑇 𝑝𝑖−𝑞‖




Minimise : approximate points

𝑒𝑘=∑𝑖=1

𝑁

‖𝑇 𝑘𝑝𝑖−𝑞 𝑗𝑘‖2 ,with𝑞 𝑗

𝑘=𝑞|min𝑞𝜖𝑄

‖𝑇 𝑘−1𝑝𝑖−𝑞‖

*Approximate would be easier




Minimise : distance between surfaces, Potmesil [2]

𝑒𝑘=∑𝑖=1

𝑁

‖𝑇 𝑘𝑝𝑖−𝑞 𝑗′ 𝑘‖2 ,with 𝑞 𝑗

′ 𝑘=(𝑇𝑘−1 ℓ 𝑖)∩𝑄

is intersection point of line with surface




Minimise : approximate using tangent plane

w here 𝑆 𝑗 is the tangent plane of 𝑄at𝑞 𝑗

𝑒=∑𝑖=1

𝑁

‖𝑇 𝑝𝑖−𝑞 𝑗′ ‖2 ,with𝑞 𝑗

′ =𝑞|min𝑞𝜖𝑆 𝑗

‖𝑇 𝑝𝑖−𝑞‖

𝑒=∑𝑖=1

𝑁

‖𝑇 𝑝𝑖−𝑞𝑖‖2




Minimise : approximate using tangent plane

𝑒𝑘=∑𝑖=1

𝑁

𝑑𝑠2 (𝑇𝑘𝑝𝑖 , 𝑆 𝑗

𝑘)

w ith 𝑆 𝑗𝑘={𝑠∨𝑛𝑞𝑗

𝑘 ∙ (𝑞 𝑗′ 𝑘−𝑠 )=0 },𝑞 𝑗

′ 𝑘=(𝑇 𝑘− 1 ℓ𝑖 )∩𝑄




Distance measures between and illustrated in the 2D case

𝒬

𝒫




Distance measures between and illustrated in the 2D case

𝒬𝑇 𝑘−1𝒫

𝑝𝑖 𝑑𝑠

𝑆 𝑗𝑘

𝑛𝑖

Minimise:



Line-Surface Intersection

0 x

z

𝑝❑Control Point (start point)𝒫𝒬

Initial:- Let be a point on - is a line normal to at - Projecting orthographically

along -axis- Set and

y

Normal Vector Line

OrthogonalProjections



Line-Surface Intersection

𝑝❑

Normal Vector Line

Control Point (start point)𝒫𝒬

𝑞0

Tangents Each iteration :- Compute - Find intersection of

with tangent plane to at

0 x

𝒫y

z

𝑞1

OrthogonalProjections

The process stop when ,

Approximate Intersection

True Intersection



The Registration Algorithm

Rewrite:

• Approximating very small value of in , the lease square algorithm can be converted to linear problem.

is the tangent plane to at




1. Select a set of and compute the surface normal at those points. Let

=2. At each iteration

a) For each control point • Apply to and to get and • Find the intersection of surface • Compute the tangent plane of at

b) Find the transformation that minimises with least square method, let




• The convergence of the procedure is tested by checking

where is a threshold set via experiment



Test Cases With Range Image Registration

= 0.046, s = 0.6523, min = -39.3, max = 20.2 (in mm)

1st Example

Histogram of the error image

Actual angle: -15.00 degree

Detected angle: -15.06 degree



Test Cases With Range Image Registration

= 0.0115, s = 0.3219, min = -7.63, max = 9.741 (in mm)

2nd Example

Actual angle: -20.00 degree

Detected angle: -19.75 degree

Histogram of the error image



Outline

• Introduction• Range Image Registration• Integration of Multiple Range Images• Results and Discussion• Conclusion



Integration of Multiple Range Images

• Multiple range images from different vantage points are needed

• Surface is not completely covered by using turn table.



Integration of Multiple Range Images

• Modelling process:1. Take 4 – 8 side view range images from different poses on a turn

table2. Take images on each top and bottom views3. Estimate rotation angles of top and bottom views images



Transformation from

Registration Process

Object –Centred Representation

Original range image

Cylindrical or

Spherical coordinateMerged Data

Select coordinate frame



Global Registration

• Register an image with the merged data.• Possible error accumulation can be avoided



Outline

• Introduction• Range Image Registration• Integration of Multiple Range Images• Results and Discussion• Conclusion



Results and Discussion

The wood blob, the plaster tooth and the derived models for them

Intensity

Wire frame

Rendered



Conclusion

• A new surface model constructing method, based on registering range images from multiple views.

• Image registration is achieved by minimising a distance measure function.

• The representation scheme may not be powerful enough to directly accommodate more complex object.



References

1. Chen, Y.; Medioni, G., "Object modeling by registration of multiple range images," Robotics and Automation, 1991. Proceedings., 1991 IEEE International Conference on , vol., no., pp.2724,2729 vol.3, 9-11 Apr 1991

2. Michael Potmesil, “Generating Models for Solid Objects by Matching 3D surface Segments”. In Proceedings of the International Joint Conference on Artificial Intelligence, page 1089-1093, Karlsuche, West Germany, August 1983



Thank you

object modelling by registration of multiple range images

Documents

object modeling

multiple views

range image registration

movement limitation

goal of registration

jirapong manitinstitute

grard medioni

complete model