automated reconstruction of industrial sites frank van den heuvel tahir rabbani
TRANSCRIPT
![Page 1: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/1.jpg)
Automated Reconstruction of Industrial Sites
Frank van den HeuvelTahir Rabbani
![Page 2: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/2.jpg)
Overview
• Introduction• Automation: how does it work?• Sample project off-shore platform• Accuracy• Future• Conclusions
![Page 3: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/3.jpg)
The groupPhotogrammetry & Remote Sensing• “Development of efficient techniques for the
acquisition of 3D information by computer-assisted analysis of image and range data“
![Page 4: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/4.jpg)
The projectServices and Training through Augmented Reality (STAR)
• EU fifth framework – IST programme• “Develop new Augmented Reality techniques
for training, on-line documentation, maintenance and planning purposes in industrial applications”
• AR-example: virtual human in video
![Page 5: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/5.jpg)
The projectServices and Training through Augmented Reality (STAR)
• Partners: Siemens, KULeuven, EPFL, UNIGE, Realviz
• TUDelft: “Automated 3D reconstruction of industrial installations from laser and image data”
![Page 6: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/6.jpg)
Automated reconstruction procedureOverview (1/3)• Segmentation• Grouping points of surface patches
![Page 7: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/7.jpg)
Automated reconstruction procedureOverview (2/3)• Segmentation• Grouping points of surface patches
• Object Detection• Finding planes and cylinders
![Page 8: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/8.jpg)
Automated reconstruction procedureOverview (3/3)• Segmentation• Grouping points of surface patches
• Object Detection• Finding planes and cylinders
• Fitting• Final parameter estimation
![Page 9: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/9.jpg)
Segmentation – step 1
• Estimation of surface normals using K-nearest neighbours (here K=10 points)
![Page 10: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/10.jpg)
Segmentation – step 2
• Region growing using:• Connectivity (K-nearest
neighbours) • Surface smoothness
(angle between normals)
![Page 11: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/11.jpg)
Detection – Planes
• Plane detection using Hough transform• Find orientation as maximum on Gaussian
sphere
![Page 12: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/12.jpg)
Detection – Cylinders
• Cylinder detection using Hough transform in 2 steps:• Step 1: Orientation
![Page 13: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/13.jpg)
Detection – Cylinders
• Cylinder detection using Hough transform in 2 steps:• Step 1: Orientation
![Page 14: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/14.jpg)
Detection – Cylinders
• Cylinder detection using Hough transform in 2 steps:• Step 1: Orientation (2 parameters)• Step 2: Position and Radius (3 parameters)
u,v search space at correct Radius
![Page 15: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/15.jpg)
Example: detection of two cylinders
• Point cloud segment
![Page 16: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/16.jpg)
Example: detection of two cylinders
• Surface normals
![Page 17: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/17.jpg)
Example: detection of two cylinders
• Normals on Gaussian sphere
![Page 18: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/18.jpg)
Example: detection of two cylinders
• Orientation of first cylinder (next: position)
![Page 19: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/19.jpg)
Example: detection of two cylinders
• Remove first cylinder points from segment
![Page 20: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/20.jpg)
Example: detection of two cylinders
• Procedure repeated for second cylinder
![Page 21: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/21.jpg)
Example: detection of two cylinders
• Result: two detected cylinders
![Page 22: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/22.jpg)
Fitting
• Complete CSG model + constraint specification
• Final least-squares parameter estimation of CSG model
![Page 23: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/23.jpg)
Fitting
• Final least-squares parameter estimation of CSG model• Minimise sum of squared distances• Enforce constraints
![Page 24: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/24.jpg)
Results on platform modelling
• Scanned by Delftech in 2003• Subset of 17.7 million points used by TUD:• Automated detection of 2338 objects• R.M.S. of residuals 4.3 mm
![Page 25: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/25.jpg)
Results on platform modelling
![Page 26: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/26.jpg)
Results on platform modellingStatistics
• Points: 17.7 million• Points in segments: 14.2 million(80%)• Points on objects:9.3 million (53%)• Detected:• Planar patches: 946• Cylinders: 1392
• Data reduction:• Object parameters 9798• 500 Mb to 0.1 Mb
![Page 27: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/27.jpg)
0 2 4 6 8 10 12 14 16 18 200
10
20
30
40
50
60
70
80
90
100
Residual (mm)
%ag
e of
poi
nts
Cumulative histogram of residuals
Results on platform modelling Accuracy• Residual analysis:
• RMS: 4.3 mm• 83% < 5 mm• 96% < 10 mm
![Page 28: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/28.jpg)
Accuracy
• Data precision:• Scanner: 6 mm (averaging: 3 mm)
• Scanner dependent
• Model precision:• Discrepancies models - real world: 0.1-10 mm ?
• Limited production accuracy• Deformations• Imperfections in segmentation
![Page 29: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/29.jpg)
Accuracy
• Object deformation or segmentation limitations?
Fitting after initial segmentation
Max.residual 21 mm
Fitting after rejecting large residuals
Max. residual 9 mm
![Page 30: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/30.jpg)
Future – automation
• Reconstruction using laser data:• Segmentation, primitive detection (available)• Correspondence between primitives >
registration• Model improvement:
• Constraint detection (piping structure)• Recognition of complex elements in a database
• Integration with digital imagery
![Page 31: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/31.jpg)
Future – integration with imagery
• Instrumentation developments• Scanners with integrated high-resolution digital
camera
• Accuracy improvement• Imagery complementary: Laser for surfaces, image for
edges• Integrated fitting of models to laser and image data
![Page 32: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/32.jpg)
Future – integration with imagery
• Instrumentation developments• Scanners with integrated high-resolution camera
• Accuracy improvement• Imagery complementary: Laser for surfaces, image for
edges• Integrated fitting of models to laser and image data
• Flexibility of image acquisition: Completeness• Non-geometric information (What is there?)
![Page 33: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/33.jpg)
Future – integration with imagery
![Page 34: Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani](https://reader030.vdocuments.net/reader030/viewer/2022032607/56649ece5503460f94bdb430/html5/thumbnails/34.jpg)
Conclusions
• Bright future for automation using laser data• More research to be done:• Automated registration• Integration with digital imagery• Using domain knowledge for automated
modelling:• Closer connection to the model users needed:• Domain knowledge for automation• Utilisation of research results