http:// 1 clustering features to recognize structure for purposes of 3d reconstruction from lidar...
TRANSCRIPT
http://www.nshorter.com 1
Clustering Features to Recognize Structure for Purposes of 3D Reconstruction from LiDAR
and other sources
Nicholas Shorter
Machine Learning Research Group Seminar Series
Email: [email protected]
Website: http://www.nshorter.com
http://www.nshorter.com 2
Presentation Layout
1. LIDAR Overview
2. Problem Statement
3. Applications
4. Existing Approaches
5. Data Features
6. Masters Research
7. Masters Research Results
http://www.nshorter.com 3
LIDAR Overview• Data Collection
– Plane Equipped with GPS, INS & LIDAR – LIDAR – Light Detection and Ranging (active sensor)
• Collection of 3D points• Laser sent out from Emitter, reflects off of Terrain, Returns to
Receiver– Receiver measures back scattered electromagnetic radiation
(laser intensity)• Time Difference Determines Range to Target
http://www.toposys.com/
http://www.nshorter.com 4
LIDAR Captured Characteristics
• Range to Target (elevation from INS & LiDAR)
• Longitude and Latitude (GPS)• First and Last Return Pulses
– First – shrubbery, vegetation, power lines, birds and buildings
– Last – buildings (unless vegetation is really dense, then vegetation too)
• Returned Laser Intensity
http://www.nshorter.com 5
LiDAR Noise• Essential to understand forms of noise corrupting
data in order to realize how to deal with the noise• Geolocation results from LiDAR, GPS and INS
sensor systems– Accuracy Limitations – Offset and Drift in both GPS and INS– Misalignment between INS and LiDAR
• Atmosphere – Intensity and Path Distortion• Shadowing Effect from Tall buildings• Artifacts from non uniform sampling from
multiple strips
http://www.nshorter.com 6
Problem Statement
• Input– LIDAR (Light Detection and Ranging) Data
• Collection of Irregularly Distributed 3-Dimensional Points
– Aerial Photograph
• Output – Semi to complete Automatic (minimal user
intervention) development of 3-Dimensional Virtual Model
http://www.nshorter.com 7
Applications for 3D Reconstruction
• Military Applications– Automatic Target Recognition– Reconstructed Models of Opponent Terrain
(UAV?)
• Tourism/Entertainment– Virtual Walkthrough of Theme Park
• Commercial– Change Detection (Natural Disasters)– Network Planning for Mobile Communication– Noise Nuisance (Universal Studios, 408
Expressway - walls)– Urban Planning
http://www.nshorter.com 8
Existing 3D Reconstruction Research
• Data Sources– LIDAR – Aerial Imagery– GIS Ground Plans
• Model Based Reconstruction– Pre-defined models with parameters– Minimize error between models and data
• Data Driven– Group Coplanar Pts– Identify Break Lines– Derive Model to Minimize Error
http://www.nshorter.com 9
Triangulation Based Methods
• Triangular Irregular Network (TIN)– Series of Non-Overlapping Triangles Modeling
given Surface
• TIN 3D Reconstruction Methods– Clustering approach
• Spherical Normal Vectors of Triangles
– TIN region growing approach• Merge Triangles to Same Region if Normal Vectors
within Threshold
http://www.nshorter.com 10
Triangulated Irregular Networks (TIN)
• Advantages of TIN– Generation Fully Automatic– Space Uniquely Defined– Cells Indexed
• Spatial Searches, Triangle Elevation Differences
• Disadvantages of TIN– Noise Causes Triangle Normal Vectors to deviate from Ideal
http://www.nshorter.com 11
Existing 3D Reconstruction Methods
• Most still under development• Most Methods Use ‘Grided’ (Interpolated)
LIDAR Data – Advantages
• Less Computationally Complex
• DTM & DSM Thresholding to distinguish Building from Non Building
• Use of additional conventional methods
– Disadvantages• Decrease in Accuracy
• Uncertainty from Building and Ground Interpolation
http://www.nshorter.com 12
Building Detection Features• Investigate Returned Laser Intensity• Vegetation Points
– Significant first and last return elevation diff.– Corresponding green aerial image color– Nearest points have diff. elev. or adjacent triangles
have significantly diff. norm vectors• Common Building Points
– Spatially close in terms of long. and lat.– Bounded by aerial img. edges and exterior wall tri.– Bounded building does not contain terrain pts – or - – Triangulation of all points, building points connected
http://www.nshorter.com 13
Building Reconstruct Cluster Features
• Surface change - normal vector orientation difference between adjacent triangles
• (X,Y,Z) - Longitude, Latitude, Elevation• Edge - Aerial imagery edge detection• Color - Aerial imagery corresponding color
(building surface differs from clutter)• Triangle planar coef, pt. height diff., same normal
vector, or planar equation• Feedback – difference between reconstruction and
raw LiDAR and Aerial Image Edges
http://www.nshorter.com 14
Masters Research
• Greedy Insertion Triangulation – Implemented Noise Filtering Technique
• Proposed FSART normal vector clustering
• Proposed Planar Regression to combat Category Proliferation
• Realized simple planar reconstruction algorithm (MSEE Thesis)
http://www.nshorter.com 15
Fuzzy SART Clustering
• Fuzzy Simplified Adaptive Resonance Theory (SART) Advantages – – Not as Sensitive to Input Order (CG FA)
– No “Mandatory” Preprocessing Techniques
– Activation Function Forms Spherical Arcs
– Activation Function is Measure not Estimate
– Long Term Weights have Intuitive Meanings
– Only 2 User Defined Parameters with Clear Meanings• Tau = Necessary Time to Learn Pattern
• VDMT = Vigilance Parameter
http://www.nshorter.com 16
Fuzzy SART Activation Function
• Activation Function
• Module Degree of Match
• Angle Degree of Match
• Pattern Encoding Shape
http://www.nshorter.com 17
Fuzzy SART Preprocessing
• Scaled Each Dimension to Same Max Value– Each Dimension Receives Equal Weight
• Translated Input Dimension Space to Center at Origin
http://www.nshorter.com 18
Planar Regression
• Equation of a Plane:–
• Solving for Z:–
• Substitutions for Coefficients:–
• Use Method of Least Squares and Choose Estimates for Coefficients to minimize SSE:–
http://www.nshorter.com 19
Planar Regression Continued…
• Derivative with Respect to Each Coefficient– – –
• System of Equations:
–
–
–
http://www.nshorter.com 20
Planar Regression Pictorial
• VDMT biased against absorbing points into an existing cluster (VDMT = 0.7)
• Planar Regression implemented to join like clusters below a perpendicular threshold
http://www.nshorter.com 21
Data Set Specifications
• Donated by Simone Clode & Franz Rottensteiner• Collected by AAMHatch• LIDAR - Approximately 1 point per 1.3m2 point
density (need at least 1 pt. per 1.5m2 for 3D rec.)– First & Last Return Pulses & Laser Intensity Recorded
• Aerial Photography – 15 cm pixel resolution• Data depicts urban and residential area of
Fairfield, Australia
http://www.nshorter.com 22
Results
• 4 Actual Building Test Cases Presented– Aerial Photographs Included for Visual
Comparison– Outputs for Various Stages of Algorithm
presented with different levels of optimization
http://www.nshorter.com 23
Aerial Photographs
http://www.nshorter.com 24
3D Scatter Plots
http://www.nshorter.com 25
Result Set 1
• No Scaling, Shifting, Filtering nor Planar Regression
http://www.nshorter.com 26
Result Set 2
• Scaling, Shifting, No Filtering Nor Planar Regression
http://www.nshorter.com 27
Result Set 3
• Scaling, Shifting, Filtering, No Planar Regression
http://www.nshorter.com 28
Result Set 4 – Roof Segmentation
• Scaling, Shifting, Filtering, Planar Regression
http://www.nshorter.com 29
Result Set 5 - Final
http://www.nshorter.com 30
Conclusions – Fuzzy SART Preprocessing
• Fuzzy SART Preprocessing Techniques Proposed– Scaling Dimensions to same Maximum giving
each dimension equal weight– Translating Input Space to Further Separate
Data– Above Measures Improved Clustering
http://www.nshorter.com 31
Conclusions – Heuristic Processing
• Triangle Differences for Determining Roof Triangles
• Planar Regression for Triangulation Clustering Proposed– Results Show Improvement Using Method
http://www.nshorter.com 32
Conclusions
• ART TIN Based Clustering Showed Promising Results for Commercial Buildings
• Like Other Methods, Had Difficulties with House Roof Segmentation– Houses fractions of a size of Buildings– Houses had as much as 5 times Roof Planes as
Buildings
http://www.nshorter.com 33
Acknowledgements• Data Contributors
– Dr. Simone Clode, Dr. Franz Rottensteiner, AAMHatch
– Mr. John Ellis, AeroMap– Mr. Steffen Firchau, TopoSys– Mr. Paul Mrstik, Terra Point
• Advisor (PhD Committee Chair)– Dr. Takis Kasparis
• PhD Committee Members– Dr. Michael Georgiopoulos , Dr. Georgios
Anagnostopoulos, Dr. Andy Lee, and Dr. Wasfy Mikhael