uav-based damage mapping and the fp7-project …
TRANSCRIPT
![Page 1: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/1.jpg)
UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT RECONASS
Norman Kerle. ESA DepartmentITC-OOA-Group
(with materials by Markus Gerke, Jorge Fernandez and Anand
Vetrivel)
![Page 2: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/2.jpg)
2
ITC/ University Twente
International Institute for Geoinformation science and Earth Observation – independent for 60 years
Faculty of University of Twente 3 years ago
Various degree courses and certificates in Disaster management Earth sciences Geoinformatics Governance Land administration Natural resources Urban planning Water resources
GLOSSY meeting - Wageningen - 9 May 2014
1 20 40 976Netherlands
1105Indonesia
6010
Student numbers
100 400300200
![Page 3: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/3.jpg)
3
ITC/ University Twente
Houses the United Nations University- ITC Centre for Spatial Analysis and Disaster Risk Management Training, education and curriculum
development
Knowledge development and research collaboration
Advisory services
In collaboration with many partners
GLOSSY meeting - Wageningen - 9 May 2014
www.unu-drm.nl
![Page 4: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/4.jpg)
4
Me @ ITC
Geographer with study background in Hamburg (Germany), Ohio State (US) and Cambridge (UK)
Since early 1990s work in the disaster field, with focus on remote sensing
PhD in volcano remote sensing (lahars)
Advanced image analysis, and focus on object-oriented analysis (OOA), increasingly with oblique data from UAV
GLOSSY meeting - Wageningen - 9 May 2014
http://www.itc.nl/ooa-grouphttp://www.itc.nl/about_itc/resumes/kerle.aspx
![Page 5: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/5.jpg)
Following disasters, rapid and comprehensive damage information is critical
Assessment is challenging
RATIONALE & MOTIVATION
5GLOSSY meeting - Wageningen - 9 May 2014
![Page 6: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/6.jpg)
RATIONALE
6
VERTICAL IMAGE-BASED ASSESSMENT HAS LIMITATIONS
Port-au-Prince, Haiti
![Page 7: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/7.jpg)
Oblique data are promising, but traditionally have limitations
Early experiments with Pictometry data that may overcome those
Not a UAV, but a MAV, 5 views
Use of data ideally in an automated manner, using object-basedapproaches
RATIONALE
7
PROMISE OF OBLIQUE DATA
GLOSSY meeting - Wageningen - 9 May 2014
![Page 8: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/8.jpg)
5 digital cameras (1 nadir, 4 oblique)
GSD 15cm (nadir images, flying height ca. 1000m)
overlap for stereo coverage
PICTOMETRY
8
BASICS OF THE SYSTEM
GLOSSY meeting - Wageningen - 9 May 2014
![Page 9: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/9.jpg)
Which Pictometry features allow damage mapping? (elevation,geometrics features, textures, etc.)
Identification of planar (=intact) and spectrally homogenous facades androof sections
Supervised classification of image segments
Effect of training on the classification
OBJECTIVES
9GLOSSY meeting - Wageningen - 9 May 2014
![Page 10: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/10.jpg)
Orthorectification of oblique images and disparity images (depth maps)
Feature extraction from ortho images and DSM
Assumption: intact buildings have planar surfaces
Find evidence for planarity through
Disparity maps (geometric homogeneity)
Image information (radiometric homogeneity)
3D point cloud segmentation to find planar surfaces
DAMAGE MAPPING: METHODOLOGY
10GLOSSY meeting - Wageningen - 9 May 2014
![Page 11: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/11.jpg)
5 classes were defined:a. Intact roofb. Broken roof/ rubblec. Intact facaded. Bare grounde. Vegetation
Training data by two interpreters Classification using a total of 22 features
in machine learning approach Final step: combination per building
EARTHQUAKE DAMAGE ASSESSMENTCLASSIFICATION OF IMAGE SEGMENTS
11
Direct damage indicators
D1
D2
D3
D4
D5
![Page 12: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/12.jpg)
EARTHQUAKE DAMAGE ASSESSMENTCLASSIFICATION PER BUILDING
12
Test on a 6 block area of Port au Prince, Haiti
Depth images from stereo pairs (for different views)
3D point cloud
GLOSSY meeting - Wageningen - 9 May 2014
![Page 13: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/13.jpg)
PARALLAX DISPARITY FOR IMAGE PAIR – MERGE OF MULTIPLE VIEWS
13
Planar areas have limited disparity
Rubble and broken features appear noisy
GLOSSY meeting - Wageningen - 9 May 2014
![Page 14: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/14.jpg)
DAMAGE MAPPING RESULTS: WESTERN VIEW EXAMPLE
14
Intact roofBroken roof/ rubbleIntact facadeBare groundVegetation
Ortho image
Detected damage
GLOSSY meeting - Wageningen - 9 May 2014
![Page 15: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/15.jpg)
15
Results using trainingdata by 2 analysts
For details seePE&RS paper, Gerke & Kerle, 2011
![Page 16: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/16.jpg)
ACCURACY ASSESSMENT & CONCLUSIONS
16
Overall accuracy of approx. 70% (63% per building) Errors at image top where heights were missing
Problem: damage indicators do not add up linearly
GLOSSY meeting - Wageningen - 9 May 2014
![Page 17: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/17.jpg)
ONGOING RESEARCH: USING UAV DATA & ECOGNITIONFOR THE CLASSIFICATION INSTEAD OF TRAINING DATA
17
Limitations: Expensive Pictometry data Single view, limited spatial resolution, limited control Adding damage indicators Avoid use of training data
Solution: Get your own UAV and eCognition
Aibot X6 (Aibotix )GLOSSY meeting - Wageningen - 9 May 2014
![Page 18: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/18.jpg)
OBJECTIVES
18
Obtain data with a pole-based camera and a UAV Identify damage indicators in multi-perspective images with OOA Create a rough 3D building model with images and damage features
draped Decision tree for semantic reasoning
GLOSSY meeting - Wageningen - 9 May 2014
![Page 19: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/19.jpg)
DATA ACQUISITION
19GLOSSY meeting - Wageningen - 9 May 2014
![Page 20: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/20.jpg)
DATA COLLECTION IN ITALY, GERMANY, NETHERLANDS
20GLOSSY meeting - Wageningen - 9 May 2014
![Page 21: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/21.jpg)
METHODOLOGY
Buildinglevel
Collapsed(D5)
Non-collapsed(D1-D4)
D5
Building’ssurroundings
Rubblepiles
Roof Facade
Stan-dingCollapsed
D4 Non-inclined
No rubblepiles
Facade’sgeometry
Inclined
Point cloud
Segmentation
UAV images
Colors indicate orientation: red is horizontal, blue vertical.Here change in color signals sloping roof
GLOSSY meeting - Wageningen - 9 May 2014 21
![Page 22: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/22.jpg)
METHODOLOGY
22
UAV images
Buildinglevel
Point cloud
Segmentation
Collapsed(D5)
Non-collapsed(D1-D4)
Building’ssurroundings
D5 D4
Rubblepiles
No rubblepiles
Facade’sgeometry
Roof Facade
StandingCollapsed
Non-inclinedInclined
Extracted features:
3D constructvisualization Random
scenarios
Per-view scoreD1-D2-D3-D4
Per-building decision tree
Uncertainty evaluation
D4 D3 D2 D1
Facadeassessment
Expertanalysis
OOA
View/facadelevel
D1
Standing
Non-inclined
![Page 23: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/23.jpg)
23
DAMAGE DETECTION WITH OOA
GLOSSY meeting - Wageningen - 9 May 2014
![Page 24: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/24.jpg)
24
PER-BUILDING ASSESSMENT
Challenges:• How to process damage
evidence?• How to deal with occlusion?
GLOSSY meeting - Wageningen - 9 May 2014
![Page 25: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/25.jpg)
25
EXPERT-BASED ASSESSMENT
ID EMS-98 scoreFacade 1 D0Facade 2 D3Façade 3 OccludedFaçade 4 D4Roof 1 D1
![Page 26: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/26.jpg)
EXPERT-BASED ASSESSMENT
26
Problem: simple building primitive Preferable: based on actual DSM
GLOSSY meeting - Wageningen - 9 May 2014
![Page 27: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/27.jpg)
ITC, University of Twente 27
Multi-view images from UAVof complex structures
3D point clouds Segmentation for 3D damage assessment
Rich 3D model data from, e.g., from ItalyChallenge is how to use such complex information
GLOSSY meeting - Wageningen - 9 May 2014
![Page 28: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/28.jpg)
ITC, University of Twente 28
Multi-view images from UAV
Demolished factory building. 300 images taken, combined into a 3D point cloud.
GLOSSY meeting - Wageningen - 9 May 2014
![Page 29: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/29.jpg)
RECONASS
29
2 FP7 projects on damage mapping
RECONASS: Reconstruction and Recovery Planning: Rapid and Continuously Updated Construction Damage, and Related Needs Assessment (start: 1 Dec 2013; 3 ½ years)
INACHUS: Technological and Methodological Solutions for Integrated Wide Area Situation Awareness and Survivor Localisation to Support Search and Rescue Teams (start: late 2014; 4 years)
GLOSSY meeting - Wageningen - 9 May 2014
![Page 30: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/30.jpg)
OBJECTIVES OF RECONASS
30
Provide rapid assessment on structural integrity of high-value buildings (ministries, hospitals), based on wireless sensor networks, finite element modelling, and external surveillance with UAV technology
3.5 years, 10 partners (German THW as disaster response organization, ITC for geoinformatics, and 8 engineering organizations)
ITC covers one work package (Synergistic Damage Assessment with Air- and Space-borne Remote Sensing), with full-time AIO researcher
4 objectives
GLOSSY meeting - Wageningen - 9 May 2014
![Page 31: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/31.jpg)
OBJECTIVES
31
Detailed and element-specific damage assessment on per building level using multi-view oblique aerial images
Synergistic use of UAV and wireless sensor network for complete, reliable and accurate damage assessment.
Integration of satellite- and airborne data-based information for synoptic damage assessment
Extension of sensor network with chemical and biological sensors for critical hazard monitoring
GLOSSY meeting - Wageningen - 9 May 2014
![Page 32: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/32.jpg)
TASKS
32
- Consider specific damage- Joint use of spectral/ color image information
(texture, minute features), and 3D point cloud spalling
Crack
GLOSSY meeting - Wageningen - 9 May 2014
![Page 33: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/33.jpg)
TASKS
33
- Relate to information from accelerometers, strain sensors, position tags and thermometers
- Relate to data or derived deformation/instability information?- Semantic relationship between sensor reading and damage as
observed outside?- Use UAV to patch gaps/ failed nodes?
![Page 34: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/34.jpg)
TASKS
34
- First damage data available from satellites
- Limitations: vertical view, lack of familiarity of analyst with disaster area
- Use UAV information to validate and calibrate the damage mapsDamage map of Port-au-Prince, Haiti
Source: SERTIT
GLOSSY meeting - Wageningen - 9 May 2014
![Page 35: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/35.jpg)
TASKS
35
- If more than 1 building is equipped with sensors, extra insights can be generated
- piggy bag additional sensors, such as chemical or biological
- Combine with environmental/atmospheric data to assess such threats
![Page 36: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/36.jpg)
FROM RECONASS TO INACHUS
36
4 year project with > 20 partners Provide search & rescue forces with current and increasingly-
detailed intelligence on the situation Nested integration initial satellite assessment Hotspot identification Scenario modelling using synthetic data UAV deployment (area-based monitoring) Dasymetric population modelling Sensors and robots
GLOSSY meeting - Wageningen - 9 May 2014
![Page 37: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/37.jpg)
GLOSSY meeting - Wageningen - 9 May 2014
37
Legislation remains unclear (no-go in various countries)
Reliability is a major issue
FINAL NOTES
![Page 38: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/38.jpg)
GLOSSY meeting - Wageningen - 9 May 2014
38
FINAL NOTES
![Page 39: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/39.jpg)
SUMMARY
39
Good progress in image-based damage mapping (2D to 3D, single- to multi-view, cognitive and semantic analysis)
UAVs promising in term of image quality and sensor control Progress in photogrammetry and OOA has been critical FP7 projects give us space for further developments
Limitations Recognizing and understanding damage (physical vs functional) Legal issues with UAV deployment Maturity of UAVs
GLOSSY meeting - Wageningen - 9 May 2014
![Page 40: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/40.jpg)
40
Questions? For ITC’s OOA work see www.itc.nl/ooa-group Same for full references Papers also on https://www.researchgate.net/profile/Norman_Kerle For RECONASS see www.reconass.eu
Or email: [email protected]
GLOSSY meeting - Wageningen - 9 May 2014
![Page 41: UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT …](https://reader033.vdocuments.net/reader033/viewer/2022042811/6268cf6c9c877353735b730b/html5/thumbnails/41.jpg)
RECONASS team
ITC, University of Twente41
http://www.itc.nl/resumes/kerle https://www.researchgate.net/profile/Norman_Kerle Email: [email protected]
http://www.itc.nl/resumes/gerke https://www.researchgate.net/researc
her/73705446_Markus_Gerke/ Email: [email protected]
Email: [email protected]
Markus Gerke
Norman Kerle
Anand Vetrivel
GLOSSY meeting - Wageningen - 9 May 2014