webinar on introduction to 3d gis data gepsus set• vermessung avt gmbh – founded in 1970 –...

www.terra-messflug.at

Webinar on Introduction to 3D GIS Data – GEPSUS set

2-5 December 2014

Trento, Imst, Podgorica

• Presentation of Terra Messflug and AVT

• Introduction to aerial photogrammetry

• Workflow for the production of Trento dataset

• Webinar organization

• Discussion

Outline

02.12.2014 GEPSUS Introduction 2

• Vermessung AVT GmbH – Founded in 1970

– Head-office in Imst, Austria, 65 employees

– Activities: cadaster, topography, geodesy, photogrammetry, aerial and terrestrial laser scanning, geoinformation

Vermessung AVT


• Terra Messflug – 100% subsidiary of Vermessung AVT

– Focused in aerial digital image acquisition and processing in Central Europe

• Aircraft: Cessna T303

• Optical sensor: UltraCam Xp

• GSD da 3 cm a 30 cm

– Other services:

• Oblique image acquisition

• Lidar data acquisition (partners)

• Lidar data processing

• Thermal image acquisition

• Thermal image processing

Terra Messflug


• Photogrammetry is the science of making measurements from photographs

– 'photo': light

– 'gram‘: drawing

– 'metry‘: measurement

• Measurements: coordinates, distances, areas, volumes

• Photogrammetric products:

– Digital Terrain / Surface models

– Orthoimages

– 2D / 3D Vectors

– Other GIS data

What is photogrammetry


• According to the platforms, we distinguish: – Spaceborne photogrammetry: the

camera is mounted on a spacecraft

– Aerial photogrammetry: the camera is mounted on an airplanes, helicopter of UAV

– Close-range (terrestrial) photogrammetry: the camera is close to the object (hand, tripod)

What is photogrammetry


Optical sensors on airborne platforms


• Analogue cameras: – Images are stored on films

– Films are developed

– Images are printed on paper or dia

– Image are processed using digital/analytical image stations

• Digital cameras: – Images are acquired using CCD or CMOS sensors

– Images are stored on a digital device

– Images are processing using digital photogrammetric stations

Analogue / Digital


PA

ST

T

OD

AY

• The optical sensor consists of: – Optical system (lenses)

– One or more CCD or CMOS matrices (frames) in the focal plane

– In case of more matrices, the single images are assembled

– Multiple bands for multispectral acquisition

Frame cameras


• For the extraction of 3D information (planimetric and height), stereo photogrammetry is used

• Stereo images: images of the same object acquired from different directions -> cameras are separate by a base B

Stereo photogrammetry


O1: position of camera 1 O2: position of camera 2 P´: homologous point P in image 1 P´´: homologous point P in image 2 B: base of the stereopair

• In aerial photogrammetry: in blocks – Min along-flight overlap between images: 60%

– Min side overlap between strips: 30%

How are stereo images acquired?


strip

strip

strip

strip

Orientation = „Reconstruction of the rays from the object point P to the image point P´, P´´, etc. ”

In general, to extract 3D information from images, we need to know the position and viewing direction of the camera during image acquisition.

Orientation: Interior O. and Exterior O.

with:

Interior Orientation = Reconstruction of the path of the ray inside the

acquisition device = camera geometry

Exterior Orientation = Reconstruction of the position during image

acquisition = camera position and attitude

What is sensor orientation


1. Describe the relationship between image and ground coordinates of the same point through equations that contain image and ground point coordinates, interior and exterior orientation parameters -> collinearity equations

2. Write collinearity equations for a certain number of points with known ground coordinates (ground control points - GCPs) and other tie points (TPs) with unknown ground coordinates

3. Build a system of equations (2 collinearity equations for each point). The unknowns of the system are parameters describing the exterior and interior orientation of the camera

4. Solve the system with least-square adjustment using GCPs and TPs. The images are oriented.

5. Once the images are oriented, we can calculate the 3D coordinates of any homologous point in the images

How do we orient images


Assumption:

- Images are the result of a central projection

- Images are generated on the focal plane of the camera

Geometric model and interior orientation


A

D

B

C

A’

D’

B’

C’

LENS (projection

centre)

OBJECT SPACE

Focal place

IMAGE SPACE

Focal length

Principal Point PP

)()()(

)()()(

)()()(

)()(`

C33C23C13

C32C22C120

C23C23C13

C31C21C110

ZZrYYrXXr

ZZrYYrXXrfyy

ZZrYYrXXr

ZZrYYrXXrfxx

R(C,C,C): rotation matrix from image space to object space containing orientation angles

CCC ZYX : camera position in object space

y]x[ : image coordinates

with: X

Y

Z x

y z

P

PC(XC, YC, ZC)

P‘

C

C C

x0, y0 : principal point; f : focal length

f

y

x

kR

Z

Y

X

Z

Y

X

CCC

C

C

C

),,(

• Relation between image and ground coordinates

External orientation


Collinearity equations

x0, y0

Ground reference system

Focal plane

Lens or mirror system

Image acquisition device,

e.g. frame- or line- sensor

Object, e.g. earth surface

c = Focal distance

Source of error:

-Imperfectness during the manufacturing process of

the device and the lens system, e.g. imperfect

grinding and decentration of the lenses from the

optical axis

-Unevenness of the focal plane

Additional sources of error for digital sensors:

-Pixel spacing and positioning

-Problems during the read-out process

-Noise

-Blooming

Problem: Image acquisition using a camera or other devices is not a perfect projection

=> The ray from the object space to the image space is not straight. ε1 ε2

ε3

Solution: Modeling and consideration of the errors:

ε1 = Lens distortions

ε2 = Shift of the principle point

ε3 = Error in the focal length

Interior orientation


- Position of principal point(s)

- Change of focal length

- Radial distortion

- Decentering

- Shear

- Scale

• Additional parameters for the estimation of

x21222412P sxaypyx2px2rxkrkrxf

fxx

yP syaxpyrpyxykrkryf

fyy

2

22

12

4

1

2 22

pp xxx

pp yyy

2p

2p

2 yxr



y)ZZ(r)YY(r)XX(r

)ZZ(r)YY(r)XX(rfyy

x)ZZ(r)YY(r)XX(r

)ZZ(rYYr)XX(rfxx

C33C23C13

C32C22C120

C23C23C13

C31C21C11`0

• Extended collinearity equations

pp xxx

pp yyy

2p

2p

2 yxr



x21222412P sxaypyx2px2rxkrkrxf

fxx

yP syaxpyrpyxykrkryf

fyy

2

22

12

4

1

2 22

with

• Laboratory calibration aims at estimated the interior orientation parameters of the camera.

• The calibration target includes a number of signalized points with known position (GCPs), distributed in the camera field of view at different depth.

Camera calibration


• Images are taken and the parameters estimated in a bundle adjustment

• A set of parameters is estimated for each lens

C:/Users/Daniela Poli/Documents/DanielaPoli/Sales/Trento_Graphithec_2014/GEPSUS/Bid_2014_03/Webinar/CameraCalibration_UCxp_V80_ShortVersion.pdf

• During image acquisition, geopositioning and inertial systems are mounted and synchronized with the camera. In case of Terra: – Double frequency GPS NovAtel OEM I/IV

– Inertial system IMU-Ild from IGI (Ingenieur-Gesellschaft für Interfaces mbH, http://www.igi.eu/ ), Germany

• GPS / IMU measure directly the trajectory of the airplane

Approximation for exterior orientation


• After the flight, the trajectories are processed with differential technique (DGPS) with master stations (max distance from images 50km) and the approximate position and attitude of each image are estimated.

http://www.igi.eu/http://www.igi.eu/

• The geometric calibration of IMU system is executed through these steps: – Visual test for damages of the case and connectors

– Verification of the faultless function of each subsensor

– Determination of special coefficients for trajectory modelling.

• In case of Terra, the calibration is executed by IGI and coefficients for data processing with AEROoffice are provided

• Statistics on the accuracy of the position and attitude estimation are given.

IMU calibration


C:/Users/Daniela Poli/Documents/DanielaPoli/Sales/Trento_Graphithec_2014/GEPSUS/Bid_2014_03/Webinar/IGI_IMUKalib_2013-12-11.pdf

Workflow for Trento dataset

Pre-flight

• Flight planning

• Permission requests

• Camera calibration

• GPS/IMU calibration

Image acquisition

• Aerial flight

• Data storage

Image preparation

• Radiometric processing

• Trajectory estimation

Image orientation

• GCPs measurement

• Aerial triangulation

3D Modeling

• Dense image matching

• 3D point cloud generation

• Classification

• Generation of DSM and DTM

Orthophoto production

• True-orthophoto

Flight planning – operational area


Home airport:

• Zell am See (Austria)

• Aalen (Germany)

Operating distance:

• 1.500 km / 2.000 km

Imst

Aircraft Cessna C303


Characteristic

values:

• Twin-engine

• 125 – 400 km/h

• 7.600 m max.

flying altitude

• 6.5 hours max.

endurance

Chief-Pilot: Marko Schnelzer • Aircraft owner

• Pilot and camera operator for > 20 years

• > 7.000 operational hours of photo flight missions

• One of the most experienced pilots in alpine environments

Pilot: Mario Königstorfer • Pilot for 15 years

• > 2.700 operational hours of photo flight missions

License: Air Worthiness Certificate – CAMO

In Italy: license from ENAC and ENAV national authorities

Pilot’s experience and license


• Image acquisition is planned in order to fulfill the project requirements: – Ground sample distance: 10cm

– Overlap: 80% along, 60% side. This is required for dense image matching and the generation of a dense 3D point cloud

• A rough digital terrain model is used to calculate the image footprints on the terrain

Flight plan


• Effect of DTM during flight planning

Flight plan


Rovereto, 2013, GSD:5cm, overlap 80%, 60%

without terrain model

(flat terrain supposed) with terrain model

Flight plan Trento

02.12.2014 GEPSUS Introduction 28 Coverage of flight lines

Camera: • UltraCam Xp (digital frame camera)

Data-units: • 3 data-units (fully redundant storage on 2 data-units)

• Storage capacity for 6.500 images each

INS / IMU: IGI AeroControl IMU II-d 256 Hz

GPS: NovAtel OEM V (multi-frequency-receiver)

VRF, IFR, Radar

CCNS4

Telescope and T-AS

Equipment in aircraft


Camera: UltraCam XP


Specifications:

• 17.310 x 11.310 pixels

(196Mp)

• max. GSD: 2.8 cm

• Focal length: 100.5 mm

• Geometric accuracy: 2 µm

• Channels: PAN, RGB, NIR

• Colour depth: >12 bit

• Colour depth during

processing: 8 or 16 bit

• Regular calibration and

maintenance

Photo flight mission


• Consideration of all relevant air-traffic regulations during flight planning and during flight execution

Flight management: Air-traffic control


• Good weather conditions are prerequisite for good images

• Up to hourly checks of weather conditions (forecast and real-time)

• allows for efficient medium-term flight mission planning

• allows for quick reaction in changing weather conditions

Flight management: weather conditions

02.12.2014 GEPSUS Introduction 33 Weather forecast: e.g. Deutscher Wetterdienst

Real-time weather check using webcams

Trento flight in brief


Digital camera Vexcel Ultracam-Xp Average GSD 10 cm Along-track overlap 80% Side overlap 60% Number of strips 9 Number of images 397 Spectral channels Red, Green, Blue, Near Infrared Radiometric resolution 12 bit Date of acquisition 23 September 2013 Area covered by the flight 55 km² Urban area covered by the flight 18 km²

• Manpower, hard- and software

• 16 experts involved in flight management, image processing, photogrammetry and subsequent analysis

• High performance fiber-channel IBM SAN storage (ca. 100 T) • High performance computation cluster • ca. 25 IBM Quad-/dualcore high-performance PCs • Calibrated and 3D monitors

• Topoflight • Microsoft UltraMap • Waypoint GrafNav, IGI AeroOffice • Inpho Geo-Imaging (3 licenses) • OrthoMaster, Orthovista, Orthovista Seam Edit, Applications Master • Match-T, SCOP++ • Global Mapper, ArcGIS • DAT/EM Summit Evolution (2 licenses) • CAD: Bentley Microstation (4 licenses), Autodesk AutoCad Map (6 licenses)

Overview infrastructure Imst


Hardware:

• High-performance IBM SAN storage system (100 TB)

• High-performance computer cluster for image processing (up to 32 CPU-cores in parallel)

• Transportable data storage devices (12 TB)

Data storage and processing


• Differential correction of GNSS positioning using permanet receiving GNSS stations and processing of INS

Computation of flight trajectory


Utilised software:

• AeroOffice

• GrafNav

Software:

• Microsoft UltraMap

• DragonFly image format: visualisation of huge image blocks (> 5000 images)

• project-based-colour-balancing to achieve homogeneous radiometric setting for the entire project area

Radiometric processing


• Manual radiometric adjustments are done for each image to optimise image quality

• Utilised software: UltraMap Radiometry

Manual radiometric adjustment


PBCB substantially improves image block homogeneity

Project Based Colour Balancing (PBCB)


before PBCB after PBCB

• Each project is evaluated regarding radiometric homogeneity of the entire project area

Quality control in geometrical context


Image block consisting of >140 images

• GSD = 10 cm

• Overlap = 60%, 30%

• The purpose of aerial triangulation is to improve the trajectory measured by the instruments onboard and estimate the position and attitude of each image.

• Ground control points were surveyed with centimeter precision and used as input for aerial triangulation

• Software used: Match-T by Inpho

Aerial triangulation


• Software suite: Inpho Geo-Imaging 5.6



Licences and

components

• 3 licences

• OrthoMaster

• Orthovista

• Orthovista Seamedit

• Ground control points for absolute orientation of images



Procedure

1. Planning in office

2. Field measurements

3. Computation

• Quality control



Procedure

• Statistics from software

• GCPs

• Image matching: establishment of consistent correspondences between primitives extracted in two or more images, and estimation of the corresponding 3D coordinates, using collinearly models or projective models.

• Many algorithms proposed, classified, for example, in – Stereo matching / multi-view matching

– Area-based / Feature-based matching

– Sparse / dense matching

Image Matching


• Innovative methodology for the generation of dense 3D point clouds

• Point density up to pixel level – Trento dataset: up to 100 pt/m2 (matching every pixel)

• Competitive to Lidar, in particular in urban areas

• Software used: SURE by nFRAMES (http://nframes.com/ )

• The 3D points are triangulated and interpolated in regular surfaces (DSM)

Dense Image Matching (DIM)


http://nframes.com/

3D point clouds


DSM

DTM

Surface measurement and reconstruction:

• DEM = Digital Elevation Model = any kind of elevation model

• DSM = Digital Surface Model = surface with vegetation and man-made objects

• DTM = Digital Terrain Model = surface without vegetation and man-made objects

Digital surface models


DIM in Trento


DIM in Trento (with RGB information)


• Ground / above-ground point classification

• Generatin of digital surface model (DSM) and digital terrain model (DTM)

DSM / DTM


Aerial image:

Not possible to measure correct

distances, due to perspective effects

Orthophoto:

Measured distances, areas

and positions are like on a

map

Aerial image:

Overlapping of image and vector data

is not consistent / correct

Orthophoto:

Correct overlap between image and vector

data

Orthophoto generation


Aero triangulation

• Highly precise matching and orientation of images (incorporating GPS/INS and ground GCPs)

Digital elevation model

• DTM for standard orthophotos

• DSM for true-orthophotos

Orthophoto

• Geometric and radiometric corrected image block

Orthophoto computation


• Quality control

Orthophoto


Problems in DEMs and necessary corrections: Manual seam line edit:

• Influence of sidelap between adjacent flights lines on image leaning

Orthophoto


2009 - 60/30 %

Example from orthophoto production in Denmark

2013 - 80/60 % 2009 - 60/30 % 2013 - 80/60 %

• Images are projected on the DSM

• In addition to terrain distortions, building distortions are corrected

• Occluded areas between buildings are detected and filled by taking the missing image parts from neighboring images.

• DSM-based orthophotos can be produced only if the aerial images have sufficient overlap in along and side direction. This is the case of Trento images.

DSM-based orthophotos


• Difference between…

DSM/DTM-based orthophotos


…DTM-based orthophotos

• Images rectified on terrain

• Buildings facades and roofs appear

displaced

… and DSM-based orthophotos

• Images rectified on terrain and objects on

terrain

• Building roofs appear on the footprints

• Good for 3D visualization

Use of 2 (or more) oriented images and measure (MANUALLY or AUTOMATICALLY) identical

points in the images for the determination of 3D object coordinates of the homologues points

Known: orientation of (at least) two images

Measured: image coordinates of a given point in both images (x1, y1, x2, y2)

measurement of image coordinates manually

y

ZZrYYrXXr

ZZrYYrXXrcyy

x

ZZrYYrXXr

ZZrYYrXXrcxx

)()()(

)()()(

)()()(

)()()(

033023013

032022012

0

033023013

031021011

0

ij

j

ij

s

l

ss

ll

h

Av

v

0

0

or

X

Y

Z

P (X, Y, Z)

x

y

2 x

Result: X, Y, Z coordinates

3D measurements


Principle:

• Oriented and overlapping images are visualized using a Digital Photogrammetric Station (DPS)

• Using special devices (shutter-glasses or polarized glasses), the human eye gets a 3D

impression of the object, which can be used to measure 3D coordinates

Advantages:

• Low redundancy

• Explicit measurements of significant and important object details

• Low data volume

• Post processing needless (blunder and errors free)

• High accuracy

Disadvantages:

• High amount of manual work

Commercial DPS packages:

e.g. Leica Photogrammetric Suite (LPS), SocetSet, ZI-

Image Station, etc.

3D stereo measurements


• Example stereoscopic analysis (Norway)

Stereoscopic plotting


Hardware:

• 2 workstations with 3

moitors each: 2 x 27´´ 3D

and 1 touch-screen monitor

• Wire-less shutter glasses

DTMs and DSMs (LAS and GeoTIFF formats)

Orthophotos (GeoTIFF format)

GIS

• Import geodata

• Visualization

• Overlapping

• Processing

• Analysis

• Export

Photogrammetric products for GEPSUS


Operation possible in GIS environment

Reference system: UTM32N, WGS84

Webinar organization - calendar


Day Time H Topic Responsible Material

2/12/2014 9:30-11:30 2 Introduction to aerial

photogrammetry: instruments,

calibration and processing workflow

Terra Messflug PPT

2/12/2014 12:30–14:30 2 Flight planning and image acquisition Terra Messflug Demo

3/12/2014 9:30-11:30 2 Trajectory estimation, Ground survey

and Aerial Triangulation

Terra Messflug Demo

3/12/2014

12:30–14:30 2 Orthophoto and True-Orthophotos

generation

Terra Messflug PPT + Demo

4/12/2014

9:30-11:30 2 3D point cloud extraction and digital

surface modelling

Terra Messflug PPT + Demo

4/12/2014

12:30–14:30 2 3D point classification Terra Messflug PPT + Demo

5/12/2014

9:30-11:30 2 Case study using Trento dataset Graphitech Demo

• Combination of theory and live demonstrations

• Demonstration with Trento dataset and examples from other projects

• Proposed exercises with Trento dataset and Global Mapper http://www.bluemarblegeo.com/products/global-mapper-download.php

• Skype platform

Webinar organization - method

http://www.bluemarblegeo.com/products/global-mapper-download.phphttp://www.bluemarblegeo.com/products/global-mapper-download.phphttp://www.bluemarblegeo.com/products/global-mapper-download.phphttp://www.bluemarblegeo.com/products/global-mapper-download.phphttp://www.bluemarblegeo.com/products/global-mapper-download.phphttp://www.bluemarblegeo.com/products/global-mapper-download.php