chapters 1 –8: overview - purdue university

CE 59700: Digital Photogrammetric Systems Ayman F. Habib1

Chapters 1 – 8: Overview• Chapter 1: Introduction• Chapters 2 – 4: Data acquisition• Chapters 5 – 8: Data manipulation

– Chapter 5: Vertical imagery– Chapter 6: Image coordinate measurements and refinement– Chapter 7: Mathematical model and bundle block adjustment– Chapter 8: Theory of orientation & photogrammetric

triangulation

• This chapter will cover more details about the bundle adjustment procedure


CE 59700: Chapter 9

Photogrammetric Bundle Adjustment


Overview• Photogrammetric point positioning• Photogrammetric bundle adjustment

– Structure of the design and normal matrices– Sequential estimation of the unknown parameters– Sequential building of the normal equation matrix– Rules of thumb for the expected precision from a bundle

adjustment procedure• Special cases:

– Resection, intersection, and stereo-pair orientation


xaya

axc

zcycPerspective Center

Xm

Zm

Ym

AXA

YA

ZA

( , , , , )( , , , , )

a x A A A

a y A A A

x f X Y Z IOPs EOPsy f X Y Z IOPs EOPs

==

Photogrammetric Point Positioning


Camera

A

a 'a

GNSS /INS Y

Z

XY

Z

X

Indirect Geo-referencingDirect Sensor Orientation

Photogrammetric Point Positioning


Collinearity Equations: Stochastic Model

11 21 31

13 23 33

12 22 32

13 23 33

2 1

( ) ( ) ( )( ) ( ) ( )( ) ( ) ( )( ) ( ) ( )

~ (0, )

A O A O A Oa p x x

A O A O A O

A O A O A Oa p y y

A O A O A O

xo

y

r X X r Y Y r Z Zx x c dist er X X r Y Y r Z Zr X X r Y Y r Z Zy y c dist er X X r Y Y r Z Z

eP

eσ −

− + − + −= − + +− + − + −− + − + −= − + +− + − + −


Distortion Model• distx = Δx Radial Lens Distortion + Δx Decentric Lens Distortion

+ Δx Atmospheric Refraction + Δx Affine Deformation+ etc….

• disty = Δy Radial Lens Distortion + Δy Decentric Lens Distortion+ Δy Atmospheric Refraction + Δy Affine Deformations + etc….


Distortion Parameters

)....(

)....(6

34

22

1DistortionLensRadial

63

42

21DistortionLensRadial

+++=Δ

+++=Δ

rkrkrkyy

rkrkrkxx

2 2 2 2D ecen te rin g L en s D is to rtio n 3 1 2

2 2 2 2D ecen te rin g L en s D is to rtio n 3 1 2

(1 ) { ( 2 ) 2 }

(1 ) {2 ( 2 )}

x p r p r x p x y

y p r p x y p r y

Δ = + + +

Δ = + + +

where: r = {(x - xp)2 + (y - yp)2}0.5

p

p

yyyxxx

−=

−=


Collinearity Equations: Involved Parameters• Image coordinates (xa, ya)• Ground coordinates (XA, YA, ZA)• Exterior Orientation Parameters (XO, YO, ZO, ω, φ, κ)• Interior Orientation Parameters:

– xp, yp, c – Camera-related distortion parameters that compensate for

deviations from the assumed perspective geometry• Radial lens distortion (RLD), Decentering lens distortion (DLD), Affine

deformation (AD)


Bundle Block Adjustment


Bundle Block Adjustment

60% Overlap and 20% side lap


Bundle Block Adjustment• Direct relationship between image and ground

coordinates• We measure the image coordinates in the images of the

block.• Using the collinearity equations, we can relate the image

coordinates, the corresponding ground coordinates, the IOPs, and the EOPs.

• Using a simultaneous least squares adjustment, we can solve for the:– The ground coordinates of tie points,– The EOPs, and– The IOPs (Camera Calibration Procedure).


Bundle Block Adjustment: Concept• The image coordinate measurements and the IOPs define

a bundle of light rays.• The EOPs define the position and the attitude of the

bundles in space.• During the adjustment: The bundles are rotated (ω, φ, κ)

and shifted (Xo, Yo, Zo) until:– Conjugate light rays intersect as well as possible at the

locations of object space tie points.– Light rays corresponding to ground control points pass through

the object points as close as possible.


Bundle Block Adjustment: Concept

Ground Control PointsTie Points


Least Squares Adjustment

vectornoise the ofmatrix covariancevariance vectornobservatiotheingcontaminatnoise

unknownsofvectormatrixdesign

vectornobservatio

nnPnemxmnA

nyPeexAy

o

o

×××××+=

−

−

12

12

11

1),0(~

σ

σ

Gauss Markov ModelObservation Equations


Least Squares Adjustment

{ }

)(/)~~(ˆˆ~

)(ˆ)(ˆ

2

12

1

mnePexAye

PAAxDPyAPAAx

To

To

TT

−=−=

==

−

−

σ

σ


Non Linear System

o

o

o

X

o

oX

o

o

oX

o

XaA

XaYy

exAy

eXXXaXaY

X

eXXXaXaY

XaeXaY

∂∂=

−=

+=

+−∂∂=−

+−∂∂+≈

−+=

)(:Where

)()(

parametersunknowntheforvalueseapproximatis:Where

terms)orderhigherignore(We)()(

ExpansionSeriesTayloruseWefunctionlinearnontheis)(

)(

• Iterative solution for the unknown parameters• When should we stop the iterations?


Example (4 Images in Two Strips)

1

3 4

5 6

2 1

3 4

5 6

2

7

5 6

3 4

8 7

5 6

3 4

8

Control PointTie Point

I II

III IV


Balance Between Observations & Unknowns• Number of observations:

– 4 x 6 x 2 = 48 observations (collinearity equations)• Number of unknowns:

– 4 x 6 + 3 x 4 = 36 unknowns• Redundancy:

– 12• Assumptions:

– IOPs are assumed to be known and errorless.– Ground coordinates of the control points are errorless.


Structure of the Design Matrix (BA)• Y = a(X) + e e ~ (0, σ2 P-1)

• Using approximate values for the unknown parameters (Xo) and partial derivatives, the above equations can be linearized leading to the following equations:

• y48x1 = A48x36 x36x1 + e48x1 e ~ (0, σ2 P-1)


Structure of the Design Matrix

y =

I II III IV 3 4 5 6

A =


Structure of the Normal Matrix

I II III IV

3 4 5 6

=

2212

1211

NNNN

T


Sample Data

• 2 cameras.• 4 images.• 16 points.

• All the points appear in all the images• Two images were captured by each camera


Structure of the Normal Matrix: Example

N C


Observation Equations

),0(~ 12111

−×××× += PeexAy onmmnn σ

[ ] 12

1211

122111

123

116

2316

1232311616

××

××

+

=

++=

×

×

××

××××

nn

nn

exx

AAy

exAxAy

m

m

mnmn

mmnmmn

• n ≡ Number of observations (image coordinate measurements)• m ≡ Number of unknowns:

• m1 ≡ Number of images 6 m1 (EOPs of the images)• m2 ≡ Number of tie points 3 m2 (ground coordinates of tie points)• m = 6 m1 + 3 m2


Normal Equation Matrix

[ ]

=

=

=

=

=

×

×

××

××

×+

+×+

123

116

21

23231623

23161616

2121

2

1

2

1

2

11)36(

2212

1211

212

1)36()36(

m

m

mmmm

mmmm

CC

PyAPyA

yPAA

C

NNNN

N

AAPAA

N

T

T

T

T

mm

T

T

T

mmmm


Normal Equation Matrix• N11 is a block diagonal matrix with 6x6 sub-blocks along

the diagonal.• N22 is a block diagonal matrix with 3x3 sub-blocks along

the diagonal.

=

×

×

×

×

××

××

123

116

123

116

23231623

23161616

2

1

2

1

2212

1211

ˆˆ

m

m

m

m

mmmm

mmmm

CC

xx

NNNN

T

• Question: Under which circumstances will we deviate from this structure?


Reduction of the Normal Equation Matrix

12312323231161623

11612323161161616

2222112

1212111

ˆˆ

ˆˆ

×××××

×××××

=+

=+

mmmmmmm

mmmmmmm

CxNxN

CxNxNT

( )( ) ( )( )

123231616161161616116

116161616231232316161616232323123

12312323231232316161611616161623

2121

1111

111

11

111221

121

1112222

22222121

1111

1112

ˆˆ

ˆ

ˆˆ

××××××

×××××××××

×××××××××

−−

−−−

−−

−=

−−=

=+−

mmmmmmmmm

mmmmmmmmmmmmmmm

mmmmmmmmmmmmmm

xNNCNx

CNNCNNNNx

CxNxNNCNNTT

T

• Solving for x2 first:

• 3m2 < 6m1• Remember: N11 is a block diagonal matrix with 6x6 sub-blocks

along the diagonal.



12312323231161623

11612323161161616

2222112

1212111

ˆˆ

ˆˆ

×××××

×××××

=+

=+

mmmmmmm

mmmmmmm

CxNxN

CxNxNT

( )( ) ( )( )

116162323231232323123

123222323161161623232323161616116

11611616232323123232323231161616

1121

2221

222

21

221211

121

2212111

11121

2221

2212111

ˆˆ

ˆ

ˆˆ

××××××

×××××××××

×××××××××

−−

−−−

−−

−=

−−=

=−+

mmmmmmmmm

mmmmmmmmmmmmmmm

mmmmmmmmmmmmmm

xNNCNx

CNNCNNNNx

CxNNCNNxN

T

T

T

• Solving for x1 first:

• 6m1 < 3m2• Remember: N22 is a block diagonal matrix with 3x3 sub-blocks

along the diagonal.



( ) 112

1221211

21 1623232323161616116

}ˆ{ −−×××××

−= To mmmmmmmmm

NNNNxD σ

( ) 112

1111222

22 2316161616232323123

}ˆ{ −−×××××

−=mmmmmmmmm

NNNNxD Toσ

• Variance covariance matrix of the estimated parameters:


Building the Normal Equation Matrix• We would like to investigate the possibility of

sequentially building up the normal equation matrix without fully building the design matrix.

• (xij, yij) image coordinates of the ith point in the jth image

[ ]ij

i

j

ijijij

ijiijjijij

exx

AAy

exAxAy

122

12112

12221112

13

16

3262

13321662

××

××

+

=

++=

×

×

××

××××



[ ]

[ ]

=

=

+

=

×

×

×

××

×

×

×

×

××××

××××

×

×

×

×

××

×

×

×

×

××

ijij

ijij

i

j

ijijijij

ijijijij

ij

ij

ij

i

j

ijij

ij

ij

ij

i

j

ijijij

yPAyPA

xx

APAAPAAPAAPA

yPAA

xx

AAPAA

exx

AAy

ijT

ijT

ijT

ijT

ijT

ijT

ijT

T

ijT

T

122

121

2

1

2212

2111

122

1

2

121

2

1

122

12112

23

26

13

16

32236223

32266226

23

26

13

16

3262

23

26

13

16

3262



192

1

192

1

992212

1211

122

121

2

1

2212

2111

13

16

23

26

13

16

32236223

32266226

×××

×

×

=

=

×

×

×

×

×

×

××××

××××

ij

ij

i

j

ijij

ijij

ijij

ijij

i

j

ijijijij

ijijijij

CC

xx

NNNN

yPAyPA

xx

APAAPAAPAAPA

T

ijT

ijT

ijT

ijT

ijT

ijT

• Note: We cannot solve this matrix for the:• The Exterior Orientation Parameters of the jth image, and• The ground coordinates of the ith point.



=

×

×

×

×

××

××

123

116

123

116

23231623

23161616

2

1

2

1

2212

1211

ˆˆ

m

m

m

m

mmmm

mmmm

CC

xx

NNNN

T

• Question: How can we sequentially build the above matrices?• Assumption: All the points are common to all the images.


N11 - Matrix

=

=

=

=

=

×

2

1

2

3

2

2

2

1

)1616(

111

111

111

111

11

000

000

000

000

m

i

m

i

m

i

m

i

im

i

i

i

mm

N

N

N

N

N

• If all the points are not common to all the images:• The summation should be carried over all the points

that appear in the image under consideration.


N22 - Matrix

=

=

=

=

=

×

1

2

1

3

1

2

1

1

)2323(

122

122

122

122

22

000

000

000

000

m

j

m

j

m

j

m

j

jm

j

j

j

mm

N

N

N

N

N

• If all the points are not common to all the images:• The summation should be carried over all the images

within which the point under consideration appears.


N12 - Matrix

=×

12131211

32232313

22322212

12312111

)2316(

12121212

12121212

12121212

12121212

12

mmmmm

m

m

m

mm

NNNN

NNNNNNNNNNNN

N

• If point “i” does not appear in image “j”:• (N12)ij = 0


C - Matrix

=

=

=

=

=

×

2

1

2

3

2

2

2

1

116

11

11

11

11

1

m

i

m

i

m

i

m

i

im

i

i

i

m

C

C

C

C

C

=

=

=

=

=

×

1

2

1

3

1

2

1

1

123

12

12

12

12

2

m

j

m

j

m

j

m

j

jm

j

j

j

m

C

C

C

C

C

• Once again, we assumed that all the points are common to all the images.


Precision of Bundle Block Adjustment• The precision of the estimated EOPs as well as the

ground coordinates of tie points can be obtained by the product of:– The estimated variance component, and– The inverse of the normal equation matrix (cofactor matrix).

• The precision depends on the following factors:– Geometric configuration of the image block

• Base-Height ratio– Image scale– Image coordinate measurement precision


Precision of Bundle Block Adjustment• Precision of a single model: If we have

– Bundle block adjustment with additional parameters that compensate for various distortions

– Regular blocks with 60%overlap and 20% side lap– Signalized targets

space image in thegiven are valuesprecision Thesecameras)(SWA distance principalcameratheof%004.0

cameras) WA and(NA distance principalcameratheof%003.03

±=±=±=

Z

Z

XY m

σσ

μσ


Camera Classification

α

• α < 75° Normal angle camera (NA)• 100° > α > 75° Wide angle camera (WA)• α > 100° Super wide angle camera (SWA)


Precision of Bundle Block Adjustment

yY

xX

cZcZ

σσ

σσ

=

=

X

Z

Y

x

zy

PC

AXA

YA

ZA

X = x * Z / cY = y * Z / c



A

a a´

x x´

B

c

H

x

Px = x - x´

a

Px / B = c / HH = B c / Px

Flight Direction ≡ x-axis

Vertical Precision



xpZ BZ

cZ σσ =

Bc

Z

Vertical Precision


Advantages of Bundle Block Adjustment• Most accurate triangulation technique since we have direct

transformation between image and ground coordinates.• Straight forward to include parameters that compensate for

various deviations from the collinearity model.• Straight forward to include additional observations:

– GNSS/INS observations at the exposure stations– Object space distances

• Can be used for normal, convergent, aerial, and close range imagery

• After the adjustment, the EOPs can be set on analog and analytical plotters as well as digital photogrammetric workstations for compilation purposes.


Photogrammetric Compilation


Disadvantages of Bundle Block Adjustment• Model is non linear: approximations as well as partial

derivatives are needed.• Requires computer intensive computations• Analog instruments cannot be used (they cannot measure

image coordinate measurements).• The adjustment cannot be separated into planimetric and

vertical adjustment.


Bundle Adjustment: Final Remarks• Elementary Unit: Images• Measurements: Image coordinates• Mathematical model: Collinearity equations• Instruments: Comparators, analytical plotters, and

Digital Photogrammetric Workstations (DPW)• Required computer power: Very large• Expected accuracy: High


Special Cases• Resection• Intersection• Stereo-pair orientation• Relative orientation (Discussed in Chapter 8)

– Dependent Relative Orientation (DRO), and– Independent Relative Orientation


Resection• We are dealing with one image.• We would like to determine the EOPs of this image

using GCPs.• Q: What is the minimum GCP requirements?

– At least 3 non-collinear GCPs are required to estimate the 6 EOPs.

– At least 5 non-collinear (well distributed in 3-D) GCPs are required to estimate the 6 EOPs and the 3 IOPs (xp, yp, c).

• Critical surface:– The GCPs and the perspective center lie on a common

cylinder.


Resection

interior orientation

exterior orientation


Resection - Critical Surface

• Question: Which parameter of the EOPs cannot be determined?


Intersection• We are dealing with two images.• The EOPs of these images are available.• The IOPs of the involved camera(s) are also available.• We want to estimate the ground coordinates of tie points

in the overlap area.• For each tie point, we have:

– 4 Observation equations– 3 Unknowns– Redundancy = 1

• Non-linear model: approximations are needed


Intersection

Object Point (A)

PCPC

a´a


Intersection: Linear Model

BOl Or

Vl Vr

ZYX



−−−

=

−−−

=

−−−

=

cyyxx

RV

cyyxx

RV

ZZYYXX

B

pr

pr

r

pl

pl

l

OO

OO

OO

rrr

lll

lr

lr

lr

),,(

),,(

κφω

κφω

μ

λ

• These vectors are given w.r.t. the ground coordinate system.



−−−

−

−−−

=

−−−

+=

cyyxx

Rcyyxx

RZZYYXX

VBV

pr

pr

pl

pl

oo

oo

oo

rl

rrrlll

lr

lr

lr

),,(),,( κφωκφω μλ

• Three equations in two unknowns (λ, μ).• They are linear equations.



−−−

+

=

cyyxx

RZYX

ZYX

pl

pl

O

O

O

lll

l

l

l

),,( κφωλ

−−−

+

=

cyyxx

RZYX

ZYX

pr

pr

O

O

O

rrr

r

r

r

),,( κφωμ

Or:


Stereo-pair Orientation• Given:

– Stereo-pair: two images with at least 50% overlap– Image coordinates of some tie points– Image and ground coordinates of control points

• Required:– The ground coordinates of the tie points– The EOPs of the involved images

• Mini-Bundle Adjustment Procedure


Stereo-pair Orientation• Example:

– Given:• 1 Stereo-pair• 20 tie points• No ground control points

– Question:• Can we estimate the ground coordinates of the tie points as well as the

exterior orientation parameters of that stereo-pair?– Answer:

• NO


Summary• Photogrammetry: Definition and applications• Photogrammetric tools:

– Rotation matrices– Photogrammetric orientation: interior and exterior orientation– Collinearity equations/conditions

• Photogrammetric bundle adjustment– Structure of the design and normal matrices

• Special cases:– Resection, intersection, and stereo-pair orientation

chapters 1 –8: overview - purdue university

Documents