Optimal Estimation of Optimal Estimation of Deterioration from Deterioration from

Diagnostic Image SequenceDiagnostic Image SequenceDimitry Gorinevsky

Consulting Professor of EE, Stanfordgorin@stanford.edu

joint work with Seung Jean Kim and Stephen Boyd, EE, Stanford

Shawn Beard, Acellent Inc. Grant Gordon, Honeywell

Fu-Kuo Chang, AA, Stanfordsupported by NSF GOALI grant

Algorithmic Approach

• Least Squares (Gauss, circa 1800)

– broadly used in aerospace systems• Quadratic Programming – QP (circa 1980)

– somewhat used in aerospace systems

min2 →−bAx

0 subject tomin2

1

≤−

→+

dCxxqHxx TT

2March 2007

Application

• Structural Health Monitoring – SHM – Inspections make 25% of aircraft life cycle cost

• SHM sensing system– not in this work

• Signal processing, estimation

3March 2007

Application: Structural Monitoring

A300-600 crash in NYAA 587 B-52 introduced in 1955

4March 2007

New Aircraft• SHM Drivers

– Maintenance cost reduction

– Composite aircraft• Safety-critical

function: decides flightworthiness

5March 2007

Airbus 380Airbus 380

Boeing 787Boeing 787

SHM Integration

• Honeywell avionics integration

Operations

Maintenance

Fleet

Data

On Ground Central

Maintenance Computer

CMC

Aircraft IVHM

B777, Primus,…

OnboardFlight Ctrls

Member Systems

…

Propulsion Utilities

Avionics Cabin

SHM

Supply Chain

Gorinevsky, Gordon, Beard, Kumar & Chang, IWSHM’056March 2007

Diagnostic Image Data

7March 2007

• Example SHM data• Series of images

– 3-D data• Damage trend

distorted by noise

Problem: Estimate underlying damage

Bayesian Estimation

• Data:

• Bayes rule

• Observation model:• Prior (trend model):• Maximum A posteriori Probability estimation

)}(),...,1({}{)}(),...,1({}{TXXX

TYYY==

Observed

Underlying trend

c XPXYPYXP ⋅⋅= })({}){|}({}){|}({

}){|}({ XYP})({XP

( ) ( ) min}{log}{|}{log →−−= XPXYPL8March 2007

Monotonic walk model

• Monotonic walk (univariate)

• A simple and fundamental model.• Monotonic deterioration, never an

improvement• Palmgren-Miner rule - linear damage

accumulation

0)(),()()1( ≥+=+ tvtvtxtx

Gorinevsky, ACC’04; Samar, Gorinevsky, & Boyd, CDC04,05,06

9March 2007

MAP Problem

10March 2007

• Data model

• MAP loss index

• This is a QP problem

( ) ( ) min)1()()()(2 21

2 →−−+−= ∑∑==

T

t

T

ttxtxrtxtyqL

)()()( tetxty +=

})({XP

}){|}({ XYP

)()()1( tvtxtx +=+)exp(~ xrrv ⋅−⋅

),0(~ 1−qNe

0≥v

0)1()( subject to ≥−− txtx

11March 2007

First-order Monotonic Regression

SAMPLE NUMBER

0 20 40 60 80 100 120 140 160 180−2

0

2

4

6

LS SMOOTHING FOR FIRST−ORDER REGRESSION MODEL

SAMPLE NUMBER

0 20 40 60 80 100 120 140 160 180−2

0

2

4

6

FIRST−ORDER MONOTONIC REGRESSION

SAMPLE NUMBER

r = 1r = 10r = 100

Bayesian Model of Image Data

Observation model

– X(t) is an underlying damage map

– Y(t) is a diagnostic image– B is a blur operator

)()(**)( tetXBtY +=

}){|}({ XYP

),0(~)( 1−qNte jk

12March 2007

Markov Random Field Prior

t-1

t n1

n2

Prior model – Damage accumulation

– Spatial continuity (regularization)

)()()1( tVtXtX +=+

})({XP

)exp(~)( xrrtv jk ⋅−⋅0)( ≥tv jk

)()(**)( tWtXRtX +=

),0(~)( ΞNtwjk

13March 2007

3-D MAP Estimation

• MAP Loss Index

∑=

−=T

tF

tXBtYL1

2)(**)(21

min)1()()(**),(21

21

1→−−++ ∑∑

==

T

t

T

ttXtXrtXRtX

)1()( subject to −≥ tXtX

Observation ModelObservation Model

Prior ModelPrior Model

• This is a QP problem (very large)

14March 2007

Tuning of the MRF Model

• Tune the regularization operator R• Steady state solution analysis

min**,** 2 →+− XRXXBYF

**** eXBY +=

( )( ) *

1*

1

e

XXTT

TTe

BRBB

BBRBB−

−

++

+= Signal RecoverySignal Recovery

Noise AmplificationNoise Amplification

15March 2007

Spatial Frequencies

• LSI approximation – True for a large image, away from the

boundaries • 2-D Fourier analysis

( ) ),(~),(OTF 2121 vvXvvbX =B

( ) ),(~),(OTF 2121 vvXvvrX =R

xvvFvvr ⋅= ),(),( 2121 array of FIR array of FIR coefficientscoefficients

(decision vector)(decision vector)16March 2007

Spatial Frequency Domain Design

• Spatial frequency-domain specs

• LP problem for designing a FIR operator R

svrvbvb

vbvb≤−

+1

)()()()()(

*

*

noisegvrvbvb

vb≤

+ )()()()(

*

Signal Recovery Signal Recovery ErrorError

Noise AmplificationNoise Amplification

min→sGorinevsky, Boyd, & Stein, ACC’03, IEEE TAC

17March 2007

Dynamical Loopshaping• Low frequency:

– high loop gain L(ω) ≈ 1/ω

– performance

• High frequency:– small loop gain L(ω)– robustness

• Bandwidth – performance

achieved in a limited frequency band: ω ≤ ωB

0 dB

ωgc

|L(iω)|

ωB

Performance

Robustnessdynamicalbandwidth

dynamical frequency

Difficult problem modern robust control18March 2007

Regularization Operator Design • Spatial loopshaping – noncausal• Solving LP on a spatial frequency grid

19March 2007

0 0.5 1 1.50

0.5

1

MA

GN

ITU

DE

SIGNAL GAIN IN THE ESTIMATOR

0 0.5 1 1.5

0

0.5

1

FREQUENCY = (v12 + v

22)1/2

MA

GN

ITU

DE

NOISE GAIN IN THE ESTIMATOR

MRF operator R

Regularization Operator Design

Noise Noise Amplification Amplification

LPLP--based based design

20March 2007

design

Number of Number of FIR delays FIR delays

FIR FIR operator operator BB

FIR FIR operator operator RR

Optimization Problem

∑=

+−=T

tF

tXRtXtXBtYL1

2 )(**),()(**)(21

min)1()(, →−+ XTXr 1)1()( subject to −≥ tXtX

• Sparse large-scale QP• Very structured• 1-2 millions of variables and constraints• Does not fit into memory with standard

sparse QP solvers21March 2007

Optimizer Software

• Algorithm and software developed by Seung Jean Kim, EE

• Matlab implementation solves the 1M size problem in a few tens of mins on a PC

• Can be yet sped up by a factor of 10

Kim, Koh, Lustig, Boyd, & Gorinevsky, IEEE JSTSP - submitted22March 2007

Optimization Approach• Interior-point method:

– Uses logarithmic barrier functions – Requires 10-50 steps till convergence

• Preconditioned Conjugate Gradient (PCG) method to solve for the step direction

• Iterative approximate solution using PCG– Not exact, but provides a search direction– Requires a good preconditioner (got one)

gX −=∆⋅H

23March 2007

Experimental SHM Data

• Aircraft skin panel

• Acellent SHM system

• Impacts at the same location

• Data collected between the impacts

Impact locationImpact location

24March 2007

SHM Data

25March 2007

After 3 impacts

After 6 impacts

After 9 impacts

20 deg C 40 deg C

After 3 impacts

After 6 impacts

After 9 impacts

20 deg C 40 deg C

• Collected at 20°C and in a thermal chamber at 40°C

• Partial temperature compensation applied

26March 2007

Filtering Results

• Experimental data: 701,784= 171x171x24 pixels

• QP-based trending

• Recovers the 3-D signal

Conclusions

• Practical approach to SHM data trending • Design parameters:

– Blur model B– Noise amplification gain (in design of R)

• Off-line software: LP-based design of R• On-line software: specialized QP solver

Accumulated Sequence of Observed Damage Maps {Y(1), …, Y(N)}

Optimization Problem for Spatio-temporal Filtering and Trending

Specialized Large-scale Sparse QP Solver

Sequence of Deblurred and Denoised Maps {X(1), …, X(N)}

New structure damage

map

27March 2007