Inversion via Bayesian Adaptive Multi-Modality Processing

(AMMP)

Inversion via Bayesian Adaptive Multi-Modality Processing

(AMMP)

Leslie Collins Electrical and Computer Engineering

Duke UniversityWork supported by DARPA/ARO MURI

Leslie Collins Electrical and Computer Engineering

Duke UniversityWork supported by DARPA/ARO MURI

OutlineOutline

• Problem Background and Setup• Adaptive Feature Selection• AMMP

– HSTAMIDS results– NIITEK/GEM-3 results– Georgia Tech results

• Confidence-based Fusion

• Problem Background and Setup• Adaptive Feature Selection• AMMP

– HSTAMIDS results– NIITEK/GEM-3 results– Georgia Tech results

• Confidence-based Fusion

Adaptive Feature Selection

Subsurface Sensing (Landmines, UXO, Subsurface Structures)

• Simulate seismic, EMI, and MAG data• Large number of features available• Pre-screener needed based on RDOF feature set• Issue: best features function of sensor

implementation, test site, etc.• In some scenarios, need to adaptively select

features• Ultimately: AMMP applied to results of pre-

screener

NIITEK Radar Results

• Features selected based on training from test lanes

• Variable performance on off-road scenarios

Manual Feature Selection

RDOF Feature Selection• Guide initial selection of features• Adaptively prune features based on input from

multiple sensors• Carin et al. proposed RDOF for induction sensing

– Induction, mag, seismic still produce many features– JCFO verified by Carin on acoustic data – not yet

applied to EMI/MAG/Seismic data

• In year 2, evaluated JCFO/adaptive feature selection on: – GPR and EMI for landmines– EMI and MAG for UXO– Next year: Simulated data for subsurface structures

JCFO• Classification of a feature vector, x,

performed using kernel-based technique

• For a binary classifier with labels, l, of +1 and –1, training data T and weights w

• A Laplacian sparseness prior is placed on the weights of the training samples xn

01

( ) ( , )N

n nn

c w K w=

= +∑x x x

( )

1( 1/ , , )1

( 1/ , , ) 1 ( 1/ , , )

cp le

p l p l

−= =+

= − = − =

xx T w

x T w x T w

JCFO, Cont.

feature1 feature2 feature3 feature4…… feature17

Object1:

Object2:

feature1 feature2 feature3 feature4…… feature17

… …….Object128:Object128: feature1 feature2 feature3 feature4…… feature17

Theta[1] Theta[2] Theta[3] Theta[4]…… Theta[17]

Beta[1]

Beta[2]

Beta[128]

Associate each training sample with parameter Beta, and each feature with parameter Theta

example: training data of 128 samples, 17 features for each sample

Training Objective : Estimating parameters of Theta and BetaFeature Selection: Sparsity in Theta implies that it finds only a few features highly relevant for classification. (making some Theta[i]=0)

Kernel function selection: Sparsity in Beta implies that it finds a small subset of data highly representative of the different classes (making some Beta[i]=0)

…

Find the maximum a posterior (MAP) estimate of Theta and Beta using EM algorithm

Manual Physics-Based versus Random Feature Selection

(EMI+MAG)

Physics-Based versus JCFO for EMI and MAG (Area 2)

JCFO picks 4 features and used 5 of 128 training vectors!

Physics-Based versus JCFO for EMI and MAG (Area 1)

AMMP

Traditional Versus Collaborative/AMMP Approach

Sensor Algorithm Threshold

Sensor Algorithm Threshold

Data or Feature Fusion

Algorithm Fusion

DecisionFusion

Sensor Algorithm

Traditional:

Collaborative/AMMP:

Sensor Algorithm

Adaptive Control

AMMP Bayesian ProcessingAMMP Bayesian Processing

• Two modes of adaptation– Statistical parameters tracked and updated (e.g.

covariance matrix)– Priors on uncertain parameters modified based

on context (e.g. size, depth of radar response indicates an anti-tank mine, EMI library modified accordingly)

• Two modes of adaptation– Statistical parameters tracked and updated (e.g.

covariance matrix)– Priors on uncertain parameters modified based

on context (e.g. size, depth of radar response indicates an anti-tank mine, EMI library modified accordingly)

1 1

0 0

( / , ) ( / )( )

( / , ) ( / )

f H f H d

f H f H dΛ = ∫

∫r Θ Θ Θ

rr Ω Ω Ω

AMMP for Landmine Detection AMMP for Landmine Detection

• Prior work suggests adaptively pruning EMI/MD library using signature magnitude improved processor performance: LM vs HM

• Multi-modality processing – suggests adaptively pruning EMI/MD library using

GPR magnitude: AP vs AT– suggests adaptively pruning GPR library using

EMI/MD discrimination algorithms: mine type– New work: incorporate uncertainty in pruning

algorithm into processing algorithm• Sensor fusion at data level or decision level

• Prior work suggests adaptively pruning EMI/MD library using signature magnitude improved processor performance: LM vs HM

• Multi-modality processing – suggests adaptively pruning EMI/MD library using

GPR magnitude: AP vs AT– suggests adaptively pruning GPR library using

EMI/MD discrimination algorithms: mine type– New work: incorporate uncertainty in pruning

algorithm into processing algorithm• Sensor fusion at data level or decision level

MD Signature LibraryMD Signature LibraryResponse Library

LM HMSig 1 Sig 1Sig 2 Sig 2Sig 3 Sig 3

Sig M-1Sig M

Sig N-1Sig N

APAP

AT

AT

*Sources of uncertainty

1θ

2θ

( )21

1 , 1 ,1 1

1 11 2

( / ) ( )[ ( / , ) ( )]

AP, AT

t i

i i

N

x i x j ji j

f H p f t H p tθ

θ θθ

θ θ= =

=

= =

∑ ∑r r( )

1

itN θ

Application to Field Data: HSTAMIDS

(Handheld, co-located MD and GPR)

Application to Field Data: HSTAMIDS

(Handheld, co-located MD and GPR)

A B C D E F G H I J K L1 PMD-6

1.75"CL-2 2.5"

TAB-1 1.125"

CL-5RP 3.875"

VAL-69 2.75"

CL-5RW 3.25"

CL-5RW 2.0"

TM-46 3.5"

CL-5RP 2.5" Blank CL-3

3.375"VS2.2 1.0"

1

2 CL-1 .9375"

CL-3 1.875"

CL-2 0.5"

CL-5IP 1.5"

CL-5RS 2.125"

CL-5IP 2.875"

CL-5IS 0.5"

CL-5RS 3.75"

CL-5IP 1.0" CL-4 3.0" M-14

1.6875"CL-1

0.875" 2

3VS2.2 2.875"

CL-2 1.625"

CL-3 0.25" Blank CL-4

1.875" Blank CL-3 3.5" CL-3 0.75" Blank CL-2 4.0" CL-3 2.0" CL-3

2.25" 3

4 CL-2 0.75"

CL-4 0.75"

M409 2.375"

CL-3 2.25"

TS-50 1.6875"

CL-5IW 1.5"

M409 2.5" Blank CL-1

2.25"CL-2

1.875"CL-4 1.25" Blank 4

5 CL-1 1.5" CL-1 3.125"

CL-4 4.0"

CL-4 2.25"

CL-1 1.75"

M-14 2.0625"

CL-1 0.625"

CL-2 3.75" CL-2 2.5" Blank CL-1 3.0" CL-4 3.0" 5

6 M-19 1.25" CL-4 2.5" Blank CL-1

2.75"TMA-4 2.125"

CL-3 2.25"

CL-4 3.725 Blank CL-1 4.0" CL-4 3.0" TM-62P3

3.0625"CL-2

2.9375" 6

7 CL-2 2.25"

M-14 0.5"

CL-5IS 2.5" Blank CL-4 0.5" CL-3 2.0" M409

1.3125"CL-3 2.25"

CL-2 0.75"

TS-50 2.25"

CL-3 3.25" Blank 7

8 Blank CL-3 3.5"

TS-50 1.25"

CL-2 2.875"

TM-62P3 2.125"

CL-1 2.625" CL-3 0.5" CL-3

3.25"M-19

3.8125" CL-4 0.5" CL-1 1.375"

M409 0.5" 8

9 CL-4 0.625"

M-19 2.5625"

CL-1 0.5" CL-4 1.5" CL-2 0.5" CL-1

3.625" CL-1 1.5" VS.50 2.125"

CL-4 1.25" CL-3 2.5" Blank CL-2

0.875" 9

10 TM-62P3 1.1875"

CL-4 3.25" Blank CL-5IW

1.25"CL-5RP 1.375"

CL-5IS 2.25"

TMA-4 1.25"

CL-4 2.25"

M-14 1.625"

CL-4 2.75" CL-3 0.5" CL-1 3.5" 10

11 CL-2 1.625" CL-1 2.5" CL-1

1.875"CL-5RW

2.5"VAL-69 1.75"

CL-5RS 1.875" Blank CL-2 1.75" CL-2 4.0" CL-1

1.375"M409 1.5"

TMA-4 1.375" 11

12 CL-5RP 0.875"

CL-5IP 3.625"

VS2.2 3.25"

CL-5IW 0.75"

CL-5RW 1.625"

CL-5RW 3.5"

CL-5IS 1.6875"

CL-5RW 2.5"

CL-5IS 1.25" Blank CL-2

3.25"CL-3 3.25" 12

13 TM-46 1.0625"

CL-5RS 0.75" CL-3 1.0" CL-2

1.875" CL-1 0.5" CL-2 1.875"

CL-5RW 1.0"

TM-46 2.25"

CL-5RP 3.0" Blank CL-3

1.25"CL-4

2.625" 13

14 CL-5IP 0.5"

CL-5RS 2.125"

CL-5IP 2.875"

TAB-1 2.0625"

CL-3 0.875"

CL-5IW 2.25"

CL-5RS 1.5"

CL-5IW 2.125"

TS-50 1.375"

CL-2 1.25"

M-14 1.25"

14

15 Blank VAL-69 0.75"

CL-5RP 1.25" CL-1 3.75" TS-50

0.25"CL-3

2.375"CL-4 1.25"

CL-4 3.125"

CL-4 0.875"

VS-50 1.875" CL-1 2.5" 15

16 Blank CL5-IS 3.0"

CL-3 1.75"

CL-4 1.25" Blank CL-1

2.625"TAB-1 1.125"

CL-3 3.375"

CL-1 0.75"

TAB-1 2.25"

16

17 Blank Blank CL-2 2.5" CL-2 0.5" PMD-6 2.3125" CL-4 3.0" CL-3 1.0" Blank CL-3

1.875" 17

18 Blank CL-4 0.875" Blank Blank Blank VS-50

0.5"CL-4

1.375" Blank 18

19 CL-2 3.25"

CL-2 2.375"

CL-1 1.625"

CL-2 0.75" Blank CL-3

2.25" 19

20 Blank CL-4 3.5" TMA-4 3.6.25" CL-1 4.0" Blank 20

21 Version 02822.01 (AMD) CL-1 1.0" CL-2 3.75"

CL-4 0.875"

CL-2 3.125" 21

22 Blank CL-3 2.0" TMA-4 3.375"

CL-1 2.75" 22

A B C D E F G H I J K LC

A B C D E F G H I J K L1 PMD-6

1.75"CL-2 2.5"

TAB-1 1.125"

CL-5RP 3.875"

VAL-69 2.75"

CL-5RW 3.25"

CL-5RW 2.0"

TM-46 3.5"

CL-5RP 2.5" Blank CL-3

3.375"VS2.2 1.0"

1

2 CL-1 .9375"

CL-3 1.875"

CL-2 0.5"

CL-5IP 1.5"

CL-5RS 2.125"

CL-5IP 2.875"

CL-5IS 0.5"

CL-5RS 3.75"

CL-5IP 1.0" CL-4 3.0" M-14

1.6875"CL-1

0.875" 2

3VS2.2 2.875"

CL-2 1.625"

CL-3 0.25" Blank CL-4

1.875" Blank CL-3 3.5" CL-3 0.75" Blank CL-2 4.0" CL-3 2.0" CL-3

2.25" 3

4 CL-2 0.75"

CL-4 0.75"

M409 2.375"

CL-3 2.25"

TS-50 1.6875"

CL-5IW 1.5"

M409 2.5" Blank CL-1

2.25"CL-2

1.875"CL-4 1.25" Blank 4

5 CL-1 1.5" CL-1 3.125"

CL-4 4.0"

CL-4 2.25"

CL-1 1.75"

M-14 2.0625"

CL-1 0.625"

CL-2 3.75" CL-2 2.5" Blank CL-1 3.0" CL-4 3.0" 5

6 M-19 1.25" CL-4 2.5" Blank CL-1

2.75"TMA-4 2.125"

CL-3 2.25"

CL-4 3.725 Blank CL-1 4.0" CL-4 3.0" TM-62P3

3.0625"CL-2

2.9375" 6

7 CL-2 2.25"

M-14 0.5"

CL-5IS 2.5" Blank CL-4 0.5" CL-3 2.0" M409

1.3125"CL-3 2.25"

CL-2 0.75"

TS-50 2.25"

CL-3 3.25" Blank 7

8 Blank CL-3 3.5"

TS-50 1.25"

CL-2 2.875"

TM-62P3 2.125"

CL-1 2.625" CL-3 0.5" CL-3

3.25"M-19

3.8125" CL-4 0.5" CL-1 1.375"

M409 0.5" 8

9 CL-4 0.625"

M-19 2.5625"

CL-1 0.5" CL-4 1.5" CL-2 0.5" CL-1

3.625" CL-1 1.5" VS.50 2.125"

CL-4 1.25" CL-3 2.5" Blank CL-2

0.875" 9

10 TM-62P3 1.1875"

CL-4 3.25" Blank CL-5IW

1.25"CL-5RP 1.375"

CL-5IS 2.25"

TMA-4 1.25"

CL-4 2.25"

M-14 1.625"

CL-4 2.75" CL-3 0.5" CL-1 3.5" 10

11 CL-2 1.625" CL-1 2.5" CL-1

1.875"CL-5RW

2.5"VAL-69 1.75"

CL-5RS 1.875" Blank CL-2 1.75" CL-2 4.0" CL-1

1.375"M409 1.5"

TMA-4 1.375" 11

12 CL-5RP 0.875"

CL-5IP 3.625"

VS2.2 3.25"

CL-5IW 0.75"

CL-5RW 1.625"

CL-5RW 3.5"

CL-5IS 1.6875"

CL-5RW 2.5"

CL-5IS 1.25" Blank CL-2

3.25"CL-3 3.25" 12

13 TM-46 1.0625"

CL-5RS 0.75" CL-3 1.0" CL-2

1.875" CL-1 0.5" CL-2 1.875"

CL-5RW 1.0"

TM-46 2.25"

CL-5RP 3.0" Blank CL-3

1.25"CL-4

2.625" 13

14 CL-5IP 0.5"

CL-5RS 2.125"

CL-5IP 2.875"

TAB-1 2.0625"

CL-3 0.875"

CL-5IW 2.25"

CL-5RS 1.5"

CL-5IW 2.125"

TS-50 1.375"

CL-2 1.25"

M-14 1.25"

14

15 Blank VAL-69 0.75"

CL-5RP 1.25" CL-1 3.75" TS-50

0.25"CL-3

2.375"CL-4 1.25"

CL-4 3.125"

CL-4 0.875"

VS-50 1.875" CL-1 2.5" 15

16 Blank CL5-IS 3.0"

CL-3 1.75"

CL-4 1.25" Blank CL-1

2.625"TAB-1 1.125"

CL-3 3.375"

CL-1 0.75"

TAB-1 2.25"

16

17 Blank Blank CL-2 2.5" CL-2 0.5" PMD-6 2.3125" CL-4 3.0" CL-3 1.0" Blank CL-3

1.875" 17

18 Blank CL-4 0.875" Blank Blank Blank VS-50

0.5"CL-4

1.375" Blank 18

19 CL-2 3.25"

CL-2 2.375"

CL-1 1.625"

CL-2 0.75" Blank CL-3

2.25" 19

20 Blank CL-4 3.5" TMA-4 3.6.25" CL-1 4.0" Blank 20

21 Version 02822.01 (AMD) CL-1 1.0" CL-2 3.75"

CL-4 0.875"

CL-2 3.125" 21

22 Blank CL-3 2.0" TMA-4 3.375"

CL-1 2.75" 22

A B C D E F G H I J K LC

JUXOCO Cal Grid

• 4 types metallic clutter:- CL1: < 3g- CL2: 3 to 10g- CL3: 11 to 40g - CL4: > 40g

• Non-metallic clutter: - Wood - Plastic - Stone

• Blank Ground

• 12 Mine types (2-5 of each AT, AP, LM, HM)

HM/LM MD Energy ComparisonHM/LM MD Energy Comparison

Note difference in downtrack/crosstrack extent, as well aslevel of response

Note difference in downtrack/crosstrack extent, as well aslevel of response

Example of One Feature from MDExample of One Feature from MD

Mix of AT/AP, LM/HM

Energy Detector• 100% Detection • 73% False Alarm

Feature-Based Detector• 100% Detection • 76% False Alarm

Feature-Based Detector w/Region Processing

• 100% Detection • 24% False Alarm

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Pfa

Pd

HSTAMIDS MD Algorithm Performance

Feature/RegionEnergyFeature

Performance of HSTAMIDS MDPerformance of HSTAMIDS MD

Can reduction of AP/AT, HM/LM uncertainty improve performance?

AT/AP GPR Energy ComparisonAT/AP GPR Energy Comparison

Note difference in downtrack/crosstrack extentNote difference in downtrack/crosstrack extent

AT/AP GPR Energy ComparisonAT/AP GPR Energy Comparison

as well aslevel of response

as well aslevel of response

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Pfa

Pd

HSTAMIDS MD Algorithm Performance

Feature/RegionEnergyFeatureAMMP

Performance with Bayesian AMMPPerformance with Bayesian AMMPEnergy Detector

• 100% Detection • 73% False Alarm

Feature-Based Detector• 100% Detection • 76% False Alarm

Feature-Based Detector w/Region Processing

• 100% Detection • 24% False Alarm

Feature-Based Detector w/ Bayesian AMMP

• 100% Detection • 17% False Alarm

Application to Field Data: NIITEK GPR and Geophex’s

GEM-3 Metal Detector

Application to Field Data: NIITEK GPR and Geophex’s

GEM-3 Metal Detector

NIITEK GPR Data

GEM-3 Data

10000 20000Frequency

-5

-4

-3

-2

-1

0

1

2

3

4

5

6

7

8

9

PP

M

A3

10000 20000Frequency

-5

-4

-3

-2

-1

0

1

2

3

4

5

6

7

8

9

PP

M

A5

10000 20000Frequency

-5

-4

-3

-2

-1

0

1

2

3

4

5

6

7

8

9

PP

M

A1Clutter

10000 20000Frequency

-2-1.75

-1.5-1.25

-1-0.75

-0.5-0.25

00.25

0.50.75

11.25

1.51.75

2

PP

M

B25

10000 20000Frequency

-1.5

-1.25

-1

-0.75

-0.5

-0.25

0

0.25

0.5

0.75

1

PP

M

B22

10000 20000Frequency

-1.5

-1.25

-1

-0.75

-0.5

-0.25

0

0.25

0.5

0.75

1

PP

M

B20 VAL69PMA-3

Clutter Clutter

10000 20000Frequency

-5-4-3-2-10123456789

B2VS50

AMMP Processing Results

Application to Lab Data: Georgia Tech’s EMI, GPR and Seismic

Sensors

Application to Lab Data: Georgia Tech’s EMI, GPR and Seismic

Sensors

EMI/GPR/Seismic Processing

• Data collected by Georgia Tech• Progress

– First, consider single sensor processing– Compare AMMP to various fusion algorithms– Conclude: need a more difficult test set for

AMMP to show its worth!

Burial Scenario #1

1.8m by 1.8m Scan Region

Rocks (3 and4 cm deep)

Dry Sand(5cm deep)

MINESVS-2.2

(7cm deep)

TS-50(1.5cm deep)

w/ Nail

M-14(0.5cm deep)

VS-50(1cm deep)

PFM-1(1.5 cm deep)

VS-1.6(6.5cm deep)

SeismicSources

Cans (3 and2.5 cm deep)

AssortedMetal Clutter (2 to 4 cm deep)

Shells(4cm deep)

ThreadedRod(3.5cm deep)

Penny(5.5cm deep)

Nails(4cm deep)

Ball Bearing(3.5cm deep)

Shells(5.5cm deep)

Burial Scenario #2

1.8m by 1.8m Scan Region

SeismicSourcesMINES

VS-50(1.3cm deep)

VS-2.2(5.4cm deep)

M-14(1cm deep)

TS-50(1.3cm deep)

PFM-1(0.6cm deep)

VS-50(0.5cm deep)

VS-1.6(5.1cm deep)

Rocks(2, 2.2, 2.5,and 1.3cm deep)

Can(2.2cm deep)

AssortedMetalClutter(<3cm deep)

CD(29cm deep)

EMI Responses

EMI Processing Results

GPR Responses

GPR Responses

GPR Processing Results

Initial Seismic Data/Results

Fusion

Confidence Fusion

Traditional Sensor Fusion

• Voting, Bayesian Decision, Bayesian Decision Statistic, Bayesian Decision Confidence (e.g. Hoballah and Varshney, Krzystofowicz and Long, Dasarathy, Blum, Kassam, and Poor, Chen and Varshney, Hall and Llinas)

• Confidence useful because normalized• Usually assume sensors/decisions are independent• Have not found Bayesian confidence fusion for

the correlated case – so have developed this metric

Confidence Fusion

• Confidence = probability of correct decision

• Compare results when confidence obtained rigorously and when confidence is assigned using a sigmoid function

*

1 2 1 2*

**

*

* * *, , 1 2

Pr( / ) Pr( ) ( / ) Pr( )Pr( / )Pr( ) ( )

( / ) Pr( )Pr( / , )( )N N

d d d dd d

d dd d d d d d N

H H f H Hc Hf

fc Hf

λ λ

λ λλλ λ

λ λ λ

=

=

= = ≈

= ≈λ λ

λ H HλK K K

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

PD

PFA

Sensor fusion performance

Majority Vote FusionBayesian Decision FusionSigmoid Confidence FusionDecision Statistic Confidence Fusion

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

PD

PFA

Individual sensor performance

Sensor 1Sensor 1 Performance PointSensor 2Sensor 2 Performance PointSensor 3Sensor 3 Performance Point

Accomplishments

• Adaptive Feature Selection (JCFO) applied to UXO problem

• AMMP successfully applied to two sets of field data (HSTAMIDS, NIITEK)

• Multi-sensor data from Georgia Tech processed – traditional fusion to date

Future Work

• JCFO application to landmines• JCFO application to subsurface structures

– Simulation– Data?

• AMMP application to EMI+GPR+Seismic• Adaptively modify sensor parameters• AMMP application to subsurface structures