progress meeting m3a presentation of td3

IIM-TS M3A – ESRIN 12 December 2007

Progress Meeting M3APresentation of TD3


Selection Procedure

Prototype selection is based on a 3-step procedure

Qualitative pre-screeningof algorithms

Quantitative evaluationof algorithms

Final ranking andprototype selection

TD2

Selected prototype


Selection Criteria

8 classes of parameters are considered:

1. Scientific Background and Technical Soundness

The selected algorithms should be based on a solid theoretical background that guarantees the accuracy of its results also at an operational level. The guidelines for rating are as follows:

• The methodology is solid;

• The methodology is technical convincing;

• The methodology is at the state-of-the-art;

• The methodology is published in high quality journals;

• The methodology is included in several other scientific publications or project technical reports.


Selection Criteria

2. Robustness and Generality

• The method is suitable to be used with different kind of images;

• The method shows high performances on different images and different test areas;

• There are software implementations or examples for the implementation available;

• The algorithm can be used in combination with other methodologies.


Selection Criteria

3. Novelty

In order to get a high score, an algorithm should have been published or reported for the first time relatively recently in the literature. The guidelines for rating the novelty are:

• The publications are after 2003 and introduce a novel, convincing and adequately tested solution to an existing problem;

• The publications in remote sensing are after 1998;

• The method is not implemented in commercial SW packages.


Selection Criteria

4. Operational Requirements

Operational requirements are evaluated in terms of computational complexity, time effort, cost etc. The guidelines for rating of operational perspectives are as follows:

• The requested modifications to KIM architecture are few;

• The algorithm works fast (e.g., near real time);

• The processing time scaling is likely to be linear with image size;

• The hardware and disk-storage requirements are low.


Selection Criteria

5. Accuracy

Both absolute and relative accuracy in all operative conditions will be evaluated. The guidelines for rating the accuracy are:

• The algorithm matches the end-user requirements and can be optimized according to them;

• The accuracy does not depend on the availability/amount of prior information.


Selection Criteria

6. Range of Applications

The number and kinds of applications that an algorithm can address is evaluated:

• The algorithm is suitable for a high number of application areas;

• The algorithm has a high number of estimated final users for the application areas;

• The algorithm has a high impact on the considered application areas.


Selection Criteria

7. Level of Automation

From an operational point of view, it is preferable that an algorithm is able to run in a completely automatic way. The main guidelines for rating of the perspectives for automation are:

• The number of parameters to be defined by the operator is low;

• The physical meaning of parameters is clear;

• The method is automatic;

• Ground truth or prior information is not requested.


Selection Criteria

8. Specific end-users requirements

From an operational point of view, capability of an algorithm to satisfy and meet different possible end-users requirements is an important parameter of evaluation. The main guidelines for driving this ranking are:

• The algorithm is flexible in meeting different possible accuracy requirements;

• The algorithm can be reasonably included in an operational procedure.


Step 1: Qualitative pre-screening of algorithms

• A pre-screening of the algorithms identified and described in TD2 is carried out in order to identify the most relevant methodologies with respect to the IIM-TS project objectives.

• The preliminary qualitative evaluation is driven from the same selection criteria used also in the next quantitative steps. In this step a high level analysis of these criteria is conducted in order to identify techniques that clearly cannot reach a satisfactory ranking on several categories of parameters.

• These techniques are discarded and not further considered in the next steps.

Selection Procedure


Pre-screening of algorithmsBinary Change Detection

Binary ChangeDetection Technique

Detection Algorithm

Reference

Change vector Analysis (CVA)Image Differencing (ID)

Vegetation Index Differencing (VID)

Principal Component Analysis (PCA)

Empirical thresholding

Singh (1989) Townshend et al. (1995)

Fung et al. (1990) Muchoney (1994)Fung (1987)

Thresholding based on the Bayes decision

theory

Bruzzone et al. (2000, 2002)Kittler et al. (1986)

Fuzzy thresholdingPal et al. (2000, 2001) Di Zenzo

(1998)

Context-based approaches

Bruzzone et al. (2000) Ghosh et al. (2007)

MultiscaleBovolo et al. (2005) Inglada et al.

(2007)

Reduction registration noise

Bruzzone et al. (1997, 2003)

Multispectral data




Detection Algorithm ReferenceKind of Data

Image Rationing (IR)

Empirical thresholdingSingh (1989) Rignot et al. (1992)

Cihlar et al. (1993)

SAR

Thresholding based onthe Bayes decision theory

Bazi et al. (2004, 2005, 2006)

Fuzzy thresholdingPal et al. (2000, 2001)

Di Zenzo (1998)

Context-based approaches Bazi et al. (2005)

MultiscaleBovolo et al. (2005) Inglada et al.

(2007)

Multivariate Alteration Detection

Nielsen et al. (1997, 1998)

Correlation coefficientContrast Ratio

Ellipticity

Empirical thresholdingDierking et al (2000, 2002)

Kersten et al. (2005)Polarimetric

SAR dataTest Statistics Conradsen et al. (2003)

Context based Molinier et al. (2007)

SAR and Polarimetric SAR data




DetectionAlgorithm

ReferenceKind of Data

Kullback Leibler distance (KLD)Normalized information distance

(NID)Mutual Information (I)

Variational Information (VI)Mixed Information (MI)

Single scaleInglada et al. (2003) Meila

(2003)Gueguen et al. Multispectral

and SARMultiscale Inglada et al. (2007)

Multimodal Datcu et al.

Feature and area based techniques

Thresholdingand refinement

Dell’Acqua et al. (2004, 2006)Della Ventura et al. (1990)

Multispectral and SAR

Multivariate Alteration Detection (MAD)Nielsen et al. (1997, 1998)

Liao et al. (2005)

Multispectral, SAR

multisensor

Multisensor techniques

Consensus theory Bruzzone et al. (2000)Multispectral

and SARMultivariate Alteration Detection

Nielsen et al. (1997, 1998)Liao et al. (2005)

Multisensor data


Pre-screening of algorithmsMulticlass Change Detection

Multiclass Change Detection Technique


Supervised Post-classification ComparisonSingh(1989) Howarth et al. (1981)Hall et al. (1991) Xu et al. (1990)

MultispectralSAR

Multisensor

Supervised Direct-Multidate ClassificationSingh (1989) Schowengerdt (1983)Hall et al. (1991) Burns et al. (1981)

Supervised Compound ClassificationBruzzone et al. (2001)

Serpico et al.

Partially Supervised approachesBruzzone et al. (1997) Cossu et al.

(2005)Fernández Prieto et al. (2001)

Unsupervised approaches

Bovolo et al. (2007) Byrne et al. (1980)

Richards et al. (1993) Häme et al. (1998) Henry et al. (2006) Multispectral

SAR

Multisensor techniquesBruzzone et al. (1999) Baraldi et al.

(2006)Macrì Pellizzeri et al. (2003)


Pre-screening of algorithmsShape Change Detection

Shape Change Detection Technique


Shape Measures ComparisonsLi et al. (2003) Gamba et al.

(2007)Guindon (1997) Multispectral,

SARand

Multisensor

Differential Snakes Agouris et al. (2001)

Polygon Detection and UpdatingBarr et al. (1997) Yang et al.

(2001)Masek et al. (2000)


Pre-screening of algorithmsTrend Analysis of Temporal Series of Images

Trend Analysis Technique Reference Kind of data

Principal component analysisEastman et al. (1993) Hall-Beyer

(2001)Rigina et al. (2003)

Single series of Multispectral

or SAR images

Kalman filtering Joyce et al. (2001)

Regression techniques

Hansen et al. (2001) DeFries et al. (1997)

Engle et al. (1987) Johansen et al. (1995)

NDVI

Hayes et al. (2001) Wilson et al. (2002)

Rigina et al. (2003) Nemani et al. (2003)

Fuller (1998)

Neural networksBrivio et al. (2001) Bruzzone et al.

(2004)Townshend et al. (2001)

Long-term Compositing Techniques

Holben (2001)

Satellite linear-based index Rauste et al. (2007)

Fourier and Wavelet AnalysisAzzali et al. (2000) Anyambe et al.

(1996Li et al. (2000) Andres et al. (1994)

Pixel-based techniques


Pre-screening of algorithmsTrend Analysis of Temporal Series of Images

Trend Analysis Technique Reference Kind of data

Integration of information from neighboring pixels and time series

transition probabilitiesBoucher et al. (2006) Single series of

Multispectral or SAR imagesThree-dimensional (or spatio-

temporal) clusteringYamamoto et al. (2001)

Heas et al. (2005)

Maximum-likelihood detectors Lombardo et al. (2002)Single series of

SAR images

Global vegetation model McCloy et al. (2004)Pairs of

Multispectral time series

Statistical analysis of areas of interest (by GIS or object analysis)

Hazel (2001) Gamba et al. (2007)

Turker et al. (2003)

Single series of Multispectral or

SAR images

Context-based techniques


Pre-screening of algorithmsPre-processing Multispectral Data

Kind ofPre-

processing

Properties

ReferenceAlgorithmPrinciple

Co-registration Semi-manualGoshtasby (1988) Ton et al.(1989)

Brown (1992) Li et al. (1992)CP or structures

matching

Atmosphericcorrections

AbsoluteHäme (1991) Olsson (1995) Conese et al. (1993)

Richter(1997) Finlayson et al. (2003)Meaningful physical units identification

Relative(empirical)

Coppin et al. (1994) Hall et al. (1991) Eckhardt et al. (1990) Peddle et al. (2003) Tokola et al.

(1999)Pixel-based

Cloudsdetection

Automatic orsemi-

automatic

McIntire et al. (2002) Di Vittorio et al. (2002) Spatial coherence

Di Vittorio et al. (2002)Adaptive

thresholding

Murtagh et al. (2003) Bayesian methods

McIntire et al. (2002) Tian et al. (1999)Arriaza et al. (2003)

Neural networks

Pan-sharpening Automatic

Garzelli et al. (2006) Injection

Chavez et al. (1991) High Pass filtering

Garguet-Duport et al. (1996) Yocky et al. (1996) Zhou et al. (1998) Aiazzi et al. (1999, 2000)

Alparone et al. (1998)Multiresolution

Tu et al. (2001, 2004) HIS

Aiazzi et al. (2006) MTF


Pre-screening of algorithmsPre-processing SAR Data

Kind ofPre-processing

Properties ReferenceAlgorithmPrinciple

Image focusing Automatic

Cafforio et al. (1991)

Curlander et al. (1991) Range doppler

Raney et al. (1994) Chirp scaling

Co-registration AutomaticForoosh et al. (2002) Phase Correlation

Stone et al. (2001) Fourier transform

Ortho-rectification and georeferencing

Sensor dependent

(parametric)

Hemmleb et al. (1997)Novak (1992) Meier et al. (1993) Range-Doppler

Camera ModelGCP matchingSensor

independent(non-parametric)

Rosenholm et al. (1998)Tao et al. (2001) Zhou et al. (2005)

Atmospheric corrections

AutomaticRauste et al. (2007) DEM

Datcu et al. (in press) Bayesian theory

Semi-automaticDe Grandi et al. (2003) Rauste et al.

(1999)Correlation

Manual De Grandi et al. (2004) GCP


Pre-screening of algorithmsPre-processing SAR Data

Kind ofPre-processing

Properties ReferenceAlgorithmPrinciple

Radiometric calibration and normalization

Automatic

Holecz et al. (1994) Radar Equation

Ulander (1996)Normalization on pixel

surface

Small et al. (2004) Normalization on pixel area

MosaickingAfek et al. (1998) Du et al. (2001)

Guindon (1995, 1996, 1997)Linear regression

Filteringand segmentation

Automatic

Richards et al. (1999)Low-pass or edge-

preservingimage-smoothing filters

Frost et al. (1982) Lee (1980)Kuan et al. (1985) Solbø et al. (2004)

Lopes et al. (1990, 1993)Trouvé et al. (2003) Aspert et al.

(2007)

Adaptive despeckling procedures

Multitemporal filtering

Bruniquel et al. (1997) Ciuc et al. (2001)

Coltuc et al. (2000) Perona et al. (1990)


Pre-screening of algorithmsPre-processing Multisensor Data

Kind of Pre-processing

Properties

ReferenceAlgorithmPrinciple

Mosaicking Automatic

Afek et al. (1998) Du et al. (2001)

Guindon (1995, 1996, 1997)Linear regression

Du et al. (2001)Last, mean, Feathering

Gradient

Co-registration Automatic

Brown (1992)Zitová et al. (2003)

Similarity measures and geometric transformation

Thépaut (1998) Orbit information

Wu et al. (1990)Djamdji et al. (1993)

Multiresolution

Ventura et al. (1990)Dai et al. (1999) Ali et al.

(2002)Feature based

Inglada et al. (2004) Multisensor


Selection Procedure

Step 2: Quantitative evaluation of algorithms

• Algorithms that pass the pre-screening in step 1 are analyzed in greater detail with a quantitative evaluation.

• This analysis is based on different parameters (scientific and technical analysis, possible impacts on the application and end-users, etc).

• For each algorithm (or cluster of algorithms) a method sheet is filled in, which reports details of the algorithm and individual scores for each parameter considered.


Method Sheets Organization

Method name Reference to TD2

Method category Applications

Method description

Main scientific papersCitations

Citations/year

Data type

Optical Radar Multisensor

LR MR VHR

Method suitable toSingle image Multitemporal

imagesData fusion

Existing software

Pre-processing reqs Comment X/S

Geometric correction

Radiometric correction

Other pre-processing

Remarks

Algorithm characteristics



Technical considerations

Evaluation Criteria Y/N

Scientific background and technical soundness

The methodology is solid

The methodology is technical convincing

The methodology is at the state-of-the-art

The methodology is published in high quality journals

The methodology is included in several other scientific publications or project technical reports

Robustness and generality

The method is suitable to be used in different research and application environments

The method is suitable to be used with different kind of images

The method shows high performances on different images and different test areas

There are software implementations or examples for the implementation available

The algorithm can be used in combination with other methodologies

Novelty

The publications are after 2003 and introduce a novel, convincing and adequately tested solution to an existing problem

The publication in remote sensing are after 1998

The method is not implemented in commercial SW packages

Evaluation



Technical considerations

Evaluation Criteria Y/N

Operational requirements

The requested modifications to KIM architecture are few

The algorithm works fast (e.g., near real time)

The processing time scaling is likely to be linear with image size

The hardware and disk-storage requirements are low

Accuracy

The algorithm matches the end-user requirements and can be optimized according to them

The accuracy does not depend on the availability/amount of prior information

Range of applications

The algorithm is suitable for a high number of application areas

The algorithm has a high number of estimated final users for the application areas

The algorithm has a high impact on the considered application areas

Level of automation

The number of parameters to be defined by the operator is low

The physical meaning of parameters is clear

The method is automatic

Ground truth or prior information is not requested

Specific end-users requirements

The algorithm is flexible in meeting different possible accuracy requirements

The algorithm can be reasonably included in an operational procedure

Evaluation


Selection Procedure

Step 3: Final ranking and prototype selection

• According to an analysis of methods sheets a final score is given to each algorithm and method.

• This value is used for ranking algorithms according to their relevance with respect to IIM-TS objectives;

• The algorithms to be prototyped are identified on the basis of the score and of a final discussion of the ranking.


Total score computation

• 1 point is given to each considered class of parameters for each positive answer in the corresponding category of the method sheet. Then the category score is normalized.

• Few points are assigned to each method according to the number of citations per year of the algorithms in scientific papers (or in technical reports) following this table:

Citations/year Points

0 0

1-4 1

5-8 2

8-12 3

> 12 4

Total Score Computation



The score achieved for each single class is properly weighted in order to take into account its relevance with respect to the goals of the project. The following equation is used:

Total value = w1 * “Scientific Background and Technical Soundness”+ w2 * “Robustness and Generality”+ w3 * “Novelty”+ w4 * “Operational Requirements”+ w5 * “Accuracy”+ w6 * “Range of Applications”+ w7 * “Level of Automation”+ w8 * “Specific End-users Requirements”+ w9 * Citation scoreThe final score indicates the relevance of the method with respect to the prototyping

procedure within IIM-TS project.



Selection criteriaWeight variable

Weight value

Scientific Background and Technical Soundness

w1 5

Robustness and Generality w2 5

Novelty w3 3

Operational Requirements w4 4

Accuracy w5 2

Range of Applications w6 3

Level of Automation w7 4

Specific End-users Requirements w8 2

citation score w9 4

wn (n = 1,…9) is the weight assigned to the n-th category of criteria, and represents the relative relevance of the considered criterion with respect to the others:


Table of Ranking

Method Category(main)

Method Name

ScientificBackground

and Technical

Soundness

Robustness

andGeneralit

y

NoveltyOperation

alReq.

Accuracy

RangeOf

Appl.

LevelOf

Autom.

SpecificEnd-usersReq.

Citation

score Total

score

5 5 3 4 2 3 4 2 4

Binary CDUnsupervised Bayesian

framework to CD5 5 3 4 2 3 4 2 4 32

Binary CD & pre-processing

Kullback-Leibler divergencefirst order

4 5 3 4 2 3 3.5 1 1 28.5

Binary CD MAD+MAF or MNF 5 5 1 4 2 3 4 1 2 27Binary CD Pre-

processingImage split 4 5 3 4 2 3 4 2 0 27

SAR pre-processing, CD & Trend Analysis

SAR preprocessing and multisensor rule-based

classifier4 5 2 4 2 3 4 1 1 26

Trend analysisFourier and Wavelet

Analysis5 3 1 4 2 3 4 1 2 25

Multi-class CDAutochange change

detectionand identification

5 5 2 4 2 3 2 1 1 25

Trend analysisHot spot monitoring via

GIS fusion4 5 3 4 2 3 2 1 0 24

Trend analysis Spatio-temporal clustering 4 4.5 3 2 2 3 4 1 1 24.5

Multi-class CDDirect Multidate

Classification4 5 1 3.5 1 3 2 1 4 24.5

Shape CD Shape change index 3 4 2.5 4 2 3 3.5 1 0 23

Multi-class CDPost-classification

comparison4 4 0 4 1 3 2 1 4 23

Trend analysisClassification of

long temporal series4 5 3 3 1 3 2 1 1 23


Table of Ranking

Method Category

(main)Method Name

ScientificBackground

and Technical

Soundness

Robustness

andGeneralit

y

NoveltyOperation

alReq.

Accuracy

RangeOf

Appl.

LevelOf

Autom.

SpecificEnd-usersReq.

Citation

score Total

score

5 5 3 4 2 3 4 2 4


Multi-modal change map generation

3 4 2 4 2 3 4 1 0 23

Binary CD & Pre-processing

Mixed Information Measure (comparison & co-

registration)3 4 2 4 2 3 4 1 0 23


The similarity metric based on Kolmogorov complexity

(comparison & co-registration)

3 3.5 2 4 2 3 4 1 0 22.5

Multi-class CD Compound Classification 5 5 1 3 1 2 2 1 2 22Binary CD & pre-

processingSAR polarimetric change

indices3 4 2 4 2 3 2 1 1 22


Kullback-Leibler divergence second order (comparison &

co-registration)3 4 2 4 1 3 4 1 0 22

Trend analysisPhenological changemonitoring (autumn

coloration)3 3 3 4 2 2 3 1 0 21


Design of the Architecture

• The selection of the prototype algorithms among those with the highest scores in the ranking should be finalized taking into account the possible synergy between different techniques.

• The final selection should be also based on an adequate balancing among techniques belonging to the different classes.


Design of the Architecture Raw SAR images

FocusingGeometric corrections

Radiometric correctionsRadiometric normalization

Mutitemporal filteringMosaiking Segmentation

Time varying segmentationPre

-pro

ce

ss

ing Registration

Ortho-rectificationMosaiking

Radiometric correctionsCloud detection

Topographic correctionsPan-sharpeningImage filtering

Feature extraction

Raw optical images

Pre

-pro

ce

ss

ing

DifferenceRatio

Information theoretically similatityMeasures (KL divergence, etc.)

Polarimetric change indeces(correlation, etc.)

Thresholding based on the Bayes decision theory

Context-based approachesMultiscale approachesMultimodal approaches

Multivatiarte Alteration Detection

Binary Change Detection

Post-classification ComparisonDirect-Multidate Classification

Compound ClassificationUnsupervised approaches

Multisensor techniques

Multiclass Change Detection

Shape MeasuresComparisons

Shape Change Detection

Neural networksSatellite linear-based index

Fourier and Wavelet AnalysisSpatio-temporal clustering

Statistical analysis of areas of interest(by GIS or object analysis)

Trend Analysis



• Pre-processing chain for multispectral images (geometric corrections and radiometric corrections)

• Pre-processing chain for SAR data (geometric corrections and radiometric corrections)

• Binary change detection:• Set of measures for image comparison (difference, magnitude of

the difference vector, ratio, log-ratio, KL, similarity measures)• Image splitting• Bayesian framework for the analysis of the results of the

comparison (minim error and cost decision rules, Gaussian model, Generalize Gaussian model (?), MRF context-sensitive decision, manual or automatic initialization?)



• Multiclass change detection:• Unsupervised method based on autochange algorithm• Supervised methods based on MDC and PCC (need for a

distribution-free classification module)• Rule based multisensor classifier

• Trend analysis of time series:• Spatio-temporal clustering (data mining)• Tools for FT and WT• Hot spot monitoring via GIS fusion

• Shape change detection measure

progress meeting m3a presentation of td3

Documents