gaussian process regression within an active learning scheme
DESCRIPTION
University of Trento Dept . of Information Engineering and Computer Science Italy. GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME. Edoardo Pasolli [email protected]. Farid Melgani [email protected]. IGARSS 2011. Introduction. Supervised regression approach. - PowerPoint PPT PresentationTRANSCRIPT
GAUSSIAN PROCESS REGRESSION WITHINAN ACTIVE LEARNING SCHEME
IGARSS 2011
University of TrentoDept. of Information Engineering and
Computer ScienceItaly
Edoardo Pasolli
Farid [email protected]
n.it
July 28, 2011
Introduction
Supervised regression approach
2
Pre-processing
Feature extraction
RegressionImage/Signal Prediction
Training sample
collection
Training sample quality/quantity
Human expert
Impact on prediction errors
Introduction
Active learning approach for classification problems
3
Trainingof classifier
Active learningmethod
Model of classifier
Learning (unlabeled) set
Labeling of selected samples
Selected samplesafter labeling
Insertion in training set
f1
f2
f1
f2
f1
f2
Selected samples from
learning (unlabeled) set
f2
f1
f2
f1
Human expert
Training (labeled) set
Class 1 Class
3
Class 2
Objective
Propose GP-based active learning strategies for biophysical parameter estimation problems
4
Gaussian Processes (GPs)
Predictive distribution
5
2**** ,~,,| NXf xy
yk 12
** , IXXK nt *
12***
2* ,, kkxx
IXXKk n
t
fy
IN n2,0~ XXKGPf ,,0~
IXXKGPy n2,,0~
noise variance
covariance matrix defined by covariance function
',xxk
Gaussian Processes (GPs)6
Example of predicted function
: predicted value
*: standard deviation of predicted value
*
: training sample
Proposed Strategies7
GPRegression
U’s: Selected
unlabeled samples
Labeling
SelectionInsertion intraining set
Human expert
L: Training set
U: Learning set
L’s: Labeled samples
Proposed Strategies
Minimize covariance measure in feature space (Cov)
8
2
2
2
2
'exp,
l'k f
xxxx : squared
exponential covariance functionsignal
variance length-scale
: training sample: covariance function with respect to training sample
',xxk'x
Proposed Strategies
Minimize covariance measure in feature space (Cov)
9
: covariance measure with respect to all training samples
covf
covf
: covariance function with respect to training sample
ik xx,
ix
: training sample
n
iikf
1cov ,xxx xx
xcov
* minarg f
: selection of samples with minimum values of
Proposed Strategies
Maximize variance of predicted value (Var)
10
kkxxx122 ,
IKk n
t
: predicted value
x: standard deviation of predicted value
x
: training sample
Proposed Strategies
Maximize variance of predicted value (Var)
11
xx 2var f xx
xvar
* maxarg f
varf
varf: variance
: training sample
: selection of samples with maximum values of
Experimental Results
Data set description (MERIS) Simulated acquisitions Objective: estimation of chlorophyll
concentration in subsurface case I + case II (open and coastal) waters
Sensor: MEdium Resolution Imaging Spectrometer (MERIS)
# channels: 8 (412-618 nm) Range of chlorophyll concentration: 0.02-54
mg/m3
12
Experimental Results
Data set description (SeaBAM) Real aquisitions Objective: estimation of chlorophyll
concentration mostly in subsurface case I (open) waters
Sensor: Sea-viewing Wide Field-of-view (SeaWiFS)
# channels: 5 (412-555 nm) Range of chlorophyll concentration: 0.02-32.79
mg/m3
13
Experimental Results
Mean Squared Error
14
MERIS SeaBAM
Experimental Results
Standard Deviation of Mean Squared Error
15
MERIS SeaBAM
Experimental Results
Detailed results
16
Accuracies on 4000 test samples
Method#
trainingsamples
MSE σMSE R2 σR2
Full 1000 0.086 - 0.991 -Initial 50 1.638 0.869 0.849 0.070RanCovVar
1500.5850.3780.184
0.4060.1050.054
0.9380.9610.980
0.0450.0100.005
RanCovVar
3000.2370.2120.095
0.0840.1770.005
0.9750.9770.990
0.0080.0180.000
MERIS
Experimental Results
Detailed results
17
Accuracies on 4000 test samples
Method#
trainingsamples
MSE σMSE R2 σR2
Full 1000 0.086 - 0.991 -Initial 50 1.638 0.869 0.849 0.070RanCovVar
1500.5850.3780.184
0.4060.1050.054
0.9380.9610.980
0.0450.0100.005
RanCovVar
3000.2370.2120.095
0.0840.1770.005
0.9750.9770.990
0.0080.0180.000
MERIS
Experimental Results
Detailed results
18
Accuracies on 459 test samples
Method#
trainingsamples
MSE σMSE R2 σR2
Full 460 1.536 - 0.806 -Initial 60 5.221 2.968 0.526 0.215RanCovVar
1602.9722.2101.818
1.0380.0740.029
0.6820.7450.784
0.0690.0070.003
RanCovVar
3102.0621.6011.573
0.6870.0100.003
0.7530.8000.803
0.0660.0010.000
SeaBAM
Experimental Results
Detailed results
19
Accuracies on 459 test samples
Method#
trainingsamples
MSE σMSE R2 σR2
Full 460 1.536 - 0.806 -Initial 60 5.221 2.968 0.526 0.215RanCovVar
1602.9722.2101.818
1.0380.0740.029
0.6820.7450.784
0.0690.0070.003
RanCovVar
3102.0621.6011.573
0.6870.0100.003
0.7530.8000.803
0.0660.0010.000
SeaBAM
Conclusions
In this work, GP-based active learning strategies for regression problems are proposed
Encouraging performances in terms of convergence speed stability
Future developments extension to other regression approaches
20
GAUSSIAN PROCESS REGRESSION WITHINAN ACTIVE LEARNING SCHEME
IGARSS 2011
University of TrentoDept. of Information Engineering and
Computer ScienceItaly
Edoardo Pasolli
Farid [email protected]
n.it
July 28, 2011