data assimilation
Post on 06-Feb-2016
88 Views
Preview:
DESCRIPTION
TRANSCRIPT
Data Assimilation
Tristan Quaife, Philip Lewis
What is Data Assimilation?
A working definition:
The set techniques the combine data with some underlying process model to provide optimal estimates of the true state and/or parameters of that model.
What is Data Assimilation?
It is not just model inversion.
But could be seen as a process constraint on inversion (e.g. a temporal constraint)
e.g. Use EO data to constrain estimates of terrestrial C fluxes
Terrestrial EO data: no direct constraint on C fluxes
Combine with model
Data Assimilation is Bayesian
• Bayes’ theorem:
P(A|B) =P(B|A) P(A)
P(B)
What does DA aim to do?
Use all available information about The underlying model The observations The observation operator
Including estimates of uncertainty and the current state of the system
To provide a best estimate of the true state of the system with quantified uncertainty
Kalman Filter DA: MODIS LAI product
Data assimilation into DALEC ecological model
Lower-level product DA
Ensure consistency between model and observations
Assimilate low-level products (surface reflectance)
Uncertainty better quantified
Need to build observation operator relating model state (e.g. LAI) to reflectance
Example of Oregon (MODIS DA)
Quaife et al. 2008, RSE
Modelled vs. observed reflectance
Red NIRNote BRF shape in red: can’t simulate with 1-D canopy (GORT here)
NEP results
No assimilation
Assimilating MODIS
(red/NIR)
DALEC model calibrated from flux measurements at tower site 1
Integrated flux predictions
Flux (gC.m-2)
Assimilated data
3yr totalStandardDeviation
NEP
No assimilation 240.2 212.2
MODIS B1 & B2 373.0 151.3
Williams et al. (2005)
406.0 27.8
GPP
No assimilation 1646.4 834.5
MODIS B1 & B2 2620.3 96.8
Williams et al. (2005)
2170.3 18.1
Flux (gC.m-2)
Assimilated data
TotalStandardDeviation
NEP
Assimilation exc. snow
373.0 151.3
Assimilation inc. snow
404.8 129.6
Williams et al. (2005)
406.0 27.8
GPP
Assimilation exc. snow
2620.3 96.8
Assimilation inc. snow
2525.6 42.7
Williams et al. (2005)
2170.3 18.1
Mean NEP for 2000-2002
15 65
gC/m2/year
4.5 km
Flux Tower
Spatial average = 50.9
Std. dev. = 9.7
(gC/m2/year)
NEP – Site2 (intermediate) parameters, with/without DA
Model running at Site 2, Oregon
Site 1 model EO-calibrated at site 2NEP observations from Site 2
Shows ability to spatialise
Data assimilation
Low-level DA can be effective
‘easier’ data uncertainties
Can be applied to multiple observation types
Requires Observation operator(s)
RT models
Requires other uncertainties
Ecosystem Model
Driver (climate)
Observation operator
Specific issues in land EOLDAS
No spatial transfer of information Require full spatial coverage Atmosphere dealt with by an instantaneous retrieval
(i.e. no transport model) All state vector members influence observations
We are not interested in other variables!
Sequential Smoothers Variational
Nominal classification of DA
Kalman Filter Variants - EKF
Ensemble Kalman Filter Variants – Unscented EnKF
Particle filters Lots of different types true MCMC technique
Sequential methods
• Propagation step:
x = Mx-
P = MP-MT + Q• Analysis step:
x* = x + K( y – Hx )
K = PHT( HPHT+R )-1
The Kalman filter
State vectorModel
Covariancematrix
Stochastic forcingKalman
gainObservation
vectorObservation covariance
matrix
Observation operator
The Kalman Filter
• Linear process model
• Linear observation operator
• Assumes normally distributed errors
• Propagation step:
x = m(x-)
P = M'P-M'T + Q• Analysis step:
x* = x + K( y – h(x) )
K = PH'T( H'PH'T+R )-1
The Extended Kalman filter
Jacobian matrix
Jacobian matrix
The Extended Kalman Filter
• Linear process model
• Linear observation operator
• Assumes normally distributed errors
• Problem with divergence
• Propagation step:
X = m(X-) + Q
no explicit error propagation• Analysis step:
X*= X + K( D – HX )
K = PHT( HPHT+R )-1
The Ensemble Kalman filter
State vector ensemble
Perturbed observations
The Ensemble Kalman Filter
• P estimated from X
• Non linear observations using augmentation:
Xa = h(X) X
The Ensemble Kalman Filter
No assimilation
Assimilating MODIS surface
reflectance bands 1 and 2
The Ensemble Kalman Filter
• Avoids use of Jacobian matrices
• Assumes normally distributed errors
– Some degree of relaxation of this assumption
• Augmentation assumes local linearisation
Particle Filters
• Propagation step:
X = m(X-) + Q
• Analysis step:
e.g. Metropolis-Hastings
Particle Filters
Particle Filters
No available observations
Particle Filters
• Fully Bayesian
– No underlying assumptions about distributions
• Theoretically the most appealing choice of sequential technique, but…
• Our analysis show little difference with EnKF
• Potentially requires larger ensemble
– But comparing 1:1 is faster than EnKF
Sequential techniques
• General considerations:
– Designed for real time systems
– Only consider historical observations
– Only assimilates observations in single time step
– Can lead to artificial high frequency components
• Extension of sequential techniques
• All observations effect every time step
• Analogous to weighting on observations
– [ smoothing-convolution / regularisation ]
• Difficult to apply in rapid change areas
Smoothers
Smoothers - regularisation
x = (HTR-1H + λ2BTB)-1HTR-1y
B is the required constraint. It imposes:
Bf = 0and the scalar λ is a weighting on the constraint.
Constraint matrix
Lagrange multiplier
Regularisation
Regularisation
Quaife and Lewis (2010) Temporal constraints on linear
BRDF model parameters. IEEE TGRS, in press.
Regularisation
• Lots of literature on the selection of λ
– Cross validation etc
• Permits insight into the form of Q
Variational techniques
• Expressed as a cost function
• Uses numerical minimisation
• Gradient descent requires differential
• Traditionally used for initial conditions
– But parameters may also be adjusted
3DVAR
J(x) = ( x-x- ) P-1 ( x-x- )T +
( y-h(x) ) R-1 ( y-h(x) )T
Background
Observations
3DVAR
• No temporal propagation of state vector
– OK for zero order approximations
– Unable to deal with phenology
4DVAR
J(x) = ( x-x- ) P-1 ( x-x- )T +
( y-h(xi) ) R-1 ( y-h(xi) )TΣi
Time varying state vector
Variational techniques
• Parameters constant over time window
• Non smooth transitions
• Assumes normal error distribution
• Size of time window?
• For zero-order case 3DVAR = 4DVAR
– 4DVAR for use with phenology model
• Absence of Q - propagation of P?
Building an EOLDAS
• Lewis et al. (RSE submitted)
• Sentinel-2
EOLDAS
Assimilation
Assume model
Uncertainty known
EOLDAS
Base level noise
Cross validation
Cross validation
EOLDAS
Cross validation
Double noise
EOLDAS
Double noise
Conclusions - technique
• DA is optimal way to combine observations and model
• Range of options available for DA
• Sequential
• Smoothers
• Variational
• Require understanding of relative uncertainties of model and observations
• Require way of linking observations and model state
• Observation operator (e.g. RT)
References
• P. Lewis et al. (2010 submitted) RSE, An EOLDAS
• T. Quaife, P. Lewis, M. DE Kauwe, M. Williams, B. Law, M. Disney, P. Bowyer (2008), Assimilating Canopy Reflectance data into an Ecosystem Model with an Ensemble Kalman Filter, Remote Sensing of Environment, 112(4),1347-1364.
• T. Quaife and P. Lewis (2010) Temporal constraints on linear BRF model parameters IEEE Transactions on Geoscience and Remote Sensing doi: 10.1109/TGRS.2009.2038901
• http://www.ecmwf.int/newsevents/training/rcourse_notes/DATA_ASSIMILATION/ASSIM_CONCEPTS/Assim_concepts11.html
• http://www.cs.unc.edu/~welch/kalman/
top related