1 bayesian methods for parameter estimation and data assimilation with crop models part 2:...
TRANSCRIPT
1
Bayesian methods for parameter estimation and data assimilation with crop models
Part 2: Likelihood function and prior distribution
David Makowski and Daniel Wallach
INRA, France
October 2006
2
• Notions in probability - Joint probability
- Conditional probability
- Marginal probability
• Bayes’ theorem
Previously
PyP
PyPyP
3
Objectives of part 2
• Introduce the notion of prior distribution.
• Introduce the notion of likelihood function.
• Show how to estimate parameters with a Bayesian method.
4
Part 2: Likelihood function and prior distributions
Estimation of parameters ()
Parameter = numerical value not calculated by the model and not observed.
Information available to estimate parameters
- A set of observations (y).
- Prior knowledge about parameter values.
5
Part 2: Likelihood function and prior distributions
Two distributions in Bayes’ theorem
• Likelihood function = function relating data to parameters.( )P y
• Prior parameter distribution = probability distribution describing our initial knowledge about parameter values.
( )P
6
Part 2: Likelihood function and prior distributions
Measurements
Prior Information about parameter values
Bayesian methodCombined info about parameters
7
ExampleEstimation of crop yield θ by combining a measurement with expert knowledge.
Measurement y = 9 t/ha ± 1
about 5 t/ha ± 2
Expert
Field with unknown yield
Part 2: Likelihood function and prior distributions
Plot
8
ExampleEstimation of crop yield θ by combining a measurement with expert knowledge.
Part 2: Likelihood function and prior distributions
• One parameter to estimate: the crop yield θ.
• Two types of information available:
- A measurement equal to 9 t/ha with a standard error equal to 1 t/ha.
- An estimation provided by an expert equal to 5 t/ha with a
standard error equal to 2 t/ha.
9
Part 2: Likelihood function and prior distributions
Prior distribution
• It describes our belief about the parameter values before we observe the measurements.
• It is based on past studies, expert knowledge, and litterature.
10
Part 2: Likelihood function and prior distributions
Example (continued)
Definition of a prior distribution
θ ~ N( µ, ² )
42
5exp42
1
2exp
2
1 2
2
2
2
P
• Normal probability distribution.
• Expected value equal to 5 t/ha.
• Standard error equal to 2 t/ha
11
Part 2: Likelihood function and prior distributions
Example (continued)
Plot of the prior distribution
0 2 4 6 8 10
Theta (t/ha)
0.0
0.1
0.2
0.3
0.4
0.5
Den
sity
12
Part 2: Likelihood function and prior distributions
Likelihood function
• A likelihood function is a function relating data to parameters.
• It is equal to the probability that the measurements would have been observed given some parameter values.
• Notation: P(y | θ)
13
Part 2: Likelihood function and prior distributions
Example (continued)
Statistical model
y | θ ~ N( θ, σ² )
y = θ + with ~ N( 0, σ² )
14
Part 2: Likelihood function and prior distributions
Example (continued)
Definition of a likelihood function
• Normal probability distribution.
• Measurement y assumed unbiaised and equal to 9 t/ha.
• Standard error σ assumed equal to 1 t/ha
y | θ ~ N( θ, σ² )
2
9exp
2
1
2exp
2
1 2
2
2
2
yyP
15
Part 2: Likelihood function and prior distributions
Example (continued)
Definition of a likelihood function
4 6 8 10 12
Theta (t/ha)
0.0
0.1
0.2
0.3
0.4
De
nsi
ty
Maximum likelihood estimate
16
Part 2: Likelihood function and prior distributions
Maximum likelihood
Likelihood functions are also used by frequentist to implement the maximum likelihood method.
The maximum likelihood estimator is the value of θ maximizing P(y | θ) .
17
0 2 4 6 8 10 12
Theta (t/ha)
0.0
0.1
0.2
0.3
0.4
De
nsi
ty
Part 2: Likelihood function and prior distributions
Likelihood functionPrior probability distribution
18
Part 2: Likelihood function and prior distributions
Example (continued)
Analytical expression of the posterior distribution
θ | y ~ N( µpost, post² )
1 8.2post B B y
8.01 22 Bpost
5
422
2
B
19
Part 2: Likelihood function and prior distributions
0 2 4 6 8 10 12
Theta (t/ha)
0.0
0.1
0.2
0.3
0.4
0.5
De
nsi
ty
Prior probability distribution Likelihood function
Posterior probability distribution
20
1. Result is a probability distribution (posterior distr.)
2. Posterior mean is intermediate between prior mean and observation.
3. Weight of each depends on prior variance and measurement error.
4. Posterior variance is lower than both prior variance and measurement error variance.
5. Used just one data point and still got estimator.
Part 2: Likelihood function and prior distributions
Example (continued)
Discussion of the posterior distribution
21
Frequentist versus Bayesian
Part 2: Likelihood function and prior distributions
Bayesian analysis introduces an element of subjectivity: the prior distribution.
But its representation of the uncertainty is easy to understand
- the uncertainty is assessed conditionally to the observations,
- the calculations are straightforward when the posterior distribution is known.
22
Part 2: Likelihood function and prior distributions
Which is better?
Bayesian methods often lead to
- more realistic estimated parameter values,
- in some cases, more accurate model predictions.
Problems when prior information is wrong and when one has a strong confidence in it.
23
Part 2: Likelihood function and prior distributions
Difficulties for estimating crop model parameters
Which likelihood function ?
- Unbiaised errors ?
- Independent errors ?
Which prior distribution ?
- What do the parameters really represent ?
- Level of uncertainty ?
- Symmetric distribution ?
24
Part 2: Likelihood function and prior distributions
Practical considerations
• The analytical expression of the posterior distribution can be derived for simple applications.
• For complex problems, the posterior distribution must be approximated.
25
Next part
Importance sampling, an algorithm to approximate the posterior probability distribution.