IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Reading Seminar on Classics
presented byBassoum Abou
Article
Estimation of the Mean of a Multivariate Normal Distribution
Suggested by C. Robert
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Plan
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Plan
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Plan
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Plan
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Plan
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Plan
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Plan
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Plan
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Presentation of the article
Estimation of the Mean of a Multivariate Normal Distribution
Authors: Charles M Stains
Source: The Annals of Statistics, Vol.9, No. 6(Nov., 1981),1135-1151
Implemented in FORTRAN
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Presentation of the article
Estimation of the means of independant normal random variable isconsidered, using sum of squared errors as loss.The central problem studied in this paper is tha tof estimating themean of multivariate normal distribution with the squared length ofthe error as as loss when the covariance matrix is khown to be theidentity matrix.
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Lemme 1
Let fuction Y be a fuction N(0, 1) real random variable and letg : R → R be an indefinite integral of a Lebesgue
measurement fuction g′ , essentially the derivate of g . Supposealso that E|g(Y)| <∞ . Then
E(g′(Y)) = E(Yg(Y))
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Definition 1
A function h : Rp → R will be called almost differentiable ifthere exist a function ∇h : Rp → Rp such that for all z ∈ Rp
h(x + z)− h(x) =∫
z.∇h(x + tz)dt
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Lemme 2
If h : Rp → R is a almost differentiable function withEξ‖X‖ <∞ then Eξ∇h(X) = Eξ(X − ξ)h(X)
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Theorem 1
Consider the estimate X+g(X) for ξ such that g : <p → <p
is an almost differentiable function for wich
EξΣ|∇igi(X)| <∞
Then for each i ∈ (1, ........, p)
Eξ(Xi + gi(X)− ξi)2 = 1 + Eξ(g2
i (X) + 2∇gi(X))
and consequently
Eξ‖X + g(X)− ξi‖2 = p + Eξ‖g(X)‖2 + 2∇g(X))
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Theorem 2
Let f : <p → <+ ∩ (0) be an almost differentiable functionfor wich ∇f : <p → <p can be taken to be almostdifferentiable, and suppose also that
Eξ( 1f (X)Σ|∇2
i f (X)|) <∞
and
Eξ‖∇logf (X)‖2 <∞
then
Eξ‖X +∇logf (X)− ξ‖2 = p + 4Eξ(∇2√
(f (X)√(f (X)
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
General
Let X be a random vector in <p , conditionally narmally distributedgiven ξ with conditionnal mean ξ , with the identity asconditionnal covariance matrix. Then the unconditionnal density of Xwith respect to Lebesgue measure in <p is given by
f (x) = 1(2Π)
p2
∫e−12 |x− ξ|dΠ(ξ)
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Formal bayes estimates
The Bayes estimate φn(X) of ξ which is defined by the conditionthat φ = φn minimizes
E‖ξ − φ(X)‖2 = EEx‖ξ − φ(X)‖2 = E(‖ξ−φ(X)‖2
∫e−|x−ξ|dΠ(ξ)
2∫e−|x−ξ|dΠ(ξ)
2)
is given by
φn(X) = Exξ = X +∇logf (X)
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Nox let us compare the unbiased estimate of risk of the normal Bayesestimate φn(X) of ξ given by Theorem 2 with the the formalposterior risk E‖ξ − φ(X)‖2 . From Theorem 2 , the unbiasedestimate of the risk is given by
ρ(X) = p + 2∇2f (X)f (X) −
‖f (X)‖2
f 2(X)
For the formal posterior risk we have
Ex‖ξ − φ(X)‖2 = p + ∇2f (X)f (X) − ‖∇logf (X)‖2
and we have at the end
Ex‖ξ − φ(X)‖2 = ρ(X)− ∇2f (X)f (X)
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
If f is superharmonic then the formal posterior risk Ex‖ξ − φ(X)‖2 isan overestimated of the estimate φn(X) given by the last formula inthe sense that
Ex‖ξ − φ(X)‖2 ≥ ρ(X)
Now if the prior measure Π has a superharmonic density π , the f isalso superharmonic and thus φn(X) is a minimax estimate of ξ
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Let us look at estimates of the form
ξ = X − λ(X)AX
then the risk of the estimate ξ defined by the previous formula with
λ(X) = 1xT Bx
is given by
Eξ‖X − 1XT BX AX − ξ‖2 = p− Eξ( XT A2X
(XT BX)2 )
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Application to symetric moving averages
Let X1, ...,Xp be independently normally distribution with meansξ1, ......, ξp and variance 1, and suppose we plan to estimate the ξi by
ξ̂i = Xi − λ(X){Xi − 12(Xi−1 + Xi+1)}
where it is understood that X0=Xp and Xp+1=X1 and simalary for the ξ’s. This is the special case of
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Aij =
−12 si j− i ≡ 1 ( mod p )
1 si j− i ≡ 0 ( mod p )0 otherwise
The characteristics roots and vectors of A, the solution αj and yj of
Ayj = αjyj
where αj real and Rp are given with j varying over the intergers suchthat
− p2 ≤ j < p
2
by
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Application
αj = 1− cos(2π jp)
and for i ∈ {1.....}
yij =
1√p if j = 0
(−1)i√
p if j = −p2√
2p cos 2πij
p if −p2 < j < 0√
2p sin 2πij
p if 0 < j < p2
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Application
this being the ith coordinate of yi. The matrix A can be expressed asA = yαyT where α is the diagonal matrix and the matrix B, given is by
B = {tr(A)I − 2A−1}A2 = y(pI − 2α)−1α2yT
It is unreasonable to use a three-term moving average with weightmore extreme than (1
3 ,13 ,
13) . Thus it seems appropriate to modify our
estimate to
ξ̂ = X − λ1(X)AX
where
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
Application
λ1(X) = min( 1XT BX ,
23)
The unbiased estimate of the improvement in the risk is changed from∆(X) = XT A2X
(XT BX)2 given is
∆1(X) =
{∆(X) if XTBX > 3
24p3 −
49∑{1
2(Xi−1 + Xi+1)} if XTBX ≤ 32
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
We consider the James Stein estimate ξ̂0 = (1− p−2‖X‖2 )X
Let ξ̂ = X + g(X) where g : <p → <p is defined by
gl(X) =
{− a∑
(X2l ∧Z2
k )Xl if |X| ≤ Zk
− a∑(X2
l ∧Z2k )
ZksgnXl if |X| > Zk
And the risk is
Eξ‖ξ̂ − ξ‖2 = p− (k − 2)2Eξ( 1∑(X2
l ∧Z2k )
)
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
We observed that the estimated improvement in the risk for theestimate ξ̂k is
∆k(X) = (k−2)2∑(X2
l ∧Z2k )
and the estimated improvement in the risk for the James Steinestimate is
∆(X) = (p−2)2
E∑
(X2j
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
We would use the estimate ξ̂0 = X + g(X) whereg : <p → <p is homogeneous of degree -1 . We consider, for
the present problem the modified estimate ξ̂ = X + cSg(X) where c isconstant to be determined. Let Y = X
σ , η = ξσ , S∗ = S
σ2 .From theorem 1 we obtain
Eξ,σ‖X + Sn+2 g(X)− ξ‖2 = σ2E
[p + n
n+2(‖g(Y)‖2 + 2∇∗.g(Y))]
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
1 Introduction
2 Basic Formulas
3 Basic Formal Bayes Estimates
4 Choice of a scalar factor
5 Application
6 Another estimates
7 The case of unknown variance
8 Conclusion
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
ConclusionDifferent approaches to obtaining improved confidence sets for ξare described by Morris(1977), Faith (1978) and .
IntroductionBasic Formulas
Basic Formal Bayes EstimatesChoice of a scalar factor
ApplicationAnother estimates
The case of unknown varianceConclusion
References
Anderson, T.W ( 1971) The Statistical Analysis of Time Series.Wiley, New York.
BERGER, J J.(1980) A robust generalized Bayes estimor andconfidence region for a multivariate normal mean. Ann. Statist .8.
EFRON B. and Morris, C. ( 1971) . Limiting the risk of Bayesand empirical Bayes estimator, Part I: The Bayes case. J. Amer.Statist. Assoc . 66 807-815.
Thank you for your attention !!!Retour.