covariance estimation from large ensembles of …...covariance estimation from large ensembles of...
TRANSCRIPT
![Page 1: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/1.jpg)
COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF
SIMULATIONSLinda Blot
Fundamental Cosmology Meeting
Teruel - 12/09/2017
![Page 2: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/2.jpg)
MOTIVATIONS
➤ LSS data can constrain cosmological parameters with precision comparable to CMB data and are complementary
➤ Non-linearities are important modelling is harder
EUCLID collaboration
![Page 3: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/3.jpg)
FROM OBSERVATIONS TO CONSTRAINTS
➤ Ideal world: full multivariate probability distribution of the observable for all the models
➤ If we assume multivariate Gaussian -> mean and covariance
➤ Estimation of the covariance:
✦ internal: from data
✦ external: from simulations
✦ model: from the theoretical model
![Page 4: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/4.jpg)
FROM OBSERVATIONS TO CONSTRAINTS
➤ Ideal world: full multivariate probability distribution of the observable for all the models
➤ If we assume multivariate Gaussian -> mean and covariance
➤ Estimation of the covariance:
✦ internal: from data
✦ external: from simulations
✦ model: from the theoretical model
cov(k1, k2) =2
Nk1
P 2(k1)�k1,k2 +
1
V
Z
�k1
Z
�k2
d3k01
Vk1
d3k02
Vk2
T (k01,�k0
1,k02,�k0
2)
non-linear regime, bias + mask
![Page 5: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/5.jpg)
FROM OBSERVATIONS TO CONSTRAINTS
➤ Ideal world: full multivariate probability distribution of the observable for all the models
➤ If we assume multivariate Gaussian -> mean and covariance
➤ Estimation of the covariance:
✦ internal: from data
✦ external: from simulations
✦ model: from the theoretical model
Sample covariance}
dcov(k1, k2) =1
Ns � 1
NsX
i=1
[
ˆPi(k1)� ¯P (k1)][ ˆPi(k2)� ¯P (k2)]
![Page 6: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/6.jpg)
DEUS PARALLEL UNIVERSE RUN SIMULATIONS
Set A Set BSet C
WMAP-7 ΛCDM Set A: 12288 simulations
Set B: 96 simulations Set C: 512 simulations
Total: ~1.5M cpu hours on the ADA supercomputer at IDRIS
AMADEUS application:
• IC: optimised version of MPGRAFIC (Prunet 2008)
• N-body: improved version of RAMSES (Teyssier 2002)
• Halo finder: PFOF (Roy et al.2014)
![Page 7: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/7.jpg)
DEUS PARALLEL UNIVERSE RUN SIMULATIONS
![Page 8: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/8.jpg)
MASS RESOLUTION EFFECT
➤ Matter power spectrum
Heitmann et al. 2010Rasera et al. 2014
![Page 9: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/9.jpg)
100
101
102
cov(
k,k
)/(P
2 lin(k
)/N
k/2
)
A
B
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
�0.4�0.2
0.00.20.40.60.81.0
�co
v(k,k
)z = 0.00
z = 0.30
z = 0.50
z = 1.00
z = 2.00
MASS RESOLUTION EFFECT
➤ Matter power spectrum variance
LB et al. 2015
![Page 10: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/10.jpg)
Map the spectrum from the PDF of set A into the one of set B using only the first two moments
MASS RESOLUTION EFFECT CORRECTION
P̂ corr
A (k) =hP̂A(k)� P̄A(k)
i �ˆPB(k)
�ˆPA(k)
+ P̄B(k)
Prob
ability
P(k)
AB
![Page 11: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/11.jpg)
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.40.80.91.01.11.21.31.4
�B(k
)/�
A(k
)
z = 0.00
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.40.80.91.01.11.21.31.4
z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.40.80.91.01.11.21.31.4
�B(k
)/�
A(k
)
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4k (h/Mpc)
0.80.91.01.11.21.31.4
z = 1.000.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
k (h/Mpc)
0.80.91.01.11.21.31.4
�B(k
)/�
A(k
)
z = 2.00
100
101
102
cov(
k,k
)/(P
2 lin(k
)/N
k/2
)
A
B
corrected
�0.4�0.2
0.00.20.4
�co
v A(k
,k)
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
�0.4�0.2
0.00.20.4
�co
v B(k
,k)
z = 0.00
z = 0.30
z = 0.50
z = 1.00
z = 2.00
CORRECTED MATTER POWER SPECTRUM VARIANCE
P̂ corr
A (k) =hP̂A(k)� P̄A(k)
i �ˆPB(k)
�ˆPA(k)
+ P̄B(k)
Blot et al. 2015
![Page 12: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/12.jpg)
0.03 0.2 0.4 0.6 0.8 1.0k1 (h/Mpc)
0.03
0.2
0.4
0.6
0.8
1.0
k 2(h
/Mpc)
z = 2.00
�0.30
�0.15
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.2 0.4 0.6 0.8 1.0k1 (h/Mpc)
0.03
0.2
0.4
0.6
0.8
1.0k 2
(h/M
pc)
z = 1.00
�0.30
�0.15
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.2 0.4 0.6 0.8 1.0k1 (h/Mpc)
0.03
0.2
0.4
0.6
0.8
1.0
k 2(h
/Mpc)
z = 0.00
�0.30
�0.15
0.00
0.15
0.30
0.45
0.60
0.75
0.90
CORRELATION MATRIX
r(k1, k2) =cov(k1, k2)p
cov(k1, k1) cov(k2, k2)
Blot et al. 2015
![Page 13: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/13.jpg)
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
S3
z = 0.00 z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
S3
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
z = 105.00
PDF OF THE MATTER POWER SPECTRUM
Skewness
𝛘2 distribution with Nk d.o.f. → Gaussian for Nk >> 1
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.00 z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
z = 105.00
Blot et al. 2015
![Page 14: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/14.jpg)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.00 z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
z = 105.00
PDF OF THE MATTER POWER SPECTRUM
Kurtosis
𝛘2 distribution with Nk d.o.f. → Gaussian for Nk >> 1
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.00 z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
z = 105.00
Blot et al. 2015
![Page 15: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/15.jpg)
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
512
(328.125)3
5123
2 x 1010
Cosmo
Symmetric variation of parameters wrt fiducial WMAP7
![Page 16: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/16.jpg)
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
0.0 0.2 0.4 0.6 0.8 1.0k
10�1
100
101
102
103
104
105
106
107
108
109
�2 (
P(k
))
z = 2.0
z = 1.0
z = 0.0
fiducial
w = �1.2
w = �0.8
Preliminary
0.0 0.2 0.4 0.6 0.8 1.0k
0
100
200
300
400
500
600
700
800
�2 (
P(k
))/�
2 L(P
(k))
fiducial w=-1 Ωm = 0.2573
![Page 17: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/17.jpg)
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00 w = �1.2
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00 w = �0.8
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00
�0.02
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00 w = �1.2
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00 w = �0.8
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00
�0.01
0.00
0.01
0.02
0.03
0.04
0.05
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00 w = �1.2
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00 w = �0.8
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00
�0.002
0.000
0.002
0.004
0.006
0.008
0.010
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
Preliminary
![Page 18: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/18.jpg)
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
0.0 0.2 0.4 0.6 0.8 1.0k
10�1
100
101
102
103
104
105
106
107
108
109
�2 (
P(k
))
z = 2.0
z = 1.0
z = 0.0
fiducial⌦m = 0.3100
⌦m = 0.2046
Preliminary
0.0 0.2 0.4 0.6 0.8 1.0k
0
500
1000
1500
2000
2500
3000
�2 (
P(k
))/�
2 L(P
(k))
fiducial w=-1 Ωm = 0.2573
![Page 19: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/19.jpg)
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00 ⌦m = 0.3100
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00 ⌦m = 0.2046
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00
�0.04
0.00
0.04
0.08
0.12
0.16
0.20
0.24
0.28
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00 ⌦m = 0.3100
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00 ⌦m = 0.2046
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00
0.00
0.04
0.08
0.12
0.16
0.20
0.24
0.28
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00 ⌦m = 0.3100
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00 ⌦m = 0.2046
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00
0.00
0.04
0.08
0.12
0.16
0.20
0.24
Preliminary
![Page 20: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/20.jpg)
FOR THE FUTURE
➤ Impact on cosmological parameter constraints of
➤ Skewness of the PDF
➤ Cosmological dependence of the likelihood (covariance)
![Page 21: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/21.jpg)
CONCLUSIONS
➤ Covariances are affected by numerical systematics
➤ Non-linearities skew the PDF of the matter power spectrum
➤ Covariance depend on cosmological parameters
![Page 22: COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting ... • IC: optimised version](https://reader030.vdocuments.net/reader030/viewer/2022040105/5e8e552a48a4934df962c7cc/html5/thumbnails/22.jpg)
COVARIANCE WITH APPROXIMATE METHODS
2.61 2.62 2.63 2.64 2.65
0.435
0.440
0.445
b1
f¥s8
kmax=0.25Fisher Forecast
MinervaCOLAPinocchio
Euclid Likelihood fitting WP