MCMC for Hard (but not too hard) Inverse Problems

Shane Reese, Dept of Statistics, BYUDave Higdon, Statistical Sciences, LANLDave Moulton, Applied Math, LANLJasper Vrugt, Hydrology, LANLColin Fox, Physics, Otago

Posterior Exploration for Computationally Intensive Forward Models

Computer Models at Los Alamos National Laboratory

hydrologycosmology

ocean circulation

agent-based models

extreme physicsnumber of simulations100101105106+102104103discoverysensitivity analysisresponse surfacecalibration & predictionforward Monte CarloMCMCutilizationCFDExample: Electrical Impedance Tomography

Conductivity: 24x24 lattice of pixels, each with a resistance between 2.5 and 4.5. Here =4, =3.Current I injected into one electrode, -I/15 extracted from remaining 15 electrodes. Pairwise Difference Image Priors m = 24x24 lattice with1st order neighborhoods

Prior realization with (,s)=(.5,.3)

More commonly: u(d) = -d2 (GMRF) u(d) = -|d| (L1 MRF) scene-based, or template priorPosterior DistributionAlgorithms:

Posterior realizations for conductivity obtained using MCMC

Single-site Metropolis (ssm)Multivariate random walk Metropolis (rwm)Differential evolution (DE-MCMC)Using fast, approximate forward modelsDelayed rejection MetropolisPosterior augmentation

Single Site Metropolis

Doesnt need very smart proposalsComputationally demanding (m updates/scan)

Single site Metropolis: traces of 3 pixels

Computational effort (simulator evaluations x m)

Posterior mean imageMultivariate Random Walk MetropolisHighly multivariate single update changes all pixelsHard to choose z in high-dimensional settings

Random Walk Metropolis: traces of 3 pixels

Differential Evolution MCMC

Highly multivariate single update changes all pixelsLike RWM but z info held in the P copiesDifferential Evolution MCMCProposal direction depends on randomly chosen pairSimilar (in spirit?) to Christen & Foxs proposalDE-MCMC ter-Braak (2006)

xpxqxrxpxp=xp+(xq-xr)+z

Differential Evolution MCMC: traces of 3 pixels

SSM for equivalent computational effortDE-MCMC for chain 2DE-MCMC for chain 1DE-MCMC for chain 3Differential Evolution MCMC: posterior for blue pixel in each of the 400 chains

Mean and sd from each of the P=400 chainsInitialized using a large SSM runMCMC Using Fast, Approximate Forward ModelsCoarsened representationsLow-fidelity forward modelsPhysical SystemObservationsF0F1F2FAFBDelayed Acceptance MetropolisPretests using the fast solver - from Christen & Fox (2004)Can be used along with any previous mcmc schemeOptimal acceptance rate?

Augmented Posterior

Final RemarksHard to beat single site Metropolis for this type of problemUsing fast, approximate solvers can speed things up by a factor of 2-5.Currently working on embedding an augmented MCMC scheme within a multigrid solverUse of template/scene priorsDidnt account for modeling error hereAlternative prior formulations

Example: Electrical Impedance Tomography