markov chain monte carlo and stochastic origin ensembles...

Markov chain Monte Carlo and stochastic

origin ensembles methodsComparison of a simple application for a Compton imager detector

Pierre-Luc Drouin

DRDC – Ottawa Research Centre

Defence Research and Development CanadaScientific Report

DRDC-RDDC-2016-R124

September 2016

c© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence,

2016

c© Sa Majesté la Reine (en droit du Canada), telle que réprésentée par le ministre de la Défense

nationale, 2016

Abstract

This document aims to summarise Markov Chain Monte Carlo (MCMC) methods, in par-

ticular, the Metropolis-Hastings algorithm and the Stochastic Origin Ensembles (SOE)

method, in a concise and notation-consistent manner. These methods are commonly used to

perform model parameter estimation for a population, based on a measured sample, through

the sampling of the probability distribution for these parameters. A simple application of

SOE is then demonstrated using simulation data from a Compton imager detector.

Significance for defence and security

In Radiation and Nuclear (RN) defence, many detection technologies rely on model pa-

rameter estimation to perform threat detection, identification and/or localisation. Such al-

gorithms are often required when the direct observation of the parameters of concern is not

possible. Some detection systems, such as Compton imagers and muon tomography sys-

tems, need to make extensive use of such algorithms compared to traditional technologies.

This document summarises a set of algorithms which represent good candidates for the es-

timation of the parameters for these new technologies. Hardware and software development

of such systems is currently ongoing at DRDC – Ottawa Research Centre.

DRDC-RDDC-2016-R124 i

Résumé

Ce document vise à résumer les méthodes de Monte-Carlo par Chaînes de Markov, et en

particulier, l’algorithme de Metropolis-Hastings et la méthode des Ensembles d’Origine

Stochastique (EOS), d’une manière concise et tout en utilisant une notation consistante.

Ces méthodes sont fréquemment utilisées afin d’estimer les paramètres d’un modèle pour

une population, basé sur la mesure d’un échantillon, par le biais de l’échantillonnage de la

distribution probabilistique pour ces paramètres. Une application simple de la métode EOS

est ensuite démontrée en utilisant des donnés simulées provenant d’un imageur Compton.

Importance pour la défense et la sécurité

En défence Radiologique et Nucléaire (RN), plusieurs technologies de détection dépendent

sur l’estimation de paramètres d’un modèle afin d’accomplir la détection de menaces,

leur identification et/ou leur localisation. De tels algorithmes sont souvent requis lorsque

l’observation directe des paramètres concernés n’est pas possible. Certains systèmes de

détection, tels que les imageurs Compton et les systèmes de tomographie muonique, re-

quièrent un usage intensif de tels algorithmes comparativement aux technologies tradi-

tionnelles. Ce document résume un ensemble d’algorithmes qui représentent de bons can-

didats afin d’estimer les paramètres pour ces nouvelles technologies. Du développement

d’équipement ainsi que logiciel est en cours au RDDC – Centre de recherches d’Ottawa.

ii DRDC-RDDC-2016-R124

Table of contents

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i

Significance for defence and security . . . . . . . . . . . . . . . . . . . . . . . . . . i

Résumé . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

Importance pour la défense et la sécurité . . . . . . . . . . . . . . . . . . . . . . . . ii

Table of contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

List of figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

List of tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Metropolis–Hastings algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

3 Sampling from the probability density function of an evaluated model . . . . . . 3

4 Stochastic origin ensembles method . . . . . . . . . . . . . . . . . . . . . . . . 4

5 Application of SOE for a Compton imager . . . . . . . . . . . . . . . . . . . . . 6

6 Comparison of the SOE algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 7

7 Limitations of the simple SOE algorithm . . . . . . . . . . . . . . . . . . . . . . 10

8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

DRDC-RDDC-2016-R124 iii

List of figures

Figure 1: Images of a simulated 137Cs point source located 10◦ off-axis as

produced using simple back projection (top), smoothed MLEM (left)

and SOE (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Figure 2: Images of a simulated 137Cs “C-shaped” source as produced using

simple back projection, MLEM and SOE with 2◦ (“Coarse”) and 1◦

(“Fine”) prior PDF binning. . . . . . . . . . . . . . . . . . . . . . . . . 9

List of tables

Table 1: Measured mean width for a 2D Gaussian fitted on the distributions

resulting from the three different algorithms. . . . . . . . . . . . . . . . 8

iv DRDC-RDDC-2016-R124

1 Introduction

In statistical data analysis, parameter estimation techniques are commonly used to deter-

mine model parameters for a population, based on a measured sample. Maximum-likeli-

hood estimation is a well known and widely used approach, which aims at determining

the model parameters that maximise the probability of the measured sample. Although this

technique is well suited for a large number of applications, it can be difficult to apply for

some situations in practice, for example when the number of unknown model parameters

is very large and/or when the model can be defined more easily by introducing latent (hid-

den) variables. This document first presents a summary of Markov Chain Monte Carlo

(MCMC) methods, in particular, the Metropolis–Hastings algorithm, which can be used to

perform parameter estimation in such situations. The Stochastic Origin Ensembles (SOE)

method, which represents a particular application of the Metropolis-Hastings algorithm, is

then described, and results from a simple application of this method are then compared to

alternative algorithms.

As shown in this document, advanced radiation imaging systems, such as Compton im-

agers, greatly benefit from using models which use a large number of latent variables.

These imagers allow for a quick localisation of point or extended radioactive sources that

can be partially shielded by the material of a cluttered environment. Such a task can repre-

sent a real challenge when attempted using traditional detectors.

2 Metropolis–Hastings algorithm

The Metropolis–Hastings algorithm [1, 2] is a random walk Monte Carlo method, which is

a subclass of the MCMC methods, and that samples from a Probability Density Function

(PDF) f (�θ), through the usage of a state vector�θ which is randomly moving in the parame-

ter space of the PDF. A Markov process is uniquely defined by the expression of a transition

PDF f (�θ →�θ′) which provides the probability density of transiting from state�θ to state�θ′.A Markov chain, defined as a sequence of such state vectors, is thus “memoryless”, since

the probability of transition to the following state depends only on the current state. A

Markov process reaches a unique stationary distribution π(�θ) = f (�θ) asymptotically when

the two following conditions are met:

• the condition of detailed balance,

π(�θ) f (�θ →�θ′) = π(�θ′) f (�θ′ →�θ), (1)

• and the Markov process must be ergodic, meaning that the process must be aperiodic

and must be able to return to any given state�θ in a finite number of steps (irreducible

process).

DRDC-RDDC-2016-R124 1

The Metropolis-Hastings algorithm ensures that Condition (1) is met by enforcing it di-

rectly through the relationship between f (�θ) and f (�θ →�θ′):

f (�θ) f (�θ →�θ′) = f (�θ′) f (�θ′ →�θ) (2)

f (�θ →�θ′)

f (�θ′ →�θ)=

f (�θ′)

f (�θ). (3)

The transition PDF f (�θ →�θ′) is then expressed as the product between a proposal PDF

g(�θ →�θ′) and an acceptance PDF h(�θ →�θ′):

f (�θ(trans.)→ �θ′) = f (�θ

prop.→ �θ′,�θ

accept.→ �θ′)

= f (�θprop.→ �θ′) f (�θ

accept.→ �θ′|�θ

prop.→ �θ′)

=g(�θ →�θ′)h(�θ →�θ′). (4)

In order to fulfill the ergodicity condition, g(�θ →�θ′) must be aperiodic and both g(�θ →�θ′)

and h(�θ →�θ′) must allow f (�θ →�θ′) to be irreducible. From Equations (3) and (4), the

condition on h(�θ →�θ′) is

h(�θ →�θ′)

h(�θ′ →�θ)=

g(�θ′ →�θ) f (�θ′)

g(�θ →�θ′) f (�θ). (5)

The Metropolis-Hastings algorithm uses the following expression for h(�θ →�θ′) to satisfy

the above constraint:

h(�θ →�θ′) = min

{1,

g(�θ′ →�θ) f (�θ′)

g(�θ →�θ′) f (�θ)

}. (6)

Equation (6) satisfies the irreducibility condition when g(�θ→�θ′) is irreducible. The Metro-

polis-Hastings algorithm thus relies uniquely on g(�θ →�θ′) to satisfy the ergodicity con-

dition, while the expression for h(�θ →�θ′) ensures that the condition of detailed balanced

can be reached. Note also that due to the form of Equation (6), the Metropolis-Hastings

expression for the transition PDF f (�θ →�θ′) is insensitive to the normalisation or scaling

of f (�θ), meaning that the expression for f (�θ) needs only to be known up to a scaling factor

that does not depend on�θ, which can greatly simplify computation in practice.

Using the above results and an initial state�θs with s= 0, the Metropolis-Hastings algorithm

can be executed as follows:

1. Draw a random proposed state �θ′ according to the chosen ergodic proposal PDF

g(�θs →�θ′).

2 DRDC-RDDC-2016-R124

2. Computeg(�θ′→�θs) f (�θ′)

g(�θs→�θ′) f (�θs). If the resulting value is greater or equal to either 1 or other-

wise, to a random number drawn uniformly in the interval ]0,1], the proposed state

is accepted,�θs+1 =�θ′, s is incremented by 1 and the algorithm continues to Step 1.

3. �θs+1 =�θs, s is incremented by 1 and the algorithm continues to Step 1.

This procedure allows the distribution of states to converge to f (�θ) once the equilibrium is

reached. However, the choice for the initial state�θ0 as well as the proposal PDF g(�θ →�θ′)can greatly affect the required number of steps required to reach this regime. Also, unless

g(�θ →�θ′) has a negligible dependency on�θ and the acceptance of�θ′ is very likely, there

can be a significant autocorrelation within the MCMC chain, such that consecutive states

cannot be considered to be statistically independent. The usage of correlated states for

estimator computation can lead to biased results. However, choosing a function g(�θ →�θ′)

having a weak dependency on�θ often results in a very small probability of acceptance for�θ′. It can thus be advantageous to accept a higher dependency within g(�θ →�θ′) to increase

the probability of acceptance. A “reduced” chain of states with low autocorrelation can then

be obtained by sampling the original chain at a fixed interval which is sufficiently large for

the desired intent. A Metropolis-Hastings algorithm thus often involves the rejection of a

number of “burn-in” steps required to reach the equilibrium, followed by a sampling of

the remaining steps to insure a sufficiently low autocorrelation. The determination of the

number of burn-in steps and the sampling rate is discussed in detail in [3].

In the case of multi-dimensional state vectors, a common method to easily increase the level

of acceptance consists in a proposal function that randomly updates a single component of�θat each step. This implies a sampling rate of the chain which is greater than the dimension

of the state vector to achieve statistical independence between the states of the reduced

chain.

3 Sampling from the probability density function of an

evaluated model

MCMC methods can be used to sample from the PDF of a model which is evaluated using

a data sample. Let the conditional probability density of a single measured event e, as

computed by a model characterised by a set of parameters�θ, be defined as f (�xe|�θ), where

�xe represents the coordinate of the event in the measurement parameter space. Assuming

the statistical independence of the events within a measured sample, the PDF of a sample,

its likelihood function, can be expressed as

L(sample|�θ) = f (sample|�θ) =nevents

∏e=1

f (�xe|�θ), (7)


where nevents is the number of measured events. In the common case of a measurement

where the number of measured events is Poisson distributed, an extended likelihood func-

tion is defined as

Le(sample|�θ) = fe(sample|�θ) = P(nevents|ν) f (sample|�θ)

=e−ννnevents

nevents!

nevents

∏e=1

f (�xe|�θ), (8)

where ν is a parameter, a subcomponent of�θ, for the Poisson Probability Mass Function

(PMF) P(n|ν) = e−ννn

n!, which corresponds to the average number of measured events.

Estimation methods, such as Maximum Likelihood (ML) and Maximum Likelihood Ex-

pectation Maximisation (MLEM), provide estimators�̂θ that maximise the likelihood L(e) of

the measured sample. ML methods can estimate the variance of the estimators through dif-

ferent methods such as contour lines in the likelihood parameter space. In contrast, MCMC

methods can be used to sample from the f(e)(�θ|sample) PDF directly, such that the result-

ing posterior distribution can be used to evaluate any metric. These posterior distributions

contain more information than provided by ML methods and are thus suited to characterise

estimator uncertainties in the case of non-Gaussian statistics.

f (�θ) is often expressed as a function of the likelihood function, through the usage of Bayes’

theorem:

f(e)(�θ|sample) =L(e)(sample|�θ)fp(�θ)

fp(sample), (9)

where f(e)(�θ|sample) represents the MCMC PDF f (�θ). In the above expression, fp(�θ) is

the prior PDF for the model parameters, while the PDF fp(sample) is the prior for the

measured sample.

Sampling simplifications with the Metropolis-Hastings algorithm

As previously mentioned, for the Metropolis-Hastings algorithm, sampling of a PDF can

be performed as long as the PDF is known up to a scaling factor which does not depend on

the sampled parameters. This can simplify the task, notably when expressing the PDF as

a function of the likelihood function. From Equation (9), it becomes no longer relevant to

evaluate the fp(sample) expression and the PDF is now more simply interpreted as

f(e)(�θ|sample) ∝L(e)(sample|�θ) fp(�θ). (10)

4 Stochastic origin ensembles method

The Stochastic Origin Ensembles method [4] represents a particular application of the Me-

tropolis-Hastings algorithm for a scenario where the parameter vector�θ can be subdivided


in a set of�φ subcomponents

�θ ≡(�φ1,�φ2, . . . ,�φnevents

), (11)

each one respectively associated to a corresponding event in the measured sample. The�φcomponents can represent hidden variables of the process and are considered to be inde-

pendent from each other since they are associated to independently measured events. A

priori, these components nonetheless follow an unknown distribution, and this distribution

is approximated using the distribution of the�φ components within�θ. The fp(�θ) prior PDF

is then computed by evaluating the probability density of each subcomponent in this dis-

tribution. Usually, the posterior sample of�θ parameter values, as provided by the resulting

Markov chain, is then used to generate a distribution in the�φ parameter space.

Due to the statistical independence of the�φ subcomponents, fp(�θ) can be expressed as

fp(�θ) =nevents

∏e=1

f (�φe), (12)

where f (�φ) is the unknown distribution which is approximated using the distribution of the�φ components within�θ:

f (�φ)≈ fb(�φ|�θ). (13)

In the above expression, fb(�φ|�θ) is a binned PDF in the�φ parameter space, which is pop-

ulated using the �θ sample. The approximation of f (�φ) using fb(�φ|�θ) does not affect the

convergence of the Markov chain to the stationary distribution; it is guarantied by the

Metropolis-Hastings algorithm.

We define a linearised bin index k for the binned�φ parameter space and let n(k|�θ) represent

the number of components within the �θ sample that fall in bin k. If the function b(�φ)

provides the bin index k associated to the �φ coordinate, then fb(�φ|�θ) is proportional to

fb(�φ|�θ) ∝ n(b(�φ)|�θ). (14)

Using Equations (12) to (14), we then have

fp(�θ)≈nevents

∏e=1

fb(�φe|�θ) ∝nevents

∏e=1

n(b(�φe)|�θ) =nbins

∏k=1

n(k|�θ)n(k|�θ), (15)

where nbins is the total number of bins within fb(�φ|�θ). With the SOE algorithm, the ratiof (�θ′)

f (�θ)from the Metropolis-Hastings algorithm is thus given by

f (�θ′)

f (�θ)≈

L(sample|�θ′)

L(sample|�θ)

nbins

∏k=1

n(k|�θ′)n(k|�θ′)

n(k|�θ)n(k|�θ), (16)


and similar to the case of an extended likelihood. If the implementation of the algorithm is

such that a single component of�θ is updated by the chosen proposal function g(�θ →�θ′),then the above expression can be simplified even further. We define o′ and d′ as the origin

and destination bin indices for the proposed transition, respectively. Thus, we have, when

o′ �= d′:

n(o′|�θ′) =n(o′|�θ)−1 (17)

n(d′|�θ′) =n(d′|�θ)+1, (18)

such thatnbins

∏k=1

n(k|�θ′)n(k|�θ′)

n(k|�θ)n(k|�θ)=

[n(o′|�θ)−1]n(o′|�θ)−1[n(d′|�θ)+1]n(d

′|�θ)+1

n(o′|�θ)n(o′|�θ)n(d′|�θ)n(d′|�θ). (19)

This finally gives

f (�θ′)

f (�θ)=

L(sample|�θ′)

L(sample|�θ)

⎧⎨⎩

1, if o′ = d′

[n(o′|�θ)−1]n(o′|�θ)−1[n(d′|�θ)+1]n(d

′|�θ)+1

n(o′|�θ)n(o′|�θ)n(d′|�θ)n(d′|�θ), otherwise

. (20)

The above ratio can thus be used within the Metropolis-Hastings algorithm as described in

the previous section to sample from f (�θ) when the model parameters are associated to the

events in the measured sample.

5 Application of SOE for a Compton imager

A simple application of the SOE method can be performed for a Compton imager [5],

where one wants to determine the distribution of �φ, defined as the angular origin of the

detected radiation. For a simplified physical model where the PDF for the angular origin

of a given detected event consists in a ring resulting from back projection of the Compton

cone [6], the likelihood of the angular origin for the event is constant along that ring and

otherwise null. This model assumes that the energy of the analysed photons is perfectly

known. If one chooses a function g(�θ →�θ′) which only proposes origins along these rings,

the expression for h(�θ →�θ′) can be easily evaluated, since the ratio of likelihood values

in Equation (20) is equal to 1 and the proposal function is symmetrical. The resulting

algorithm to generate the Markov chain for this simple Compton imager SOE is thus given

by:

1. Pick an initial origin for�θ, using random positions along the back projected rings of

the measured events.

2. Compute the values for n(k|�θ) by filling an histogram, and record the bin index for

each event.


3. Draw a random event index e uniformly out of the nevents events in the measured

sample.

4. Pick a random position along the back projected ring for the selected event and de-

termine the proposed destination bin d′.

5. Computef (�θ′)

f (�θs)as given by Equation (20) (where the ratio of likelihood values is equal

to 1), using the destination bin d′ and the recorded current bin index for the event e

as the o′ bin. If the resulting value is greater or equal to either 1 or otherwise, to a

random number drawn uniformly in the interval ]0,1], the proposed state is accepted,

n(o′|�θ′) and n(d′|�θ′) and the current bin index for event e are updated,�θs+1 =�θ′, s is

incremented by 1 and the algorithm continues to Step 3.

6. �θs+1 =�θs, s is incremented by 1 and the algorithm continues to Step 3.

7. The resulting chain must be sampled at least every nevents steps.

6 Comparison of the SOE algorithm

In this section, results from the application of the SOE algorithm to Compton imager sim-

ulation data are presented and compared to alternate algorithms. The algorithm was tested

using 1453 detected gamma rays from a 137Cs point source located 10◦ off-axis. The prior

SOE histogram was binned using 50×50 bins from −90◦ to 90◦ in each direction. A to-

tal of 100 000 steps were generated, including 50 000 burn-in steps, and one step every

100 steps was then used to generated the posterior distribution. Figure 1 shows the result-

ing distribution, along with corresponding results which were obtained in [6] using simple

back projection and smoothed MLEM. Corresponding angular resolution results for this

simulation are shown in Table 1, where a 2D Gaussian distribution was fitted using a range

of ±10◦ around the peak. When comparing MLEM and SOR results to back projection re-

sults, there is an obvious improvement for both the source image and the angular resolution

values, as these two algorithms allow the elimination of the circular patterns produced by

the back projection of the Compton cones. The mean Gaussian width is reduced by more

than 50%, which is a very significant improvement considering that the three algorithms are

not using assumptions regarding the source’s spatial distribution. When comparing MLEM

and SOE results, Figure 1 shows slight clustering of events outside the source location with

the MLEM technique, which is not present with SOE. The computed angular resolution is

slightly better with SOE, and the fit range for the 2D Gaussian excluded most of the visible

clusters in MLEM’s results.

An infinitesimally thin “C-shaped” 137Cs source was also imaged, using the same simu-

lated dataset that was processed in [6]. The SOE algorithm was run twice to observe the

effect of different prior PDF binning. The results are presented in Figure 2. Similarly to


��

��

� �

� �

� �

� �

�

�

�

�

�

�

�

!

"�

"�

"�

"

"!

��

Figure 1: Images of a simulated 137Cs point source located 10◦ off-axis as produced using

simple back projection (top), smoothed MLEM (left) and SOE (right).

Table 1: Measured mean width for a 2D Gaussian fitted on the distributions resulting

from the three different algorithms.

Algorithm Resolution

[◦]

Simple back projection 7.1±0.1

MLEM 3.2±0.1

SOE 3.1±0.1


��

��

� �

� �

� �

� �

�

�

�

�

��

�

��

�

��

�

��

� ! "#$%&' ( )*+,-

��

��

� �

� �

� �

� �

�

�

�

�

�

�

��

�

��

�

��

�� !�" #$%&'( ) *+,-.

��

��

� �

� �

� �

� �

�

�

�

�

�

�

�

�

�

�

�

!" #$%&'() $*&+( , #'-./

��

��

� �

� �

� �

� �

�

�

�

�

�

�

�

�

�

��

��

� ! "#$%& �'()*% + ,-./0

Figure 2: Images of a simulated 137Cs “C-shaped” source as produced using simple back

projection, MLEM and SOE with 2◦ (“Coarse”) and 1◦ (“Fine”) prior PDF binning.


the results obtained with the point source, the figure shows a drastic improvement of the

reconstructed image when comparing SOE with the back projection method, which trans-

lates into a higher contrast. When qualitatively comparing SOE results with MLEM, we

observe a dependence of the outcome on the binning that is chosen for the prior PDF, as

one would expect since this effectively approximates the probability density to be uniform

within a given bin. Correlations between image bins are thus reduced as the prior PDF bin-

ning becomes finer, but this is done at the expense of increased statistical noise, since the

prior PDF is approximated using the measured sample itself. When comparing image blur,

the MLEM result appears to fall between the two SOE results. The SOE result with finer

binning could benefit from a postsmoothing filter in order to reduce the statistical noise

effects. Similar structures have been observed with the MLEM method when the number

of iterations becomes too large.

7 Limitations of the simple SOE algorithm

The simple SOE method based on a back projection model which was described in Sections

5 and 6 has the advantage of being analytical and easy to compute. However, it has many

of the flaws of the other methods based on this model:

• it assumes that the detector measures energies exactly, such that back projected

events lie along an infinitesimal ring rather than a broad uncertainty band;

• the detector angular response is assumed to be uniform;

• energy deposition is assumed to occur in the centroid of the detector pixels;

• the analysed measured sample is assumed to be pure, without any contamination

such as multiple scattering within the scatter plane, backscattering off the absorber

plane or undetected energy escaping either plane;

• the source is assumed to be at an infinite distance from the detector; and

• the lateral position of the scatter pixel with respect to the origin of the back projection

angular space is neglected.

This list of approximations can contribute to various unwanted effects, such as image blur,

distortion, artificial image structures, etc. A more realistic model which would avoid so

many approximations is thus desirable. Except for the last two items, the above approxi-

mations will be addressed in the future.


8 Conclusion

In this document, the Metropolis-Hastings algorithm was presented, following a sum-

mary of MCMC methods. The SOE method was then explained, as a special case of the

Metropolis-Hastings algorithm, and a simple implementation for a Compton imager was

demonstrated and compared to alternate algorithms. When compared to the back projec-

tion method, the SOE method showed a drastic improvement of the reconstructed image.

The latter method produced results which were comparable to the results obtained with

MLEM. Compared to MLEM, SOE showed less clustering of events outside the simulated

point source location.


This page intentionally left blank.


References

[1] Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E.

(1953), Equation of State Calculations by Fast Computing Machines, The Journal of

Chemical Physics, 21(6), 1087–1092.

[2] Hastings, W. K. (1970), Monte Carlo Sampling Methods Using Markov Chains and

Their Applications, Biometrika, 57(1), 97–109.

[3] Raftery, A. E. and Lewis, S. M. (1995), The Number of Iterations, Convergence

Diagnostics and Generic Metropolis Algorithms, In Practical Markov Chain Monte

Carlo (W.R. Gilks, D.J. Spiegelhalter and S. Richardson, eds.), pp. 115–130,

Chapman and Hall.

[4] Sitek, A. (2008), Representation of photon limited data in emission tomography using

origin ensembles, Physics in medicine and biology, 53(12), 3201–3216.

[5] Andreyev, A. (2009), Stochastic image reconstruction method for Compton camera,

In Nuclear Science Symposium Conference Record (NSS/MIC), 2009 IEEE,

pp. 2985–2988, IEEE.

[6] Ueno, R. (2016), Development of the GEANT4 Simulation for the Compton

Gamma-Ray Camera, (DRDC-RDDC-2016-C138) Defence Research and

Development Canada – Ottawa Research Centre.


This page intentionally left blank.


DOCUMENT CONTROL DATA(Security markings for the title, abstract and indexing annotation must be entered when the document is Classified or Protected.)

1. ORIGINATOR (The name and address of the organization preparingthe document. Organizations for whom the document was prepared,

e.g. Centre sponsoring a contractor’s report, or tasking agency, areentered in section 8.)

3701 Carling Avenue, Ottawa ON K1A 0Z4,

Canada

2a. SECURITY MARKING (Overall security marking ofthe document, including supplemental markings if

applicable.)

UNCLASSIFIED

2b. CONTROLLED GOODS

(NON-CONTROLLED GOODS)

DMC A

REVIEW: GCEC DECEMBER 2012

3. TITLE (The complete document title as indicated on the title page. Its classification should be indicated by the appropriateabbreviation (S, C or U) in parentheses after the title.)

Markov chain Monte Carlo and stochastic origin ensembles methods

4. AUTHORS (Last name, followed by initials – ranks, titles, etc. not to be used.)

Drouin, P.-L.

5. DATE OF PUBLICATION (Month and year of publication ofdocument.)

September 2016

6a. NO. OF PAGES (Totalcontaining information.Include Annexes,

Appendices, etc.)

22

6b. NO. OF REFS (Totalcited in document.)

6

7. DESCRIPTIVE NOTES (The category of the document, e.g. technical report, technical note or memorandum. If appropriate, enter

the type of report, e.g. interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period iscovered.)

Scientific Report

8. SPONSORING ACTIVITY (The name of the department project office or laboratory sponsoring the research and development –include address.)

DRDC – Ottawa Research Centre

3701 Carling Avenue, Ottawa ON K1A 0Z4, Canada

9a. PROJECT OR GRANT NO. (If appropriate, the applicable

research and development project or grant number underwhich the document was written. Please specify whetherproject or grant.)

9b. CONTRACT NO. (If appropriate, the applicable number under

which the document was written.)

10a. ORIGINATOR’S DOCUMENT NUMBER (The officialdocument number by which the document is identified by the

originating activity. This number must be unique to thisdocument.)

DRDC-RDDC-2016-R124

10b. OTHER DOCUMENT NO(s). (Any other numbers which maybe assigned this document either by the originator or by the

sponsor.)

11. DOCUMENT AVAILABILITY (Any limitations on further dissemination of the document, other than those imposed by securityclassification.)

Unlimited

12. DOCUMENT ANNOUNCEMENT (Any limitation to the bibliographic announcement of this document. This will normally correspondto the Document Availability (11). However, where further distribution (beyond the audience specified in (11)) is possible, a wider

announcement audience may be selected.)

Unlimited

13. ABSTRACT (A brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. It is highlydesirable that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with an indication of thesecurity classification of the information in the paragraph (unless the document itself is unclassified) represented as (S), (C), or (U). It is

not necessary to include here abstracts in both official languages unless the text is bilingual.)

This document aims to summarise Markov Chain Monte Carlo (MCMC) methods, in particular,

the Metropolis-Hastings algorithm and the Stochastic Origin Ensembles (SOE) method, in a

concise and notation-consistent manner. These methods are commonly used to perform model

parameter estimation for a population, based on a measured sample, through the sampling of the

probability distribution for these parameters. A simple application of SOE is then demonstrated

using simulation data from a Compton imager detector.

14. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Technically meaningful terms or short phrases that characterize a document and could

be helpful in cataloguing the document. They should be selected so that no security classification is required. Identifiers, such asequipment model designation, trade name, military project code name, geographic location may also be included. If possible keywordsshould be selected from a published thesaurus. e.g. Thesaurus of Engineering and Scientific Terms (TEST) and that thesaurus identified.

If it is not possible to select indexing terms which are Unclassified, the classification of each should be indicated as with the title.)

Markov chain Monte Carlo

Metropolis-Hastings

Stochastic Origin Ensembles

Parameter estimation

Maximum likelihood

Maximum Likelihood Expectation Maximisation

markov chain monte carlo and stochastic origin ensembles...

Documents