![Page 1: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/1.jpg)
1
Cluster Monte Carlo Cluster Monte Carlo Algorithms Algorithms
Jian-Sheng WangJian-Sheng WangNational University of SingaporeNational University of Singapore
Cluster Monte Carlo Cluster Monte Carlo Algorithms Algorithms
Jian-Sheng WangJian-Sheng WangNational University of SingaporeNational University of Singapore
2
Outlinebull Introduction to Monte Carlo and
statistical mechanical modelsbull Cluster algorithmsbull Replica Monte Carlo
3
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
4
Stanislaw Ulam (1909-1984)
S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 2: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/2.jpg)
2
Outlinebull Introduction to Monte Carlo and
statistical mechanical modelsbull Cluster algorithmsbull Replica Monte Carlo
3
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
4
Stanislaw Ulam (1909-1984)
S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 3: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/3.jpg)
3
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
4
Stanislaw Ulam (1909-1984)
S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 4: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/4.jpg)
4
Stanislaw Ulam (1909-1984)
S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 5: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/5.jpg)
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 6: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/6.jpg)
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 7: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/7.jpg)
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 8: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/8.jpg)
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 9: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/9.jpg)
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 10: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/10.jpg)
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 11: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/11.jpg)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 12: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/12.jpg)
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 13: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/13.jpg)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 14: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/14.jpg)
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 15: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/15.jpg)
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 16: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/16.jpg)
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 17: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/17.jpg)
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 18: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/18.jpg)
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 19: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/19.jpg)
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 20: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/20.jpg)
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 21: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/21.jpg)
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 22: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/22.jpg)
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 23: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/23.jpg)
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 24: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/24.jpg)
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 25: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/25.jpg)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 26: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/26.jpg)
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 27: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/27.jpg)
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 28: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/28.jpg)
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 29: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/29.jpg)
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 30: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/30.jpg)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 31: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/31.jpg)
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 32: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/32.jpg)
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 33: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/33.jpg)
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 34: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/34.jpg)
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 35: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/35.jpg)
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 36: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/36.jpg)
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 37: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/37.jpg)
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 38: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/38.jpg)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 39: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/39.jpg)
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 40: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/40.jpg)
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 41: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/41.jpg)
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 42: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/42.jpg)
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 43: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/43.jpg)
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 44: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/44.jpg)
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 45: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/45.jpg)
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 46: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/46.jpg)
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 47: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/47.jpg)
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 48: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/48.jpg)
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 49: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/49.jpg)
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
![Page 50: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore](https://reader030.vdocuments.net/reader030/viewer/2022012916/56649f3c5503460f94c5b165/html5/thumbnails/50.jpg)
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems