Computational Computational Methods in PhysicsMethods in Physics

PHYS 3437 PHYS 3437Dr Rob ThackerDr Rob Thacker

Dept of Astronomy & Physics Dept of Astronomy & Physics (MM-301C)(MM-301C)

thacker@ap.smu.cathacker@ap.smu.ca

Today’s LectureToday’s Lecture

Use MC techniques to simulate a Use MC techniques to simulate a random walkrandom walk

A few slides on random number A few slides on random number generatorsgenerators

Mobility of molecules Mobility of molecules

Consider a single molecule suspended in Consider a single molecule suspended in air, on average how far will it travel in 1s?air, on average how far will it travel in 1s?

Naïve calculation: mvNaïve calculation: mv22/2=(5/2)kT for a /2=(5/2)kT for a diatomic gas, hencediatomic gas, hence

k=1.38x10k=1.38x10-23-23 J s J s-1-1 and take T=300 K, take and take T=300 K, take mass to be that of Nmass to be that of N22~28m~28mpp=4.68x10=4.68x10-26-26 kg kg

Implies a velocity of v=665 m sImplies a velocity of v=665 m s-1-1

m

kTv

5

Distance travelledDistance travelled

So does a molecule move 2/3 km on So does a molecule move 2/3 km on average in 1s?average in 1s?

No – collisions ensure that the motion No – collisions ensure that the motion changes direction an enormous changes direction an enormous number of times per secondnumber of times per second

Result – average molecule moves < 1 Result – average molecule moves < 1 m in a secondm in a second

The path of a particle frequently The path of a particle frequently interrupted by collisions is called a interrupted by collisions is called a random walkrandom walk

Random walks & Random walks & diffusiondiffusion

Gas diffusion can be approximated on the Gas diffusion can be approximated on the atomic scale by modelling many atomic scale by modelling many thousands of particles undergoing thousands of particles undergoing random walksrandom walks

We do not need to know any specific We do not need to know any specific details about the collisionsdetails about the collisions just that they occur randomly just that they occur randomly After each collision the new direction is After each collision the new direction is

randomrandom We’ll just look at the properties of one We’ll just look at the properties of one

random walk in this lecture thoughrandom walk in this lecture though

Scattering angle – random Scattering angle – random points on a spherepoints on a sphere

After the collision After the collision the incident the incident particle i is particle i is scattered in a scattered in a random directionrandom direction

The scattering The scattering directions should directions should be equally be equally distributed on the distributed on the surface of a spheresurface of a sphere

This is This is notnot the same the same as picking as picking and and randomly!randomly!

i

t

Picking points randomly on Picking points randomly on a spherea sphere

Left result is from randomly choosing Left result is from randomly choosing [0,2[0,2] ] and and [0,[0,] ]

Notice that points are more clustered around Notice that points are more clustered around the polethe pole

Spherical geometrySpherical geometry If one uses spherical polar If one uses spherical polar

coordinates to describe the coordinates to describe the surface, the area element on surface, the area element on the sphere is dthe sphere is d=sin =sin d d d d

Think about the geometry near Think about the geometry near the poles – when the poles – when =0 the =0 the coordinate is completely coordinate is completely degeneratedegenerate

Since dSince d contains a factor of contains a factor of sin sin we cannot take a uniform we cannot take a uniform distribution of distribution of & &

However since –d(cos However since –d(cos )=sin )=sin dd we can write |d we can write |d|=d(cos |=d(cos dd

This means we take a uniform This means we take a uniform distribution of distribution of where where [0,2[0,2]]

We also take a uniform We also take a uniform distribution of cos distribution of cos [-1,1] [-1,1]

sin d

d

d=sin d d

sin

Back of the envelope Back of the envelope calculation for mean free calculation for mean free

pathpath Let Let be the cross-sectional be the cross-sectional

area of each particlearea of each particle Let n=number of particles Let n=number of particles

per unit volume of side length l per unit volume of side length l What distance What distance is required so is required so

that the accumulated cross that the accumulated cross section of particles is ½lsection of particles is ½l22?? i.e.i.e. half of the time the incident particle half of the time the incident particle

will collide with another particle after will collide with another particle after having travelled having travelled

l

Average free streaming Average free streaming lengthlength

Given the distance Given the distance and cross sectional area l and cross sectional area l22 the total number of particles in the volume the total number of particles in the volume ll22 is nis nll22

Assuming the particles are distributed on a Assuming the particles are distributed on a “lattice” that has no overlaps the total “lattice” that has no overlaps the total accumulated cross section of all these accumulated cross section of all these particles is just particles is just nnll22

Need to find Need to find such that such that nnll22 =l =l22/2/2 Simple rearrangement gives Simple rearrangement gives =1/(2=1/(2n)n) Note that the factor of ½ is a result of us Note that the factor of ½ is a result of us

assuming that there are no overlapsassuming that there are no overlaps

Lattices are not realistic in Lattices are not realistic in this casethis case

If we think about particles being arranged on a lattice then If we think about particles being arranged on a lattice then there are a number of possible paths with an infinite path there are a number of possible paths with an infinite path length between collisionslength between collisions

Of course this is improbably unlikely to ever occur in a gas – Of course this is improbably unlikely to ever occur in a gas – the particles will be distributed randomlythe particles will be distributed randomly

The random distribution will have a number of overlaps which The random distribution will have a number of overlaps which means the factor of proportionality is 1/√2 rather than ½means the factor of proportionality is 1/√2 rather than ½

HenceHence

n

2

1

Estimate of the free Estimate of the free streaming lengthstreaming length

GivenGiven

We plug in We plug in =(10=(10-9-9))22 m m22

n~Nn~NAA/(22 l) = 6.02×10/(22 l) = 6.02×102323 / 0.022 m / 0.022 m33 ~ 2.7×10 ~ 2.7×102525 m m--

33

Find that Find that ~2.6×10~2.6×10-8-8 m m Diameter of NDiameter of N22 molecule ~ 3.1×10 molecule ~ 3.1×10-10-10 m m This is roughly 83 molecular diameters – so not This is roughly 83 molecular diameters – so not

too large!too large! Given this information we can now put together a Given this information we can now put together a

Monte Carlo random walk routineMonte Carlo random walk routine

n

2

1

MC random walk MC random walk algorithmalgorithm

1)1) Choose Choose and cos and cos at at randomrandom

2)2) Let the particle move Let the particle move , go to 1, go to 1Need to keep track of particles original Need to keep track of particles original

position (i.e. (0,0,0)) current position, position (i.e. (0,0,0)) current position, and the number of collisions Nand the number of collisions NImplementation hint: leaves distances Implementation hint: leaves distances in units of in units of . That way you can scale . That way you can scale results after the simulation is results after the simulation is completed for different completed for different ..

MC ExperimentMC Experiment We perform m experiments, and allow up We perform m experiments, and allow up

to N=1000 collisions (for example)to N=1000 collisions (for example) Typically we take m~500 or more, for Typically we take m~500 or more, for

exampleexample For each N we calculate the average For each N we calculate the average

distance <d>distance <d>RMSRMS// that the particle has that the particle has migrated from the origin migrated from the origin The average is taken over the m pathsThe average is taken over the m paths <d><d>RMS RMS is given byis given by

Plot up <d>Plot up <d>RMSRMS// and look at results and look at results

n

iiRMS d

nd

1

21

The underlying law is very obvious here

152

8-

2

105.1102.6

1

RMS

RMS

dN

Nd

Simulating that many collisions is incrediblydifficult right now, although the US has computerscapable of almost 1015 operations per second

Simple explanation of √N Simple explanation of √N scaling in 1dscaling in 1d

Consider a random walk in 1d, you can go forwards or backConsider a random walk in 1d, you can go forwards or back Each step in building up the path is given by a random Each step in building up the path is given by a random

variable xvariable xii, x, xii is either +1 or -1 is either +1 or -1

Clearly the average value of xClearly the average value of xii is zero: <x is zero: <xii>=(1+ -1)/2=0>=(1+ -1)/2=0

The average value of xThe average value of xii22 is 1: <x is 1: <xii

22>=(1+1)/2=1>=(1+1)/2=1 Path endpoint d is given byPath endpoint d is given by

The mean of the square of number of paths is thenThe mean of the square of number of paths is then

The cross terms <xThe cross terms <xiixxjj> i≠j average to zero: -1×1+-1×-> i≠j average to zero: -1×1+-1×-1+1×1+1×-1=01+1×1+1×-1=0

Hence the RMS distance isHence the RMS distance is

n

iixd

1

nxnxxd i

n

ii

n

ii

2

1

2

2

1

2 termscross

ndd RMS 2

Different classifications of Different classifications of random-“ness”random-“ness”

Truly RandomTruly Random Exhibiting true randomness – values cannot be Exhibiting true randomness – values cannot be

predicted in any waypredicted in any way PseudorandomPseudorandom

Appearance of randomness but having a deterministic Appearance of randomness but having a deterministic patternpattern

Period of pattern can be exceptionally longPeriod of pattern can be exceptionally long Must produce “good” approximation to a random Must produce “good” approximation to a random

deviatedeviate Quasi(sub)-randomQuasi(sub)-random

Having a set of non-random numbers in a randomized Having a set of non-random numbers in a randomized orderorder

Useful when we want a series of points to have Useful when we want a series of points to have certain propertiescertain properties e.g. optimal space filling, like Sobol, Halton sequencese.g. optimal space filling, like Sobol, Halton sequences

Hardware generatorsHardware generators Linux operating system example: “/dev/random”Linux operating system example: “/dev/random”

Gathers information from hardware within the Gathers information from hardware within the computer through the operating system – “noise”computer through the operating system – “noise” creates an “entropy pool” that is used to create the random creates an “entropy pool” that is used to create the random

numbersnumbers Avoids tracking things such as network traffic that Avoids tracking things such as network traffic that

can be manipulated by outsiderscan be manipulated by outsiders Was designed to be a true random generator but Was designed to be a true random generator but

recent research (March 2006) has shown it has a few recent research (March 2006) has shown it has a few weaknessesweaknesses

Other inputs for random information:Other inputs for random information: Readings from a Geiger counter – based on a quantum process, Readings from a Geiger counter – based on a quantum process,

completely unpredictable in theorycompletely unpredictable in theory Detected noise from a radio receiverDetected noise from a radio receiver Thermal or quantum-mechanical noise, amplified to provide a random Thermal or quantum-mechanical noise, amplified to provide a random

voltage source.voltage source.

Humour: LavaRndHumour: LavaRnd

Lava Lamps!Lava Lamps! SGI created a random SGI created a random

generator based upon generator based upon random bits that were random bits that were extracted from images of extracted from images of the erupting blobs inside the erupting blobs inside six Lava Lite lampssix Lava Lite lamps Project is no longer Project is no longer

functioningfunctioning A new project A new project

http://www.lavarnd.org/http://www.lavarnd.org/ uses the same idea but uses the same idea but relies on noise in the ccd relies on noise in the ccd of digital camerasof digital cameras

Commercial RNGsCommercial RNGs

Operate from USB or Operate from USB or Serial connectionsSerial connections Standard mode is to deliver Standard mode is to deliver

one byte of data at a timeone byte of data at a time Current models pass Current models pass

“DIEHARD” battery of tests“DIEHARD” battery of tests Very fast generationVery fast generation Prices range anywhere Prices range anywhere

from ~$100 to more than from ~$100 to more than $1,000 per unit$1,000 per unit

Types of pseudo random Types of pseudo random generatorgenerator

Linear Congruential Generators Linear Congruential Generators Based on a simple iterative processBased on a simple iterative process

Lagged Fibonnaci Generators Lagged Fibonnaci Generators Utilize a similar concept to the Fibonacci Utilize a similar concept to the Fibonacci

sequencesequence Shift Register Generators Shift Register Generators

Similar in principle to LFGsSimilar in principle to LFGs Combined Generators Combined Generators

Can be designed to offer best of combined Can be designed to offer best of combined methodsmethods

Linear Congruential Linear Congruential GeneratorsGenerators

(LCG)(LCG) A Linear Congruential Generator (LCG) is defined asA Linear Congruential Generator (LCG) is defined as

IIj+1j+1 = (I = (Ijj a + c) mod m a + c) mod m

Where –Where –IIjj – current number [I – current number [I0 0 – seed]– seed]IjIj+1+1 – next number – next numbera - multipliera - multiplierc - incrementc - incrementm – modulus m – modulus

The pThe period is clearly (at most) the value of the modulus, eriod is clearly (at most) the value of the modulus, m.m.

mod m=remainder afterdividing by m

Park and Miller Park and Miller “minimal” LCG“minimal” LCG

Really Lewis, Goodman & Miller (1968) – algorithm was Really Lewis, Goodman & Miller (1968) – algorithm was extensively reviewed in a later paper by Park and Millerextensively reviewed in a later paper by Park and Miller

By choosing the period and multiplier carefully LCGs of By choosing the period and multiplier carefully LCGs of the form Ithe form Ij+1j+1 = I = Ijj a mod m can be made to behave very a mod m can be made to behave very well well This type of form cannot use 0 as the initial seedThis type of form cannot use 0 as the initial seed

LG & M suggest choosingLG & M suggest choosing a=7a=755 and m=2 and m=23131-1-1

Implementing it in a code is straightforward apart from Implementing it in a code is straightforward apart from one very important issueone very important issue The multiplication of IThe multiplication of Ijj by by aa will frequently produce an overflow will frequently produce an overflow Rather than using a 64bit multiplication we can use “Schrage’s Rather than using a 64bit multiplication we can use “Schrage’s

method” to avoid overflow (see slide after summary)method” to avoid overflow (see slide after summary)

RANDURANDU RANDU is an infamous LCG that was

widespread on IBM mainframes in 1960s And adopted elsewhere!

Definition: Ij+1=(65539I j) mod 231

Straightforward algebra shows thatIj+2=6Ij+1-9Ij

Describes a series of planes in 3d (15)

Many scientific papers were invalidated due to the poor properties of RANDU

Quick & dirty estimatesQuick & dirty estimates Sometimes you just need a variable to “slightly” Sometimes you just need a variable to “slightly”

randomize things on a very quick timescalerandomize things on a very quick timescale LCG generators provide a good way of doing this LCG generators provide a good way of doing this

provided we choose m,a,c appropriatelyprovided we choose m,a,c appropriately But we want to avoid overflow, which constrains But we want to avoid overflow, which constrains

variable valuesvariable values NR gives a list of suitable values for m,a,cNR gives a list of suitable values for m,a,c

Example FORTRAN code for a [0:1] deviate:Example FORTRAN code for a [0:1] deviate:j=mod(j*ia+ic,im)j=mod(j*ia+ic,im)ran=float(j)/float(im) ran=float(j)/float(im)

Even faster generator are possible if we first Even faster generator are possible if we first check that overflow is a well behaved process check that overflow is a well behaved process low order bits only survivelow order bits only survive Then can use something like Then can use something like i=1664525*i+1013904223i=1664525*i+1013904223

Lagged Fibonnaci Lagged Fibonnaci GeneratorsGenerators

LFGs are based on the concept of the LFGs are based on the concept of the Fibonnaci sequence & are closely related to Fibonnaci sequence & are closely related to MRGsMRGs IInn=I=In-1n-1+I+In-2n-2 1, 1, 2, 3, 5, 8, …1, 1, 2, 3, 5, 8, … The idea can be generalized to any operator (as The idea can be generalized to any operator (as

opposed to addition) and to any pair of previous opposed to addition) and to any pair of previous values n-j, n-k as opposed to n-1 and n-2values n-j, n-k as opposed to n-1 and n-2

Currently popular LFG’s include Additive Currently popular LFG’s include Additive LFG’s based upon the following formulaLFG’s based upon the following formula IInn=I=In-jn-j+I+In-kn-k (mod 2 (mod 2mm ) where 0<j<k ) where 0<j<k k and l are called “lags” – the current value is k and l are called “lags” – the current value is

determined by values from l and k places agodetermined by values from l and k places ago

Survey of Numerical Survey of Numerical Recipes routinesRecipes routines

You may well come across codes containing these routines, You may well come across codes containing these routines, so it is worthwhile knowing what they areso it is worthwhile knowing what they are

Ran0 – Park-Miller LCGRan0 – Park-Miller LCG Minimal generatorMinimal generator Fails statistical tests for N>few 10Fails statistical tests for N>few 1077, much less than the period, much less than the period

Ran1 – Park-Miller plus a Bays-Durham shuffle routineRan1 – Park-Miller plus a Bays-Durham shuffle routine Tries to avoid “low-order” serial correlationsTries to avoid “low-order” serial correlations Improves statistical performance significantlyImproves statistical performance significantly

Ran2 – Longer period generator (10Ran2 – Longer period generator (101818) based upon ) based upon L’Ecuyer’s method of combining sequences (with shuffle)L’Ecuyer’s method of combining sequences (with shuffle) Believed to produce “perfect” random numbersBelieved to produce “perfect” random numbers $1000 waiting if you can prove it doesn’t$1000 waiting if you can prove it doesn’t

Ran3 – Knuth’s subtractive methodRan3 – Knuth’s subtractive method Very robust & fastVery robust & fast

Ran4 – Data encryption standard based generatorRan4 – Data encryption standard based generator

Mersenne TwisterMersenne Twister The Mersenne Twister by Matsumoto & Nishimura The Mersenne Twister by Matsumoto & Nishimura

(1998) is a comparatively new algorithm with many (1998) is a comparatively new algorithm with many useful properties useful properties An enormous period, 2An enormous period, 21993719937-1, (~10-1, (~1060006000) – significantly larger ) – significantly larger

than other generatorsthan other generators Provides equidistribution to at least 623-dimensions, and Provides equidistribution to at least 623-dimensions, and

probably higherprobably higher The algorithm can be implemented efficiently on RISC The algorithm can be implemented efficiently on RISC

hardwarehardware it avoids multiplication and divisionit avoids multiplication and division

Requires little memory Requires little memory 624 words needed for the working area624 words needed for the working area

Its name derives from the fact that period length is Its name derives from the fact that period length is chosen to be a Mersenne primechosen to be a Mersenne prime

The GSL library includes an implementation of MTThe GSL library includes an implementation of MT MT home page contains C, FORTRAN, as well as 32bit MT home page contains C, FORTRAN, as well as 32bit

and 64bit versionsand 64bit versions

http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html

SummarySummary Monte Carlo methods can be applied to Monte Carlo methods can be applied to

simulate physical systems that undergo simulate physical systems that undergo essentially random interactionsessentially random interactions

Can show with numerical experiments Can show with numerical experiments that random walk distance is proportional that random walk distance is proportional to √N where N is the number of collisionsto √N where N is the number of collisions

Hardware alternatives are available if you Hardware alternatives are available if you really need true random numbersreally need true random numbers

LCGs are useful for generating quick and LCGs are useful for generating quick and dirty sequences, but are outclassed by dirty sequences, but are outclassed by more modern algorithmsmore modern algorithms

Use the Mersenne-Twister if you want Use the Mersenne-Twister if you want really good random numbers and don’t really good random numbers and don’t have any time constraintshave any time constraints

Next LectureNext Lecture

Introduction to simple parallel Introduction to simple parallel programmingprogramming

Schrage’s MethodSchrage’s Method Suppose we have generated an value ISuppose we have generated an value Ijj=z, and our =z, and our

next step is create the product az and then take az next step is create the product az and then take az mod mmod m

If m=aq, for some q, then we see immediately that az If m=aq, for some q, then we see immediately that az mod aq=a(z mod q)mod aq=a(z mod q) Thus we can do the modulo step before we take Thus we can do the modulo step before we take

the product az and avoid overflowthe product az and avoid overflow Schrage’s Method is exactly this idea extended to Schrage’s Method is exactly this idea extended to

allow m=aq+r allow m=aq+r The resulting identity (z≥0) is The resulting identity (z≥0) is

For the Park Miller example we takeFor the Park Miller example we take

qzrqzaraqaz /)mod()mod(

2836mod

127773/

amr

amq

NON-EXAMINABLE