1

OutlineOutline

generation of random variates convolution composition acceptance/rejection

generation of uniform(0, 1) random variates linear congruential generators generator in ARENA

tests of generators

2

ConvolutionConvolution

X = b1Y1 + ... + bnYn

generate variates of Y1 to Yn

sum biYi to get X

Example 2.6.2 for Binomial

Example 2.5.3 for Triangular

Example 2.6.3 for Erlang (k, )

3

CompositionComposition

. ..

, ..

,

, 11

kk ppw

ppw

Y

Y

X

equivalent form in distribution

F(x) = p1F1(x ) + ... + p1Fk(x )

use a zero-one uniform variate to determine the “type” and then

generate the corresponding Y variate

4

Acceptance / RejectionAcceptance / Rejection

generate a variate from the uniform distribution on a disc of unit radius

1o generate a variate of (X, Y) such that X, Y X, Y ~ i.i.d. uniform [-1, 1]

2o accept (x, y) to be the desirable variate if x2 + y2 1; else go to 1o

result: uniform in {(x, y)| x2 + y2 1}

5

Acceptance / Rejection Acceptance / Rejection – Discrete Distribution– Discrete Distribution

X ~ {pi}; Y ~ {qi} such that pi/qi c for all i

1o Generate y from Y ~ {qi}.

2o Generate u from U. 3o If cqyu < py, set x = y and stop; else go to 1o.

similar procedure applicable to continuous distribution with {pi}, {qi} replaced by the

corresponding density functions

primarily for continuous distributions whose F-1 is

hard to find

6

Pseudo-Random NumbersPseudo-Random Numbers

linear congruential generator Zi = (a Zi-1 + c) mod M

Ui = Zi/M

Z0 = seed (initial value)

for large M and suitably chosen a, c, M Zi approximately ~ discrete uniform [0, M]

Ui approximately ~ uniform [0, 1]

7

Comments on Comments on Linear Congruential GeneratorsLinear Congruential Generators

{Zi}: a deterministic sequence given Z0

sequence can change with seeds sequences may not of full cycles (< M) random numbers from LCG lie on planes of a

hyper unit cube

0

20

40

60

80

100

120

0 20 40 60 80 100 120

Z_n

Z_{

n+1}

1數列

8

Comments on Comments on Linear Congruential GeneratorsLinear Congruential Generators

all right to use multiplicative LCG, i.e., c = 0 full period for suitable choice of M and a

Common Multiplicative LCG

a M

44, 485, 709, 377, 909 248

75 231-1

216+3 231-1

397204094 231-1

9

The Current (2000) Arena RNGThe Current (2000) Arena RNG

use some but not all idea of LCG CMRG - Combined multiple recursive generator

An = (1403580 An-2 – 810728 An-3) mod 4294967087

Bn = (527612 Bn-1 – 1370589 Bn-3) mod 4294944443

Zn = (An – Bn) mod 4294967087

Seed = a six-vector of first three An’s, Bn’s

Two simultaneous recursions

Zn / 4294967088 if Zn > 0

4294967087 / 4294967088 if Zn = 0Un =

10

The Current (2000) Arena RNG – The Current (2000) Arena RNG – PropertiesProperties

extremely good statistical properties good uniformity in up to 45-dimensional hypercube

cycle length = 3.1 1057

to cycle, all six seeds must match up on 600 MHz PC: 8.4 1040 millennia to exhaust

only slightly slower than old LCG

11

The Current (2000) Arena RNG – The Current (2000) Arena RNG – Streams and SubstreamsStreams and Substreams

automatic streams and substreams 1.8 1019 streams of length 1.7 1038 each a stream: 2.3 1015 substreams of length 7.6 1022 each

default stream is 10 (historical reasons) possible to specify a different stream

e.g., EXPO(6.7, 4) to use stream 4 ARENA automatically advances to next substream in each

stream for each replication helps synchronize for variance reduction

12

Statistical Tests of Zero-One Statistical Tests of Zero-One Random Number GeneratorsRandom Number Generators

check whether data are from i.i.d. unif[0, 1] quick test: mean and variance goodness of fit: 2 test; K-S test i.i.d.: j-lag correlation; run-up length

philosophy of tests: whether empirical evidence supports the statistical property under consideration

13

Quick Tests by ExcelQuick Tests by Excel

0.6605 0.3731 0.6692 0.9157 0.5950 0.4234 0.5147 0.4666 0.1528 0.2280

0.6533 0.1212 0.5849 0.1915 0.3402 0.5994 0.6569 0.4878 0.7650 0.2408

0.6449 0.4019 0.9497 0.7242 0.1563 0.2178 0.9821 0.4164 0.2228 0.6927

0.6699 0.1716 0.6164 0.0416 0.7201 0.7576 0.9351 0.2245 0.9758 0.8835

0.3257 0.3577 0.2211 0.4077 0.0976 0.1244 0.1969 0.9783 0.9285 0.2933

0.8170 0.5852 0.5681 0.1267 0.0202 0.5911 0.7173 0.5570 0.9073 0.3972

0.7975 0.0811 0.8872 0.3850 0.5900 0.0784 0.4811 0.5412 0.2339 0.8267

0.2329 0.1273 0.4226 0.6894 0.7127 0.3524 0.2805 0.2825 0.1765 0.9473

0.7176 0.5913 0.9445 0.7336 0.5003 0.2953 0.7923 0.2322 0.4370 0.9498

0.1109 0.6171 0.5854 0.4158 0.2626 0.6870 0.8552 0.0022 0.6483 0.6830

mean?sample

variance?confidence Interval?

14

Goodness of Fit Test – Goodness of Fit Test – 22 TestTest

idea: the actual number of sample points in a given range should be close to the expected number in some sense

M ranges (categories) ei: expected # of sample values in the ith range

ai: actual # of sample values in the ith range

15

Goodness of Fit Test – Goodness of Fit Test – 2 2 TestTest

M

i i

ii

e

ea

1

2)( approximately 2 distribution of M-1 degree of freedom

lose one degree of freedom for each estimated parameter

16

Theory and Main Idea of Theory and Main Idea of 22 Goodness of Fit Test Goodness of Fit Test

(X1, X2, ..., Xk) ~ Multinomial (n; p1, p2, ..., pk)

2

1

2

1)(

knk

i i

iik np

npXQ

17

Goodness-of-Fit TestGoodness-of-Fit Test

“better” to have ei = ej for i not equal to j

for this method to work, ei 5

choose significant level decision:

if , reject H0; otherwise, accept H0.22

18

Goodness of Fit Test – Goodness of Fit Test – KK--S S TestTest

F(x)

x

empirical distribution

compare with the distribution of unif[0, 1]

19

Goodness of Fit Test – Goodness of Fit Test – KK--S S TestTest

F(x)

x

largestdifference

find the “distance” between the

empirical data and the uniform [0, 1]

distributionthe largest difference follow

certain well defined distribution

for n 20,

D0.05 1.36/(n)0.5

D0.01 1.63/(n)0.5

20

jj-lag Correlation Test-lag Correlation Test

i.i.d. uniform [0, 1] random variables: covariance stationary random variables

3)(12 jiij UUE estimate j by: 3

termsof #

....)(12ˆ 21111

jjj

jUUUU

if the data are truly uniform [0, 1]:

)1,0(~)ˆ(

ˆ ;

1) termsof (#

7) termsof (#13)ˆ( ;0)ˆ(

2Normal

VVE

j

jjj

cov(UiUi+j)/V(Ui)

21

Run-up Length TestRun-up Length Test

run ups

can show that the lengths of runs are i.i.d. after deleting the numbers that start runs

values of random variates

order of random variates

new length of runs becomes: 3, 4, 1, 0, 2, 1, 0, 2new length of runs becomes: 3, 4, 1, 0, 2, 1, 0, 2

discard

22

Run-up Length TestRun-up Length Test

,...2,1 ;)!1(

)(

kk

kkLP

✦ distribution of lengths (after deleting numbers that start runs):

✦ can use goodness of fit test to check the distributions of the run lengths