A Mathematical Look at the Torah Codes

Pieter Trapman

November 19, 2001

Contents

1 Introduction 2

2 What it is all about 32.1 A brief description of the procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 A brief description of some critiques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 The procedure 73.1 The definition of the distance function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 The overall proximity measures P1, P2, P3 and P4 . . . . . . . . . . . . . . . . . . . . . . . . 83.3 The sample of word pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.4 The significance test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.5 The control texts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4 Discussion of the results 104.1 Proofs for fraud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 Critique on the method 125.1 The permutation test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125.2 Sensitivity to small part of the data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135.3 Critique on the distance function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

6 Some variations and remarks 156.1 About the variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156.2 The variations of the metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

7 Comments about the list of word pairs 227.1 Critique on the list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227.2 Results of some variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237.3 Experiments done by McKay et al. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

8 Conclusion and personal opinion 24

9 Acknowledgement 25

10 References 25

1

1 Introduction

In September 1994 the article Equidistant Letter Sequences in the book of Genesis was published in StatisticalScience, a peer-reviewed scientific journal. This article is written by Doron Witztum, Eliyahu Rips and YoavRosenberg. I will refer to this article as WRR1994. This article is remarkable because, if the conclusions ofWitztum et al. are correct, it has big religious consequences.

The first paragraph of this article is: “It has been noted that when the Book of Genesis is written astwo-dimensional arrays, equidistant letter sequences spelling words with related meanings often appear inclose proximity. Quantitative tools for measuring this phenomenon are developed. Randomization analysisshows that the effect is significant at the level of 0.00002.” In a more familiar way of saying: “In the bookof Genesis a remarkable phenomenon occurs. The probability that this phenomenon could occur by chanceis less than 0.002%.”

The text that is used is the Hebrew text of Genesis. This indicates that the research has a religiousbackground. Most statisticians working on the Torah codes are Jews. Many Jews believe that the Torahwas directly dictated to Moses on Mount Sinai. The Torah is, in their point of view, a divine text and theexistence of the code must prove the divinity of the Torah. Worth noting is that all the research of Witztumet al. is done in the book of Genesis and not in the other books of the Torah. This is strange because thereis not a general argument which makes Genesis more attractive than the other books of the Torah.

The phenomenon, described in the article, has everything to do with equidistant letter sequences (ELSs).Witztum et al. describe an ELS as follows: “a sequence of letters in the text whose position, not countingspaces, form an arithmetical progression; that is, the letters are found at the positions

n, n+ d, n+ 2d, . . . , n+ (k − 1)d.

We call d the skip, n the start and k the length of the ELS. d can be positive as well as negative. Thesethree parameters uniquely identify the ELS, which is denoted (n, d, k).”

To understand this definition think about a text as a two-dimensional array written on a very large paper.This text must be written without spaces and the same number of letters on all rows. (except perhaps forthe last line). ELSs appear usually as straight lines on this paper. All ELSs are straight lines when we forma cylinder by pasting the end of the first line together with the beginning of the second line and so on.

Witztum et al. looked for ELSs that spell out the names of famous Jewish rabbis and their dates of deathand (if known) the dates of birth. The research question was: “Are the names of the rabbis extraordinarilyclose to the corresponding dates?” For answering this question, four measures and a distance function areintroduced. In later research Harold Gans, a former cryptologist of the National Security Agency (NSA),looked at the combination of some famous rabbis with their birth- and dying places.

The journalist Michael Drosnin has published a book called The Biblecode. This book became a realbestseller, so the research of Witztum et al. became widely known. The article from Statistical Sciencewas printed in total in this book. It may be that this article is the most widely spread scientific article ofthe world. Witztum, Rips and Rosenberg have publicly stated that Drosnin’s way of using their results istotally wrong. There are many critical notes to the theory of Torah codes. Because of the popularity ofDrosnin’s book, it is difficult to see whether the criticism is against Drosnin’s results or against the work ofWitztum et al.

As a kind of joke, some mathematicians have used Drosnin’s methods to show that the murder of Drosninis predicted in Moby Dick. For this see the next fragment of Moby Dick. This picture will also give a betterunderstanding of what ELSs are.

In the picture on page 3, Drosnin’s name appears across the line “ him to have been killed”. This partof Moby Dick also reveals that Drosnin will be killed by “driving a nail into his heart” that “slices out aconsiderable hole”. The murder will take place in “Cairo” or “Athens”. The motive of the murder is clear,about this subject are the ELSs that spell out the words “treasure hunter”, “liar”, “lies” and “rort”. Notethat analysing this part of Moby Dick is done in the same way as Michael Drosnin analyses the Torah in hisbook. It is not the method of Witztum et al.

Many religious groups have used the “Biblecode” for their own purposes. Christians used the code toshow that Jesus is “predicted” as the Christ in The Torah. Orthodox Jewish groups used the code to showthat Jesus is a false prophet. The statistical methods, used for these researches, are very questionable. Andwith a little puzzling you can get almost every result you want.

2

It is not clear what the consequences are when the code turns out to be real. Drosnin says that it willbe the proof there is a supernatural being or at least there was one. His argument seems to be correct, butthere is no proof, because there is no positive evidence in this direction. There could be other explanationsfor this phenomenon. If the results of the research of Witztum et al. turns out to be correct, we may saythat the atheistic world has a brand new mystery.

In September 1999 a new article about the Thora code was published in Statistical Science. This articlewas written by Brendan McKay, Dror Bar-Natan, Maya Bar-Hillel and Gil Kalai (this article will be calledMBBK1999). In this article the authors want to show that the methods that are used to get the results ofWRR1994 are not right. Their claim is, that the appearance of ELSs in Genesis is not different from whatone can expect.

There is also a big discussion on the internet. Doron Witztum has defended his work against the claimsof McKay et al. On the other side there are many attempts to falsify the claims made in WRR1994. Aproblem in the whole discussion is that many scientists involved in this discussion do not hesitate in usingad hominum arguments. The integrity of other scientists is frequently doubted. This makes it difficult toinvestigate the truth about the “Biblecode”.

In this report I want to look at the Torah codes from a mathematical point of view. I will take a lookat the measures that are used, the distance function that is used and some alternative distance functionsproposed by McKay et al. I will also look at statistical theory, concerning this subject.

2 What it is all about

As reported in the introduction, Witztum et al. published an article about the Torah codes in StatisticalScience, in 1994, but research about this subject was already done a long time before 1994. In the secondWorld War it was rabbi Weissmandel who wrote the text of the Torah in 10× 10 matrices in order to readsecret messages. At that time there was no method to investigate whether these secret codes are due tochance or whether something extraordinary is in the text. Now with the use of computers, we can givea better judgement on this question. Because the phenomenon was known long before the real scientificresearch was done, some problems occur. The most important problem is that of “tuning”. In the nextsubsection I will give a better understanding of what is going on. After that I will discuss the problem oftuning.

3

2.1 A brief description of the procedure

The phenomenon of the Torah-codes is about Equidistant Letter Sequences (ELSs) spelling out meaningfulmessages. In the previous section ELSs were described. It must be a priori known what kind of messages wewill look for. Some list of words must be made. Or better: Some list of word pairs must be made. In thisway it can be investigated if ELSs, spelling out related words, are extraordinary close to each other.

Witztum et al. took a set of words from Margaliot’s The Encyclopedia of Great Men in Israel. Two listsof word pairs were made. One list contained all rabbis, whose entries in this Encyclopedia cover more thanthree columns of text, and at least a birth or death date for these rabbis, and some data for these rabbis.The second list contained all rabbis, whose entries covers one and a half to three columns, and some data.The data are the death dates and (if known) the birth dates of the rabbis. 34 rabbis are on the first list and32 on the second list. In Hebrew dates are written in letters, so it is not very unusual that these dates occurin a text.

One may think that the data are obtained mechanically. This is not the case. The set of rabbis is closed,but there is some freedom. The rabbis all have various appellations in literature. Some of these are wrong,so must be deleted, other appellations are very discutable and can be deleted too. The boundary of whichappellations should be deleted and which appellations should be kept is rather subjective. For this sakeexperts in the field of rabbinical history have made several lists of appellations. But these lists are stillformed according to their personal judgement. No mechanical rules are given.

For the birth and death dates, the same problem occurs. In The Encyclopedia of Great Men in Israelsome dates are wrong, other dates are very uncertain, and so on. So experts must make a list of correctdates. But again there is some subjectivity in this research.

To investigate whether the phenomenon of the Torah-codes is real, one has to do some calculations onthese words and the positions of the letters in these words when the words are spelled out as ELSs. For thispurpose a distance function, c(w,w′), between two words w and w′, is defined. This function is given in thenext section. This distance function is based on another distance function, that is defined to investigate thedistance between letters in a text. The function c(w,w′) takes the distance between letters within an ELSas well as the distance between the two ELSs that are of concern, into account. The ELSs that should bemeasured are the ELSs that spell out appellations of the rabbis and the ELSs that spell out the dates ofbirth and death of these rabbis.

The distance function that is used to investigate the distance between letters and ELSs is the commonEuclidian distance when the text is written as two dimensional arrays. From this distance function betweenELSs, the function c(w,w′) is defined. The words, w and w′ are a word pair consisting of one appellation andone date. To calculate the value of the function, all ELSs, spelling out these words, are taken into account.Note that we look for word pairs, not for rabbi-date pairs. A rabbi-date pair should take all appellations ofthe same rabbi in the same function.

When the distance functions and measures are defined, I can define some overall measures that give someinsight in the results of the research. Originally Witztum et al. used two measures, P1 and P2. Because ofreasons given later, another two measures were introduced. These new measures are the same as P1 and P2,except that the appellations starting with “rabbi” are ignored. These measures are called P3 and P4.

The list of word pairs can be made in different ways. To get a good interpretation of the results,some permutations of the data are made, so the dates of one rabbi are combined with the appellations ofanother rabbi. 32! of these variations can be made. Because it is impossible to calculate the result for32! permutations, one million permutations are chosen at random. Each of these one million permutationsgive values for P1−4. These values can be ordered in the same way as a set of real numbers can be ordered.The rank of the identity permutation seems to be a good measure to gain some information about thesignificance of the phenomenon. In later research 200 million permutations are used.

The four given measures all give different values. Call the least permutation rank ρ0, and the numberof permutations (including the original permutation) n. The significance level is computed by 4

nρ0. Thiscomputation is allowed because of the Bonferroni inequality.

The results of the research are very impressive. The least permutation rank for the identity permutationwas 4 out of a million. So the significance level is 0.000016.

4

To investigate if the given result is not only due to the sample of word pairs, the same test is done onsome other Hebrew texts. The results for these other texts are not surprising at all and will be given insection 4.

2.2 A brief description of some critiques

The claims of Witztum et al. are rather extraordinary, so one can expect critiques on the results of theresearch. Some new experiments are done by other scientists and some comments are made about the testprocedure.

The list of rabbis is closed but the list of appellations and dates is certainly not closed, as we have seenin the previous subsection. Variations of appellations and dates have significant effects on the results. Adiscussion about this critique is given in sections 6 and 7.

The most important theological objection is that the phenomenon does not appear in the other books ofthe Torah. It is totally unclear why Genesis should have an other status than the other four books. One mayeven expect the same properties for the text, because the whole Torah is believed to be dictated by God. Thecontents of each book is not even generally accepted. Some rabbis take the first verses of Deutronomy (asnumbered in the King James translation) as the last verses of Numbers. Some claims by Rabbi Weissmandelare about ELSs in Numbers and Leviticus. This is not about famous rabbis but about some Jewish holidays.

The Torah has been copied very accurately in history. Errors could hardly occur in this manual copying,but still some errors are possible. McKay et al. referred to nine other versions of the Hebrew Torah. Thetexts are almost the same; the meaning of the text is the same in all versions, but on the level of letters atmost 43 differences between the version used by Witztum et al. and the other versions, can be found in thetext of Genesis. Using another version of Genesis make the results much less impressive. In some cases thewhole phenomenon is deleted.

Some critiques are about the fact that the ranks are different for the four different measures. Thisimplies that according to one experiment one permutation is better than another permutation and accordingto another experiment (with a different measure) this permutation is worse than the other. This seems to becontradictory. This is not necessarily the case, because it is never claimed that the given measures exactlymeasure the same quantities as the quantities that are in the text (the code). However extraordinary lowpermutation ranks for the different measures still show something strange is happening in the Torah (or inthe sample of word pairs). A critique of this kind is given in section 4.

If one gives a row of thousand numbers between one and six. One can think that it is impossible thatexactly this row appears when a dice is thrown a thousand times. But the case is that this row is generatedby throwing a dice a thousand times. All rows of numbers have a very small probability to occur, but theprobability that one of the rows occurs is one. The conclusion that some miracle occurred by throwingexactly that row is an example of something that looks like tuning.

Tuning is selective biasing towards a positive result. Tuning can occur without any bad intentions of thescientists that execute the research. So accusations of tuning do not automatically mean that the researchersare not honest. Many experiments are executed to show that something extraordinary, that is seen, is notdue to chance. Even if the phenomenon is due to chance, the experiment probably will show that it isnot. Many remarkable things can be seen in each text, while nothing strange is happening. In the researchabout the Torah codes rabbi Weissmandel already had noted that some famous rabbis are hidden as ELSsin Genesis. But he could have seen the names of presidents of the United States of America, the names ofthe kings of England or one of many other possibilities too. That one of these things gives extraordinaryresults can be explained by chance only. If one only looks at the observed strange phenomenon, it can beconcluded that it is not due to chance.

Witztum et al. claimed to have taken the problem of tuning into account and made the second list ofrabbis totally a priori. Rabbi Weissmandel only looked for very famous rabbis. Very much is to say aboutthese rabbis. So the entries of these rabbis cover more than three columns in The Encyclopedia of GreatMen in Israel. The first experiments done by Witztum et al. are done on this list of rabbis too, but to avoidthe risk of tuning, they composed a second list of rabbis on which no experiment was run yet. The list ofrabbis is a priori. So tuning seems to be impossible.

But there are some problems, as we have seen earlier. The list of rabbis is a priori, but the list of

5

appellations and dates is not. The list of appellations can be made optimal to the test. It is also possiblethat some of the names in the second list are already observed as ELS by rabbi Weissmandel. This observationcould have made rabbi Weissmandel aware of the extraordinary appearance of famous rabbis as ELSs. Thisrabbi had not made his work very scientifically and he had not used The Encyclopedia of Great Men inIsrael or any other objective source to select the rabbis. Why should he? Without a computer it is hardlyimpossible to do any computation that gives meaningful results. Because of this fact and because of a lecturegiven by Eliyahu Rips in 1985, it can be doubted whether the second list is as a priori as claimed in thearticle.

Persi Diaconis, The Harvard professor in Statistics, proposed to use a third sample of word pairs. Thisproposal is rejected. McKay et al. made some new samples. The results of these samples are not extraordinaryat all. In section 7 these experiments and the results are given.

Another critique on the statistical part of the experiment is that no alternative hypothesis is given inWRR1994. It is necessary to define a null-hypothesis in a statistical experiment. But when this hypothesisis rejected one can investigate whether something different is happening here and what sort of thing thatis. It is not a disaster that no alternative hypothesis is stated, but doing so could give some more clarity onthe results. Now, strictly speaking, the only conclusion drawn from the results of the experiments is thatthe occurrence of the famous rabbis is not totally random. But this is not the same as saying that their isa code in Genesis. The non-randomness can be due to some patterns in every meaningful text, because ofproperties of the language itself.

The null hypothesis H0 stated in WRR1994 is that the permutation rank of each of the statistics of P1−4

has a discrete uniform distribution in [0, 1]. Rejecting H0 does not automatically lead to the conclusion thatthe text of Genesis has special properties. The result can also be due to the choice of word pairs.

Consider the function c(w,w′). The distribution of c(w,w′) for random words w and w′, in fixed text isapproximately uniform. However any two such distances are dependent random variables. This is crystalclear for c(w,w′) and c(w,w′′), where there is an argument w in common. As we shall see later thesedependencies have great influence on P2. For this reason the a priori distribution of P2 is not the same fordifferent permutations. So the a priori rank order of the identity permutation is not uniformly distributed.This problem is serious enough to have some doubts about the conclusions given by Witztum et al.

To show that the results are no reflections of something that is in the text, McKay et al. give somevariations in the procedure used by Witztum et al. The procedure was a priori known, but if little variations,that respect the boundaries of the experiment, destroy the proof for an extraordinary phenomenon, the resultsof the original experiment should be suspected. Variations that respect the boundaries of the experimentare variations that take the features of the phenomenon into account. So experiments that are stronglyinfluenced by ELSs that are very distant from each other and weakly by ELSs that are close to each otherdo not respect the boundaries. Some variations chosen by McKay et al. can be thrown in the waste basketbecause of crossing the boundaries. But certainly not all variations can be ignored.

Persi Diaconis also contributed a lot to other parts of the statistical research. The interpretation of theresults of the permutation test is not trivial. Diaconis proposed some adaptations in the method in order toavoid some problems. Because of miscommunication with Robert Aumann, who represented Witztum et al.in the scientific world, the method of Diaconis is not executed. It must be said that Diaconis only gave abrief sketch of his proposal for the procedure, and he himself agreed with the test done by Witztum et al.Later Diaconis as well as Aumann admitted that this agreement was due to a misunderstanding. In section7 Diaconis’ proposal is tackled and some variations are given.

McKay et al. did some research on War and Peace of Tolstoy. They changed some of the appellationsand concluded that the same phenomenon that is seen by Witztum et al. in Genesis occurs also in War andPeace. More details about this experiment are given in section 7.

The last research that has to be mentioned is that of Gil Kalai. He claims that the results of WRR1994are too good to be true. The results for the second list of rabbis are much too close to the results for thefirst list, even if the distributions are exactly the same. This suggests fraud by Witztum et al. and makesthe whole research scientifically unacceptable. The proof of this fraud will be given in a later section.

6

3 The procedure

In this section we will take a look at the procedure that Witztum et al. used for their research. Some thingshave to be done before getting results and in this section the different steps in the procedure will be givenas different subsections. The steps we have to take are the following:

We have to define a notion of distance between two words. Then we can lend meaning to the idea ofwords in “close proximity”. We must also define statistics that express how close the related words are.Then we have to choose a sample pair of related words on which the test can be run. These words must beknown in advance. Now we can determine whether the statistics are unusually small for the chosen sample.

3.1 The definition of the distance function

In this subsection I want to discuss the metric c(w,w′) that Witztum et al. have used for the distance intheir article. They used a fixed text G = g1g2g3....gL, where gi is the letter on the ith position in the textand L is the length of the text. For Genesis L = 78, 064.

To say something meaningful about words which appears with equal spacing in the text, Witztum et al.also looked for slightly unequal spaced, perturbed ELSs. These perturbed ELSs are the same as normal ELSsexcept for the last three letters. The last three spacings may be larger or smaller by up to 2. For this we lookat a word w = w1w2....wk with k greater than 5. We also consider a triple (x, y, z) with −2 ≤ x, y, z ≤ 2.This triple is the perturbation. An (x, y, z)-ELS is a triple (n, d, k) such that gn+(i−1)d = wi for 1 ≤ i ≤ k−3,gn+(k−3)d+x = wk−2, gn+(k−2)d+x+y = wk−1 and gn+(k−1)d+x+y+z = wk.

Note that it is necessary that k ≥ 5 because otherwise some properties are undefined. When k = 4 it ispossible to change d and get the same perturbed ELS with different (x, y, z). When k ≤ 3, x is not definedbecause there are less than 3 spaces. For all the ELSs, that are used, it holds that 5 ≤ k ≤ 8. The upperbound 8 is not introduced without any reason. For a greater upper bound ELSs hardly occur by chance.Even if the code is real, the number of perturbed ELSs will be much lower than it is with an upper bound8. So it is not possible anymore to compare the unperturbed ELSs with the perturbed ones.

The unperturbed ELSs are the (0, 0, 0)-ELSs. So these ELSs are just a special case of the (x, y, z)-ELSs.The other values of (x, y, z) represent the non-zero perturbations of the last three letters from their naturalpositions. In total there are 53 = 125 of these perturbations, including the (0, 0, 0)-perturbation.

For the distance function (which will be defined later) it is important to know what we will measure. Wecan use n+ (k− 1)d+ x+ y+ z for the last letter of a perturbed ELS, or we can use n+ (k− 1)d. From theprograms used by Witztum et al. we can see that they used the form n + (k − 1)d. So they treat the lastletter as if the ELS is unperturbed.

Now we define the distance function ∆(t, h). This function measures the Euclidian distance betweentwo letters of the letter matrix that occurs when we write the text in arrays of length h. If the text waswritten in one long row the distance would be the difference between the position numbers, t. The distanceis the equivalent of the Euclidian distance function when we take into account that the text is written on acylinder of circumference h. Define the integers ∆1 and ∆2 as the quotient and remainder of t divided by h,respectively. (Thus, t = ∆1h + ∆2 and 0 ≤ ∆2 ≤ h− 1.) Then

∆(t, h)2 =

∆2

1 + ∆22 if 2∆1 ≤ h,

(∆1 + 1)2 + (∆2 − h)2 otherwise.

Now we define for two (x, y, z)-ELSs, e = (n, d, k) and e′ = (n′, d′, k′) and for every h

δh(e, e′) = ∆(d, h)2 + ∆(d′, h)2 + min0≤i≤k−1,0≤i′≤k′−1

∆(|n+ di− n′ − d′i′|, h)2,

µh(e, e′) =1

δh(e, e′).

The definition of δh(e, e′) does not seem to be very unusual. The last term is the minimal distancebetween a letter of e and a letter of e′.

7

We can define a multiset H(d, d′) of values of h. For 1 ≤ i ≤ 10 the nearest integers to d/i and d′/i(1

2rounded upward) are in H(d, d′) if they are at least 2. (This is due to a programming error, there

should be no underbound, the effect of this error is neglectable). Note that H(d, d′) is a multiset; some ofits elements may be equal. We consider this definition because we want to consider those ELSs that arerelatively compact. So we look at those cylinder circumferences for which at least one of a pair of two ELSsappears in a straight line or close to a straight line. The number 10 is arbitrary. Given H(d, d′), we define:

σ(e, e′) =∑

h∈H(d,d′ )

µh(e, e′).

For any (x, y, z)-ELS e with skip d, we consider the interval I of the text with the following properties:

• I contains e;

• I doesn’t contain any other (x, y, z)-ELS of w with a skip smaller than d in absolute value.

If such an I exists, there is a unique longest I, denoted by Te. Note that we begin with an ELS not witha word. In that case we do not necessarily have a unique longest I, because a word can occur as differentELSs with the same skip. If there is no such I we define Te = ∅. In either case, Te is called the domain ofminimality of e. In a similar way we can define Te′ . The intersection Te ∩ Te′ is the domain of simultaneousminimality of e and e′. We can now define ω(e, e′) = |Te ∩ Te′ |/L. In this definition L is the length ofGenesis.

Next define a set E(x,y,z)(w) of (x, y, z)-ELSs of w. Let D be the least integer such that for a randomtext with letter probabilities equal to the relative letter frequencies in G, the expected number of ELSs ofw with absolute skip distance in [2, D] is at least 10. If this expected number is less than 10 then D = ∞.If D 6= ∞, D is equal to the product of the relative frequencies of the letters constituting w multiplied bythe total number of all ELSs with 2 ≤ d ≤ D. Then E(w) = E(x,y,z)(w) contains all those (x, y, z)-ELSs ofw with absolute skip distance in [2, D]. The number of potential ELSs with absolute skip distance in therange [2, D] is given by (D − 1)(2L− (k − 1)(D + 2)).

The programs of Witztum et al. use (D − 1)(2L − (k − 1)D) for the number of potential ELSs withabsolute skip distance in the range [2, D]. McKay et al. deliberately made the same mistake to take a fairlook at the WRR1994-research.

Now define for E(w) and E(w′) both non-empty:

Ω(x,y,z)(w,w′) =∑

e∈E(w),e′∈E(w′)

ω(e, e′)σ(e, e′).

If at least one of E(w) and E(w′) is empty, Ω(x,y,z)(w,w′) is undefined. For this definition it is importantthat the skip is at least 2, because many rabbis have names that occur in the Torah. So the ELSs with skip1 give an advantage to the perturbed ELSs that is not due to any extraordinary phenomenon in Genesis.

Next we define c(w,w′). If there are less than 10 values of (x, y, z) for which Ω(x,y,z)(w,w′) is defined orif Ω(0,0,0)(w,w′) is undefined, then c(w,w′) is undefined. Otherwise, c(w,w′) is the fraction of the definedvalues Ω(x,y,z)(w,w′) that are greater than or equal to Ω(0,0,0)(w,w′).

We can conclude that for the distance function c(w,w′) many arbitrary choices are made. If we changethe distance function at some small points the choice remains arbitrary. Other functions are not a prioripreferable because of this reason. The distance function taken by Witztum et al. does not seem to be veryfar fetched. The value of c(w,w′) may be either undefined or a fraction between 1

125and 1. A small value

indicates that w and w′ are close to eachother.

3.2 The overall proximity measures P1, P2, P3 and P4

We now define the proximity measures that are used to come to scientific results. Define N as the numberof word pairs (w,w′) in the sample for which the distance c(w,w′) is defined. Now let k be the number ofsuch word pairs for which c(w,w′) ≤ 1

5 . Define:

P1 :=

N∑

j=k

(Nj

)(1

5)j(

4

5)N−j .

8

We take a closer look at this definition. Note that if the c(w,w′) were independent random variables, thatare uniformly distributed on [0, 1], then P1 is the probability that at least k out of the N of those are lessthan or equal to 0.2. For the text of Genesis neither independence nor uniformity is proved or assumed.So P1 is simply an ordinal index that measures the number of word pairs in a given sample whose wordsare rather close to each other, or to be exact: words for which c(w,w′) ≤ 1

5 . This measure P1 ignores alldistances c(w,w′) greater than 1

5 and gives equal weight to all distances less than 15 .

Now define:

FN (X) := X(1 − lnX +(− lnX)2

2!+ . . .+

(− lnX)N−1

(N − 1)!).

with N defined as above. We can now define P2.

P2 := FN (∏

c(w,w′)).

FN (X) is chosen with the following in mind. Let x1,x2,. . . ,xN be independent random variables that areuniformly distributed over [0, 1], then the distribution of X := x1x2 . . .xN is given by Prob(X ≤ X0) =FN (X0). So P2 can intuitively be viewed as follows: If the c(w,w′) were independent random variablesthat are uniformly distributed over [0, 1], then P2 would be the probability that the product

∏c(w,w′) is

as small as it is or smaller. But again neither uniformity nor independence are assumed so P2 is an ordinalindex rather than a probability. The index P2 enables the researchers to compare the proximity of wordpairs arising from different permutations of the personalities.

The other statistics P3 and P4 are defined like P1 and P2, except that all appellations starting with thetitle “rabbi” are omitted.

3.3 The sample of word pairs

For statistical tests it is necessary that the sample on which the test is executed is closed. The sample mustalso be a priori stated, so that tuning is impossible. The choice of the word pairs are rabbis combined withtheir dates of death and (if known) the dates of birth. In Hebrew the dates are given in words of at least sixletters. For the first list, the rabbis that were chosen must take at least three columns in The Encyclopediaof Great Men in Israel by Margaliot. This yielded 34 personalities. In order to avoid any suspicion ofhaving fitted the test to the data, Witztum et al. decided to use a new sample. This sample was also takenfrom The Encyclopedia of Great Men in Israel but now the rabbis whose entries cover one and a half tothree columns of text were chosen. This yielded 32 personalities. Only the second sample is used in theresults for WRR1994. The dates and appellations are not from The Encyclopedia of Great Men in Israel.Witztum et al. used their own list of appellations and dates. At this point it is difficult to prove that thedata have not been fitted to the tests.

Personality-date pairs are not the same as word pairs. The personalities each have several appellations.The dates can also appear in various ways. As an example, in English you have “May 1”, “the first of May”and “on the first of May”. So a personality-date pair (p, p′) correspond to several word pairs (w,w′).

3.4 The significance test

We have a list of 32 personalities combined with some dates. Now we combine the personalities with thedates of other personalities. This can be done in 32! ways, including the original combination of personalitiesand dates. We call such a permutation π. The original combination (the personalities with their own dates)is called π0. Each of this permutations determines the statistics P π1 , P π2 , P π3 and P π4 . We can order the 32!results for P πi for i = 1, 2, 3, 4 in the usual way of ordering the real numbers. If the phenomenon studied, isdue to chance, it will be just as likely that P1 = P π0

i occupies any one of the 32! places. The null-hypothesisis that this is indeed the case. An alternative hypothesis is not explicitly stated in WRR1994.

It is impossible to compute the proximity measures for all 32! combinations, so to calculate the significancelevel 999,999 of this permutations (not π0) are chosen. Including π0 we have one million permutations tocompare with each other for each Pi. The rank order of Pi among these 1,000,000 numbers is defined asthe numbers of P πi not exceeding Pi. Now define ρi as the rank of Pi divided by 1,000,000; under the nullhypothesis, ρi is the probability that Pi would rank lower than it does.

9

After the calculation of the ρi the null hypothesis must be accepted or rejected. It must be avoided thatonly favorable evidence is selected. For example, suppose that ρ3 = 0.01, and the other ρi are higher. Thenthere is a temptation to only consider ρ3 and to reject the null hypothesis at the level of 0.01. But whenthe statistics are enough diverse it is quite likely that just by chance, some of them will be low. So we mustlook for another question to accept or reject the null-hypothesis. A correct question will be “Assume thenull-hypothesis. What is the probability that at least for one of the four i’s, ρi ≤ 0.01 holds?” When wedenote the event ρi ≤ 0.01 by Ei, we must find the probability not of one of the Ei’s but of E1 or E2 or E3

or E4. If the Ei’s were mutually exclusive, this probability would be 0.04; overlaps will only decrease thetotal probability. So we can safely say that this probability is less than or equal to 0.04. So if ρ3 = 0.01, andthe other ρi are higher. we can reject the null-hypothesis at the level of 0.04. Or if ρi = δ, and the otherρi are higher, then we can reject the null-hypothesis at the level of 4δ. So the overall significance level (orp-value), using the four statistics is ρ0 := 4 min1≤i≤4 ρi.

3.5 The control texts

To check if the assumptions are not extraordinarily wrong Witztum et al. used some control texts on whichthey run the same tests. They used six control texts called I, R, T , U , V and W . Text R is a randompermutation of all letters of Genesis. For W the words of Genesis are permuted. The letters in the wordshave still the same order. For V the verses are permuted, while leaving the letters in the verses unpermuted.In text U the words within each verse are permuted. The letters within the words as well as the order ofthe verses are left unpermuted. Another scientist proposed to use the first segment of War and Peace byTolstoy as control text. So control text T is the first segment of the Hebrew translation of War and Peacewith the same length as Genesis. Finally text I is the book of Isaiah, another book of the Bible. Isaiah isnot a book of the Torah and it is not believed to be directly dictated by God.

A.M. Hasofer and McKay et al. have tested the procedure on the other books of the Torah: Exodus,Leviticus, Numbers and Deutronomy. These books have the same status as Genesis. McKay et al. have usedthis test in their paper.

4 Discussion of the results

The results given in the 1994-Statistical Science paper are given in the next table.

Rank order of Pi among one million P πiText P1 P2 P3 P4

G 453 5 570 4R 619,140 681,450 364,859 573,861T 748,183 363,481 580,307 277,103I 899,830 932,868 929,840 946,261W 883,770 516,098 900,642 630,269U 321,071 275,741 488,949 491,116V 211,777 519,115 410,746 591,503

Later Witztum et al. improved their software. A.M. Hasofer has used this update to calculate the rankorders of P1 and P2 for all five books of the Torah. McKay et al. did some experiments on this subject too.In the next table the results of these experiments are given.

Rank order of Pi among one million P πiText P1 P2

Genesis 718 2Exodus 135,735 193,315Leviticus 816,860 947,387Numbers 901,660 920,919Deutronomy 790,542 759,428

Least ρ1−4 values

Text List 1 List 2Genesis 0.00004 0.00000068Exodus 0.0212 0.1010Leviticus 0.6950 0.8467Numbers 0.0046 0.6628Deutronomy 0.0664 0.7282

10

The results are the rank orders of the Pi among one million corresponding P π1 . Thus the entry 453 in thefirst table means that for 452 out of the 999,999 permutations the statistic P π

1 was smaller than P1. Fromthe table follows that for Genesis minρi = 0.000004 so ρ0 = 4 minρi = 0.000016.

Later the authors of WRR1994 as well as the authors of MBBK1999 did new experiments with 200,000,000permutations. The results of this research are even more impressing than those of the original test. The newvalues of ρ2 and ρ4 are 1.9× 10−6 and 6.8× 10−7.

The results for the control texts are totally insignificant on their own. The results for the other booksof the Torah are also insignificant, in the same way. As stated before this gives a philosophical problem toWRR. Why is the code in Genesis and not in any other book of the Torah? From Jewish perspective Genesishas exactly the same divine status as the other books of the Torah.

Now we come to a point that I have left unspoken in the section about the procedure. Why havethe statistics P3 and P4 been introduced? This is necessary because after the title “rabbi” often only thegiven names of the personality are used. As in Dutch the names Peter and Jan are very popular, nameslike Avraham and Yehudah are very popular in Hebrew. So there are several different rabbis called rabbiAvraham. Other appellations are often “nicknames”. These nicknames are not the same for different rabbi’s.If the phenomenon investigated by Witztum et al. is real the values of c(w,w′) will be misleadingly low whenwe match in π one rabbi Avraham to the dates of another rabbi Avraham. This might have resulted inmisleadingly low values of P π1 and P π2 . This will lead to smaller significance levels for P1 and P2. In the listof rabbis that is used, four rabbis are called Avraham, three rabbis with the name David, four rabbis calledHaim, two rabbis Yehudah and also two rabbis Moshe. The probability that none of the rabbis is combinedwith the results of a rabbi with the same name is very small. The probability that none of the rabbis called

Avraham is combined with the results of another rabbi Avraham or with itself is only (28!)2

32!24! = 40957192 ≈ 0.57.

So we may assume that in 999,999 permutations such combinations take place. Note that the ranks for P1

and P3 do not differ much. Neither do the ranks of P2 and P4.Another critique on the results of WRR1994 is given by Hasofer. He stated that the results are not good

enough to speak about the divinity of Genesis. The “best” measure of distance is P4. For this measure 3 outof the 999,999 permutations have better results than the original sample. This is a remarkable result. Butthis also means that in total there are estimated 32!× 3

106 ≈ 7.89 × 1029 samples better than the originalsample. A divine writer could have done better. Against this argument can be stated that Witztum et al.never claimed that their measures are the same measures that are used to write the code in the text. Theyclaim that their measures show that there is a code in Genesis. It is like measuring the tallest people bytheir length. People that are very long usually are heavy too. When a sample of people selected by length iscompared with a totally random sample of people, one sees that in the sample selected by length, the weightis significant greater than in the totally random sample. But there are some counterexamples for the claimthat the people are selected by weight.

4.1 Proofs for fraud

In MBBK1999 is said that the results in WRR1994 are too good to be true. This is based on experimentsof Gil Kalai. For the first experiment done by Witztum et al., using the list of 34 personalities in the TheEncyclopedia of Great Men in Israel taking more than 3 columns, the value of P2 is 1.29× 10−9. The valueof P2 for the Statistical Science experiment was 1.15 × 10−9. We look at the P2 value because this valuewas the first success measure. P3 and P4 were later counted for both lists and the permutation test was noteven “invented” by that time. McKay et al. used a Monte Carlo simulation of the sampling distribution ofthe ratio of two such P2 values. This simulation was done as follows: The list of total 66 rabbis from thetwo experiments of Witztum et al. is randomly partioned in a list of 34 rabbis and a list of 32 rabbis. Nowcompute the ratio of the larger to the smaller value. The median ratio was about 700. The situation that

this ratio is less than or equal to 1.29×10−9

1.15×10−9 ≈ 1.12 occurs in less than one in a hundred times. How manypermutations are used in this simulation is not given in MBBK1999 or by Kalai. Witztum et al. shouldsuspect their own results because of these data.

The similarity for the results of the two tests even points to the following way of research. The appellationsfor the rabbis in the second test are constantly adapted until the result is better than the result of the firsttest. This adaptation can take place in several ways. It is possible that successful appellations are added

11

until the results were better than the original result. It is also possible that very unsuccessful appellationsare deleted, until the score is better than the score of the first test. In the last case, it is possible that we maynot speak of fraud, but only of naivity of the researchers. The very unsuccessful appellations are reconsideredand all possible arguments for deleting such an appellation are used. In this way not every appellation canbe considered for deletion, some of the appellations are so clearly correct, that it is impossible to delete theseappellations.

On the second list are 44 appellations whose deletion decrease the P2 score, 23 appellations, whosedeletion increase the P2 score and 35 appellations that are neutral, i.e. they have no influence on the P2

value because they are not spelled as ELS in Genesis. If the first way mentioned is used. We can take aview at the appellations that by deletion weaken the result. From the 23 appellations for which this holds

are nine appellations whose deletion have an effect that is below the square of 1.29×10−9

1.15×10−9 . This is 39% of theconsidered appellations. We look for the square of the ratio, because if the result is obtained by the stoppingmethod that we consider, we may expect this to be the effect of the last step.

If the second method of frauding is used, we may assume that the deleted appellations have about thesame properties as the appellations that stay on the list. So we take a look at the same 23 appellations.And we have the same result.

Bar Natan discovered that the P2 scores of the dates of rabbis in the second list with respect to theappellations of these rabbis not chosen by Witztum et al. is significant smaller than the P2 score of the samedates with the same appellations randomly permuted. This is a clear pointing to fraud by Witztum et al.

In WRR1994 pair-distance histograms are given. For all word pairs of the experiment, the number ofdistances in a certain distance interval are given. The histograms for the two tests are very much alike. Ingeneral the similarity between two distributions is measured by the supremum norm of their difference. Thisnorm between the two distributions of pair distances is 0.05489. The probability that for a random partitionof the 66 rabbis into parts of 34 and 32 the norm is smaller or equal than 0.05489 is about 0.035. This, incombination with the strong similarity between the two P2 values, give an overall probability of 0.0006, ifthe two distributions are the same.

It is a big question, why the pair-distance distributions are so much the same. It is not clear why someonewants these distributions to be similar, and it is not clear too, how unintentional tuning in this way can takeplace.

It is certainly not the case that the two distributions should be the same, if the code is real. The rabbisof the first list are more famous than the rabbis on the second list (their entries cover more columns in TheEncyclopedia of Great Men in Israel). So it is not very unlikely that the Creator of the code offers moreattention to these more famous rabbis. But different distributions only make it less likely that the resultsare so close to each other.

In the histograms can be seen that big distances hardly appear in Genesis. This is strange, because thereare word pairs that are not given as ELSs at all. One may expect that some word pairs that are spelled asELSs are due to chance, otherwise one can expect that all words appear as ELSs. So one can expect for thebig distances behaviour like the behaviour in the text with the letters of Genesis, randomly permuted.

5 Critique on the method

5.1 The permutation test

The list of word pairs is not very good for permutations. The 32 rabbis have different numbers of appellationsand different numbers of dates of birth and death. Each rabbi-date pair consists of a different number ofword pairs. The number of these word pairs are different for different permutations. Especially for P2 agreater number of word pairs can influence the result. McKay et al. have done some calculations for thisphenomenon and it was shown that π0 produces more word-pairs (w,w′) than about 98% of all permutations.The influence of using more word pairs is not very clear from the definition of P2 and using the control textsdoes not show meaningful influence of this problem, but it will be better to use a method that does not haveproblems with the number of word pairs.

Persi Diaconis, Professor of Statistics at Harvard University, suggested a method that gives one singlevalue to each personality-date pair. Using this method we can avoid the problem. The Method is given in

12

a letter of Professor Diaconis to Robert Aumann of the Hebrew University in Jerusalem, dated at May 7,1990.

The proposal is:

• Let X be the space of persons and Y be the space of data for this person. Define a function d(x, y)from X × Y → [0, 1]. This function is not defined in this letter of Diaconis, but in an earlier letter heproposed to take the minimum distance c(w,w′) of the different word pairs in one person-date pair.The computation of d(x, y) is expensive.

• Define a statistic T (one can take P1−4 for it, but Diaconis proposed in first instance To use T =d(x1, y1) + . . .+ d(xn, yn)). This statistic can be used in the same way as it is used by Witztum et al.So we use a permutation test.

• Calculate the n × n matrix d(xi, yj) with 1 ≤ i, j ≤ n. For this matrix n2 values must be computed.So we have 322 expensive calculations. Now we can easily generate millions of cheap permutations andcalculate T (π).

For some reason Witztum et al. did not use this proposal. For MBBK1999 a variation of the permutationtest proposed by Diaconis was used and the result they get is not significant at all. McKay et al. took somekind of average of all c(w,w′) values for a given word pair (w,w′). It is not exactly discussed how this averageis taken. It is reasonable to ignore big c(w,w′) values. If the code is real, these values are probably due tochance. Witztum did the test with only the smallest c(w,w′) values. The result was: minρ1−4 = 0.000153.This result can certainly be seen as a success.

It seems better to me that Diaconis proposal is executed in the following way: Take the average over theten smallest c(w,w′) values for a given personality-dates pair. If there are less than ten c(w,w′) values thentake the average over all values. Now we have one value for every personality-dates pair and the 32 × 32matrix proposed by Diaconis can be generated. And the rest of the test can be run. The number ten isarbitrary, so we can do the experiment with some other numbers.

Another good method seems to be averaging the natural logarithm of the c(w,w) values for one personality-dates pair. In the paper Concerning the statistical test that was published in our paper in Statistical Science(Part B) by Doron Witztum, is referred to such an experiment done by Brendan McKay. The results ofthis experiment are given in Witztum’s article as 125

1,000,000 for the first list and 81,000,000 for the second list of

rabbis. The results are never published by McKay.From the c(w,w′) values per personality-date pair, P1 and P2 can be calculated. For every permutation

there are 32 values. These 32 values can be summed according to P1−4. So we have 8 results. Witztumexecuted this experiment and the best result was ρ = 0.000013. This result still is rather good. But it mustbe said that it is about 50 times worse than the results given in WRR1994. For some personality-date pairsthere are only a few c(w,w′) values. According to the definition of P1 and P3 it is not very useful to computethese values. They are almost worthless. But for the sake of completeness these values have been computed.The best result was not due to such problem cases, because for the best result only P2 and P4 are used.

Witztum et al. objected to the test done by McKay et al., because the procedure was not clear fromMBBK1999 and the statistics P1−4 are ignored completely. But P1−4 are not holy and the code should notbe dependent of those statistics. Another way of measuring should still reveal some code.

5.2 Sensitivity to small part of the data

The values of P2 and P4 are highly sensitive to the smallest values of c(w,w′). The following facts show howsmall variations can destroy the whole effect.

• The four rabbis that contribute the most strongly to the result have such an influence, that removingthese rabbis causes a jump in the significance level from 1 in 60.000 to 1 in 30. These rabbis are notmore interesting than other rabbis, historically speaking.

• One of the 102 appellations contributes a factor 10 to the result by it self. The five most influentialappellations together contribute a factor 860. These appellations are not more important than otherappellations.

13

These variations need some more remarks. Deleting the rabbis or appellations that have the greatestinfluence is not a priori. One can expect that if one delete the best results the overall result will becomeless impressive. So it would be fair to delete the worst results too. Or one can delete the best rabbis orappellations for each permutation and then compute the overall permutation rank. Another remark is thatit is not strange that some appellations have great influence. If one assumed that the code is indeed due toa supernatural Author of the Torah, one can imagine that this Author does not use all appellations for arabbi but use only few of these appellations and these appellations should have greater influence.

5.3 Critique on the distance function

Now we take a closer look at the multiset H(d, d′). For 1 ≤ i ≤ 10 the nearest integers to d/i and d′/i (12

rounded upward) are in H(d, d′) if they are at least two. Elements of this multiset may be equal.For different values of d, one may have a different size of H(d, d′). If d = d′ = 2, H(d, d′) = 2, 2 and if

d = d′ = 5, H(d, d′) = 5, 3, 2, 5,3, 2. For very small values of d and d′, the number of elements of H(d, d′)is very low. This has a negative influence on σ(e, e′) =

∑h∈H(d,d′ ) µh(e, e′) because the sum exists of less

elements. But σ(e, e′) should express success for small skips. This has serious effects on c(w,w′).To illustrate this problem consider the next example: We consider two unperturbed ELSs, e and e′ with

skip two e = (n, 2, k) and e′ = (n′, 2, k′). The position of the last letter of the first ELS differs two positionswith the position of the first letter of the second ELS. This skips are minimal over the whole book of Genesis.For these two ELSs holds:

H(2, 2) = 2, 2δ2(e, e′) = 3

µ2(e, e′) =1

3

σ(e, e′) =2

3

ω(e, e′)σ(e, e′) =2

3

In the same way for the two words spelled out by these ELSs we look at some perturbed ELSs. This timeall skips are six and the skips are minimal over 43

53 of Genesis. Now we have:

H(6, 6) = 6, 3, 2, 2,6,3, 2, 2δ6(e, e′) = 3

δ3(e, e′) = 12

δ2(e, e′) = 27

µ6(e, e′) =1

3

µ3(e, e′) =1

12

µ2(e, e′) =1

27

σ(e, e′) =53

54

ω(e, e′)σ(e, e′) =43

55

Note: 23 ≤ 43

54 . So this perturbed ELSs seems to give a better result than the unperturbed ELSs, while theunperturbed ELSs have a very small skip and appears in the letter matrix of Genesis as one straight line.with successive letters on Euclidian distance one. The perturbed ELSs are not minimal over the whole text.

Some better definition for σ(e, e′) should be used. Good alternatives seems to be to use one value for i inthe definition of H(d, d). This one is given as one of the variations in the next section. Another alternativeis to take for σ(e, e′) 20 times the mean value of the values of µh(e, e′) with h ∈ H(d, d′). For skips of fifteenand higher σ(e, e′) does not change but for smaller skips σ(e, e′) gets an higher value.

14

6 Some variations and remarks

6.1 About the variations

Many choices that Witztum et al. made are arbitrary. McKay et al. proposed some variations to show thatthe choices made for WRR1994 are extremely important for getting the results that are published.

A model for the space of variations is to consider the space of variations as a direct product X =X1 × . . . × Xn, where each Xi is the set of available choices of one parameter of the experiment. Thismodel is not exact but rather useful. The model is inexact because some variations in one parameter mayexclude variations in other parameters. McKay et al. only looked for neighbours of the original values of theparameters, that is, only one of the coordinates is changed. So it is no problem that the model is inexact.In general it is true that when two variations each give a worse result the combination of this two variationswill give a worse result too.

For results some measures should be compared. It is not very useful to compare the values of P1 andP3. From the definition it is clear that these measures are only influenced because of some combinationsin the word sample that have a c(w,w′)-value close to 0.2. P2 and P4 depends continuously on all thec(w,w′) values. The MBBK1999 research was done on the first list of rabbis (with 3 or more columns inThe Encyclopedia of Great Men in Israel) as well as on the the second list of rabbis (the list that was usedfor WRR1994). For the first list Witztum et al. only used the measures P1 and P2. For the second list allfour measures could be used. In MBBK1999 four values for each variation are given as result. The firstvalue is P2 for the first list divided by 1.76× 10−9. The second value is the least permutation rank for thefirst list, divided by 4.0× 10−5. This is the least permutation rank over P1−4. The third value is P4 for thesecond list divided by 7.9× 10−9. Finally, the fourth value is the least permutation rank for the second listdivided by 6.8× 10−7. So for the WRR1994 test the results are [1, 1, 1, 1]. In most cases P2 is the measurewith the best results for the first list and P4 for the second list. In many cases the measures P2 and P4 arenot that interesting. When we use a different measure, we get a value that may not be comparable to theoriginal value. For instance if you take the square of real positive numbers greater than one, the numbersbecome greater, but the rank order of the original numbers is equal to the rank order of the squares. Somevariations are indeed squares or square roots. So the new permutation ranks are the most interesting resultwe can look for.

6.2 The variations of the metric

In this subsection we will analyse some variations. Besides I will give the variations and the results of thesevariations. I will give some remarks about these variations.

Consider the function:

δh(e, e′) = ∆(d, h)2 + ∆(d′, h)2 + min0≤i≤k−1,0≤i′≤k′−1

∆(|n+ di− n′ − d′i′|, h)2.

This function is central in the definition of c(w,w′). For this function some choices are made. Now we takea look at some alternatives.

First define:

f = ∆(d, h),

f ′ = ∆(d′, h),

l = min0≤i≤k−1,0≤i′≤k′−1

∆(|n+ di− n′ − d′i′|, h),

µ = mean0≤i≤k−1,0≤i′≤k′−1∆(|n+ di− n′ − d′i′|, h),

m = ∆(|2n+ d(k − 1) − 2n′ − d′(k′ − 1)|

2, h),

L = max0≤i≤k−1,0≤i′≤k′−1

∆(|n+ di− n′ − d′i′|, h),

x, y = dimensions of smallest enclosing rectangle.

15

36 BRENDAN MCKAY, DROR BAR-NATAN, MAYA BAR-HILLEL, AND GIL KALAI

where the min, mean, and max are taken over 0 ≤ i ≤ k − 1 and 0 ≤ i′ ≤ k′ − 1. The

quantity m is the cylindrical distance between the midpoints of the two ELSs.

φ(e, e′) δ(e, e′) = φ(e, e′) δ(e, e′) =√

φ(e, e′)

f 2 + f ′2 + l2 [1, 1; 1, 1] (WRR) [154, 120; 10.1, 99]

f 2 + f ′2 + m2 [1.5, 3.7; 66, 92] [65, 83; 101, 650]

f 2 + f ′2 + µ2 [1.3, 5.1; 0.6, 2.3] [168, 230; 25, 410]

f 2 + f ′2 + L2 [2.4, 4.1; 1.0, 11.4] [220, 340; 40, 1000]

f 2 + f ′2 + 2l2 [2.5, 1.6; 2.8, 1.1] [210, 88; 12.1, 66]

2f 2 + 2f ′2 + l2 [1.4, 1.3; 0.6, 1.8] [61, 82; 11.7, 220]

(f + f ′ + l)2 [1.8, 1.9; 0.5, 1.0] [190, 137; 10.1, 154]

(f + f ′ + m)2 [0.6, 1.9; 17.5, 57] [98, 120; 130, 1200]

(f + f ′ + µ)2 [3.6, 8.3; 0.4, 3.7] [220, 290; 20, 550]

(f + f ′ + L)2 [7.1, 15.1; 0.5, 11.6] [430, 460; 34, 1100]

max(f, f ′, l)2 [2.4, 1.3; 2.7, 1.9] [86, 76; 6.8, 69]

max(f, f ′, m)2 [3.9, 6.8; 240, 230] [40, 58; 74, 400]

max(f, f ′, µ)2 [2.9, 9.8; 1.2, 3.0] [220, 280; 25, 310]

max(f, f ′, L)2 [2.5, 13.3; 1.1, 12.1] [380, 500; 39, 810]

µ2 [5.7, 18.6; 2.2, 4.2] [340, 360; 49, 420]

L2 [2.8, 13.6; 1.3, 12.3] [420, 530; 35, 740]

(L + l)2 [4.0, 13.8; 2.1, 7.0] [360, 380; 73, 570]

L2 + l2 [2.7, 13.4; 0.9, 5.5] [330, 450; 38, 600]

(x + y)2 [30, 44; 0.5, 16.8] [640, 550; 15.5, 630]

x2 + y2 [15.1, 33; 0.4, 9.7] [500, 610; 18.5, 620]

max(x, y)2 [9.9, 31; 0.2, 5.9] [190, 340; 31, 840]

xy [680, 140; 0.5, 71] [1.1e4, 720; 97, 3900]

x2 + y2 + l2 [8.9, 26; 0.4, 4.7] [180, 320; 24, 740]

x2 + y2 + m2 [1.5, 13.2; 2.3, 14.4] [150, 340; 26, 830]

x2 + y2 + µ2 [7.4, 24; 0.5, 5.4] [183, 310; 23, 680]

x2 + y2 + L2 [14.7, 38; 0.7, 8.2] [430, 560; 27, 720]

(x + y + l)2 [7.1, 17.4; 0.1, 1.1] [250, 290; 21, 440]

(x + y + m)2 [2.0, 13.7; 1.9, 13.5] [230, 380; 28, 705]

(x + y + µ)2 [22, 22; 0.3, 4.3] [430, 500; 22, 650]

(x + y + L)2 [10.4, 26; 0.8, 12.6] [610, 630; 37, 1100]

xy + l2 [42, 28; 0.3, 1.4] [3900, 600; 46, 211]

xy + m2 [4.0, 17.3; 3.8, 26] [670, 440; 74, 830]

xy + µ2 [11.6, 27; 0.4, 3.2] [740, 560; 49, 650]

xy + L2 [9.4, 26; 0.9, 15.0] [810, 710; 43, 1050]

Table 5. The effect of changing δh(e, e′)

16

The quantity m is the distance between the midpoints of two ELSs. We see that δh(e, e′) = f2 + f ′2 + l2.This is the square of a distance. McKay et al. have given the results for other choices for the distancefunctions, which measure the same type of compactness as the function of Witztum et al. If we assumebounded word lengths, all functions used in the next table are bounded above and below by constantmultiples of the first measure.

The new functions have some differences with respect to the original function. I think that the choicemade by Witztum et al., δh(e, e′) = f2 + f ′2 + l2, is the most rational. A negative property of the distancebetween the midpoints is that ELSs written like a cross, one vertical, one horizontal and one letter in commonhave less distance than when the ELSs are written in a sentence one following the other.

The previous table shows that Witztum et al. were extremely lucky with their choice for the distancefunction. None of the variations improved the results of WRR1994 for the permutation ranks. The squareroots are even extremely unsuccessful. For the square roots the P2 and P4 values are not very illustrative,but one can expect that the rank orders do not differ as much as they do. I cannot imagine one single reasonwhy the results of the square roots behave so extremely bad. But it is not fair to count all the variationsas independent variations. If one variation with a bad result is followed by another variation with a worseresult one can expect the total result to worsen too. Almost all variations that keep the squares of thedistances show less impressive results. Taking the square root does the same. So at least 33 variations arecombinations of two variations with worse results.

The results for the squares of distances are not extremely worse than the results of WRR1994, except foronly a few functions. The exceptions are f2 + f ′2 + m2, (f + f ′ + m)2, max(f, f ′,m)2, xy + m2, xy + L2,(x+ y)2 and xy. For the other functions the least permutation rank stays below 1 of 105 for the second list.So the result is still very good.

In all cases P4 decreases, the permutation rank of P4 increases. This indicates that the value of themeasure P4 is due to an overall tendency for c(w,w′) to decrease or increase with some measures. Thisindicates that the P4 value does not have any meaning on its own.

We can change some other things in the definitions. We have various possibilities for µh(e, e′). Themappings must have negative derivative, but there are certainly more possibilities than µ = 1

δ . In the nexttable the results of some variations in the definition of H(d, d) are given too. I have used the results ofMBBK1999 and some results from an article published on the internet by Witztum and Beremez in reply tothe article of McKay et al.

The results of minimum row-length are not interesting at all. It is due to an programming error thatthis minimum row length is included in the theory of the measures. But McKay et al. used it in their paperso the results are given here. The differences for different row-lengths are neglectable too. So this variationdoes not add much. It is seen that variations in the definition of µh(e, e′) have significant effects.

17

The effect of changing µh(e, e′) or H(d, d′)

Variation Scores

Definition of µh(e, e′):1/δ (WRR) [1; 1; 1; 1]

1/√δ [154; 120; 10.1; 99]

1/δ2 [560; 6.0; 26; 2.5]exp(−δ) [3e6; 240; 250; 33]

Definition of H(d, d′):Round 1

2 down [1.1; 1.0; 1.4; 1.5]Always round down [0.8; 0.8; 1.5; 1.6]

Always round up [1.4; 1.0; 0.4; 0.6]Remove duplicates [0.5; 0.7; 1.5; 1.7]

Use 1 value of i [2e5; 340; 31; 21]or 2 [2e4; 210; 3.4; 4.5]or 3 [2053; 91; 1.1; 1.8]or 4 [119; 16.4; 0.1; 0.2]or 5 [3.7; 0.6; 0.3; 0.2]or 6 [2.1; 0.5; 0.5; 0.5]or 7 [3.4; 2.5; 0.3; 0.3]or 8 [2.7; 1.7; 0.2; 0.2]or 9 [0.7; 0.7; 0.4; 0.5]

or 10 (WRR) [1; 1; 1; 1]or 11 [0.8; 0.9; 0.6; 0.7]or 12 [1.1; 1.2; 0.8; 0.8]or 13 [1.3; 1.3; 1.2; 1.0]or 14 [1.8; 2.0; 1.1; 0.9]or 15 [3.6; 3.3; 1.4; 1.1]or 20 [11.8; 5.9; 3.1; 3.8]or 25 [66; 15.3; 4.8; 5.4]or 50 [3600; 40; 93; 28]

Minimum row length 1 [1.0; 1.0; 1.0; 1.0]or 2 (WRR) [1; 1; 1; 1]

or 3 [0.9; 1.0; 1.3; 1.2]or 4 [0.9; 1.0; 1.0; 1.1]or 5 [0.9; 1.0; 1.2; 1.3]or 10 [1.1; 0.9; 5.4; 5.9]

McKay et al. have given some other functions for µh(e, e′). The results are:

wrong variations of µh(e, e′)

Definition of µh(e, e′) Scores

−δ [5e8; 6100; 1e8; 7e5]−δ2 [5e8; 2e4; 1e8; 7e5]−lnδ [6e8; 3000; 1e8; 8e5]

These variations are totally wrong. The dominating factors in the results are those ELSs, that have greatdistances from each other. ELSs with a very small c(w,w′) does not add anything to the results. It almostmakes no differences if they were not there at all. While ELSs with a big c(w,w′) influences the results verymuch. The whole research was done to investigate if the ELSs are in close proximity. The distant ELSs aresupposed to be there because of chance. So these results are without any meaning.

18

From the definitions, used in WRR1994, can be seen that Witztum et al. want to give an greater weightto minimal distances than to other distances. The factor Te is very important for the “code”. McKay et al.give some alternatives for this definition and how to take Te into account for the definition of Lω(e, e′). Notethat the third variation in the next table does not give more weight to the least skip at all.

variations of the definitions of domains of minimality

Variation Scores

Definition of TeUse < (WRR) [1, 1, 1, 1]

use ≤ [1.3; 1.1; 3.7; 2.7]whole text [27; 850; 2.0; 407]

Definition of Lω(e, e′)|Te ∩ Te′ | (WRR) [1; 1; 1; 1]|Te ∩ Te′ |2 [36; 1.5; 12.1; 1.1]|Te ∪ Te′ | [94; 580; 0.2; 29.1]

. . . but only if overlapping [27; 52; 0.5; 19.0]|Te||Te′ | [4.6; 1.3; 2.2; 0.8]

(|Te|+ |Te′ |)/2 [4.8; 42; 0.5; 11.9]√|Te||Te′| [2.7; 5.8; 0.8; 6.3]

min(|Te|, |Te′|) [1.1; 1.7; 0.9; 1.1]max(|Te|, |Te′|) [109; 470; 0.4; 27]

In MBBK1999 the functions σ(e, e′) and Ω(w,w′), are considered. Witztum et al. defined σ(e, e′) asfollows:

σ(e, e′) =∑

h∈H(d,d′ )

µh(e, e′).

and Ω(w,w′) for E(w) and E(w′) both non-empty:

Ω(x,y,z)(w,w′) =∑

e∈E(w),e′∈E(w′)

ω(e, e′)σ(e, e′),

Variations are given in the next table:

variations of definitions of σ(e, e′) or Ω(x,y,z)(w,w′)

Variation Scores

definition of σ(e, e′)using the sum (WRR) [1; 1; 1; 1]using the maximum [176; 6.3; 12.6; 3.9]

using H(d, d′) (WRR) [1; 1; 1; 1]include the values of h on each side of those in H(d, d′) [280; 7.9; 26; 17]

or two values on each side [420; 11.2; 21; 15]

definition of Ω(x,y,z)(w,w′)using the sum (WRR) [1; 1; 1; 1]

using the best term for Ω [4700; 13.6; 64; 1.8]. . . as well as the best for σ [2e5; 12.5; 690; 10.2]

. . . and include the values of h on each side of those in H(d, d′) [1e5; 23; 2200; 52]or two values on each side [9e4; 22; 2900; 100]

19

the effects of changing E(w)

Variation Scores

expected ELS count of 2 [7600; 7.0; 4e4; 310]or 5 [53; 1.6; 20; 19.5]or 6 [6.3; 0.8; 3.8; 0.9]or 7 [204; 8.8; 0.4; 0.5]or 8 [6.2; 2.4; 2.0; 0.8]or 9 [9.0; 4.1; 1.6; 1.0]

or 10 (WRR) [1; 1; 1; 1]or 11 [1.3; 1.3; 1.9; 1.8]or 12 [4.7; 3.6; 1.3; 0.7]or 13 [2.4; 2.5; 4.2; 0.9]or 14 [3.0; 3.0; 3.6; 0.9]or 15 [1.2; 2.9; 5.9; 2.0]or 20 [2.7; 8.3; 59; 7.1]or 25 [0.8; 4.0; 91; 15.2]or 30 [6.8; 14.1; 140; 22]or 50 [2.2; 4.1; 550; 79]or 75 [3.7; 4.5; 590; 81]or 100 [4.0; 4.7; 560; 62]

Exactly 10 ELSs [23; 2.2; 630; 7.7]Minimum skip of 1 [1.5; 2.1; 0.1; 5.0]

or 2 (WRR) [1; 1; 1; 1]or 3 [0.3; 0.7; 11.1; 5.9]or 4 [1.2; 1.6; 16.3; 7.9]or 5 [0.5; 0.8; 16.7; 11.3]or 10 [13.7; 0.6; 33; 35]

The definition of E(w) is also important for the definition of Ω(w,w′). Witztum et al. defined it accordingto a skip limit. This skip limit is got from an expected number of ELSs (See Section 2.1). This skip limit isintroduced to reduce computational effort. So variation of the expected ELS count should not lead to verydifferent scores. In the previous table is shown that the variations lead to scores that differ a lot from theoriginal score. When the expected ELS count increases the result should tend more and more to the resultthat is computed. Because the reduction (for computational effort) has decreasing effects.

McKay et al. also chose minimum skip lengths of 3, 4, 5 and 10. When these values are used one canexpect worse results because the code phenomenon relies on minimal ELSs and while using these values theminimal ELS with skip 2 are totally ignored. In the second list there is one very successful appellation,“Ha’raavi”. The minimal skip of this appellation can be expected to be 2 (because of the length of thisword and the frequencies of its letters). So the minimal ELS of this appellation is totally ignored by thesevariations.

In MBBK1999 is also looked for ELS pairs which lie within the cut-off at parameter 20 but not within thecut-off of parameter 10. The results of this variation is not interesting, because the best results are thrownin the waste-basket. For this reason it is impossible to compare.

In the next table the definition of the perturbations (x, y, z) is considered. Recall how this perturbationsare used. We look at a word w1w2....wk with k greater than 5. We also consider a triple (x, y, z) with−2 ≤ x, y, z ≤ 2. An (x, y, z)-ELS is a triple (n, d, k) such that gn+(i−1)d = wi for 1 ≤ i ≤ k−3, gn+(k−3)d+x =wk−2, gn+(k−2)d+x+y = wk−1 and gn+(k−1)d+x+y+z = wk. It is possible to choose gn+(k−3)d+x = wk−2,gn+(k−2)d+y = wk−1 and gn+(k−1)d+z = wk instead of gn+(k−3)d+x = wk−2, gn+(k−2)d+x+y = wk−1 andgn+(k−1)d+x+y+z = wk, or to perturb the third, fourth and fifth letter instead of the last three letters.Witztum suggested to perturb only the appellations or only the dates. We can also change the number ofpossible perturbations by also looking at perturbations of more than 2 places.

20

different ways to do perturbations

Variation Scores

Perturb as x, y, z instead of x, x+ y, x+ y + z [0.7; 0.1; 0.8; 2.1]Perturb letters 3, 4, 5 [0.4; 1.0; 1.3; 2.1]

Perturb up to three places [0.2; 2.4; 0.04; 1.1]or 4 places [0.2; 4.2; 0.005; 0.6]or 5 places [0.1; 5.0; 0.0007; 0.3]or 6 places [0.07; 4.8; 0.0003; 0.3]

Perturb last 2 letters [5e4; 4.5; 6700; 28]. . . up to 3 places [118; 2.4; 340; 18.6]

or 4 places [2.5; 0.6; 135; 48]Perturb only appellations [23; 7.5; 240; 34]

Perturb only dates [15000; 0.3; 1350; 7.3]

In the definition of c(w,w′) a lower bound of 10 for the number of defined Ω(x,y,z)(w,w′) is used. Thereason for this lower bound is easy to see. Imagine there are very successful convergences of ELSs in Genesis.The converging ELSs are not only very close but also very rare so there appear no perturbed ELSs for somewords, only real ELSs. Now the factor c(w,w) = 1

1 = 1. The result 1 means total failure. For denominatorbound 2, a “very successful” ELS could give the result c(w,w) = 1

2 = 0.5 Which also indicates no success.The following table shows no spectacular results, so the extreme successful ELSs do not appear. From thedata of the experiment, it turns out that only in one case of the second list, a denominator less than 25occurs. That was a result 4

19.

different denominator bounds

Variation Scores

Denominator bound2 [2.9; 1.0; 1.0; 1.0]3 [2.9; 1.2; 1.0; 1.0]4 [1.8; 1.2; 1.0; 1.0]5 [1.8; 1.2; 1.0; 1.0]

10(WRR) [1; 1; 1; 1]15 [1.0; 1.0; 1.0; 1.0]20 [1.0; 0.9; 1.1; 1.1]25 [1.0; 1.0; 1.1; 1.1]

McKay et al. give some variations in the cutoff for the definition of P1 and P3. As mentioned only thevalues of P2 for the first list, the least permutation rank for the first list, P4 for the second list and the leastpermutation rank for the second list are given in MBBK1999. So the given results are not very instructive.Witztum and Beremez give more interesting results in their article.

21

different P1 cutoffs for L1

Cutoff defining P1 P1 ρ1 P3 ρ3

0.05 475487 18.76 134 4.020.1 386357 84.42 1205 37.30.15 2639 26.13 74 6.43

0.2( WRR) 1 1 1 10.25 0.0024 0.069 0.019 0.130.33 0.0008 0.098 2.47 6.120.4 0.001 0.19 0.63 4.030.5 0.00013 0.036 0.018 0.41

different P1 cutoffs for L2

P1 ρ1 P3 ρ3

105048 18.5 5157 8.04133 1.89 6.57 0.26145 4.0 14.4 1.261 1 1 1

0.00032 0.014 0.000015 0.00190.00034 0.05 0.0001 0.0180.0083 0.21 0.0048 0.140.055 0.9 0.05 1.0

7 Comments about the list of word pairs

7.1 Critique on the list

Witztum et al. got their list of rabbis from the The Encyclopedia of Great Men in Israel, but the appellationsand the birth and death dates came from a wide variety of sources. The dates from The Encyclopedia ofGreat Men in Israel are treated like other information from other sources, some dates from this source areomitted and other dates are introduced because of other sources. Some rabbis do not have a birth date inThe Encyclopedia of Great Men in Israel, if this birth date can be found in other sources it is introduced butnot always. This gives some doubts about the claim that all information was a priori known. The researchershad some freedom to alter the data to improve the results.

The selection of the rabbis for the two lists was not executed very accurately. In each list there are rabbismissing and rabbis who are present but should not be there.

In Hebrew dates can be written in several forms. Witztum et al. only used the day and the month of thebirth- and death dates, not the years. The months can be written in several ways. A selection of this nameswas made. McKay et al. marked that this selection was not consequent because of the Biblical names ofthe months were not used. It must be said that most of the Biblical names come from other sources thanthe Torah. Specifying dates by special days such as religious holidays (like we say Christmas for the 25th ofDecember) is not done.

In Hebrew there are eight ways of writing dates. Like in English forms like “May 1st” and “1st of May”are used. Witztum et al. used only three forms. They omitted even one form that is used in The Encyclopediaof Great Men in Israel. Because of religious reasons the 15th and 16th of each month are usually writtenusing the letters representing 9 + 6 and 9 + 7 (instead of 10 + 5 and 10 + 6). The last forms were usedcenturies ago (and even that was not always the case). Now these forms are never used anymore and manyJews do not even know this way of writing 15 and 16. For WRR1994 both forms were used. Doron Witztumexplained the choice of not deleting the 10 + 5 and 10 + 6 form. These forms are deleted from the traditionalJewish use, because the same letters are used in the name of God, and This Name is that holy, that it is notallowed to use it. But in the Torah the name of God frequently appears, so the reason for ignoring theseforms does not hold any longer.

The rabbis are often known with many appellations. Like Beatrix van Oranje is called Queen Beatrix,Queen of the Netherlands, Her majesty or one of the many other possibilities, many rabbis are known withthe same diversity of appellations. Many acronyms are used. For example, “Rambam” is an acronym forRabbenu Moshe ben Maimon, also known as Maimonides (he is the most famous Jewish rabbi that livedafter the New Testament was written). Witztum et al. used less than half of all appellations that werepossible. So there must be some reason for this. In WRR1994 the authors say about their choice: “The listof appellations for each personality was prepared by Professor S. Z. Havlin, of the department of Bibliographyand Librarianship at Bar-Ilan University, on the basis of a computer search of the ’Responsa’ database atthe university.” But the method that Professor Havlin used was not mechanical. The selection method wasrather subjective. Many internet discussions are about this issue. I am no expert on this area, so I will only

22

mention this point of critique and give results of some variations of the names.

7.2 Results of some variations

As mentioned Witztum et al. allowed a very “obscure” way of writing 15 and 16. Only using the standardform gets a score of [8.7; 2.7; 33; 5.2]. The name of the month Cheshvan is an abbreviation of Marcheshvan.Only using the full name gets a score of [6.4; 1.8; 96; 51], while using both forms had no effects. The monthIyyar could been spelled with two yods (there is a good reason to do this). This variation yields a scoreof [7.2; 1.9; 3.7; 4.0]. Using both spellings leads to [0.3; 1.1; 5.5; 5.6]. On the Jewish calendar is a leap-yearmonth. In stead of using Adar 1 and Adar 2, Adar First and Adar Second can be used or use both forms.The results are: [9.2; 6.1; 1.0; 1.0] and [0.8; 0.9; 1.0; 1.0] respectively.

Witztum et al. could have used only the Biblical names of the months. (These names are not all from theTorah.) This variation gives a score of [220; 24; 3400; 2800] if only the Biblical names are used, which alsoyields that one month has no name so some rabbis must be deleted from the list. Using both forms yields ascore of [1.7; 10.5; 67; 450]. It is strange Witztum et al. did not use this variation, because they claimed toprefer biblical data.

In the first list of rabbis, Witztum et al. introduced a date not given in The Encyclopedia of Great Men inIsrael. One of the criteria for introducing a date was that a date was given in that Encyclopedia. The datecould be corrected but it must be there. The result of this variation in the first list is a score of [8.2; 4.9; 1; 1].This seems to be a good change. The result is not world shocking because the results published in WRR1994were taken from the second list only. Introducing some dates from other sources than The Encyclopedia ofGreat Men in Israel in the second list yields some other variations. McKay et al. gave some minor variationsthat did not sort very much effect. They did not publish the result of the experiment when all these variationsare used.

Some dates used for WRR1994 are according to some important sources totally wrong. In MBBK1999this problem is tackled by correcting the death date of rabbi Beirav or the death date of rabbi Teomim.These variations lead to [1; 1; 1.3; 0.8] and [1; 1; 0.9; 1.2]. The result of the combination of these variations isnot given. But it looks like this result does not differ very much from the results of WRR1994.

Writing the dates in alternative ways almost always makes the results of WRR1994 less impressive.McKay et al. give an example while introducing one extra form for the date. The score is [1.2; 2.2; 0.6; 16.4].The score of the variation that uses all date forms is not given. McKay et al. showed that the combinationof the three date forms is the best of all 28 − 1 = 255 combinations for P2 for the first list and for the leastpermutation rank of the second list (the result on which the publication of WRR1994 was based, the leastpermutation rank of the first list, the choice of Witztum et al. is sixth best. It is the third measure for P4

on the second list).

7.3 Experiments done by McKay et al.

Brendan McKay and Dror Bar-Natan used the freedom of choice of appellations to show that the methodused for WRR1994 could reveal codes in Tolstoy’s War and Peace too. They used the control text thatwas used for WRR1994, that is the first 78,064 letters of the Hebrew translation of War and Peace. 83appellations used by Witztum et al. are used for this experiment too. 20 appellations were deleted and 29other appellations were added. The choices that were made for this new list were not random but it wasnot mechanical and not very consistent too. McKay et al. do not even suggest that the data were a prioriknown, so we can expect some search for an optimum for the choice of appellations. An expert, MenachemCohen, said there is “no essential difference” between the lists in WRR1994 and MBBK1999. The resultsare very impressive and because of this one may think that Tolstoy was a supernatural being too.

McKay et al. also published an own experiment. For this experiment Simcha Emanuel, a specialist inrabbinical history at Tel-Aviv University was consulted. For the first experiment Emanuel get the names ofthe 32 rabbis that appear on the second list of WRR1994 and was asked to prepare appellations for eachof them. Emanuel had not seen the lists used in the article and was asked not to consult them, nor was hegiven any explicit guidance concerning which types of appellations to include and how to spell them. Hewas asked to use his own professional judgement to settle all issues. Emanuel consulted, during his work,a second historian David Assaf of Tel-Aviv university. Together they corrected some dates given in The

23

Encyclopedia of Great Men in Israel. The same method as used by Witztum et al. was performed on thenew list and the least permutation rank of P1−4 was 0.233.

The same experiment was carried out with a list of rabbis who takes 1 to 1.5 columns in The Encyclopediaof Great Men in Israel and whose dates of birth or death are mentioned. This list contains 26 rabbis. Theleast permutation rank of P1−4 was 0.404 for this experiment.

The third experiment by McKay et al. is described as follows:“After the above two experiments were completed, we carried out the following re-enactment of WRR’s

second experiment.

1. A list of rabbis was drawn from Margaliot’s encyclopedia by applying WRR’s criteria for their secondlist, while correcting the errors they made. Our list differed from WRR’s in dropping two rabbis andincluding three others. One rabbi who fits the selection criteria could not be included because heappears incorrectly in WRR’s first list.

2. Emanuel was shown the spelling rules and table of appellations for WRR’s first list as they firstappeared in WRR (1986). He then compiled a parallel table of appellations for our list of 33 rabbis,attempting to follow the rules and practices of WRR’s first list.

3. To mimic WRR’s processing of dates for their first list, we used the dates given by Margaliot exceptin the cases where Emanuel either found an error or found an additional date. In some cases Emanuelregarded a date as uncertain, in which case we followed WRR’s practice of leaving the date out. Overall,Emanuel changed more of Margaliot’s dates than WRR did.

4. The resulting list of word pairs was processed using WRR’s permutation test.

The result of applying WRR’s permutation test was that the least permutation rank of P1−4 was anuninteresting 0.254.”

In the new list performed by Emanuel are some fundamental differences from the list of WRR1994. Oneof the differences was that Emanuel sometimes used the one letter abbreviation for rabbi. This differencewas corrected after McKay et al. pointed out this differences to Emanuel. The permutation ranks for thistests are 0.154, 0.054, 0.089 and 0.017 for P1−4 respectively. Using the Bonferroni inequality an overallsignificance level of 0.068 is reached. This result is small but not very impressive. Because the list is adaptedto the lists used in WRR1994, one can say that the data of this list of Emanuel were not a priori as well.

Eliyahu Rips observed that the name of Theodor Herzl appears close to his birth date. McKay et al.made a list of all presidents, prime ministers and Knesset speakers of the State of Israel from 1948 to thedate the experiment was carried out (1999). In this word list are the family names and full names of thesepeople and their dates of birth and death (if known) and the date of their first inauguration. Only the resultsfor P1 and P2 are given. P3 and P4 are for clear reasons worthless. The permutation ranks were 0.155 forP1 and 0.044 for P2.

8 Conclusion and personal opinion

The Biblecode is an interesting phenomenon, which occupied the mind of many scientists. The claims are veryspectacular, and on the first sight the research of Witztum, Rips and Rosenberg seems to be very thorough,the tests were a priori given and the list of word pairs seems to be a priori known. The consequences of areal code are undeniable very great.

But a closer look at the research gives some doubt about the claims of Witztum et al. The list of wordpairs is not proven a priori. There is very much freedom in choosing the appellations of the rabbis. Theexperiment is very sensitive to some small variations. Not all variations respect the boundary of experiments.But it is not clear what the exact boundaries are. A test on a new sample of rabbis fails completely. If wealso take into account that the results of the two experiments are extremely close we can conclude that theexperiment has no value on its own. A new experiment with a new sample, that is objective and undeniablea priori, should be executed to convince people of the truth of the Biblecode.

The religious consequences of the research are a negative factor to the research, some people consider anattack towards the code as an attack towards their religion. On the other hand some people want to falsify

24

the claims of Witztum et al. that much, that they use wrong methods for their research. McKay et al. andsome other researchers made some errors, that are too big for people with their scientific background.

My personal opinion towards the codes is negative. The statistical proof for the phenomenon are notgood enough to convince me and the statistical proofs for fraud of Witztum et al. are too impressing todeny this.

As a Christian I believe that the Torah is a divine text, and each person should know what is in thesefive books (as well as in the whole Bible). But I cannot figure out one single reason why God should givesome code in the Torah. The reason cannot be that people now can predict the future, because predictionsof that kind are forbidden in the plain text of the Torah. It seems to be unlikely that God wants to proofHis existence in that very difficult way. It seems to me that He can use a more convincing method.

I think that Jews as well as Christians are not allowed to use the codes for outreach purposes. Thescientific proofs for the code are not convincing and I think it is not allowed to use any argument that isuncertain for this purposes.

9 Acknowledgement

Some people have contributed to this work. I want to thank Prof. Richard Gill for giving advice and forcorrecting some errors. Special thanks is due to the “referees” Hanne van Dijk, Barbara Raven and VictorTrip.

10 References

Many of the references are to documents published on the internet. To simplify the citation, I will abbreviatetwo important sites as follows:

M = http://cs.anu.edu.au/ bdm/dilugimW = http://www.torahcodes.co.il

• Mathematicians’ Statement on the Bible Codes. http://math.caltech.edu/code/petition.html

• M. Bar-Hillel, D. Bar-Natan, and B. D. McKay (1998). Torah codes: puzzle and solution.Chance, vol. 11, 13-19.

• D. Bar-Natan, A. Gindis, and B. D. McKay (1999). Report on new ELS tests of Torah, revisedreport. M/rejoin1.html

• D. Bar-Natan and B. D. McKay (1999). Equidistant letter sequences in Tolstoy’s “War andPeace” M/WNP/

• P. Diaconis (1986). Letter of December 30 to D. Kazhdan. This letter can be found on M/StatSci/permtest

• P. Diaconis (1990). Letter of September 5 to R. Aumann. This letter can be found on M/StatSci/permtest

• A. M. Hasofer (ca. 1997). Torah Codes: Reality or Illusion. M/opinions/hasofer.html

• A. M. Hasofer (1998). A statistical critique of the Witztum et al. paper. M/hasofer s.pdf

• G. Kalai (1997). Various articles on Bible codes. http://www.ma.huji.ac.il/ kalai/bc.html

• B. D. McKay et al. (1998). Jesus as the Son of Man. M/Jesus

• B. D. McKay (1997). Tolstoy Loves Me. M/wpmckay.html

• B. D. McKay (1998). An objective experiment of Doron Witztum. M/witztum/camps

• B. D. McKay (?). The Demise of Drosnin. M/drosnin.html

• B. D. McKay, D. Bar-Natan, M. Bar-Hillel and G. Kalai (1999). Solving the Bible CodePuzzle. Statistical Science, vol. 14, 150-173.

25

• M. Perakh (1998). Various articles on Bible codes. http://www.nctimes.net/ mark/fcodes/

• E. Rips (ca. 1985). Transcript of lecture. M/ripslect

• J. Satinover (1997). Cracking the Bible code. Morrow, New York. (In Dutch translation)

• B. Simon (1998). Various articles on Bible codes. http://wopr.com/biblecodes/

• D. Witztum (1998a). A refutation refuted. W/ref1.htm and W/ref2.htm

• D. Witztum (1998b). An Analysis of the Decisions in Our Research. W/docum4eng.htm

• D. Witztum (1999a). Concerning the Statistical Test that was Published in our Paper in StatisticalScience. W/persi2.htm

• D. Witztum (1999b). Concerning the Statistical Test that was Published in our Paper in StatisticalScience - Part B. W/persi4e.htm

• D. Witztum (2000). New Statistical Evidence for a Genuine Code in Genesis. W/emanuel/eman hb.htm

• D. Witztum and Y. Beremez (1998). The “Famous Rabbis” Sample: A New Measurement.W/new hb.htm

• D. Witztum and Y. Beremez (2000). MBBK’s Study of Variations. W/variat/var eng.htm

• D. Witztum and E. Rips (1996). “Bar Hillel and Bar Natan Inquire, Witztum and Rips Respond.”W/response.htm

• D. Witztum and E. Rips (1998). Choice of Choices. W/chance.htm

• D. Witztum, E. Rips and Y. Rosenberg (1994). Equidistant letter sequences in the Book ofGenesis. Statistical Science, vol. 9, 429-438.

• D. Witztum, E. Rips and Y. Rosenberg (ca. 1995). Equidistant letter sequences in the Bookof Genesis, II. The relation to the text. Preprint. M/Nations/WRR2/

26