Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1137–1147,Sofia, Bulgaria, August 4-9 2013. c©2013 Association for Computational Linguistics

Reconstructing an Indo-European Family Treefrom Non-native English texts

Ryo Nagata1,2 Edward Whittaker3

1Konan University / Kobe, Japan2LIMSI-CNRS / Orsay, France

3Inferret Limited / Northampton, Englandnagata-acl@hyogo-u.ac.jp, ed@inferret.co.uk

Abstract

Mother tongue interference is the phe-nomenon where linguistic systems of amother tongue are transferred to anotherlanguage. Although there has been plentyof work on mother tongue interference,very little is known about how stronglyit is transferred to another language andabout what relation there is across mothertongues. To address these questions,this paper explores and visualizes mothertongue interference preserved in Englishtexts written by Indo-European languagespeakers. This paper further explores lin-guistic features that explain why certainrelations are preserved in English writing,and which contribute to related tasks suchas native language identification.

1 Introduction

Transfer of linguistic systems of a mother tongueto another language, namely mother tongue inter-ference, is often observable in the writing of non-native speakers. The reader may be able to deter-mine the mother tongue of the writer of the fol-lowing sentence from the underlined article error:

The alien wouldn’t use my spaceship butthe hers.

The answer would probably be French or Span-ish; the definite article is allowed to modify pos-sessive pronouns in these languages, and the us-age is sometimes negatively transferred to Englishwriting. Researchers such as Swan and Smith(2001), Aarts and Granger (1998), Davidsen-Nielsen and Harder (2001), and Altenberg andTapper (1998) work on mother tongue interfer-ence to reveal overused/underused words, part ofspeech (POS), or grammatical items.

In contrast, very little is known about howstrongly mother tongue interference is transferredto another language and about what relation thereis across mother tongues. At one extreme, onecould argue that it is so strongly transferred totexts in another language that the linguistic rela-tions between mother tongues are perfectly pre-served in the texts. At the other extreme, onecan counter it, arguing that other features such asnon-nativeness are more influential than mothertongue interference. One possible reason for thisis that a large part of the distinctive language sys-tems of a mother tongue may be eliminated whentransferred to another language from a speaker’smother tongue. For example, Slavic languageshave a rich inflectional case system (e.g., Czechhas seven inflectional cases) whereas French doesnot. However, the difference in the richness cannotbe transferred into English because English has al-most no inflectional case system. Thus, one can-not determine the mother tongue of a given non-native text from the inflectional case. A similarargument can be made about some parts of gen-der, tense, and aspect systems. Besides, Wong andDras (2009) show that there are no significant dif-ferences, between mother tongues, in the misuseof certain syntactic features such as subject-verbagreement that have different tendencies depend-ing on their mother tongues. Considering these,one could not be so sure which argument is cor-rect. In any case, to the best of our knowledge, noone has yet answered this question.

In view of this background, we take the first stepin addressing this question. We hypothesize that:

Hypothesis: Mother tongue interference is sostrong that the relations in a language fam-ily are preserved in texts written in anotherlanguage.

In other words, mother tongue interference is sostrong that one can reconstruct a language fam-

1137

ily tree from non-native texts. One of the majorcontributions of this work is to reveal and visual-ize a language family tree preserved in non-nativetexts, by examining the hypothesis. This becomesimportant in native language identification1 whichis useful for improving grammatical error correc-tion systems (Chodorow et al., 2010) or for pro-viding more targeted feedback to language learn-ers. As we will see in Sect. 6, this paper revealsseveral crucial findings that contribute to improv-ing native language identification. In addition, thispaper shows that the findings could contribute toreconstruction of language family trees (Enrightand Kondrak, 2011; Gray and Atkinson, 2003;Barbancon et al., 2007; Batagelj et al., 1992;Nakhleh et al., 2005), which is one of the centraltasks in historical linguistics.

The rest of this paper is structured as follows.Sect. 2 introduces the basic approach of this work.Sect. 3 discusses the methods in detail. Sect. 4 de-scribes experiments conducted to investigate thehypothesis. Sect. 5 discusses the experimental re-sults. Sect. 6 discusses implications for work inrelated domains.

2 Approach

To examine the hypothesis, we reconstruct alanguage family tree from English texts writ-ten by non-native speakers of English whosemother tongue is one of the Indo-European lan-guages (Beekes, 2011; Ramat and Ramat, 2006).If the reconstructed tree is sufficiently similar tothe original Indo-European family tree, it will sup-port the hypothesis. If not, it suggests that somefeatures other than mother tongue interference aremore influential.

The approach we use for reconstructing a lan-guage family tree is to apply agglomerative hi-erarchical clustering (Han and Kamber, 2006) toEnglish texts written by non-native speakers. Re-searchers have already performed related workon reconstructing language family trees. For in-stance, Kroeber and Chrietien (1937) and Ellegard(1959) proposed statistical methods for measuringthe similarity metric between languages. More re-cently, Batagelj et al. (1992) and Kita (1999) pro-posed methods for reconstructing language fam-ily trees using clustering. Among them, the

1Recently, native language identification has drawn the at-tention of NLP researchers. For instance, a shared task onnative language identification took place at an NAACL-HLT2013 workshop.

most related method is that of Kita (1999). Inhis method, a variety of languages are modeledby their spelling systems (i.e., character-basedn-gram language models). Then, agglomera-tive hierarchical clustering is applied to the lan-guage models to reconstruct a language familytree. The similarity used for clustering is based ona divergence-like distance between two languagemodels that was originally proposed by Juang andRabiner (1985). This method is purely data-drivenand does not require human expert knowledge forthe selection of linguistic features.

Our work closely follows Kita’s work. How-ever, it should be emphasized that there is a signif-icant difference between the two. Kita’s work (andother previous work) targets clustering of a varietyof languages whereas our work tries to reconstructa language family tree preserved in non-native En-glish. This significant difference prevents us fromdirectly applying techniques in the literature to ourtask. For instance, Batagelj et al. (1992) use basicvocabularies such as belly in English and ventre inFrench to measure similarity between languages.Obviously, this does not work on our task; belly isbelly in English writing whoever writes it. Kita’smethod is also likely not to work well because alltexts in our task share the same spelling system(i.e., English spelling). Although spelling is some-times influenced by mother tongues, it involves alot more including overuse, underuse, and misuseof lexical, grammatical, and syntactic systems.

To solve the problem, this work adopts a word-based language model in the expectation that wordsequences reflect mother tongue interference. Atthe same time, its simple application would causea serious side effect. It would reflect the topicsof given texts rather than mother tongue interfer-ence. Unfortunately, there exists no such Englishcorpus that covers a variety of language speakerswith uniform topics; moreover the availability ofnon-native corpora is still somewhat limited. Thisalso means that available non-native corpora maybe too small to train reliable word-based languagemodels. The next section describes two methods(language model-based and vector-based), whichaddress these problems.

3 Methods

3.1 Language Model-based Method

To begin with, let us define the following symbolsused in the methods. Let Di be a set of English

1138

texts where i denotes a mother tongue i. Similarly,let Mi be a language model trained using Di.

To solve the problems pointed out in Sect. 2, weuse an n-gram language model based on a mixtureof word and POS tokens instead of a simple word-based language model. In this language model,content words in n-grams are replaced with theircorresponding POS tags. This greatly decreasesthe influence of the topics of texts, as desired. Italso decreases the number of parameters in thelanguage model.

To build the language model, the followingthree preprocessing steps are applied to Di. First,texts in Di are split into sentences. Second, eachsentence is tokenized, POS-tagged, and mappedentirely to lowercase. For instance, the first ex-ample sentence in Sect. 1 would give:

the/DT alien/NN would/MD not/RBuse/VB my/PRP$ spaceship/NN but/CCthe/DT hers/PRP ./.

Finally, words are replaced with their correspond-ing POS tags; for the following words, word to-kens are used as their corresponding POS tags:coordinating conjunctions, determiners, preposi-tions, modals, predeterminers, possessives, pro-nouns, question adverbs. Also, proper nouns aretreated as common nouns. At this point, the spe-cial POS tags BOS and EOS are added at the begin-ning and end of each sentence, respectively. Forinstance, the above example would result in thefollowing word/POS sequence:

BOS the NN would RB VB my NN butthe hers . EOS

Note that the content of the original sentence is farfrom clear while reflecting mother tongue interfer-ence, especially in the hers.

Now, the language model Mi can be built fromDi. We set n = 3 (i.e., trigram language model)following Kita’s work and use Kneser-Ney (KN)smoothing (Kneser and Ney, 1995) to estimate itsconditional probabilities.

With Mi and Di, we can naturally apply Kita’smethod to our task. The clustering algorithm usedis agglomerative hierarchical clustering with theaverage linkage method. The distance2 betweentwo language models is measured as follows. The

2It is not a distance in a mathematical sense. However,we will use the term distance following the convention in theliterature.

probability that Mi generates Di is calculated byPr(Di|Mi). Note that

Pr(Di|Mi) ≈Pr(w1,i) Pr(w2,i|w1,i)

×|Di|∏

t=3

Pr(wt,i|wt−2,i, wt−1,i) (1)

where wt,i and |Di| denote the tth token in Di andthe number of tokens in Di, respectively, since weuse the trigram language model. Then, the dis-tance from Mi to Mj is defined by

d(Mi → Mj) =1

|Dj |log

Pr(Dj |Mj)

Pr(Dj |Mi). (2)

In other words, the distance is determined basedon the ratio of the probabilities that each lan-guage model generates the language data. Becaused(Mi → Mj) and d(Mj → Mi) are not symmet-rical, we define the distance between Mi and Mj

to be their average:

d(Mi,Mj)=d(Mi → Mj)+d(Mj → Mi)

2. (3)

Equation (3) is used to calculate the distance be-tween two language models for clustering.

To sum up, the procedure of the language fam-ily tree construction method is as follows: (i) Pre-process each Di; (ii) Build Mi from Di; (iii) Cal-culate the distances between the language models;(iv) Cluster the language data using the distances;(v) Output the result as a language family tree.

3.2 Vector-based Method

We also examine a vector-based method for lan-guage family tree reconstruction. As we will seein Sect. 5, this method allows us to interpret clus-tering results more easily than with the languagemodel-based method while both result in similarlanguage family trees.

In this method, Di is modeled by a vector. Thevector is constructed based on the relative frequen-cies of trigrams. As a consequence, the distanceis naturally defined by the Euclidean distance be-tween two vectors. The clustering procedure is thesame as for the language model-based method ex-cept that Mi is vector-based and that the distancemetric is Euclidean.

1139

4 Experiments

We selected the ICLE corpus v.2 (Granger et al.,2009) as the target language data. It consists ofEnglish essays written by a wide variety of non-native speakers of English. Among them, the 11shown in Table 1 are of Indo-European languages.Accordingly, we selected the subcorpora of the 11languages in the experiments. Before the exper-iments, we preprocessed the corpus data to con-trol the experimental conditions. Because some ofthe writers had more than one native language, weexcluded essays that did not meet the followingthree conditions: (i) the writer has only one na-tive language; (ii) the writer has only one languageat home; (iii) the two languages in (i) and (ii) arethe same as the native language of the subcorpusto which the essay belongs3. After the selection,markup tags such as essay IDs were removed fromthe corpus data. Also, the symbols ‘ and ’ wereunified into ’4. For reference, we also used na-tive English (British and American university stu-dents’ essays in the LOCNESS corpus5) and twosets of Japanese English (ICLE and the NICE cor-pus (Sugiura et al., 2007)). Table 1 shows thestatistics on the corpus data.

Performance of POS tagging is an importantfactor in our methods because they are based onword/POS sequences. Existing POS taggers mightnot perform well on non-native English texts be-cause they are normally developed to analyze na-tive English texts. Considering this, we testedCRFTagger6 on non-native English texts contain-ing various grammatical errors before the exper-iments (Nagata et al., 2011). It turned out thatCRFTagger achieved an accuracy of 0.932 (com-pared to 0.970 on native texts). Although it did notperform as well as on native texts, it still achieveda fair accuracy. Accordingly, we decided to use itin our experiments.

Then, we generated cluster trees from the cor-pus data using the methods described in Sect. 3.

3For example, because of (iii), essays written by nativespeakers of Swedish in the Finnish subcorpus were excludedfrom the experiments. This is because they were collected inFinland and might be influenced by Finnish.

4The symbol ‘ is sometimes used for ’ (e.g., I‘m).5The LOCNESS corpus is a corpus of native En-

glish essays made up of British pupils’ essays, Britishuniversity students’ essays, and American universitystudents’ essays: https://www.uclouvain.be/en-cecl-locness.html

6Xuan-Hieu Phan, “CRFTagger: CRF English POSTagger,” http://crftagger.sourceforge.net/,2006.

Native language # of essays # of tokensBulgarian 294 219,551Czech 220 205,264Dutch 244 240,861French 273 202,439German 395 236,841Italian 346 219,581Norwegian 290 218,056Polish 354 251,074Russian 255 236,748Spanish 237 211,343Swedish 301 268,361English 298 294,357Japanese1 (ICLE) 171 224,534Japanese2 (NICE) 340 130,156Total 4,018 3,159,166

Table 1: Statistics on target corpora.

We used the Kyoto Language Modeling toolkit7

to build language models from the corpus data.We removed n-grams that appeared less than fivetimes8 in each subcorpus in the language mod-els. Similarly, we implemented the vector-basedmethod with trigrams using the same frequencycutoff (but without smoothing).

Fig. 1 shows the experimental results. Thetree at the top is the Indo-European family treedrawn based on the figure shown in Crystal(1997). It shows that the 11 languages are dividedinto three groups: Italic, Germanic, and Slavicbranches. The second and third trees are the clus-ter trees generated by the language model-basedand vector-based methods, respectively. The num-ber at each branching node denotes in which stepthe two clusters were merged.

The experimental results strongly support thehypothesis we made in Sect. 1. Fig. 1 revealsthat the language model-based method correctlygroups the 11 Englishes into the Italic, Ger-manic, and Slavic branches. It first mergesNorwegian-English and Swedish-English into acluster. The two languages belong to the NorthGermanic branch of the Germanic branch andthus are closely related. Subsequently, the lan-guage model-based method correctly merges theother languages into the three branches. A dif-

7The Kyoto Language Modeling toolkit: http://www.phontron.com/kylm/

8We found that the results were not sensitive to the valueof frequency cutoff so long as we set it to a small number.

1140

Polish

Italic Germanic Slavic

1

3

6

7

8

9

10

BulgarianSwedishFrench Spanish Norwegian Czech

Italian RussianDutchGerman

French

English

Spanish

English

Italian

English

Swedish

English

Norwegian

English

Dutch

English

German

English

Polish

English

Bulgarian

English

Czech

English

Russian

English

2 4

5

Indo-European family tree

Cluster tree generated by

LM-based method

1

3

4

7

6

8

10

French English

Spanish English

Italian English

Swedish English

Norwegian English

Dutch English

German English

Polish English

Bulgarian English

Czech English

Russian English

2

5

9Cluster tree generated by vector-based clustering

Figure 1: Experimental results.

ference between its cluster tree and the Indo-European family tree is that there are some mis-matches within the Germanic and Slavic branches.While the difference exists, the method stronglydistinguishes the three branches from one an-other. The third tree shows that the vector-basedmethod behaves similarly while it mistakenly at-taches Polish-English into an independent branch.From these results, we can say that mother tongueinterference is transferred into the 11 Englishes,strongly enough for reconstructing its languagefamily tree, which we propose calling the inter-language Indo-European family tree in English.

Fig. 2 shows the experimental results with na-tive and Japanese Englishes. It shows that thesame interlanguage Indo-European family treewas reconstructed as before. More interestingly,native English was detached from the interlan-guage Indo-European family tree contrary to theexpectation that it would be attached to the Ger-manic branch because English is of course a mem-ber of the Germanic branch. This implies thatnon-nativeness common to the 11 Englishes ismore influential than the intrafamily distance is9;

9Admittedly, we need further investigation to confirm thisargument especially because we applied CRFTagger, which isdeveloped to analyze native English, to both non-native andnative Englishes, which might affect the results.

Interlanguage Indo-European family tree Other family

JapaneseEnglish1

JapaneseEnglish2

3

Native English

1213

ACL 2013

Figure 2: Experimental results with native andJapanese Englishes.

otherwise, native English would be included inthe German branch. Fig. 2 also shows that thetwo sets of Japanese English were merged intoa cluster and that it was the most distant in thewhole tree. This shows that the interfamily dis-tance is the most influential factor. Based onthese results, we can further hypothesize as fol-lows: interfamily distance > non-nativeness >intrafamily distance.

5 Discussion

To get a better understanding of the interlanguageIndo-European family tree, we further explore lin-guistic features that explain well the above phe-nomena. When we analyze the experimental re-sults, however, some problems arise. It is al-most impossible to find someone who has a goodknowledge of the 11 languages and their motherlanguage interference in English writing. Besides,there are a large number of language pairs to com-pare. Thus, we need an efficient and effective wayto analyze the experimental results.

To address these problems, we did the follow-ing. First, we focused on only a few Englishesout of the 11. Because one of the authors hadsome knowledge of French, we selected French-English as the main target. This naturally madeus select the other Italic Englishes as its counter-parts. Also, because we had access to a nativespeaker of Russian who had a good knowledge ofEnglish, we included Russian-English in our fo-cus. We analyzed these Englishes and then ex-amined whether the findings obtained apply to theother Englishes or not. Second, we used a methodfor extracting interesting trigrams from the cor-pus data. The method compares three out of the11 corpora (for example, French-, Spanish-, andRussian-Englishes). If we remove instances of atrigram from each set, the clustering tree involving

1141

the three may change. For example, the removalof but the hers may result in a cluster tree merg-ing French- and Russian-Englishes before French-and Spanish-Englishes. Even if it does not change,the distances may change in that direction. We an-alyzed what trigrams had contributed to the clus-tering results with this approach.

To formalize this approach, we will denote a tri-gram by t. We will also denote its relative fre-quency in the language data Di by rti. Then, thechange in the distances caused by the removal of tfrom Di, Dj , and Dk is quantified by

s = (rtk − rti)2 − (rtj − rti)

2 (4)

in the vector-based method. The quantity (rtk −rti)

2 is directly related to the decrease in the dis-tance between Di and Dk and similarly, (rtj −rti)

2 to that between Di and Dj in the vector-based method. Thus, the greater s is, the higher thechance that the cluster tree changes. Therefore, wecan obtain a list of interesting trigrams by sortingthem according to s. We could do a similar calcu-lation in the language model-based method usingthe conditional probabilities. However, it requiresa more complicated calculation. Accordingly, welimit ourselves to the vector-based method in thisanalysis, noting that both methods generated sim-ilar cluster trees.

Table 2 shows the top 15 interesting trigramswhere Di, Dj , and Dk are French-, Spanish-, andRussian-Englishes, respectively. Note that s ismultiplied by 106 and r is in % for readability. Thelist reveals that many of the trigrams contain thearticle a or the. Interestingly, their frequencies aresimilar in French-English and Spanish-English,and both are higher than in Russian-English. Thiscorresponds to the fact that French and Spanishhave articles whereas Russian does not. Actu-ally, the same argument can be made about theother Italic and Slavic Englishes (e.g., the JJ NN:Italian-English 0.82; Polish-English 0.72)10. Anexception is that of trigrams containing the definitearticle in Bulgarian-English; it tends to be higherin Bulgarian-English than in the other Slavic En-glishes. Surprisingly and interestingly, however, itreflects the fact that Bulgarian does have the def-inite article but not the indefinite article (e.g., theJJ NN: 0.82; a JJ NN: 0.60 in Bulgarian-English).

10Due to the space limitation, other lists were not includedin this paper but are available at http://web.hyogo-u.ac.jp/nagata/acl/.

Table 3 shows that the differences in articleuse exist even between the Italic and Germanicbranches despite the fact that both have the in-definite and definite articles. The list still con-tains a number of trigrams containing articles. Fora better understanding of this, we looked furtherinto the distribution of articles in the corpus data.It turns out that the distribution almost perfectlygroups the 11 Englishes into the correspondingbranches as shown in Fig. 3. The overall use ofarticles is less frequent in the Slavic-Englishes.The definite article is used more frequently in theItalic-Englishes than in the Germanic Englishes(except for Dutch-English). We speculate thatthis is perhaps because the Italic languages have awider usage of the definite article such as its modi-fication of possessive pronouns and proper nouns.The Japanese Englishes form another group (thisis also true for the following findings). This corre-sponds to the fact that the Japanese language doesnot have an article system similar to that of En-glish.

s Trigram t rti rtj rtk

5.14 the NN of 1.01 0.98 0.784.38 a JJ NN 0.85 0.77 0.622.74 the JJ NN 0.87 0.86 0.712.30 NN of the 0.49 0.52 0.331.64 . . . 0.22 0.12 0.051.56 NNS . EOS 0.77 0.70 0.921.31 NNS and NNS 0.09 0.13 0.211.25 BOS RB , 0.25 0.22 0.141.22 of the NN 0.42 0.44 0.301.17 VBZ to VB 0.26 0.22 0.141.09 BOS i VBP 0.07 0.05 0.171.03 NN of NN 0.74 0.70 0.630.88 NN of JJ 0.15 0.15 0.250.67 the JJ NNS 0.28 0.28 0.200.65 NN to VB 0.40 0.38 0.31

Table 2: Interesting trigrams (French- (Di),Spanish- (Dj), and Russian- (Dk) Englishes).

Another interesting trigram, though not as ob-vious as article use, is NN of NN, which ranks12th and 2nd in Table 2 and 3, respectively. In theItalic Englishes, the trigram is more frequent thanthe other non-native Englishes as shown in Fig. 4.This corresponds to the fact that noun-noun com-pounds are less common in the Italic languagesthan in English and that instead, the of -phrase (NNof NN) is preferred (Swan and Smith, 2001). For

1142

s Trigram t rti rtj rtk

21.49 the NN of 1.01 0.98 0.545.70 NN of NN 0.74 0.70 0.503.26 NN of the 0.49 0.52 0.303.10 the JJ NN 0.87 0.86 0.702.62 . . . 0.22 0.12 0.031.53 of the NN 0.42 0.44 0.291.50 NN , NN 0.30 0.30 0.181.50 BOS i VBP 0.07 0.05 0.190.85 NNS and NNS 0.09 0.13 0.190.81 JJ NN of 0.40 0.39 0.310.68 . . EOS 0.13 0.06 0.020.63 a JJ NN 0.85 0.77 0.730.63 RB . EOS 0.21 0.16 0.310.56 NN , the 0.16 0.16 0.080.50 NN of a 0.17 0.09 0.06

Table 3: Interesting trigrams (French- (Di),Spanish- (Dj), and Swedish- (Dk) Englishes).

instance, orange juice is expressed as juice of or-ange in the Italic languages (e.g., jus d’orange inFrench). In contrast, noun-noun compounds orsimilar constructions are more common in Russianand Swedish. As a result, NN of NN becomes rel-atively frequent in the Italic Englishes. Fig. 4 alsoshows that its distribution roughly groups the 11Englishes into the three branches. Therefore, theway noun phrases (NPs) are constructed is a clueto how the three branches were clustered.

This finding in turn reveals that the consecu-tive repetitions of nouns occur less in the ItalicEnglishes. In other words, the length tends tobe shorter than in the others where we definethe length as the number of consecutive repeti-tions of common nouns (for example, the lengthof orange juice is one because a noun is con-secutively repeated once). To see if this is true,we calculated the average length for each English.Fig. 5 shows that the average length roughly dis-tinguishes the Italic Englishes from the other non-native Englishes; French-English is the shortest,which is explained by the discussion above, whileDutch- and German-Englishes are longest, whichmay correspond to the fact that they have a prefer-ence for noun-noun compounds as Snyder (1996)argues. For instance, German allows the concate-nated form as in Orangensaft (equivalently or-angejuice). This tendency in the length of noun-noun compounds provides us with a crucial insightfor native language identification, which we will

2

3

4

5

6

1 1.5 2 2.5 3

Rel

ativ

e fr

eque

ncy

of d

efin

ite a

rtic

le (

%)

Relative frequency of indefinite article (%)

Bulgarian

Czech

DutchFrench

German

Italian

NorwegianPolish

Russian

Spanish

Swedish

English

Japanese1

Japanese2

ItalicGermanic

SlavicJapanese

Figure 3: Distribution of articles.

0

0.5 1

Relative frequency of NN of NN (%)

FrenchItalianSpanish

Italic

PolishRussianBulgarianCzech

Slavic

EnglishDutchSwedishGermanNorwegian

Germanic

Japanese1Japanese2

Japanese

Figure 4: Relative frequency of NN of NN in eachcorpus (%).

come back to in Sect. 6.The trigrams BOS RB , in Table 2 and RB . EOS

in Table 3 imply that there might also be a certainpattern in adverb position in the 11 Englishes (theyroughly correspond to adverbs at the beginningand end of sentences). Fig. 6 shows an insight intothis. The horizontal and vertical axes correspondto the ratio of adverbs at the beginning and the endof sentences, respectively. It turns out that the Ger-man Englishes form a group. So do the Italic En-glishes although it is less dense. In contrast, theSlavic Englishes are scattered. However, the ra-tios give a clue to how to distinguish Slavic En-glishes from the others when combined with other

1143

0

0.1

Average length of noun-noun compounds

FrenchItalianSpanish

Italic

BulgarianCzechRussianPolish

Slavic

SwedishNorwegianGermanDutchEnglish

Germanic

Japanese1Japanese2

Japanese

Figure 5: Average length of noun-noun com-pounds in each corpus.

5

10

15 20 25 30

Rat

io o

f adv

erbs

at t

he e

nd (

%)

Ratio of adverbs at the beginning (%)

Bulgarian

Czech

Polish

Russian

Dutch

German

Norwegian

Swedish

French

ItalianSpanish

English

Japanese1Japanese2

ItalicGermanic

SlavicJapanese

Figure 6: Distribution of adverb position.

trigrams. For instance, although Polish-Englishis located in the middle of Swedish-English andBulgarian-English in the distribution of articles(in Fig. 3), the ratios tell us that Polish-English ismuch nearer to Bulgarian-English.

6 Implications for Work in RelatedDomains

Researchers including Wong and Dras (2009),Wong et al. (2011; 2012), and Koppel et al. (2005)work on native language identification and showthat machine learning-based methods are effec-tive. Wong and Dras (2009) propose using infor-mation about grammatical errors such as errors indeterminers to achieve better performance while

they show that its use does not improve the perfor-mance, contrary to the expectation. Related to this,other researchers (Koppel and Ordan, 2011; vanHalteren, 2008) show that machine learning-basedmethods can also predict the source language ofa given translated text although it should be em-phasized that it is a different task from native lan-guage identification because translation is not typ-ically performed by non-native speakers but rathernative speakers of the target language11.

The experimental results show that n-gramscontaining articles are predictive for identify-ing native languages. This indicates that theyshould be used in the native language identifi-cation task. Importantly, all n-grams contain-ing articles should be used in the classifier unlikethe previous methods that are based only on n-grams containing article errors. Besides, no ar-ticles should be explicitly coded in n-grams fortaking the overuse/underuse of articles into con-sideration. We can achieve this by adding a spe-cial symbol such as φ to the beginning of each NPwhose head noun is a common noun and that hasno determiner in it as in “I like φ orange juice.”

In addition, the length of noun-noun com-pounds and the position of adverbs should alsobe considered in native language identification. Inparticular, the former can be modeled by the Pois-son distribution as follows. The Poisson distribu-tion gives the probability of the number of eventsoccurring in a fixed time. In our case, the numberof events in a fixed time corresponds to the num-ber of consecutive repetitions of common nouns inNPs, which in turn corresponds to the length. Tobe precise, the probability of a noun-noun com-pound with length l is given by

Pr(l) =λl

l!e−λ, (5)

where λ corresponds to the average length. Fig. 7shows that the observed values in the French-English data very closely fit the theoretical proba-

11For comparison, we conducted a pilot study where wereconstructed a language family tree from English textsin European Parliament Proceedings Parallel Corpus (Eu-roparl) (Koehn, 2011). It turned out that the reconstructedtree was different from the canonical tree (available at http://web.hyogo-u.ac.jp/nagata/acl/). However,we need further investigation to confirm it because each sub-corpus in Europarl is variable in many dimensions includ-ing its size and style (e.g., overuse of certain phrases such asladies and gentlemen).

1144

0

0.5

1

0 1 2 3

Pro

babi

lity

Length of noun-noun compound

TheoreticalObserved

Figure 7: Distribution of noun-noun compoundlength for French-English.

bilities given by Equation (5)12. This holds for theother Englishes although we cannot show them be-cause of the space limitation. Consequently, Equa-tion (5) should be useful in native language identi-fication. Fortunately, it can be naturally integratedinto existing classifiers.

In the domain of historical linguistics, re-searchers have used computational and corpus-based methods for reconstructing language fam-ily trees. Some (Enright and Kondrak, 2011;Gray and Atkinson, 2003; Barbancon et al., 2007;Batagelj et al., 1992; Nakhleh et al., 2005) ap-ply clustering techniques to the task of languagefamily tree reconstruction. Others (Kita, 1999;Rama and Singh, 2009) use corpus statistics forthe same purpose. These methods reconstruct lan-guage family trees based on linguistic features thatexist within words including lexical, phonological,and morphological features.

The experimental results in this paper suggestthe possibility of the use of non-native texts for re-constructing language family trees. It allows us touse linguistic features that exist between words, asseen in our methods, which has been difficult withprevious methods. Language involves the featuresbetween words such as phrase construction andsyntax as well as the features within words andthus they should both be considered in reconstruc-

12The theoretical and observed values are so close that itis difficult to distinguish between the two lines in Fig. 7. Forexample, Pr(l = 1) = 0.0303 while the corresponding ob-served value is 0.0299.

tion of language family trees.

7 Conclusions

In this paper, we have shown that mother tongueinterference is so strong that the relations be-tween members of the Indo-European languagefamily are preserved in English texts written byIndo-European language speakers. To show this,we have used clustering to reconstruct a lan-guage family tree from 11 sets of non-nativeEnglish texts. It turned out that the recon-structed tree correctly groups them into the Italic,Germanic, and Slavic branches of the Indo-European family tree. Based on the resultingtrees, we have then hypothesized that the fol-lowing relation holds in mother tongue interfer-ence: interfamily distance > non-nativeness >intrafamily distance. We have further exploredseveral intriguing linguistic features that play animportant role in mother tongue interference: (i)article use, (ii) NP construction, and (iii) adverbposition, which provide several insights for im-proving the tasks of native language identificationand language family tree reconstruction.

Acknowledgments

This work was partly supported by the Digiteo for-eign guest project. We would like to thank thethree anonymous reviewers and the following per-sons for their useful comments on this paper: Ko-taro Funakoshi, Mitsuaki Hayase, Atsuo Kawai,Robert Ladig, Graham Neubig, Vera Sheinman,Hiroya Takamura, David Valmorin, Mikko Vile-nius.

ReferencesJan Aarts and Sylviane Granger, 1998. Tag sequences

in learner corpora: a key to interlanguage gram-mar and discourse, pages 132–141. Longman, NewYork.

Bengt Altenberg and Marie Tapper, 1998. The use ofadverbial connectors in advanced Swedish learners’written English, pages 80–93. Longman, New York.

Francois Barbancon, Tandy Warnow, Steven N. Evans,Donald Ringe, and Luay Nakhleh. 2007. An exper-imental study comparing linguistic phylogenetic re-construction methods. Statistics Technical Reports,page 732.

Vladimir Batagelj, Tomaz Pisanski, and Dami-jana Kerzic. 1992. Automatic clustering of lan-guages. Computational Linguistics, 18(3):339–352.

1145

Robert S.P. Beekes. 2011. Comparative Indo-European Linguistics: An Introduction (2nd ed.).John Benjamins Publishing Company, Amsterdam.

Martin Chodorow, Michael Gamon, and Joel R.Tetreault. 2010. The utility of article and prepo-sition error correction systems for English languagelearners: feedback and assessment. Language Test-ing, 27(3):419–436.

David Crystal. 1997. The Cambridge Encyclopedia ofLanguage (2nd ed.). Cambridge University Press,Cambridge.

Niels Davidsen-Nielsen and Peter Harder, 2001.Speakers of Scandinavian languages: Danish, Nor-wegian, Swedish, pages 21–36. Cambridge Univer-sity Press, Cambridge.

Alvar Ellegard. 1959. Statistical measurement of lin-guistic relationship. Language, 35(2):131–156.

Jessica Enright and Grzegorz Kondrak. 2011. The ap-plication of chordal graphs to inferring phylogenetictrees of languages. In Proc. of 5th InternationalJoint Conference on Natural Language Processing,pages 8–13.

Sylviane Granger, Estelle Dagneaux, Fanny Meunier,and Magali Paquot. 2009. International Corpus ofLearner English v2. Presses universitaires de Lou-vain, Louvain.

Russell D. Gray and Quentin D. Atkinson. 2003.Language-tree divergence times support the Ana-tolian theory of Indo-European origin. Nature,426:435–438.

Jiawei Han and Micheline Kamber. 2006. Data Min-ing: Concepts and Techniques (2nd Ed.). MorganKaufmann Publishers, San Francisco.

Bing-Hwang Juang and Lawrence R. Rabiner. 1985.A probabilistic distance measure for hidden Markovmodels. AT&T Technical Journal, 64(2):391–408.

Kenji Kita. 1999. Automatic clustering of languagesbased on probabilistic models. Journal of Quantita-tive Linguistics, 6(2):167–171.

Reinhard Kneser and Hermann Ney. 1995. Im-proved backing-off for m-gram language modeling.In Proc. of International Conference on Acoustics,Speech, and Signal Processing, volume 1, pages181–184.

Philipp Koehn. 2011. Europarl: A parallel corpus forstatistical machine translation. In Proc. of 10th Ma-chine Translation Summit, pages 79–86.

Moshe Koppel and Noam Ordan. 2011. Translationeseand its dialects. In Proc. of 49th Annual Meetingof the Association for Computational Linguistics,pages 1318–1326.

Moshe Koppel, Jonathan Schler, and Kfir Zigdon.2005. Determining an author’s native language bymining a text for errors. In Proc. of 11th ACMSIGKDD International Conference on KnowledgeDiscovery in Data Mining, pages 624–628.

Alfred L. Kroeber and Charles D. Chrietien. 1937.Quantitative classification of Indo-European lan-guages. Language, 13(2):83–103.

Ryo Nagata, Edward Whittaker, and Vera Shein-man. 2011. Creating a manually error-tagged andshallow-parsed learner corpus. In Proceedings ofthe 49th Annual Meeting of the Association for Com-putational Linguistics: Human Language Technolo-gies, pages 1210–1219.

Luay Nakhleh, Tandy Warnow, Don Ringe, andSteven N. Evans. 2005. A comparison of phyloge-netic reconstruction methods on an Indo-Europeandataset. Transactions of the Philological Society,103(2):171–192.

Taraka Rama and Anil Kumar Singh. 2009. From bagof languages to family trees from noisy corpus. InProc. of Recent Advances in Natural Language Pro-cessing, pages 355–359.

Anna Giacalone Ramat and Paolo Ramat, 2006. TheIndo-European Languages. Routledge, New York.

William Snyder. 1996. The acquisitional role of thesyntax-morphology interface: Morphological com-pounds and syntactic complex predicates. In Proc.of Annual Boston University Conference on Lan-guage Development, volume 2, pages 728–735.

Masatoshi Sugiura, Masumi Narita, Tomomi Ishida,Tatsuya Sakaue, Remi Murao, and Kyoko Muraki.2007. A discriminant analysis of non-native speak-ers and native speakers of English. In Proc. of Cor-pus Linguistics Conference CL2007, pages 84–89.

Michael Swan and Bernard Smith. 2001. Learner En-glish (2nd Ed.). Cambridge University Press, Cam-bridge.

Hans van Halteren. 2008. Source language markersin EUROPARL translations. In Proc. of 22nd Inter-national Conference on Computational Linguistics,pages 937–944.

Sze-Meng J. Wong and Mark Dras. 2009. Con-trastive analysis and native language identification.In Proc. Australasian Language Technology Work-shop, pages 53–61.

Sze-Meng J. Wong, Mark Dras, and Mark Johnson.2011. Exploiting parse structures for native lan-guage identification. In Proc. Conference on Em-pirical Methods in Natural Language Processing,pages 1600–1611.

Sze-Meng J. Wong, Mark Dras, and Mark Johnson.2012. Exploring adaptor grammars for native lan-guage identification. In Proc. Joint Conference on

1146

Empirical Methods in Natural Language Process-ing and Computational Natural Language Learning,pages 699–709.

1147