Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 313–321,Edinburgh, Scotland, UK, July 27–31, 2011. c©2011 Association for Computational Linguistics

Simple Effective Decipherment via Combinatorial Optimization

Taylor Berg-Kirkpatrick and Dan KleinComputer Science Division

University of California at Berkeley{tberg, klein}@cs.berkeley.edu

Abstract

We present a simple objective function thatwhen optimized yields accurate solutions toboth decipherment and cognate pair identifica-tion problems. The objective simultaneouslyscores a matching between two alphabets anda matching between two lexicons, each in adifferent language. We introduce a simplecoordinate descent procedure that efficientlyfinds effective solutions to the resulting com-binatorial optimization problem. Our systemrequires only a list of words in both languagesas input, yet it competes with and surpassesseveral state-of-the-art systems that are bothsubstantially more complex and make use ofmore information.

1 Introduction

Decipherment induces a correspondence betweenthe words in an unknown language and the wordsin a known language. We focus on the setting wherea close correspondence between the alphabets of thetwo languages exists, but is unknown. Given onlytwo lists of words, the lexicons of both languages,we attempt to induce the correspondence betweenalphabets and identify the cognates pairs present inthe lexicons. The system we propose accomplishesthis by defining a simple combinatorial optimiza-tion problem that is a function of both the alphabetand cognate matchings, and then induces correspon-dences by optimizing the objective using a block co-ordinate descent procedure.

There is a range of past work that has var-iously investigated cognate detection (Kondrak,2001; Bouchard-Cote et al., 2007; Bouchard-Coteet al., 2009; Hall and Klein, 2010), character-leveldecipherment (Knight and Yamada, 1999; Knightet al., 2006; Snyder et al., 2010; Ravi and Knight,

2011), and bilingual lexicon induction (Koehn andKnight, 2002; Haghighi et al., 2008). We considera common element, which is a model wherein thereare character-level correspondences and word-levelcorrespondences, with the word matching parame-terized by the character one. This approach sub-sumes a range of past tasks, though of course pastwork has specialized in interesting ways.

Past work has emphasized the modeling as-pect, where here we use a parametrically simplisticmodel, but instead emphasize inference.

2 Decipherment as Two-LevelOptimization

Our method represents two matchings, one at the al-phabet level and one at the lexicon level. A vector ofvariables x specifies a matching between alphabets.For each character i in the source alphabet and eachcharacter j in the target alphabet we define an indi-cator variable xij that is on if and only if character iis mapped to character j. Similarly, a vector y rep-resents a matching between lexicons. For word u inthe source lexicon and word v in the target lexicon,the indicator variable yuv denotes that u maps to v.Note that the matchings need not be one-to-one.

We define an objective function on the matchingvariables as follows. Let EDITDIST(u, v;x) denotethe edit distance between source word u and targetword v given alphabet matching x. Let the lengthof word u be lu and the length of word w be lw.This edit distance depends on x in the followingway. Insertions and deletions always cost a constantε.1 Substitutions also cost ε unless the charactersare matched in x, in which case the substitution is

1In practice we set ε = 1lu+lv

. lu + lv is the maximumnumber of edit operations between words u and v. This nor-malization insures that edit distances are between 0 and 1 forall pairs of words.

313

free. Now, the objective that we will minimize canbe stated simply:

∑u

∑v yuv · EDITDIST(u, v;x),

the sum of the edit distances between the matchedwords, where the edit distance function is parame-terized by the alphabet matching.

Without restrictions on the matchings x and ythis objective can always be driven to zero by eithermapping all characters to all characters, or matchingnone of the words. It is thus necessary to restrictthe matchings in some way. Let I be the size ofthe source alphabet and J be the size of the targetalphabet. We allow the alphabet matching x tobe many-to-many but require that each characterparticipate in no more than two mappings and thatthe total number of mappings be max(I, J), aconstraint we refer to as restricted-many-to-many.The requirements can be encoded with the followinglinear constraints on x:

∀i∑

j

xij ≤ 2

∀j∑

i

xij ≤ 2

∑

i

∑

j

xij = max(I, J)

The lexicon matching y is required to be τ -one-to-one. By this we mean that y is an at-most-one-to-onematching that covers proportion τ of the smaller ofthe two lexicons. Let U be the size of the sourcelexicon and V be this size of the target lexicon.This requirement can be encoded with the followinglinear constraints:

∀u∑

v

yuv ≤ 1

∀v∑

u

yuv ≤ 1

∑

u

∑

v

yuv = τ min(U, V )

Now we are ready to define the full optimizationproblem. The first formulation is called the ImplicitMatching Objective since includes an implicitminimization over edit alignments inside the com-putation of EDITDIST.

(1) Implicit Matching Objective:

minx,y

∑

u

∑

v

yuv · EDITDIST(u, v;x)

s.t. x is restricted-many-to-many

y is τ -one-to-one

In order to get a better handle on the shape of theobjective and to develop an efficient optimizationprocedure we decompose each edit distance compu-tation and re-formulate the optimization problem inSection 2.2.

2.1 Example

Figure 1 presents both an example matching prob-lem and a diagram of the variables and objective.Here, the source lexicon consists of the Englishwords (cat, bat, cart, rat, cab), andthe source alphabet consists of the characters (a,b, c, r, t). The target alphabet is (0, 1,2, 3). We have used digits as symbols in the targetalphabet to make it clear that we treat the alphabetsas disjoint. We have no prior knowledge about anycorrespondence between alphabets, or between lexi-cons.

The target lexicon consists of the words (23,1233, 120, 323, 023). The bipartite graphsshow a specific setting of the matching variables.The bold edges correspond to the xij and yuv thatare one. The matchings shown achieve an edit dis-tance of zero between all matched word pairs ex-cept for the pair (cat, 23). The best edit align-ment for this pair is also diagrammed. Here, ‘a’is aligned to ‘2’, ‘t’ is aligned to ‘3’, and ‘c’ isdeleted and therefore aligned to the null position ‘#’.Only the initial deletion has a non-zero cost sinceall other alignments correspond to substitutions be-tween characters that are matched in x.

2.2 Explicit Objective

Computing EDITDIST(u, v;x) requires running adynamic program because of the unknown editalignments; here we define those alignments z ex-plicitly, which makes the EDITDIST(u, v;x) easy towrite explicitly at the cost of more variables. How-ever, by writing the objective in an explicit form thatrefers to these edit variables, we are able to describea efficient block coordinate descent procedure thatcan be used for optimization.

EDITDIST(u, v;x) is computed by minimizingover the set of monotonic alignments between thecharacters of the source word u and the charactersof the target word v. Let un be the character at thenth position of the source word u, and similarly for

314

a

b

c

r

t

0

1

2

3

Alphabet Matching

Lexicon Matching

xij

cat

bat

rat

cart

cab

1233

120

323

023

23

yuv

Edit Distance

c

a

t

2

3

# # EditDist(u, v;x) =

minx,y

�

u

�

v

yuv · EditDist(u, v;x)

s.t. x is restricted-many-to-many

y is τ -one-to-one

Matching Problem

Insertion

� �

�Substitution

Deletion

� ·��

n

�

m

(1− xunvm)zuv,nm

+�

n

zuv,n# +�

m

zuv,#m

�s.t.minzuv

zuv is monotonic

zuv,nm

Figure 1: An example problem displaying source and target lexicons and alphabets, along with specific matchings.The variables involved in the optimization problem are diagrammed. x are the alphabet matching indicator variables,y are the lexicon matching indicator variables, and z are the edit alignment indicator variables. The index u refers toa word in the source lexicon, v refers to word in the target lexicon, i refers to a character in the source alphabet, andj refers to a character in the target alphabet. n and m refer to positions in source and target words respectively. Thematching objective function is also shown.

vm. Let zuv be the vector of alignment variablesfor the edit distance computation between sourceword u and target word v, where entry zuv,nmindicates whether the character at position n ofsource word u is aligned to the character at positionm of target word v. Additionally, define variableszuv,n# and zuv,#m denoting null alignments, whichwill be used to keep track of insertions and deletions.

EDITDIST(u, v;x) =

minzuv

ε ·(

SUB(zuv, x) + DEL(zuv) + INS(zuv))

s.t. zuv is monotonic

We define SUB(zuv, x) to be the number of sub-stitutions between characters that are not matchedin x, DEL(zuv) to be the number of deletions, andINS(zuv) to be the number of insertions.

SUB(zuv, x) =∑

n,m

(1− xunvm)zuv,nm

DEL(zuv) =∑

n

zuv,n#

INS(zuv) =∑

m

zuv,#m

Notice that the variable zuv,nm being turned on in-dicates the substitute operation, while a zuv,n# orzuv,#m being turned on indicates an insert or deleteoperation. These variables are digrammed in Fig-ure 1. The requirement that zuv be a monotonicalignment can be expressed using linear constraints,but in our optimization procedure (described in Sec-tion 3) these constraints need not be explicitly rep-resented.

Now we can substitute the explicit edit distanceequation into the implicit matching objective (1).

315

Noticing that the mins and sums commute, we arriveat the explicit form of the matching optimizationproblem.

(2) Explicit Matching Objective:

minx,y,z

[∑

u,v

yuv · ε ·(SUB(zuv, x) + DEL(zuv) + INS(zuv)

)]

s.t. x is restricted-many-to-many

y is τ -one-to-one

∀uv zuv is monotonic

The implicit and explicit optimizations are the same,apart from the fact that the explicit optimization nowexplicitly represents the edit alignment variables z.Let the explicit matching objective (2) be denotedas J(x, y, z). The relaxation of the explicit problemwith 0-1 constraints removed has integer solutions,2

however the objective J(x, y, z) is non-convex. Wethus turn to a block coordinate descent method in thenext section in order to find local optima.

3 Optimization Method

We now state a block coordinate descent procedureto find local optima of J(x, y, z) under the con-straints on x, y, and z. This procedure alternatesbetween updating y and z to their exact joint optimawhen x is held fixed, and updating x to its exact op-timum when y and z are held fixed.

The psuedocode for the procedure is given in Al-gorithm 1. Note that the function EDITDIST returnsboth the min edit distance euv and the argmin editalignments zuv. Also note that cij is as defined inSection 3.2.

3.1 Lexicon Matching Update

Let x, the alphabet matching variable, be fixed. Weconsider the problem of optimizing J(x, y, z) overthe lexicon matching variable y and and the editalignments z under the constraint that y is τ -one-to-one and each zuv is monotonic.

2This can be shown by observing that optimizing x when yand z are held fixed yields integer solutions (shown in Section3.2), and similarly for the optimization of y and z when x isfixed (shown in Section 3.1). Thus, every local optimum withrespect to these block coordinate updates has integer solutions.The global optimum must be one of these local optima.

Algorithm 1 Block Coordinate DescentRandomly initialize alphabet matching x.repeat

for all u, v do(euv, zuv)← EDITDIST(u, v;x)

end for[Hungarian]y ← argminy τ -one-to-one

[∑u,v yuveuv

]

[Solve LP]x← argmaxx restr.-many-to-many

[∑i,j xijcij

]

until convergence

Notice that y simply picks out which edit distanceproblems affect the objective. The zuv in each ofthese edit distance problems can be optimized in-dependently. zuv that do not have yuv active haveno effect on the objective, and zuv with yuv activecan be optimized using the standard edit distance dy-namic program. Thus, in a first step we compute theU · V edit distances euv and best monotonic align-ment variables zuv between all pairs of source andtarget words usingU ·V calls to the standard edit dis-tance dynamic program. Altogether, this takes timeO((∑

u lu) · (∑

v lv)).

Now, in a second step we compute the leastweighted τ -one-to-one matching y under theweights euv. This can be accomplished in timeO(max(U, V )3) using the Hungarian algorithm(Kuhn, 1955). These two steps produce y and z thatexactly achieve the optimum value of J(x, y, z) forthe given value of x.

3.2 Alphabet Matching Update

Let y and z, the lexicon matching variables and theedit alignments, be fixed. Now, we find the optimalalphabet matching variables x subject to the con-straint that x is restricted-many-to-many.

It makes sense that to optimize J(x, y, z) with re-spect to x we should prioritize mappings xij thatwould mitigate the largest substitution costs in theactive edit distance problems. Indeed, with a littlealgebra it can be shown that solving a maximumweighted matching problem with weights cij thatcount potential substitution costs gives the correctupdate for x. In particular, cij is the total cost ofsubstitution edits in the active edit alignment prob-

316

lems that would result if source character i were notmapped to target character j in the alphabet match-ing x. This can be written as:

cij =∑

u,v

∑

n,m s.t. un=i,vm=j

ε · yuv · zuv,nm

If x were constrained to be one-to-one, wecould again apply the Hungarian algorithm, thistime to find a maximum weighted matching underthe weights cij . Since we have instead allowedrestricted-many-to-many alphabet matchings weturn to linear programming for optimizing x. Wecan state the update problem as the following linearprogram (LP), which is guaranteed to have integersolutions:

minx

∑

ij

xijcij

s.t. ∀i∑

j

xij ≤ 2, ∀j∑

i

xij ≤ 2

∑

i

∑

j

xij = max(I, J)

In experiments we used the GNU Linear Program-ming Toolkit (GLPK) to solve the LP and updatethe alphabet matching x. This update yields match-ing variables x that achieve the optimum value ofJ(x, y, z) for fixed y and z.

3.3 Random RestartsIn practice we found that the block coordinate de-scent procedure can get stuck at poor local optima.To find better optima, we run the coordinate descentprocedure multiple times, initialized each time witha random alphabet matching. We choose the localoptimum with the best objective value across all ini-tializations. This approach yielded substantial im-provements in achieved objective value.

4 Experiments

We compare our system to three different state-of-the-art systems on three different data sets. We setup experiments that allow for as direct a comparisonas possible. In some cases it must be pointed outthat the past system’s goals are different from ourown, and we will be comparing in a different waythan the respective work was intended. The threesystems make use of additional, or slightly different,sources of information.

4.1 Phonetic Cognate Lexicons

The first data set we evaluate on consists of 583triples of phonetic transcriptions of cognates inSpanish, Portuguese, and Italian. The data set wasintroduced by Bouchard-Cote et al. (2007). For agiven pair of languages the task is to determine themapping between lexicons that correctly maps eachsource word to its cognate in the target lexicon. Werefer to this task and data set as ROMANCE.

Hall and Klein (2010) presented a state-of-the-art system for the task of cognate identification andevaluated on this data set. Their model explicitlyrepresents parameters for phonetic change betweenlanguages and their parents in a phylogenetic tree.They estimate parameters and infer the pairs of cog-nates present in all three languages jointly, while weconsider each pair of languages in turn.

Their model has similarities with our own in thatit learns correspondences between the alphabets ofpairs of languages. However, their correspondencesare probabilistic and implicit while ours are hard andexplicit. Their model also differs from our own ina key way. Notice that the phonetic alphabets forthe three languages are actually the same. Sincephonetic change occurs gradually across languagesa helpful prior on the correspondence is to favor theidentity. Their model makes use of such a prior.Our model, on the other hand, is unaware of anyprior correspondence between alphabets and doesnot make use of this additional information aboutphonetic change.

Hall and Klein (2010) also evaluate their modelon lexicons that do not have a perfect cognate map-ping. This scenario, where not every word in onelanguage has a cognate in another, is more realistic.They produced a data set with this property by prun-ing words from the ROMANCE data set until onlyabout 75% of the words in each source lexicon havecognates in each target lexicon. We refer to this taskand data set as PARTIALROMANCE.

4.2 Lexicons Extracted from Corpora

Next, we evaluate our model on a noisier data set.Here the lexicons in source and target languagesare extracted from corpora by taking the top 2,000words in each corpus. In particular, we used the En-glish and Spanish sides of the Europarl parallel cor-

317

pus (Koehn, 2005). To make this set up more real-istic (though fairly comparable), we insured that thecorpora were non-parallel by using the first 50K sen-tences on the English side and the second 50K sen-tences on the Spanish side. To generate a gold cog-nate matching we used the intersected HMM align-ment model of Liang et al. (2008) to align the fullparallel corpus. From this alignment we extracted atranslation lexicon by adding an entry for each wordpair with the property that the English word wasaligned to the Spanish in over 10% of the alignmentsinvolving the English word. To reduce this transla-tion lexicon down to a cognate matching we wentthrough the translation lexicon by hand and removedany pair of words that we judged to not be cognates.The resulting gold matching contains cognate map-pings in the English lexicon for 1,026 of the wordsin the Spanish lexicon. This means that only about50% of the words in English lexicon have cognatesin the Spanish lexicon. We evaluate on this data setby computing precision and recall for the number ofEnglish words that are mapped to a correct cognate.We refer to this task and data set as EUROPARL.

On this data set, we compare against the state-of-the-art orthographic system presented in Haghighiet al. (2008). Haghighi et al. (2008) presents sev-eral systems that are designed to extract transla-tion lexicons for non-parallel corpora by learninga correspondence between their monolingual lexi-cons. Since our system specializes in matching cog-nates and does not take into account additional infor-mation from corpus statistics, we compare againstthe version of their system that only takes into ac-count orthographic features and is thus is best suitedfor cognate detection. Their system requires a smallseed of correct cognate pairs. From this seed the sys-tem learns a projection using canonical correlationanalysis (CCA) into a canonical feature space thatallows feature vectors from source words and targetwords to be compared. Once in this canonical space,similarity metrics can be computed and words can bematched using a bipartite matching algorithm. Theprocess is iterative, adding cognate pairs to the seedlexicon gradually and each time re-computing a re-fined projection. Our system makes no use of a seedlexicon whatsoever.

Both our system and the system of Haghighi etal. (2008) must solve bipartite matching problems

between the two lexicons. For this data set, the lexi-cons are large enough that finding the exact solutioncan be slow. Thus, in all experiments on this dataset, we instead use a greedy competitive linking al-gorithm that runs in time O(U2V 2log(UV )).

Again, for this dataset it is reasonable to expectthat many characters will map to themselves in thebest alphabet matching. The alphabets are not iden-tical, but are far from disjoint. Neither our system,nor that of Haghighi et al. (2008) make use of thisexpectation. As far as both systems are concerned,the alphabets are disjoint.

4.3 DeciphermentFinally, we evaluate our model on a data set wherea main goal is to decipher an unknown correspon-dence between alphabets. We attempt to learn amapping from the alphabet of the ancient Semiticlanguage Ugaritic to the alphabet of Hebrew, andat the same time learn a matching between Hebrewwords in a Hebrew lexicon and their cognates in aUgaritic lexicon. This task is related to the task at-tempted by Snyder et al. (2010). The data set con-sists of a Ugaritic lexicon of 2,214 words, each ofwhich has a Hebrew cognate, the lexicon of their2,214 Hebrew cognates, and a gold cognate dictio-nary for evaluation. We refer to this task and data setas UGARITIC.

The non-parameteric Bayesian system of Snyderet al. (2010) assumes that the morphology of He-brew is known, making use of an inventory of suf-fixes, prefixes, and stems derived from the wordsin the Hebrew bible. It attempts to learn a corre-spondence between the morphology of Ugaritic andthat of Hebrew while reconstructing cognates forUgaritic words. This is a slightly different goal thanthat of our system, which learns a correspondencebetween lexicons. Snyder et al. (2010) run theirsystem on a set 7,386 Ugaritic words, the same setthat we extracted our 2,214 Ugaritic words with He-brew cognates from. We evaluate the accuracy of thelexicon matching produced by our system on these2,214 Ugaritic words, and so do they, measuring thenumber of correctly reconstructed cognates.

By restricting the source and target lexicons tosets of cognates we have made the task easier. Thiswas necessary, however, because the Ugaritic andHebrew corpora used by Snyder et al. (2010) are not

318

Model τ AccuracyHall and Klein (2010) – 90.3MATCHER 1.0 90.1

Table 1: Results on ROMANCE data set. Our system islabeled MATCHER. We compare against the phylogeneticcognate detection system of Hall and Klein (2010). Weshow the pairwise cognate accuracy across all pairs oflanguages from the following set: Spanish, Portuguese,and Italian.

comparable: only a small proportion of the wordsin the Ugaritic lexicon have cognates in the lexiconcomposed of the most frequent Hebrew words.

Here, the alphabets really are disjoint. The sym-bols in both languages look nothing alike. There isno obvious prior expectation about how the alpha-bets will be matched. We evaluate against a well-established correspondence between the alphabetsof Ugaritic and Hebrew. The Ugaritic alphabet con-tains 30 characters, the Hebrew alphabet contains 22characters, and the gold matching contains 33 en-tries. We evaluate the learned alphabet matching bycounting the number of recovered entries from thegold matching.

Due to the size of the source and target lexicons,we again use the greedy competitive linking algo-rithm in place of the exact Hungarian algorithm inexperiments on this data set.

5 Results

We present results on all four datasets ROMANCE,PARTIALROMANCE, EUROPARL, and UGARITIC.On the ROMANCE and PARTIALROMANCE data setswe compare against the numbers published by Halland Klein (2010). We ran an implementation ofthe orthographic system presented by Haghighi etal. (2008) on our EUROPARL data set. We com-pare against the numbers published by Snyder et al.(2010) on the UGARITIC data set. We refer to oursystem as MATCHER in result tables and discussion.

5.1 ROMANCE

The results of running our system, MATCHER, onthe ROMANCE data set are shown in Table 1. Werecover 88.9% of the correct cognate mappings onthe pair Spanish and Italian, 85.7% on Italian andPortuguese, and 95.6% on Spanish and Portuguese.

Model τ Precision Recall F1Hall and Klein (2010) – 66.9 82.0 73.6MATCHER 0.25 99.7 34.0 50.7

0.50 93.8 60.2 73.30.75 81.1 78.0 79.5

Table 2: Results on PARTIALROMANCE data set. Oursystem is labeled MATCHER. We compare against thephylogenetic cognate detection system of Hall and Klein(2010). We show the pairwise cognate precision, recall,and F1 across all pairs of languages from the followingset: Spanish, Portuguese, and Italian. Note that approx-imately 75% of the source words in each of the sourcelexicons have cognates in each of the target lexicons.

Our average accuracy across all pairs of languagesis 90.1%. The phylogenetic system of Hall andKlein (2010) achieves an average accuracy of 90.3%across all pairs of languages. Our system achievesaccuracy comparable to that of the phylogenetic sys-tem, despite the fact that the phylogenetic system issubstantially more complex and makes use of an in-formed prior on alphabet correspondences.

The alphabet matching learned by our system isinteresting to analyze. For the pairing of Span-ish and Portuguese it recovers phonetic correspon-dences that are well known. Our system learns thecorrect cognate pairing of Spanish /bino/ to Por-tuguese /vinu/. This pair exemplifies two com-mon phonetic correspondences for Spanish and Por-tuguese: the Spanish /o/ often transforms to a /u/ inPortuguese, and Spanish /b/ often transforms to /v/in Portuguese. Our system, which allows many-to-many alphabet correspondences, correctly identifiesthe mappings /o/→ /u/ and /b/→ /v/ as well as theidentity mappings /o/→ /o/ and /b/→ /b/ which arealso common.

5.2 PARTIALROMANCE

In Table 2 we present the results of running our sys-tem on the PARTIALROMANCE data set. In this dataset, only approximately 75% of the source words ineach of the source lexicons have cognates in each ofthe target lexicons. The parameter τ trades off pre-cision and recall. We show results for three differentsettings of τ : 0.25, 0.5, and 0.75.

Our system achieves an average precision acrosslanguage pairs of 99.7% at an average recall of34.0%. For the pairs Italian – Portuguese, and Span-

319

Model Seed τ Precision Recall F1Haghighi et al. (2008) 20 0.1 72.0 14.0 23.5

20 0.25 63.6 31.0 41.720 0.5 44.8 43.7 44.250 0.1 90.5 17.6 29.550 0.25 75.4 36.7 49.450 0.5 56.4 55.0 55.7

MATCHER 0 0.1 93.5 18.2 30.50 0.25 83.2 40.5 54.50 0.5 56.5 55.1 55.8

Table 3: Results on EUROPARL data set. Our systemis labeled MATCHER. We compare against the bilinguallexicon induction system of Haghighi et al. (2008). Weshow the cognate precision, recall, and F1 for the pair oflanguages English and Spanish using lexicons extractedfrom corpora. Note that approximately 50% of the wordsin the English lexicon have cognates in the Spanish lexi-con.

ish – Portuguese, our system achieves prefect preci-sion at recalls of 32.2% and 38.1% respectively. Thebest average F1 achieved by our system is 79.5%,which surpasses the average F1 of 73.6 achieved bythe phylogenetic system of Hall and Klein (2010).

The phylogenetic system observes the phyloge-netic tree of ancestry for the three languages andexplicitly models cognate evolution and survival ina ‘survival’ tree. One might expect the phyloge-netic system to achieve better results on this data setwhere part of the task is identifying which words donot have cognates. It is surprising that our modeldoes so well given its simplicity.

5.3 EUROPARL

Table 3 presents results for our system on the EU-ROPARL data set across three different settings of τ :0.1, 0.25, and 0.5. We compare against the ortho-graphic system presented by Haghighi et al. (2008),across the same three settings of τ , and with two dif-ferent sizes of seed lexicon: 20 and 50. In this dataset, only approximately 50% of the source wordshave cognates in the target lexicon.

Our system achieves a precision of 93.5% at a re-call of 18.2%, and a best F1 of 55.0%. Using a seedmatching of 50 word pairs, the orthographic sys-tem of Haghighi et al. (2008) achieves a best F1 of55.7%. Using a seed matching of 20 word pairs,it achieves a best F1 of 44.2%. Our system out-performs the orthographic system even though theorthographic system makes use of important addi-

Model τ Lexicon Acc. Alphabet Acc.Snyder et al. (2010) – 60.4* 29/33*MATCHER 1.0 90.4 28/33

Table 4: Results on UGARITIC data set. Our system is la-beled MATCHER. We compare against the deciphermentsystem of Snyder et al. (2010). *Note that results for thissystem are on a somewhat different task. In particular, theMATCHER system assumes the inventories of cognates inboth Hebrew and Ugaritic are known, while the systemof Snyder et al. (2010) reconstructs cognates assumingonly that the morphology of Hebrew is known, which is aharder task. We show cognate pair identification accuracyand alphabet matching accuracy for Ugaritic and Hebrew.

tional information: a seed matching of correct cog-nate pairs. The results show that as the size ofthis seed is decreased, the performance of the ortho-graphic system degrades.

5.4 UGARITIC

In Table 4 we present results on the UGARITIC dataset. We evaluate both accuracy of the lexicon match-ing learned by our system, and the accuracy of thealphabet matching. Our system achieves a lexiconaccuracy of 90.4% while correctly identifying 28 outthe 33 gold character mappings.

We also present the results for the deciphermentmodel of Snyder et al. (2010) in Table 4. Note thatwhile the evaluation data sets for our two modelsare the same, the tasks are very different. In par-ticular, our system assumes the inventories of cog-nates in both Hebrew and Ugaritic are known, whilethe system of Snyder et al. (2010) reconstructs cog-nates assuming only that the morphology of Hebrewis known, which is a harder task. Even so, the re-sults show that our system is effective at decipher-ment when semantically similar lexicons are avail-able.

6 Conclusion

We have presented a simple combinatorial modelthat simultaneously incorporates both a matchingbetween alphabets and a matching between lexicons.Our system is effective at both the tasks of cognateidentification and alphabet decipherment, requiringonly lists of words in both languages as input.

320

ReferencesA. Bouchard-Cote, P. Liang, T.L. Griffiths, and D. Klein.

2007. A probabilistic approach to diachronic phonol-ogy. In Proc. of EMNLP.

A. Bouchard-Cote, T.L. Griffiths, and D. Klein.2009. Improved reconstruction of protolanguage wordforms. In Proc. of NAACL.

A. Haghighi, P. Liang, T. Berg-Kirkpatrick, and D. Klein.2008. Learning bilingual lexicons from monolingualcorpora. Proceedings of ACL.

D. Hall and D. Klein. 2010. Finding cognate groupsusing phylogenies. In Proc. of ACL.

K. Knight and K. Yamada. 1999. A computational ap-proach to deciphering unknown scripts. In Proc. ofACL Workshop on Unsupervised Learning in NaturalLanguage Processing.

K. Knight, A. Nair, N. Rathod, and K. Yamada. 2006.Unsupervised analysis for decipherment problems. InProc. of COLING/ACL.

P. Koehn and K. Knight. 2002. Learning a translationlexicon from monolingual corpora. In Proc. of ACLworkshop on Unsupervised lexical acquisition.

P. Koehn. 2005. Europarl: A Parallel Corpus for Statis-tical Machine Translation. In Proc. of Machine Trans-lation Summit.

G. Kondrak. 2001. Identifying Cognates by Phonetic andSemantic Similarity. In NAACL.

H.W. Kuhn. 1955. The Hungarian method for the assign-ment problem. Naval research logistics quarterly.

P. Liang, D. Klein, and M.I. Jordan. 2008. Agreement-based learning. Proc. of NIPS.

S. Ravi and K. Knight. 2011. Bayesian inference for Zo-diac and other homophonic ciphers. In Proc. of ACL.

B. Snyder, R. Barzilay, and K. Knight. 2010. A statisti-cal model for lost language decipherment. In Proc. ofACL.

321