What’s in a translation rule?

Paper by Galley, Hopkins, Knight & MarcuPresentation By: Behrang Mohit

Problem

• The problem of syntax in SMT

• Yamada & Knight (2001) had transformations like child-reorderings– Addressed the SOV vs. VSO

orders– Does not address all the

syntactic movements• English Adverbs: The

government simply says …• ne … pas

Three Alternative

• Abandon Syntax– Evidence: Kohn et. Al. 2003

• Abandon English Syntax– Learn grammar from parallel corpus

• Wu (1997): ITG: binary branching rules

• Use English syntax to learn transformation rules from parallel corpus and larger fragments of the English tree structure.

A Theory of Word Alignment

• Generative process– Source string to target tree

(symbol tree)– Derivation Step: replaces

a substring of the source string with a subtree of the target tree.

– Derivation: Sequence DS.

Three Alternative Derivations

Replacing and Creating

• Each source element is replaced at exactly one step of the derivation

• Each node target tree is created at exactly one step of derivation

• Replaced(s,D)– Replaced (va, D) = 2

• Created (t,D)– Created (AUX, D) = 3

Word Alignment• Alignment: A relation between leaves of

the target tree (t) and elements of the source string (s):– iff Replaced(s,D) = created(t,D)

“Good Derivations”

• Input: source string, target tree, word alignments

• A set that induces a super alignment set for the given word alignment.– 1 & 3

),( TSA

Derivations Rules

• ne VB pas

• NP VP

• Task: given T, S and A, learn

in any

• What about inferring complex rules?

),( TSA

),( TSD A

Alignment Graph

• Target Tree, augmented with the source strings

• Span of nodes• Frontier set• Frontier graph

fragment: root and all sinks are in the frontier set– Spans of the sinks form

a partition of the span of the root.

Alignment Graph

• Target Tree, augmented with the source strings

• Span of nodes• Frontier set• Frontier graph

fragment: root and all sinks are in the frontier set– Spans of the sinks form

a partition of the span of the root.

Alignment Graph

• Target Tree, augmented with the source strings

• Span of nodes• Frontier set• Frontier graph

fragment: root and all sinks are in the frontier set– Spans of the sinks form

a partition of the span of the root.

Transformation process

• Input: Place the sinks in the order defined by the partition.

• Output: Replace sink nodes with variable corresponding to the position in input, then take the tree part of the fragment.

• These rules are in

),( TSA

Rule Extraction Algorithm

• Search the space of graph fragments for frontier graph fragments (FGF).– Search of all fragments is

exponential

• The frontier set (FS) can be found linearly

• For each node (n) in the FS, there is a unique minimal FGF, rooted at n.

Rule Extraction Algorithm

• Search the space of graph fragments for frontier graph fragments (FGF).– Search of all fragments is

exponential

• The frontier set (FS) can be found linearly

• For each node (n) in the FS, there is a unique minimal FGF, rooted at n.

Expanding from minimal fragments

• Compose new frontier graph fragment by merging to of the minimal fragments

Experiments

• French-English (Hansard)– Human alignments– GIZA++ alignments

• Chinese-English (FBIS)– GIZA++ alignments (trained on huge corpus)

• Issue: Coverage of the extracted rules.– Percentage of the parse trees in the corpus

that can be transformed by the translation rules.

Coverage of the model

Coverage of the model

• Number of expansions– Single: Yamada & Knight 2001– 17 to 43 expansions for full coverage – Alignment– Lang Diffs

Another example of multi-level reordering

Conclusion

• Previous works: child-node reordering

• This model looks at larger tree fragments

• Translation rules are both syntactically and lexically motivated.

• The rule extraction algorithm can deal with alignment and systematic parsing errors.

• Next step: defining probability distribution over the rules Decoding

Explanatory power of the model