machine translation · 2019-05-13 · rule-based machine translation a. creating rules is long and...

Machine Translation

Kairit’s NLP courseMay 8, 2019Mark Fishel

2

Plan

3

● motivation● how it works

○ rule-based / statistical / neural MT

● NMT in practice○ text domain○ low-resource settings○ our work

Warren Weaver, 1947: “When I look at an article in Russian, I say ‘This is really written in English, but it has been coded in some strange symbols. I will now proceed to decode.’”

Very brief history of MT,beginning:

4

I know you are the dream girlof millions of your fans; en → de:

5

Very brief history of MT,not so long ago:

Very brief history of MT,now:

1. post-editing○ translate automatically, fix manually

(“post-edit”)○ works best with “boring” texts (e.g. legal, technical,

etc.)■ can be 20%–90% faster than manual translation

Use-cases:

7

What human translators think:

8

2. gisting○ approximate half-erroneous text in an understandable

language is better than perfect text in an incomprehensible language

○ Google Translate, Bing Translator, jms.

Use-cases:

9

Rule-based MT

10

1. Let’s create manually a word/phrase dictionary, reordering and inflecting rules, etc. E.g.:

Rule-based machine translation

11

Dictionary:words:● house → maja● car → auto● ...

phrases:● of course → muidugi

Rules:● in X(en) → Xseesütlev

(et)

○ nt. “in Tallinn” = “Tallinnas”

● …


12



(et)


● …

Dictionary:words:● house → maja● car → auto● a/the → ???

phrases:● of course → muidugi


13



(et)


● …

Dictionary:words:● house → maja● car → auto● a/the → ???● park → park/parkima?phrases:● of course → muidugi


14


Dictionary:words:● house → maja● car → auto● a/the → ???● park → park/parkima?● cool → jahe/lahe?phrases:


(et)


● …


15




(et)


● in three hours → kolme tunniga??


16




Reeglid:● in X(en) → Xseesütlev

(et)

○ nt. “in the box” = “karbis”

● in three hours → kolme tunniga??


a. creating rules is long and boringb. the rules have to be extremely

complicated if we want the translation system to handle a variety of texts

c. new language pair = start from scratch (practically no reusability)

17

ET: ?

LV: Vai tev ir labāka ideja?

Statistical Translation

ET: ?


ET: Mul on parem idee.

LV: Man ir labāka ideja.

ET: Kas sul on tõlketekste?

LV: Vai tev ir pārtulkotu tekstu?

ET: Sul peaks olema palju tõlketekste.

LV: Tev jābūt daudz pārtulkotu tekstu.


ET: Kas









ET: Kas sul









ET: Kas sul on









ET: Kas sul on parem idee









ET: Kas sul on parem idee?









Actual translation:● segment input● translate pieces● reorder● put in context● …

ET: Kas sul on parem idee?









Neural MT, autoregression

34

Neural MT, encoder-decoder

35

Subwords

36

Neural MT, attention mechanism

37

38

Src: Google AI Blog

https://ai.googleblog.com/2016/09/a-neural-network-for-machine.html

39

Src: lielakeda.lv

http://lielakeda.lv/other/NLP/alignments/

40

Src: lielakeda.lv

http://lielakeda.lv/other/NLP/alignments/

Neural MT, self-attention(Transformer)

41

42

Src: Google AI Blog

https://3.bp.blogspot.com/-aZ3zvPiCoXM/WaiKQO7KRnI/AAAAAAAAB_8/7a1CYjp40nUg4lKpW7covGZJQAySxlg8QCLcBGAs/s640/transform20fps.gif

43

Src: Illustrated Transformer

https://jalammar.github.io/illustrated-transformer/

Text domain

44

Back-translation

45

Unsupervised translation

46

Politeness

47

Multilinguality

48

“Monolingual translation”

49

Practice: QE, PEControllability and usability

Technical requirements

50

Summary:● RBMT vs SMT vs NMT● subwords + self-attention● low-res tricks● “bonuses”● practice ≠ theory

51

52

That’s it!

machine translation · 2019-05-13 · rule-based machine translation a. creating rules is long and...

Documents