ELMo

ELMoTasks:Sen.ment

themoviewasgrossandoverwrought,butIlikedit

thismoviewasgreat!wouldwatchagain

thismoviewasnotreallyveryenjoyable

++—

‣Whyarecontext-dependentembeddingsusefulhere?

Petersetal.(2018)

ELMoTasks:NER

BarackObamawilltraveltoHangzhoutodayfortheG20meeBng.

PERSON LOC ORG

B-PER I-PER O O O B-LOC B-ORGO O O O O

‣Whyarecontext-sensi.veembeddingsusefulhere?

Petersetal.(2018)

ELMoTasks:SRL

‣ Iden.fyargumentsoftheverb

‣Whyarecontext-dependentembeddingsusefulhere?

Petersetal.(2018)

ELMo

‣ SST5=sen.ment,SQuAD=QA(cominglater)

Petersetal.(2018)

‣ Training:1BwordsoftexttakenfromMTdata(news,etc.)

‣ Largegainsacrossmanytasks

ELMo

Petersetal.(2018)

‣ Nearestneighborsofwordsincontext

ELMo

‣ Highlyflexible:canuseasanembeddinglayerfornearlyanytask

‣ Canbebigandslow:bigLSTMscansuckupGPUmemory

‣ Strictlybe]erthanword2vecifyoucanaffordit

‣ AllenNLP:callingELMoisjustoneline,plentyofmodelsfordifferenttasksbuiltontopofit

Petersetal.(2018)