BPE_

aaabcaabdeab←

① A=aa B=ab

ABC Abde Bw

② vocab-- { A , B , b.ci die}

NBinfereniey-aogmaxyc-y%cF-PYJ-t.lk=

argmax ftp.cxily)

yc-fi-l-argma-FEWGY.ua?!?f-eacnwordYET

MLE-forNBLikelihoodfunetionLCO.tn)=É,cogpcxci ! ycii ;

= É log Pcy";E- I

0 C- 1122×1 V1 S.t. Ii Oi , y = I

✗ C- 112

Pcy" '; a)= {

✗ if yci> =/- l- d QW

.

= 1cg"'

1=1 ) a -1 Icy"1=0) ( 1- 2)

LCX ) is concave .

% = փ,log-11cg

"'t- 1) a -11cg

"'=o ) (ta)]

= % ¥2,

1cg" '

= 1) log ✗+ 1-cyciko) toga-2)

I ¥2,1 ( yci ) = c)£ - 1cg"' -07,1--2--0I = ÉÉ Icy" ' =L ) /FÉ Icycity +1cg" 1=0 )= # pros. ex

. / # total

NILEforLRLCW-7-i-Z.bgPCY" ' Indi ) , y)

= É y'" tog + ( 1-of")wgIt¥×

5=1

Is Llw) concave? log ¥éI ①Plot② f-

"

G) C- 0

No closed-form solution !

Rowinfor a penny , in for a poundin :[ ?

10in" - w,)

Bowlsentence)='+a¥, 1C"

in"=Wzo) 2 in

a z for

:O ii.f- ("

in"-

-win)

pound←! 77

WIN

Errordeeomposttiona.tlpredictors

Mt¥¥É÷¥Why small norms ?

overfit

¥×.✗ ✗tox

1 W . X - W . CXt 0×11= 1 w • o✗ I

⇐ 11W 11 . 110×11w

small