home g - machine learningcs229.stanford.edu/livenotes2020spring/cs229-livenotes... · 2020. 4....

TopicsPerceptronExponential FamilyGeneralized Linear Models

Softmax RegressionMulticlass Classification

Logistic Regressiona

It

g 2 z GG L 230O Z CO

hope 1home g E'aIt Ota

0 i O T x y how ngdifferent

y how0 algo got it rightL if wrong y L EEEy wrong y 0

EnzoQ

OEl

iO O

O

want 0 70 but 0Th LOO OtLn tLu5 n On th Nu

Exponential FamiliesPDF

plugin blog exp Thy acne

y data beg exp 2TTG

n natural parameterTcg sufficient statistic

y in classblog Base measure

acq log partition function

y scalar

q vector scalar

TG vector scalar I match

b g scalar

ExampleBernoulli Binary Dataprobability of event

p g lo 494t y

exp log 994 09 exp y tog lo ta g logo a

exp log 9 t logo 0

Tks Fat

bly ITcg gn log to q Csigmoid

acq logCto log l teenlog Iten

Gaussian w fixed variance or L

ply a En exp HIIe 5k expCpu y Zai

Tk Tk Tanbcg fan expC ETG yq µacne of IzProperties natural paramsMLE w r t 2 is concave

negative laglikelihood NLL is convex

EEg n a IqacnVarly y zigzag

GLMAssumptions Design Clonolces

c yl K O Exponential familyReal GaussianBinary Bernoullicount PoissonRT Gamma ExponentialDist peta Dirichlet BayesianMystats

Il if 0Th O EIRDk E Rd

cu Test tune output ECyIn o

how IE yl x o

hoax log ply OTH

Framxi Ip

Learning Update RulehoCa

Rest

Oji o t x y hoax ajplug in appropriate holu

Terminologyq natural parameter

µ EE y R g y canonical response for

2 g A canonical link f n

god znach3 paramatrizations

Modelparameters Natural param CanonicalParamBernoulli0 Y µ ot Gaussian

pi Entg X Poisson

Designgtlearning choice

Logistic Regressionholn Elgin o 9 n teen

Assumption'Legression

ammo A Fundo

Classification

xx xxxx x x0.5FEEI.eefzxxxxaxc xx

a

n1He 2

sigmoidLota of

o

Softmax RegressionMulticlass Classification

Cross Entropy minimization

U2 g 00OO

00Lie

K E IRDy E 0,13 one hot vector 0 0,40

d

Glass C IRD f oOz

class c o o D 3 kg in i e oooo

IO l

IEnno

OOI ht.ee

l ll l

Goal a

oin

normalize

PG

t 4

goal i mm distance between 2 dumba

man Cross Entropy pp I ply tog pacyYC 0,0 03log fly

00Thlog e

E eOEn

NgCEEQa o y

Fund Oo Oo 8D Gradient Descent