OutlineLinear Regression recapLocally WeightedRegressionProbabilistic interpretationLogistic RegressionG Newton's MethodRecape featuresforXM y Eh example XM ith training

exampleN E IRDth y C IR Xo Lhe prediction

n examples D features on EUtraining

holx ogx EX featuevedoegampleXc new example

T.co t.IE Chow y T holxZ

g

sit i n

X Xz Ks

et 4

Locally Weighted Regression

Parametric learning algorithmF it fixed set of parameters ou to date

Nonparametric learning algorithmparameters grows linearly meh size of data

j

T.ie

To evaluate h at certainLocally Weighted RegressionR Fcf 0 to Menominee fit 0 to minimise

Cgi 0 TEE wi Cg an 5

Return Ota wit expfllxitzf.LI

If XI small wi LIx H large wi r O

xx x

AnTi bandwidth

Probabilistic interpretation

why Least squaresAssume ya OTW't E

Aerror unmodded effects randomnoise

E N O or

plea expc cE

Assumption E are ILDryaiydomd.ee mean

This implies thatP y In o L

In art orexec y zq3e

parametrized by

y In io n N 0th or

P y In o

L O p Fl X o

II p y In o

FI oexrC CyHzotoI5

log likelihoodeco log Lco

log 7 exp

Eflag e log exp 7

n lag EE Gi x5202

MLE Maximum Likelihood EstimationChoose 0 to maximize LCO

Minimize Ey OTHDZ TCO

classificationMake assumption P ylX O

Compute 0 by MLE

gEEO I binary classification

o's

Logistic RegressionWant he E EO I

hoa gcoex 1He 04

GG 1I c e 2

at oa

IEEEsigmoidfn or logistic fr

P y L Ix O hocxT A

tumor sizeofmalignant

tumor

Ply o IX o I holx

P yl X o hfxY.CI h T

Lfo pty IX O

pcy Ix o

II how hocxingty

Ko log LCO

y log hock ta y toga h D

Batch Gradient Descent

0g i Oj Td eco Gradient Ascent

Og g azoDescent

Concave

tent

05 0 ta ECy hocxin x

Zoeco

J

Newton's MethodHave fFundo sa fCO 0

want maximize ecov want e O 0

Aderivative

Maryann derivative 0

a slope flow

c o 09

ON OH O

f OG f O height0 base

D f Of COCO

Oath Oct f ftf Oct

f o _e O

Oleh oh l'Cl Oct

Quadratic convergence0 I 0.01 0.0001

0 Vector ftp.oh Dxcd D2

Oath i get't H polein

Hessian ft Hy IfvectorHRD't

2920g