(Congfu Xu) PhD, Associate Professor

Email: xucongfu@zju.edu.cnInstitute of Artificial Intelligence, College of Computer Science, Zhejiang University, Hangzhou 310027, P.R. China

September 11, 2003Oct. 16, 2006 SVM(Chapter8 SLT & SVM )

8.1.1 SLT & SVM

8.1

8.1.2 SLT & SVM

For God so loved the world that he gave his one and only Son, that whoever believes in him shall not perish but have eternal life. For God did not send his Son into the world to condemn the world, but to save the world through him. from JOHN 3:16-17 NIV

8.1.3 SLT&SVM

SLT&SVM

8.1.4 SLT&SVM

SVM

SVM

SLT & SVMSRM

8.2 SLT

, (),

8.3 F. Rosenblatt1958,1962

Novikoff(1962)

Tikhonov(1963), Ivanov(1962), Phillips(1962)

VanikChervonenkis(1968)VCVC

SLT()VapnikChervonenkis(1974)SRM

VapnikChervonenkis(1989),

90,,(Statistical Learning Theory,SLT)

8.4

8.4.1

G

LM

S

X

y

y

(G)xRn ,F(x)

(S)xy F(y|x)

(LM)f(x, a)aAA

8.4.2 f(x,)(),n , ,

(ill-posed problem)

,, ,,, (ERM)w

,

w,,w,

,

,,

,,,

8.5 SLT

SLT:

VC()SLTVC(Vapnik-Chervonenkis Dimension)

VC01h2hhVC

VC

VC,VC, ,VC,VCNVCn+1Sin(ax)VC

VC Open problem: ,VC

SLT,SLT,, : hVC,n

1:( ,VC

,VC,,

2, ,VC,,

SLT

,

,,VC;,,(Structural Risk Minimization)SRM

1

2SRM,

,,

8.6

8.6.1 1963Vapnik,,,(SV)

1971KimeldorfSV,

1990Grace,BoserVapnikSVM

1995Vapnik

8.6.2 SVM(0),SVM,,(margin)H1,H2

1

1

2

Minimize

Subject to

Lagrange

3Lagrange

x1 =(0, 0), y1 = +1x2 =(1, 0), y2 = +1x3 =(2, 0), y3 = -1x4 =(0, 2), y4 = -1Matlab1, 2, 3, 4wb

8.6.3 Vapnik

(xixj),,,,

SLT,Hibert-Schmidt,Mercer,

Mercer

,

8.6.4 SVM,

S

8.6.5 SVMlight - satyr.net2.private:/usr/local/binsvm_learn, svm_classifybsvm - satyr.net2.private:/usr/local/binsvm-train, svm-classify, svm-scalelibsvm - satyr.net2.private:/usr/local/binsvm-train, svm-predict, svm-scale, svm-toymySVMMATLAB svm toolbox

8.7

8.7.1 SVM

SVMNNSVMNNSVM NN NNNN

by R. Feynman from The Feynman Lectures on Physics, Addison-Wesley

SVMNN

8.7.2

SVM,

8.7.3 ,,,,SVM

11SMO(Sequential Minimal Optimization)

2

3SVM

2SVM,

3SVMSVMOne-class SVMSVMOne-against-the-restOne-against-oneMulti-class Objective FunctionsSVMDecision Directed Acyclic Graph, DDAGSVM Decision TreeSVM

SVMSVM

SVM

SVMBBSnews

A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery,1998,2(2)Vapnik V N. The Nature of Statistical Learning Theory, NY: Springer-Verlag, 1995..,2000

Introduction to Support Vector Machine.Vapnik V N. . ... . , 20001.. SVMHRRP. , 2005.

THANKS FOR YOUR PRESENCE!A righteous man may have many troubles, but the LORD delivers him from them all; he protects all his bones, not one of them will be broken. from Psalms 34:19-20 NIV