CS558 Project

Local SVM Classification based on triangulation

(on the plane)

Glenn Fung

Outline of Talk

Classification problem on the plane All of the recommended stages were applied:

Sampling Ordering:

Clustering Triangulation

Interpolation (Classification)SVM: Support vector Machines

Optimization: Number of training points increased Evaluation:

Checkerboard datasetSpiral dataset

Classification Problem in

Given m points in 2 dimensional space Represented by an m-by-2 matrix A Membership of each in class +1 or –1A i

R 2

SAMPLING:

1000 randomly sampled points

ORDERING:

Clustering A Fuzzy-logic based clustering algorithm was used. 32 cluster centers were obtained

-50 0 50 100 150 200 250-50

0

50

100

150

200

250

0 50 100 150 200

0

50

100

150

200

ORDERING:

Delaunay Triangulation Algorithms to triangulate and to get the Delaunay triangulation from HWKs 3 and 4 were used. Given a point,the random point approach is used to localize the triangle that contains it.

Interpolation:

SVM SVM : Support Vector Machine Classifiers A different nonlinear Classifier is used for each triangle

The triangle structure is efficiently used for both training and testing phases and for defining a “simple” and fast nonlinear classifier.

What is a Support Vector Machine?

An optimally defined surface Typically nonlinear in the input space Linear in a higher dimensional space Implicitly defined by a kernel function

What are Support Vector Machines Used For?

Classification Regression & Data Fitting Supervised & Unsupervised Learning

(Will concentrate on classification)

Support Vector MachinesMaximizing the Margin between Bounding

Planes

x0w= í +1

x0w= í à 1

A+

A-

jjwjj22

w

The Nonlinear Classifier

K (A;A0) : Rmân â Rnâm7à! Rmâm

K (x0;A0)Du = í

The nonlinear classifier:

Where K is a nonlinear kernel, e.g.: Gaussian (Radial Basis) Kernel :

"àökA iàA jk22; i; j = 1;. . .;mK (A;A0)ij =

The ij -entry of K (A;A0) represents the “similarity” of data points A i A jand

Reduced Support Vector Machine AlgorithmNonlinear Separating Surface: K (x0;Aö0)Döuö= í

(i) Choose a random subset matrix ofA 2 Rmân

entire data matrix A 2 Rmân

(ii) Solve the following problem by the Newtonmethod with corresponding D ú D :

2÷kp(eà D(K (A;A0)Döuöà eí );ë)k22+ 2

1kuö; í k22min(u; í ) 2 Rm+1

K (x0;Aö0)Döuö= í

(iii) The separating surface is defined by the optimal(u;í )solution in step (ii):

How to Choose in RSVM?A

A is a representative sample of the entire dataset Need not be a subset of A

A good selection of A may generate a classifier usingvery small m

Possible ways to chooseA :

Choose random rows from the entire datasetm A Choose such that the distance between its rows A

exceeds a certain tolerance Use k cluster centers of Aas AàA+ and

Obtained Bizarre “Checkerboard”

Optimization: More sampled pointsTraining parameters adjusted

Result: Improved Checkerboard

Nonlinear PSVM: Spiral Dataset94 Red Dots & 94 White Dots

Next:Bascom Hill

Some Questions

Would it work for B&W pictures (regression instead of classification?

Aplications?