cs558 project local svm classification based on triangulation (on the plane) glenn fung

CS558 Project

Local SVM Classification based on triangulation

(on the plane)

Glenn Fung

Outline of Talk

Classification problem on the plane All of the recommended stages were applied:

Sampling Ordering:

Clustering Triangulation

Interpolation (Classification)SVM: Support vector Machines

Optimization: Number of training points increased Evaluation:

Checkerboard datasetSpiral dataset

Classification Problem in

Given m points in 2 dimensional space Represented by an m-by-2 matrix A Membership of each in class +1 or –1A i

R 2

SAMPLING:

1000 randomly sampled points

ORDERING:

Clustering A Fuzzy-logic based clustering algorithm was used. 32 cluster centers were obtained

-50 0 50 100 150 200 250-50

0

50

100

150

200

250

0 50 100 150 200

0

50

100

150

200

ORDERING:

Delaunay Triangulation Algorithms to triangulate and to get the Delaunay triangulation from HWKs 3 and 4 were used. Given a point,the random point approach is used to localize the triangle that contains it.

Interpolation:

SVM SVM : Support Vector Machine Classifiers A different nonlinear Classifier is used for each triangle

The triangle structure is efficiently used for both training and testing phases and for defining a “simple” and fast nonlinear classifier.

What is a Support Vector Machine?

An optimally defined surface Typically nonlinear in the input space Linear in a higher dimensional space Implicitly defined by a kernel function

What are Support Vector Machines Used For?

Classification Regression & Data Fitting Supervised & Unsupervised Learning

(Will concentrate on classification)

Support Vector MachinesMaximizing the Margin between Bounding

Planes

x0w= í +1

x0w= í à 1

A+

A-

jjwjj22

w

The Nonlinear Classifier

K (A;A0) : Rmân â Rnâm7à! Rmâm

K (x0;A0)Du = í

The nonlinear classifier:

Where K is a nonlinear kernel, e.g.: Gaussian (Radial Basis) Kernel :

"àökA iàA jk22; i; j = 1;. . .;mK (A;A0)ij =

The ij -entry of K (A;A0) represents the “similarity” of data points A i A jand

Reduced Support Vector Machine AlgorithmNonlinear Separating Surface: K (x0;Aö0)Döuö= í

(i) Choose a random subset matrix ofA 2 Rmân

entire data matrix A 2 Rmân

(ii) Solve the following problem by the Newtonmethod with corresponding D ú D :

2÷kp(eà D(K (A;A0)Döuöà eí );ë)k22+ 2

1kuö; í k22min(u; í ) 2 Rm+1

K (x0;Aö0)Döuö= í

(iii) The separating surface is defined by the optimal(u;í )solution in step (ii):

How to Choose in RSVM?A

A is a representative sample of the entire dataset Need not be a subset of A

A good selection of A may generate a classifier usingvery small m

Possible ways to chooseA :

Choose random rows from the entire datasetm A Choose such that the distance between its rows A

exceeds a certain tolerance Use k cluster centers of Aas AàA+ and

Obtained Bizarre “Checkerboard”

Optimization: More sampled pointsTraining parameters adjusted

Result: Improved Checkerboard

Nonlinear PSVM: Spiral Dataset94 Red Dots & 94 White Dots

Next:Bascom Hill

Some Questions

Would it work for B&W pictures (regression instead of classification?

Aplications?