on utillizing lvq3-type algorithms to enhance prototype reduction schemes sang-woon kim and b. john...
TRANSCRIPT
![Page 1: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/1.jpg)
On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction
Schemes
Sang-Woon Kim and B. John Oommen*
Myongji University, Carleton University*
![Page 2: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/2.jpg)
Workshop on PRIS’ 2002
Outline of the Study
Introduction Overview of the Prototype Reduction
Schemes The Proposed Reduction Method Experiments & Discussions Conclusions
![Page 3: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/3.jpg)
Workshop on PRIS’ 2002
Introduction (1)
The Nearest Neighbor (NN) Classifier : A widely used classifier, which is simple and
yet one of the most efficient classification rules in practice.
However, its application often suffers from the computational complexity caused by the huge amount of information.
![Page 4: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/4.jpg)
Workshop on PRIS’ 2002
Introduction (2)
Solving strategies to the problem : Reducing the size of the design set without
sacrificing the performance. Accelerating the speed of computation by
eliminating the necessity of calculating many distances.
Increasing the accuracy of the classifiers designed with limited samples.
![Page 5: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/5.jpg)
Workshop on PRIS’ 2002
Motivation of the Study
In NN classifications, prototypes near the boundary play more important roles.
The prototypes need to be moved or adjusted towards the classification boundary.
The proposed approach is based on this philosophy, namely that of creating and adjusting.
![Page 6: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/6.jpg)
Workshop on PRIS’ 2002
Prototype Reduction Schemes- Conventional Approaches -
The Condensed Nearest Neighbor (CNN) : The RNN, SNN, ENN, mCNN rules
The Prototypes for Nearest Neighbor (PNN) classifiers
The Vector Quantization (VQ) & Bootstrap (BT) techniques
The Support Vector Machines (SVM)
![Page 7: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/7.jpg)
Workshop on PRIS’ 2002
A Graphical Example (PNN)
![Page 8: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/8.jpg)
Workshop on PRIS’ 2002
LVQ3 Algorithm
An improved LVQ algorithm :
Learning Parameters : Initial vectors Learning rates : Iteration numbers
Training Set = Placement + Optimizing:
![Page 9: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/9.jpg)
Workshop on PRIS’ 2002
Support Vector Machines (SVM)
The SVM has a capability of extracting vectors which support the boundary between two classes, and they can satisfactorily represent the global distribution structure.
![Page 10: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/10.jpg)
Workshop on PRIS’ 2002
Extension by Kernels
![Page 11: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/11.jpg)
Workshop on PRIS’ 2002
The Proposed Method
First, the CNN, PNN, VQ, SVM are employed to select initial prototype vectors.
Next, an LVQ3-type learning is performed to adjust the prototypes: Perform the LVQ3 with Tip to select w Perform the LVQ3 with Tip to select e Repeat the above steps to obtain the best w* and e*
Finally, determine the best prototypes by invoking the learning n times with Tip and Tio.
![Page 12: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/12.jpg)
Workshop on PRIS’ 2002
Experiments
The proposed method is tested with artificial and real benchmark design data sets, and compared with the conventional methods.
The one-against-all NN classifier is designed.
Benchmark data sets :
![Page 13: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/13.jpg)
Workshop on PRIS’ 2002
![Page 14: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/14.jpg)
Workshop on PRIS’ 2002
![Page 15: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/15.jpg)
Workshop on PRIS’ 2002
Experimental Results (3)
![Page 16: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/16.jpg)
Workshop on PRIS’ 2002
Experimental Results (4)
![Page 17: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/17.jpg)
Workshop on PRIS’ 2002
Data Compression Rates
![Page 18: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/18.jpg)
Workshop on PRIS’ 2002
Classification Error Rates (%)- Before Adjusting -
![Page 19: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/19.jpg)
Workshop on PRIS’ 2002
Classification Error Rates (%)- After Adjusting with LVQ3 -
![Page 20: On Utillizing LVQ3-Type Algorithms to Enhance Prototype Reduction Schemes Sang-Woon Kim and B. John Oommen* Myongji University, Carleton University*](https://reader036.vdocuments.net/reader036/viewer/2022082613/5697bfdf1a28abf838cb2b8f/html5/thumbnails/20.jpg)
Workshop on PRIS’ 2002
Conclusions
The method provides a principled way of choosing prototype vectors for designing NN classifiers.
The performance of a classifier trained with the method is better than that of the CNN, PNN, VQ, and SVM classifier.
The future work is to expand this study into large data set problems such as data mining and text categorization.