iaetsd an enhanced feature selection for

Abstract Irrelevant features, at the side of redundant features, strictly

have an effect on the correctness of the knowledge machines. Thus, feature set selection have to be compelled to be able to determine and take away the maximum as much of the unrelated and redundant knowledge as feasible. With this intention choosing a subset of features with relation to the target notions, feature set selection is an efficient alternative way for reducing spatial property or dimensionality (ex: subset), removing unrelated data (Ex: irrelevant data), increasing learning accuracy, and generating Qualitative result. Feature selection involves classifying a set of the foremost relevant features that generates appropriate outcome as the original entire set of features. Several feature set selection techniques are planned and studied for machine learning applications. By this criterion, an Enhanced fast clustering-based feature selection algorithm, EFAST, is employed during this paper.

The EFAST algorithmic rule works in 2 steps. In the starting step, features are classified into clusters by exploitation graph-theoretic clustering approaches. In the second step, the foremost relevant representative feature that is powerfully associated with target categories is chosen from every cluster to form a set of features.

Features in dissimilar clusters are comparatively autonomous and the clustering-based strategy of EFAST includes a high chance of generating a set of valuable and autonomous features. Keywords-EFAST Algorithmic rule, Correlations, Feature set Selection, and Graph based clustering 1 INTRODUCTION

The use of feature selection can develop accurateness, relevancy, applicability and be aware of a learning method. For this reason, several ways of automatic feature selection are developed. Some of these ways are based on the search of the features that enables the data set to be measured consistent. In an exceedingly search problem we usually tend to evaluate the search states, in the case of feature selection we measure the promising feature sets. Feature set selection is a very important subject when preparing classifiers in Machine Learning (ML) issues. Selection of Feature set is an efficient system for dimensionality reduction, elimination of inappropriate knowledge, rising learning accurateness, and improving result unambiguousness. Based on the minimum spanning tree methodology, we propose an EFAST algorithmic rule. The

algorithmic rule is a 2 step method, in that features are separated into clusters by way of using graph theoretic clustering means. Within the succeeding step, the frequently used representative feature that is robustly related to target categories is specific from every cluster to structure the ultimate subset of features. Features in distorted clusters are comparatively autonomous. The clustering-based theme of EFAST includes a high risk of designing a set of constructive and autonomous features. In our planned EFAST algorithmic rule, it needs the building of the minimum spanning tree (MST) from a subjective comprehensive graph. The separation of the MST into a forest by means of each tree signifying a cluster and the collection of representative features from the clusters. The planned feature set selection algorithmic rule EFAST was tested and the investigational results demonstrate that, evaluated with different varied forms of feature set selection algorithms, the projected algorithmic rule not solely decrease the amount of features, but also advances the performances of the famed varied forms of classifiers.

The results, on publically obtainable real-world high dimensional image, microarray, and text knowledge, established that EFAST not only produces smaller sets of features however improves the performances of the classifier. In our study, we tend to apply graph theoretic clustering schemes to features. In exacting, we tend to accept the MST based clustering algorithms, since they do not imagine that knowledge points are classified around centres or separated by a normal geometric curve and are widely used in training. Based on the MST method, we tend to suggest an Enhanced Fast clustering-bAsed feature Selection algoriThm (EFAST).

Good feature set is one that contains features extremely correlative with the target, so far uncorrelated with one another. In the above planned

An Enhanced Feature Selection for High-Dimensional Knowledge

L.Anantha Naga Prasad 1 K.Muralidhar 2

1 M.Tech, Computer Science and Engg, Anantha Lakshmi Institute of Technology&Sciences,JNTUA, Andhra Pradesh, India 2 Assistant Prof, Department of CSE, Anantha Lakshmi Institute of Technology&Sciences, JNTUA, Andhra Pradesh, India

1 mail id: [email protected] 2 mail id: [email protected]

Proceedings of International Conference on Developments in Engineering Research

ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in

INTERNATIONAL ASSOCIATION OF ENGINEERING & TECHNOLOGY FOR SKILL DEVELOPMENT22

terms this system is an efficient fast filter method which may categorize relevant features as redundancy among relevant features without pair wise correlation study, and repeatedly chooses features which exploit their mutual information with the category to expect provisionally to the reply of any feature formerly elected. In contrast to from these algorithms, our projected EFAST algorithmic rule utilizes clustering based methodology to select features. 2 RELATED WORK 2.1 EXISTING SYSTEM

In the past approach there are many algorithms that illustrate a way to maintain the knowledge into the database and how to retrieve it quicker, however the difficulty here is no one cares about the database maintenance with ease manner and safe methodology. A Distortion algorithmic rule, that creates a personal space for every and each word from the already elected transactional database, those are put together named as dataset, which is able to be acceptable for a collection of exacting words, however it'll be problematic for the set of records. An inference algorithmic rule build propagation to the higher than downside, and cut back the issues occurred within the existing distortion algorithmic rule, however here conjointly having the matter known as knowledge overflow, once the user get confused then they will never get the knowledge back. The embedded ways incorporate feature choice as a locality of the training method and are sometimes specific to given learning algorithms and as a result could also be improved than the opposite 3 teams. Typical machine learning algorithms like decision trees or artificial neural networks are samples of embedded ways. The wrapper techniques use the analytical accuracy of a planned algorithmic rule to decide the goodness of the actual subsets, the accurateness of the learning algorithms is usually high. But the simplification of the chosen feature is restricted and the process problem is high. The filter ways are autonomous of learning algorithms, with fine generality. Their process complexness is low, however the accuracy of the learning algorithms is not assured. The hybrid techniques are a mix of filter and wrapper ways by employing a filter methodology to diminish search space which will be measured by the succeeding wrapper. They primarily target on grouping filter and wrapper ways to attain the most effective potential performance with a selected learning algorithmic rule with similar time complexness of the filter ways.

Hierarchical clustering has been implemented in word choice within the context of text classification. And it is noise-tolerant and strong to feature communications, additionally as being relevant for binary or continuous knowledge solely. However, it does not discriminate between redundant

features and low numbers of training instances of the algorithmic rule.

Relief-F could be a feature choice strategy that chooses cases randomly, and altered the weights of the feature importance based on the closest neighbor. By its qualities, Relief-F is one in all the foremost prosperous methods in feature choices. 2.2 Disadvantages of Existing System The simplification of the chosen features is

restricted and hence the complexness is large. Accurateness is not guaranteed. Ineffective at deleting redundant features Performance associated problems Security problems So the attention of our new system is to boost the outturn for any basis to eliminate the knowledge security lacks there in and build a more recent system outstanding handler for handling data in an economical manner. 2.3 Proposed System

In this proposed system, The Enhanced fast clustering-based feature selection algorithmic rule (EFAST) works in 2 steps. In the first step, features are classified into clusters by exploitation graph-theoretic clustering ways. In the second step, the foremost relevant representative feature that is powerfully associated with target categories is chosen from every cluster to form a set of features.

Features in dissimilar clusters are comparatively autonomous and therefore the clustering-based strategy of EFAST includes a high probability of generating a set of valuable and autonomous features.

Inside this paper we tend to generate correlations for high dimensional knowledge supported EFAST algorithmic rule in four steps.

1. Removal of unrelated features: If we choose a Dataset 'D' with m features F= {F1, F2... Fn} and class C, mechanically features are obtainable with target relevant feature. The simplification of the chosen features is restricted and the process complexness is huge. If Fi is relevancy to the target C if there exists some si, fi, and c specified for probability p(Si=si , Fi=fi) >0, p(C=c | Si =si, Fi=fi)≠p(C=c | Si = si) otherwise feature Fi is an unrelated feature. 2. T-Relevance, F-correlation calculation: If (Fi ∈ F) then target notion C is treated as T-Relevance. If (Fi, Fj ∈ F ^ i≠j) is named F-correlation. T-Relevance among a feature and the target notion C, the correlation F-Correlation between a combine of features, the feature redundancy F-Redundancy and therefore the representative feature R-Feature of a feature cluster will be outlined.


ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in


3. MST construction by fuzzy logic : We adopt the minimum-spanning tree (MST) clustering way in competence view. during this method we calculate a neighborhood graph of occurrences, then take away any edge in the graph that is a lot shorter/longer (by fuzzy logic) than its neighbors.

4. Relevant feature calculation: When removing all the unnecessary spare edges, a Forest will be obtained. In that every tree represents a cluster. Finally it contains feature set and then calculates the accurate/relevant feature. 2.4 Problem Definition Many algorithms that illustrate a way to maintain the knowledge into the database and the way to retrieve it quicker, however the matter is no one cares about the database maintenance with ease manner and safe methodology. The systems like Distortion and congestion algorithmic rule, which makes an individual space for every and each word from the already elected transactional database, those are put together known as dataset, which is able to be appropriate for a collection of exacting words, however it will be troublesome for the cluster of records, once the user get confused then they will never get the data back. The wrapper ways use the analytical accuracy of a predetermined learning algorithmic rule to verify the goodness of the chosen subsets, the correctness of the learning algorithms is usually high. Their computational difficulty is low, however the correctness of the learning algorithms is not assured. An EFAST algorithmic rule analysis has targeted on sorting out relevant features. A famed example is Relief which weighs every feature in line with its ability to discriminate instances below completely different targets supported distance-based criteria task. Though, Relief is unproductive at removing redundant features as too analytical however extremely correlative features are seemingly each to be extremely weighted. Relief-F expands Relief, permitting this methodology to work with strident and incomplete data sets and to cope with multiclass issues, however still unable to acknowledge redundant features. 2.5 Literature Review

Literature survey is the most vital step in code development procedure. Before finding the tool it is essential to decide the time facet, financial system and company strength. Once these things are satisfied, then the subsequent step is to decide which operating system and language can be used for developing the tool.

Once the programmers commence building the tool the programmers require set of external support. This maintenance will be obtained from programmers, from books or from websites. Before building the

system the above thought are taken into account for developing the projected system. The core part of the developing projected sector considers and totally survey all the necessary needs for creating the project. For every project Literature survey is the most vital sector in code development procedure. Preceding to developing the tools and the associated planning it is necessary to decide and survey the time facet, resource constraint, man power, financial system, and company strength. Once these items are fulfilled and totally reviewed, then the subsequent step is to make a decision concerning the code specifications within the relevant system such as what kind of operating system the project would require, and what are all the essential code are required to proceed with the subsequent step such as developing the tools and the associated operations. 3. FUZZY BASED FEATURE SET SELECTION ALGORITHMS EFAST Algorithmic rule is a classic algorithm for frequent item set mining and association rule learning over transactional databases. This EFAST algorithmic rule internally contains an algorithmic rule known as Apriori, which progresss by discovering the frequent individual things in the database and enlarging them to larger and well-built item sets as long as those item sets seem sufficiently frequently in the database. The common item sets confirmed by Apriori will be accustomed to determine association rules that highlight general trends in the database. 3.1 Feature set selection algorithmic rule In machine learning, statistics feature selection called as variable selection or attribute selection or variable set selection. Is the procedure of choosing a set of relevant features to be used in model creation. The central hypothesis employing a feature selection technique is that the data contains several redundant or irrelevant features. Redundant features are those which supply no more information than the presently specific features, and irrelevant Features offer no helpful information in any background. Feature selection ways are a set of the additional general field of feature extraction. Feature extraction creates new features from functions of the novel features, whereas feature selection returns a set of the features. Feature selection techniques are usually employed in domains wherever there are several features and relatively few samples (or knowledge points). Feature selection techniques supply 3 main profits once constructing analytical models: • Improved model interpretability, • Shorter training times, • Improved generalisation by reducing over fitting. Feature selection is also useful as a part of the data analysis method, as shows which features are vital for prediction, and the way these features are connected.


ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in


3.2 Definitions In learning Machines [11], [15] Suppose 퐹to be the complete set of features, 푖 ∈퐹 be a feature,푆푖 =퐹−{퐹푖} and 푆′푖 ⊆푆푖. Let 푠’푖 be a value-assignment of all features in 푆′푖, 푓푖 a value-assignment of feature퐹푖, and 푐a value-assignment of the target concept 퐶. The definition will be formalized as follows. Definition: (Relevant feature) 퐹푖 has relevancy to the target concept 퐶 if and only if there exists some 푠′푖, 푓푖 and 푐, such that, for probability (푆′푖 =푠′푖,퐹푖 =푓푖)>0,푝(퐶=푐∣ 푆′푖 =푠′푖,퐹푖 =푓푖) ≠푝(퐶=푐∣ 푆′푖 =푠푖). or else, feature퐹푖 is an irrelevant feature. There are 2 sorts of relevant features due to different 푆′푖: (i) if 푆′푖 =푆푖, we will recognize that 퐹푖 is directly relevant to the target concept (ii) if 푆′푖 ⊊푆푖, we could get that 푝(퐶∣푆푖,퐹푖)=푝(퐶∣푆푖). Definition: (Markov blanket) The definitions of Markov blanket and redundant feature are introduced as follows, correspondingly. let 푀푖 ⊂퐹(퐹푖 ∕∈푀푖 ),푀푖 is assumed to be a Markov blanket for 퐹푖 if and only if 푝(퐹−푀푖−{퐹푖},퐶∣ 퐹푖,푀푖)=푝(퐹−푀푖−{퐹푖},퐶∣ 푀푖). Definition: (Redundant feature) let 푆 be a collection of features, a feature in 푆 is redundant if and only if it has a Markov Blanket within푆. Relevant features have strong correlation with target concept so are always essential for a finest set, while redundant features are not since their values are fully correlative with one another. Definition: symmetric uncertainty (푆푈) is derived by normalizing the entropies of feature values or target categories. SU is the evaluation of correlation between either two features or a feature and the target concept. In existing system we tend to calculate SU as 푆U(푋,푌)=2×퐺푎푖푛(푋∣푌) / 퐻(푋)+퐻(푌) Where 퐻(푋)=− Σ 푥∈푋 푝(푥)log2푝(푥) 퐺푎푖n(푋∣푌)=퐻(푋)−퐻(푋∣푌) 퐺푎푖n(Y∣X)=퐻(푌)−퐻(푌∣푋) 퐻(푋∣푌)=− Σ 푦∈푌 푝(푦) Σ 푥∈푋 푝(푥∣푦)log2푝(푥∣푦). In our paper we are proposing SU as 푆U(푋,푌)=2×퐺푎푖푛Ratio(푋∣푌) / 퐻(푋)+퐻(푌) Where 퐻(푋)=− Σ 푥∈푋 푝(푥)log2푝(푥)

Intrinsic Information is the entropy of distribution of instances into branches. 퐻(푋∣푌)=− Σ 푦∈푌 푝(푦) Σ 푥∈푋 푝(푥∣푦)log2푝(푥∣푦). Definition: (T-Relevance) The relevance between the feature퐹푖 ∈퐹and the target concept 퐶 is consigned as The T-Relevance of 퐹푖 and 퐶, and denoted by 푆U(퐹푖,퐶). If 푆U (퐹푖,퐶) is > a predetermined threshold 휃, we say that 퐹푖 is a strong T-Relevance feature.

Definition: (F-Correlation) The correlation between any pair of features퐹푖 and퐹푗 (퐹푖,푗 ∈퐹∧푖≠푗) is named as the F-Correlation of 퐹푖 and퐹푗, and denoted by 푆푈(퐹푖,퐹푗). Definition: (F-Redundancy) Let 푆 = {퐹1, 퐹2... 퐹푖... 퐹푘<∣퐹∣} be a cluster of features. if ∃퐹푗 ∃ 푆, 푆푈(퐹푗, 퐶) ≥ 푆푈(퐹푖, 퐶) ∧ 푆푈(퐹푖, 퐹푗) > 푆푈(퐹푖, 퐶) is often corrected for every 퐹푖 ∈ 푆 (푖 ≠ 푗), then 퐹푖 are Redundant features with reverence to the given 퐹푗. Definition: (R-Feature) A feature 퐹푖 ∈ 푆 ={퐹1, 퐹2, ..., 퐹푘} (푘 < |퐹|) is a representative feature of the cluster 푆 ( i.e. 퐹푖 is a R-Feature ) if and only if, 퐹푖 = argmax퐹푗∈푆 푆푈(퐹푗, 퐶). By this we will say a) Irrelevant features have no/weak correlation with Target concept b) Redundant features are assembled in a cluster and a representative feature will be taken out of the Cluster. 3.3 EFAST Algorithm by Gain Ratio Algorithm: EFAST Inputs: D(f1,f2,…fm, C) – given data set Output: S – Feature Selection // irrelevant feature removal For i=1 to m do T-Relevance=SU(Fi, C) //SU is calculated based on Gain Ratio If T-Relevance >Ɵ then // here Ɵ is the Threshold value S= S U {Fi } // MST Construction G= NULL For each pair of features { fi, fj } ⊂S do F-Correlation = SU(fi, fj) Add fi and/or fj to G with the F-Correlation as its weight. Minspantree = Prim(G); //using Prim algorithm to construct MST // clustering and Enhanced Feature Selection Forest = Minspantree For each edge Eij ∈Forest do If SU(fi, fj) < SU(fi ,C) ^ SU(fi, fj) < SU(fi ,C) then Forest= Forest - Eij S= 휙 For each tree Ti ∈ Forest do Fj

R = argmax Fk Ti SU(Fk, C) S= S U { Fj

R } Return s


ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in


3.4 About Gain Ratio Information is measured in terms of bits

Given a probability distribution, the info requisite to predict an event is the distribution’s entropy

Entropy offers the knowledge needed in bits (this will involve fractions of bits!)

We can calculate entropy as nnn ppppppppp logloglog),,,entropy( 221121

Gain ratio: a modification of the information gain that reduces its bias on high-branch attributes Gain ratio ought to be

Large when knowledge is equally spread

Small when all knowledge belong to 1 branch

Gain ratio takes range and size of branches under consideration when selecting an attribute It corrects the information gain by

taking the intrinsic information of a split under consideration (i.e. what quantity data can we have to be compelled to tell that branch and instance belongs to)

We can calculate Gain Ratio as Intrinsic information: entropy of distribution

of instances into branches Gain ratio (Quinlan’86) normalizes info gain

by:

Ex: gainratio (“Attribute”) =gain (“Attribute”) Intrinsicinfo (“Attribute”) Ex: gainratio (“Id”) =0.94/3.8=0.24 4 Frame Work

Our projected feature set selection framework structure involves irrelevant feature removal and redundant feature elimination by using fuzzy logic. It offers internal logical schema to form clusters with the assistance of EFAST Algorithmic rule. Smart feature subsets contain features extremely correlative with the class, yet unrelated with each other. [20] Frame work Analysis, it involves (i) building the minimum spanning tree (MST) from a weighted complete graph (ii) the partitioning of the MST into a forest with every tree representing a cluster and (iii) the selection of representative features from the clusters.

Fig. 1: Framework of the Fuzzy Based The outline of Frame work is characterized as

Fig. 2: EFAST Architecture

Data Sets

We Use Gain Ratio for Effectiveness

Irrelevant Feature Removal

MST Construction

Clustering of MST

Enhanced Feature Selection


ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in


5 Algorithmic Rule Analyses The planned EFAST algorithmic rule rationally consists of 3 steps: (i) Removing irrelevant features, (ii) Constructing a MST from relative ones, (iii) Separating the MST and selecting Representative features.

For a data set 퐷 with 푚 features 퐹 = {퐹1, 퐹2... 퐹푚} and class 퐶, we tend to compute the T-Relevance 푆푈(퐹푖, 퐶) value for every feature 퐹푖 (1 ≤ 푖 ≤ 푚). In the first step. The features whose 푆푈(퐹푖, 퐶) values are larger than a predefined threshold 휃 contain the target-relevant feature subset 퐹′ = {퐹′1, 퐹′2... 퐹′푘} (푘 ≤ 푚). In the second step, we 1st calculate the F-Correlation푆푈(퐹′푖, 퐹′푗) value for every pair of features 퐹′푖 and 퐹′푗(퐹′푖, 퐹′푗∈ 퐹′∧ 푖 ≠ 푗). Then, viewing features 퐹′푖 and 퐹′푗 as vertices and 푆푈(퐹′푖, 퐹′푗) (푖 ≠ 푗) as the weight of the edge between vertices 퐹′푖 and 퐹′푗 , a weighted complete graph 퐺 = (푉,퐸) is built wherever 푉 = {퐹′푖 | 퐹′i∈퐹′ ∧ 푖 ∈ [1, 푘]} and 퐸 = {(퐹′푖, 퐹′푗) ∈ (퐹′푖, 퐹′푗 ∈ 퐹′ ∧ 푖, 푗 ∈[1, 푘] ∧ 푖 ≠ 푗}. In the third step, we 1st eliminate the edges 퐸 = {(퐹′푖, 퐹′푗) ∈ (퐹′푖, 퐹′푗∈ 퐹′ ∧ 푖, 푗 ∈ [1, 푘] ∧ 푖 ≠ 푗}, whose weights are smaller than both of the T-Relevance 푆푈(퐹′푖, 퐶) and 푆푈(퐹′푗, 퐶), from the MST. Each removal ends up in 2 disconnected trees 푇1 and 푇2. The complete graph 퐺 reflects the correlations among all the target-relevant features. If graph 퐺 has 푘 vertices and 푘(푘−1)/2 edges. For this we build a MST, which connects all vertices such that the sum of the weights of the edges is the minimum, using the famous Prim algorithmic rule [14].The weight of edge (퐹′푖, 퐹′푗) is F-Correlation 푆푈(퐹′푖, 퐹′푗). If (퐹′푖, 퐹′푗∈ 푉 (푇)), 푆푈(퐹′푖, 퐹′푗) ≥ 푆푈(퐹′푖, 퐶) ∨푆푈(퐹′푖, 퐹′푗) ≥ 푆푈(퐹′푗, 퐶) this property assurances the features in 푉 (푇) are redundant. Suppose the MST shown in Fig.3 is generated from a complete graph 퐺. So as to cluster the features, we 1st pass through all the six edges, and then decide to remove the edge (퐹0, 퐹4) as its weight 푆푈(퐹0, 퐹4) = 0.3 is smaller than both 푆푈(퐹0, 퐶) = 0.5 and 푆푈(퐹4, 퐶) = 0.7. This makes the MST is clustered into 2 clusters denoted as 푉 (푇1) and 푉 (T2). From Fig.3 we identify that 푆푈(퐹0, 퐹1) > 푆푈(퐹1, 퐶), 푆푈(퐹1, 퐹2) > 푆푈(퐹1, 퐶) ∧ 푆푈(퐹1, 퐹2) >푆푈(퐹2, 퐶), SU(F1, F3) > SU(F1, C) ∧ SU(F1, F3) >SU(F3, C). We also recognized that there is no edge exists

between F0,F2 and F0,F3 and F2,F3.

Fig.3: Clustering Step

After removing all the redundant edges, a forest is achieved. Every tree Tj ∈ Forest denotes a cluster that is signified as V(Tj). As examined above, the features in every cluster are redundant, thus for every cluster V (Tj) we tend to like a representative feature Fj

R whose T-Relevance SU (Fj

R,C )is the high. All FjR (j =

1...|Forest|) comprise the final feature subset ∪ FjR.

5.1 Time complexity analysis. The computation of SU values for T-Relevance and F-Correlation, that has linear complexness in provisos of the number of instances in a given data set. The first part of the algorithmic rule includes a linear time complexness O (m) as a result of the number of features m. If k (1 ≤ k ≤ m) features are elected as relevant ones in the 1st half, when k = 1, only 1 feature is chosen. Therefore, there is no need to continue the rest parts of the algorithmic rule, and therefore complexness is O(k). If 1 < k ≤ m, the second part of the algorithm first of all constructs a complete graph from relevant features and therefore complexness is O(k2), and then MST complexness is O(k2). The third part partitions the MST and chooses the representative features with the complexness of O(k). Thus when 1 < k ≤ m, the complexity of the algorithmic rule is O(m+k2). This means when k ≤√m, EFAST has linear complexness O(m), where as the worst complexness is O(m2) when k = m. However, k is drastically set to be lower bound of √m*lg m in the implementation of EFAST. Therefore the complexness is O (m * lg2m), which is usually less than O (m2) since lg2m < m. it makes EFAST has an enhanced runtime performance with high dimensional knowledge.


ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in


6 knowledge Source To evaluate the performance and effectiveness of EFAST algorithmic rule we are using publicly existing data sets. The numbers of features of the data sets vary from 35 to 49152 with a mean of 7874. The dimensionality of the 53.3% data sets exceeds 5000, of which 26.6% data sets have more than 10000 features. The data sets cover a collection of application domains like images, text and microarray data categorization. Table 1: sample benchmark data sets __________________________________________ Data ID Data Name F I T Domain __________________________________________ 1 chess 37 3196 2 Text 2 mfeat-fourier 77 2000 10 Image, Face 3 coil2000 86 9822 2 Text 4 elephant 232 1391 2 Microarray, Bio 5 tr12.wc 5805 313 8 Text 6 leukemia1 7130 34 2 Microarray, Bio 7 PIX10P 10001 100 10 Image, Face 8 ORL10P 10305 100 10 Image, Face __________________________________________ 7 Results and Analysis 7.1 Main Form

Now we have to upload any dataset i.e. Here we are uploading “chess” data set.

7.2 Loading Data Set

7.3 Calculating Entropy, Gain And GainRatio

7.4 calculating T-Relevance and Relevant Attributes


ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in


7.5 calculating F correlation

7.6 Generating MST

7.7 Relevant Feature Calculation

7.8 MST using InformationGain

7.9 clustering Inf.Gain Based MST

7.10 MST using Gain Ratio

7.11 clustering Gain Ratio Based MST

By Observing 7.8 to 7.11 we can say that Gain Ratio makes effective clustering than Information Gain as proposed in the existing system. We can represent the same thing in graphical form also. In 7.12 we are showing the graphical chart illustration of information gain vs. gain ratio. From the above discussions and experimental results we can conclude that Gain Ratio gives Enhanced Feature Selection than Information Gain.


ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in


7.12 Graphical Representation of Inf.Gain Vs Gain Ratio

Analysis on Chess Data Set Construction of MST using Inf.Gain

Construction of MST using Gain Ratio

Construction of MST Clustering using Inf.Gain

Construction of MST Clustering using Gain Ratio

Construction of MST Clustering using Inf.Gain

Construction of MST Clustering using Gain Ratio

8 Conclusions In this paper, we have presented a completely unique relevant clustering-based EFAST algorithmic rule for high dimensional knowledge. The algorithmic rule involves a) removing unrelated features, b) building a MST from relative ones, and c) divisioning the MST and selecting representative features. a cluster consists of features. Each cluster is referred as a single feature and thus dimensionality is severely reduced. The proposed algorithm gets the most fractions of chosen features, the enhanced runtime, and the finest classification accuracy. For the future work, we plan to find out different types of correlation calculations, relevance measures and revise some formal properties of feature space. ACKNOWLEDGEMENTS The authors would be fond of to the editors and the anonymous commentators for their intuitive and helpful observations and suggestions that resulted in significant developments to the current work.


ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in


REFERENCES [1] Almuallim H. and Dietterich T.G., Algorithms for Identifying Relevant Features, In Proceedings of the 9th Canadian Conference on AI, pp 38-45, 1992. [2] Almuallim H. and Dietterich T.G., Learning boolean concepts in the presence of many irrelevant features, Artificial Intelligence, 69(1-2), pp 279-305, 1994. [3] Arauzo-Azofra A., Benitez J.M. and Castro J.L., A feature set measure based on relief, In Proceedings of the fifth international conference on Recent Advances in Soft Computing, pp 104-109, 2004. [4] Baker L.D. and McCallum A.K., Distributional clustering of words for text classification, In Proceedings of the 21st Annual international ACM SIGIR Conference on Research and Development in information Retrieval, pp 96-103, 1998. [5] Battiti R., Using mutual information for selecting features in supervised neural net learning, IEEE Transactions on Neural Networks, 5(4), pp 537-550, 1994. [6] Bell D.A. and Wang, H., A formalism for relevance and its application in feature subset selection, Machine Learning, 41(2), pp 175-195, 2000. [7] Biesiada J. and Duch W., Features election for high-dimensionaldatała Pear-son redundancy based filter, AdvancesinSoftComputing, 45, pp 242C249, 008. [8] Butterworth R., Piatetsky-Shapiro G. and Simovici D.A., On Feature Se-lection through Clustering, In Proceedings of the Fifth IEEE international Conference on Data Mining, pp 581-584, 2005. [9] Cardie, C., Using decision trees to improve case-based learning, In Proceedings of Tenth International Conference on Machine Learning, pp 25-32, 1993. [10] Chanda P., Cho Y., Zhang A. and Ramanathan M., Mining of Attribute Interactions Using Information Theoretic Metrics, In Proceedings of IEEE international Conference on Data Mining Workshops, pp 350-355, 2009. [11] Chikhi S. and Benhammada S., ReliefMSS: a variation on a feature ranking Relief algorithm. Int. J. Bus. Intell. Data Min. 4(3/4), pp 375-390, 2009. [12] Cohen W., Fast Effective Rule Induction, In Proc. 12th international Conf. Machine Learning (ICML’95), pp 115-123, 1995. [13] Dash M. and Liu H., Feature Selection for Classification, Intelligent Data Analysis, 1(3), pp 131-156, 1997. [14] Dash M., Liu H. and Motoda H., Consistency based feature Selection, In Proceedings of the Fourth Pacific Asia Conference on Knowledge Discovery and Data Mining, pp 98-109, 2000.

[15] Das S., Filters, wrappers and a boosting-based hybrid for feature Selection, In Proceedings of the Eighteenth International Conference on MachineLearning, pp 74-81, 2001. [16] Dash M. and Liu H., Consistency-based search in feature selection. Artificial Intelligence, 151(1-2), pp 155-176, 2003. [17] Demsar J., Statistical comparison of classifiers over multiple data sets, J. Mach. Learn. Res., 7, pp 1-30, 2006. [18] Dhillon I.S., Mallela S. and Kumar R., A divisive information theoretic feature clustering algorithm for text classification, J. Mach. Learn. Res., 3, pp 1265-1287, 2003. [19] Dougherty, E. R., Small sample issues for microarray-based classification. Comparative and Functional Genomics, 2(1), pp 28-34, 2001. [20] Fayyad U. and Irani K., Multi-interval discretization of continuous-valued attributes for classification learning, In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pp 1022-1027, 1993.

L.Anantha Naga Prasad M.Tech, Computer Science and Engg, Anantha Lakshmi Institute of Technology&Sciences, JNTUA, Andhra Pradesh, India. [email protected]. His current research interests include data mining/machine learning, information retrieval, computer networks, and software engineering.

K.Muralidhar Assistant Professor, Department of CSE, Anantha Lakshmi Institute of Technology&Sciences, JNTUA, Andhra Pradesh, India. [email protected]. His current research Interests include data mining/machine learning, computer networks, and software engineering.


ISBN NO : 378 - 26 - 13840 - 9

www.iaetsd.in
