the yes program technical writing guidelines & tips michael georgiopoulos michael georgiopoulos

The YES ProgramThe YES Program

Technical Writing Technical Writing Guidelines & TipsGuidelines & Tips

Michael GeorgiopoulosMichael Georgiopoulos

12/03/0912/03/09 YES Program YES Program

Paper OutlinePaper Outline

AbstractAbstract IntroductionIntroduction Literature ReviewLiterature Review Main Part of The PaperMain Part of The Paper

Algorithm DescriptionAlgorithm Description Algorithm Justification/AnalysisAlgorithm Justification/Analysis Experimental Design Experimental Design Databases UsedDatabases Used Experimental Results and ObservationsExperimental Results and Observations

Summary/ConclusionsSummary/Conclusions ReferencesReferences Other Parts (Acknowledgments to NSF, others)Other Parts (Acknowledgments to NSF, others)


AbstractAbstract

In the abstract you write, in a few sentences, In the abstract you write, in a few sentences, what you have done, why have you done it, how what you have done, why have you done it, how well you have done it, and you explain very well you have done it, and you explain very briefly how it compares to the state-of-the-art briefly how it compares to the state-of-the-art

Example of an AbstractExample of an Abstract (GFAM paper) (GFAM paper)This paper focuses on the evolution of Fuzzy ARTMAP neural network classifiers, using genetic evolution of Fuzzy ARTMAP neural network classifiers, using genetic algorithmsalgorithms, with the objective of improving generalization performance objective of improving generalization performance (classification accuracy of the ART network on unseen test data) and alleviating the ART category proliferation problem alleviating the ART category proliferation problem (the problem of creating more than necessary ART network categories to solve a classification problem). We refer to the resulting architecture as GFAM. We demonstrate through extensive experimentation that GFAM exhibits good exhibits good generalization and is of small sizegeneralization and is of small size (creates few ART categories), while consuming reasonable consuming reasonable computational effort. In a number of classification problems. computational effort. In a number of classification problems. In some cases, GFAM produces the optimal GFAM produces the optimal classifierclassifier. Furthermore, we compare the performance of GFAM with other competitive ARTMAP classifiers compare the performance of GFAM with other competitive ARTMAP classifiers that have appeared in the literature and addressed the category proliferation problem in ART. We illustrate that

GFAM produces improved results over these architectures, as well as other competitive classifiers.


Introduction/Motivation Introduction/Motivation

In the introduction you discuss in more detail, In the introduction you discuss in more detail, what you what you have done, why have you done it, and how well have have done, why have you done it, and how well have you done ityou done it..

In the Introduction you place your work in the In the Introduction you place your work in the context of context of previous workprevious work that has been conducted in the field. that has been conducted in the field.

In the introduction you should show In the introduction you should show sufficient knowledge sufficient knowledge of the previous literatureof the previous literature that has tackled a similar that has tackled a similar problem as the one that you are tackling, and you should problem as the one that you are tackling, and you should also explain why your approach should be preferred.also explain why your approach should be preferred.

The introduction should end with a paragraph that The introduction should end with a paragraph that explains the organization of the paperexplains the organization of the paper by section by section number, which is followed by one sentence of what each of number, which is followed by one sentence of what each of these sections contain. these sections contain.


Example of Introduction Example of Introduction piecespieces1st Paragraph (Why 1st Paragraph (Why ART?)ART?) The Adaptive Resonance Theory (ART) was developed by The Adaptive Resonance Theory (ART) was developed by

GrossbergGrossberg (1976). One of the most celebrated ART (1976). One of the most celebrated ART architectures is Fuzzy ARTMAP (FAM) (Carpenter et al, architectures is Fuzzy ARTMAP (FAM) (Carpenter et al, 1992), which has been successfully used in the literature for 1992), which has been successfully used in the literature for solving a variety of classification problems. Some of the solving a variety of classification problems. Some of the advantages that FAM possesses is that it can solve advantages that FAM possesses is that it can solve arbitrarily complex classification problems, it converges arbitrarily complex classification problems, it converges quickly to a solution (within a few presentations of the list quickly to a solution (within a few presentations of the list of input/output patterns belonging to the training set), it of input/output patterns belonging to the training set), it has the ability to recognize novelty in the input patterns has the ability to recognize novelty in the input patterns presented to it, it can operate in an on-line fashion (new presented to it, it can operate in an on-line fashion (new input/output patterns can be learned by the system without input/output patterns can be learned by the system without re-training with the old input/output patterns), and it re-training with the old input/output patterns), and it produces answers that can be explained with relative ease.produces answers that can be explained with relative ease.


Example of Introduction Example of Introduction piecespieces22ndnd Paragraph (What is the Paragraph (What is the Problem?)Problem?) One of the limitations of FAM that has been repeatedly reported in One of the limitations of FAM that has been repeatedly reported in

the literature is the category proliferation problem. This refers to the literature is the category proliferation problem. This refers to the creation of a relatively large number of categories to represent the creation of a relatively large number of categories to represent the training data. Categories are the hidden nodes (or units) in a the training data. Categories are the hidden nodes (or units) in a FAM neural network. Each node is mapped to a specific class. The FAM neural network. Each node is mapped to a specific class. The creation of a large number of categories means poor compression of creation of a large number of categories means poor compression of the training data. Quite often the category proliferation problem, the training data. Quite often the category proliferation problem, observed in FAM, is connected with the issue of overtraining. Over-observed in FAM, is connected with the issue of overtraining. Over-training happens when FAM is trying to learn the training data training happens when FAM is trying to learn the training data perfectly at the expense of degraded generalization performance perfectly at the expense of degraded generalization performance (i.e., classification accuracy on unseen data) and also at the expense (i.e., classification accuracy on unseen data) and also at the expense of creating many categories to represent the training data (leading of creating many categories to represent the training data (leading to the category proliferation problem). Also, it has been related to to the category proliferation problem). Also, it has been related to several limitations of FAM, such as the representative inefficiency of several limitations of FAM, such as the representative inefficiency of the hyperbox categories or the excessive triggering of the match the hyperbox categories or the excessive triggering of the match tracking mechanism due to existence of noise.tracking mechanism due to existence of noise.


Example of Introduction Example of Introduction piecespieces3rd Paragraph (What have 3rd Paragraph (What have we done?)we done?) In this paper, we propose the use of genetic algorithms In this paper, we propose the use of genetic algorithms

(Goldberg, 1989) to solve the category proliferation problem, (Goldberg, 1989) to solve the category proliferation problem, while improving the generalization performance in FAM. We while improving the generalization performance in FAM. We refer to the resulting architecture as GFAM. In our work here, refer to the resulting architecture as GFAM. In our work here, we use GAs to evolve simultaneously the weights, as well as we use GAs to evolve simultaneously the weights, as well as the topology of the FAM neural networks. We start with a the topology of the FAM neural networks. We start with a population of trained FAMs, whose number of nodes in the population of trained FAMs, whose number of nodes in the hidden layer and the values of the interconnection weights hidden layer and the values of the interconnection weights converging to these nodes are fully determined (at the converging to these nodes are fully determined (at the beginning of the evolution) by the ART's training rules. To this beginning of the evolution) by the ART's training rules. To this initial population of FAM networks, GA operators are applied initial population of FAM networks, GA operators are applied to modify these trained FAM architectures (i.e., number of to modify these trained FAM architectures (i.e., number of nodes in the hidden layer, and values of the interconnection nodes in the hidden layer, and values of the interconnection weights) in a way that encourages better generalization and weights) in a way that encourages better generalization and smaller size architectures. smaller size architectures.


Example of Introduction piecesExample of Introduction pieces44thth Paragraph (How well have Paragraph (How well have we done?)we done?)

Our results show that GFAM performed well on a number of Our results show that GFAM performed well on a number of classification problems, and on a few of them it performed classification problems, and on a few of them it performed optimally. Furthermore, GFAM networks were found to be optimally. Furthermore, GFAM networks were found to be superior to a number of other ART networks (ssFAM, ssEAM, superior to a number of other ART networks (ssFAM, ssEAM, ssGAM, safe micro-ARTMAP) that have been introduced into the ssGAM, safe micro-ARTMAP) that have been introduced into the literature to address the category proliferation problem in ART. literature to address the category proliferation problem in ART. More specifically, GFAM gave a better generalization More specifically, GFAM gave a better generalization performance and a smaller than, or equal, size network (in almost performance and a smaller than, or equal, size network (in almost all problems tested), compared to these other ART networks, all problems tested), compared to these other ART networks, requiring reduced computational effort to achieve these requiring reduced computational effort to achieve these advantages. More specifically, in some instances the difference in advantages. More specifically, in some instances the difference in classification performance of GFAM with these other ART classification performance of GFAM with these other ART networks quite significant (as high as 10%). Also, in some networks quite significant (as high as 10%). Also, in some instances the ratio of the number of categories created by these instances the ratio of the number of categories created by these other ART networks, compared to the categories created by other ART networks, compared to the categories created by GFAM was large (as high as 5).GFAM was large (as high as 5).


Example of Introduction piecesExample of Introduction pieces55thth Paragraph (Organization of Paragraph (Organization of the Paper)the Paper)

The organization of this paper is as follows: In section 2 we The organization of this paper is as follows: In section 2 we present a literature review relevant to some of the issues present a literature review relevant to some of the issues addressed in this paper. In section 3 we emphasize some of the addressed in this paper. In section 3 we emphasize some of the basics of the Fuzzy ARTMAP architecture. In section 4 we basics of the Fuzzy ARTMAP architecture. In section 4 we describe all the necessary elements pertinent to the evolution of describe all the necessary elements pertinent to the evolution of the Fuzzy ARTMAP architecture. In Section 5, we describe the the Fuzzy ARTMAP architecture. In Section 5, we describe the experiments and the datasets used to assess the performance of experiments and the datasets used to assess the performance of GFAM, we assess the performance of GFAM, and we offer GFAM, we assess the performance of GFAM, and we offer performance comparisons between the GFAM and other ART performance comparisons between the GFAM and other ART architectures that were proposed as solutions for the category architectures that were proposed as solutions for the category proliferation problem in FAM. Also, in Section 5 we compare proliferation problem in FAM. Also, in Section 5 we compare GFAM with other non-ART based classifiers (although the GFAM with other non-ART based classifiers (although the comparison is not comprehensive). In Section 6, we summarize comparison is not comprehensive). In Section 6, we summarize our contribution, findings, and we provide directions for future our contribution, findings, and we provide directions for future research. research.


Literature Review SectionLiterature Review Section

It is a good idea to have a It is a good idea to have a separate separate sectionsection, , immediately after the introduction, immediately after the introduction, that discusses the that discusses the literature reviewliterature review in in more detail. The literature review needs to more detail. The literature review needs to be be ThoroughThorough,, RelevantRelevant, and, and It has to be related with the problem that you are It has to be related with the problem that you are

addressing (for example in the outlier detection addressing (for example in the outlier detection strategies by Gramajo and Fox we should refer to strategies by Gramajo and Fox we should refer to the A-priori algorithm paper, where the idea of the A-priori algorithm paper, where the idea of frequent itemsets was introduced)frequent itemsets was introduced)


Main Paper (Algorithm Main Paper (Algorithm Description)Description)

In this part of the paper you describe the In this part of the paper you describe the algorithm that you have invented (such algorithm that you have invented (such as an efficient FIM algorithm for outlier as an efficient FIM algorithm for outlier detection problems) and you providedetection problems) and you provide Pseudo-CodePseudo-Code Step-by-Step Description Step-by-Step Description (with (with

explanations), and anexplanations), and an ExampleExample (explains how your algorithm (explains how your algorithm

operates)operates)


Main Paper (Algorithm Main Paper (Algorithm Explanations)Explanations)

In this part of the paper you explain how In this part of the paper you explain how and why you came up with this algorithm and why you came up with this algorithm and you emphasize some of the good and you emphasize some of the good properties of the properties of the algorithm…Words, words, words…


Main Paper (Experimental Main Paper (Experimental Design)Design)

In this part of the paper you describe In this part of the paper you describe how you are going to conduct your experiments to compare the merits of to compare the merits of your algorithm with the merits of other your algorithm with the merits of other techniques that have shown up in the techniques that have shown up in the literature and address similar problems. literature and address similar problems.

If your algorithm is a classifier, your If your algorithm is a classifier, your typical measures of merit are: are: Generalization PerformanceGeneralization Performance Size of your ClassifierSize of your Classifier Time Complexity to Design your ClassifierTime Complexity to Design your Classifier


Main Paper (Experimental Main Paper (Experimental Design)Design)An ExampleAn Example

We conducted a number of experiments to assess We conducted a number of experiments to assess the performance of the genetically engineered the performance of the genetically engineered Fuzzy ARTMAP (GFAM) architecture. There were Fuzzy ARTMAP (GFAM) architecture. There were two objectives for this experimentation. two objectives for this experimentation. The first objective is to The first objective is to find good (default) values for

the ranges of two of the GA parameters, the , the probability of deleting a category,, and the probability probability of deleting a category,, and the probability of mutating a category,. The default values were of mutating a category,. The default values were identified by conducting experiments with 19 databases. identified by conducting experiments with 19 databases. This effort is described in detail in section 5.2.This effort is described in detail in section 5.2.

The second objective is to The second objective is to compare the GFAM performance (for the default parameter values) to that (for the default parameter values) to that of of popular ART architectures that have been that have been proposed in the literature with the intent of addressing proposed in the literature with the intent of addressing the category proliferation problem in FAM, such as the category proliferation problem in FAM, such as ssFAM, ssEAM, ssGAM, and micro-ARTMAP. ssFAM, ssEAM, ssGAM, and micro-ARTMAP.


Main Paper (Databases)Main Paper (Databases)

In this part you In this part you describe the databases that that you use to compare your algorithm with other you use to compare your algorithm with other techniques. In choosing the databases for your techniques. In choosing the databases for your experiments you should consider databases experiments you should consider databases that that Have different number of data-pointsHave different number of data-points Have different dimensionality for the input patternsHave different dimensionality for the input patterns Have different number of Labels for the output Have different number of Labels for the output

patternspatterns Correspond to classification problems of varying Correspond to classification problems of varying

degrees of difficultydegrees of difficulty


Main Paper (Databases)Main Paper (Databases)An ExampleAn Example

We experimented with both artificial and real databases. We experimented with both artificial and real databases. Table 1 shows the specifics of these databases.Table 1 shows the specifics of these databases.

Gaussian DatabasesGaussian Databases (Database Index 1-12): These are (Database Index 1-12): These are artificial databases, where we created 2-dimensional data artificial databases, where we created 2-dimensional data sets, Gaussianly distributed, belonging to 2-class, 4-class, sets, Gaussianly distributed, belonging to 2-class, 4-class, and 6-class problems. In each one of these databases, we and 6-class problems. In each one of these databases, we varied the amount of overlap of data belonging to different varied the amount of overlap of data belonging to different classes. In particular, we considered 5%, 15%, 25%, and classes. In particular, we considered 5%, 15%, 25%, and 40% overlap. Note that 5% overlap means the optimal 40% overlap. Note that 5% overlap means the optimal Bayesian Classifier would have 5% misclassification rate on Bayesian Classifier would have 5% misclassification rate on the Gaussianly distributed data. There are a total of the Gaussianly distributed data. There are a total of 3×4=12 Gaussian databases. We name the databases as 3×4=12 Gaussian databases. We name the databases as “G#c-##” where the first number is the number of classes “G#c-##” where the first number is the number of classes and the second number is the class overlap. For example, and the second number is the class overlap. For example, G2c-05 means the Gaussian database is a 2-class and 5% G2c-05 means the Gaussian database is a 2-class and 5% overlap database. overlap database.


Main Paper (Experimental Main Paper (Experimental Results)Results)

This part is This part is one of the most important parts of the paper. Your objective here is to present your results . Your objective here is to present your results with with tables (as needed), and (as needed), and figures (as needed), and (as needed), and make appropriate observations of how well your make appropriate observations of how well your algorithm performsalgorithm performs

It is important in this part to compare your algorithm It is important in this part to compare your algorithm with other similar techniques that have appeared in the with other similar techniques that have appeared in the literature and have literature and have enough and convincing evidence that your algorithm works better (in some aspects) than that your algorithm works better (in some aspects) than other techniquesother techniques

It is always advantageous It is always advantageous that your comparisons rely on your own implementations of all the other of all the other algorithms that you compare your new algorithm with algorithms that you compare your new algorithm with (quite often this is not feasible)(quite often this is not feasible)


Main Paper (Experimental Main Paper (Experimental Results)Results)An ExampleAn Example

See GFAM paperSee GFAM paper


Main Paper Main Paper (Summary/Conclusions)(Summary/Conclusions)

In the Summary/Conclusions you emphasize In the Summary/Conclusions you emphasize what you have done and you provide the and you provide the major observations from your work, as well as from your work, as well as and contributions of your workand contributions of your work

Occasionally you provide Occasionally you provide directions of future directions of future research research that you, or others might pursuethat you, or others might pursue


Main Paper Main Paper (Summary/Conclusions)(Summary/Conclusions)An ExampleAn Example

In this paper, we have introduced yet another method of solving In this paper, we have introduced yet another method of solving the category proliferation problem in ART. This method relies on the category proliferation problem in ART. This method relies on evolving a population of trained Fuzzy ARTMAP (FAM) neural evolving a population of trained Fuzzy ARTMAP (FAM) neural networks. We refer to the resulting architecture as GFAM. networks. We refer to the resulting architecture as GFAM.

We have experimented with a number of databases that helped We have experimented with a number of databases that helped us identify good default parameter settings for the evolution of us identify good default parameter settings for the evolution of FAM. We defined a fitness function that gave emphasis to the FAM. We defined a fitness function that gave emphasis to the creation of a small size FAM networks which exhibited good creation of a small size FAM networks which exhibited good generalization. In the evolution of FAM trained networks, we generalization. In the evolution of FAM trained networks, we used a unique (and needed) operator; the delete category used a unique (and needed) operator; the delete category operator. This operator allowed us to evolve into FAM networks operator. This operator allowed us to evolve into FAM networks of smaller size. The network identified at the end of the of smaller size. The network identified at the end of the evolutionary process (i.e., last generation) was the FAM network evolutionary process (i.e., last generation) was the FAM network that attained the highest fitness value. Our method for creating that attained the highest fitness value. Our method for creating GFAM resulted in a FAM network that performed well on a GFAM resulted in a FAM network that performed well on a number of classification problems, and on a few of them it number of classification problems, and on a few of them it performed optimally. performed optimally.


Main Paper Main Paper (Summary/Conclusions)(Summary/Conclusions)An Example (Continued)An Example (Continued)

GFAM was found to be superior to a number of other ART networks GFAM was found to be superior to a number of other ART networks (ssFAM, ssEAM, ssGAM, safe micro-ARTMAP) that have been (ssFAM, ssEAM, ssGAM, safe micro-ARTMAP) that have been introduced into the literature to address the category proliferation introduced into the literature to address the category proliferation problem in ART. More specifically, GFAM gave a better generalization problem in ART. More specifically, GFAM gave a better generalization performance (in almost all problems tested) and a smaller than or performance (in almost all problems tested) and a smaller than or equal size network (in almost all problems), compared to these other equal size network (in almost all problems), compared to these other ART networks, requiring reduced computational effort to achieve these ART networks, requiring reduced computational effort to achieve these advantages. advantages.

In particular, in some instances the difference in classification In particular, in some instances the difference in classification performance of GFAM and these other ART networks was quite performance of GFAM and these other ART networks was quite significant (as high as 10%). Furthermore, in some instances the ratio significant (as high as 10%). Furthermore, in some instances the ratio of the number of categories created by these other ART networks, of the number of categories created by these other ART networks, compared to the categories created by GFAM, was large (as high as 5). compared to the categories created by GFAM, was large (as high as 5). Finally, some comparisons were also drawn between GFAM and dFAM, Finally, some comparisons were also drawn between GFAM and dFAM, FasART, and dFasART, and other classifiers that led us to the FasART, and dFasART, and other classifiers that led us to the conclusion that GFAM achieves good classification accuracy by conclusion that GFAM achieves good classification accuracy by creating an ART network whose size compares very favorably with the creating an ART network whose size compares very favorably with the size of the other classifiers.size of the other classifiers.

Obviously, the introduced method to evolve trained FAMs can be Obviously, the introduced method to evolve trained FAMs can be extended to other ART architectures, such as EAM, and GAM, amongst extended to other ART architectures, such as EAM, and GAM, amongst others, without any significant changes in the approach followed. others, without any significant changes in the approach followed.


ReferencesReferences

This part is obvious, in the sense that you will This part is obvious, in the sense that you will be providing a be providing a list of all the references that that are related to your work, and all the references are related to your work, and all the references that influenced your workthat influenced your work

The reference list has to be The reference list has to be complete, and , and each paper needs to be each paper needs to be appropriately cited according to the standards of the publication to according to the standards of the publication to which your contribution is submittedwhich your contribution is submitted


AppendicesAppendices

You may choose to have one or more You may choose to have one or more appendices, if your paper requires it. In an appendices, if your paper requires it. In an appendix you put appendix you put information that is that is importantimportant but you decided not to put it in the but you decided not to put it in the main part of the paper, so as to main part of the paper, so as to avoid disturbing the flow of the paper. of the paper.

For example an appendix could be For example an appendix could be A A glossary of terms A A proof of a theorem proof of a theorem that is lengthythat is lengthy Pieces of codePieces of code that you want the reader to know of that you want the reader to know of


AcknowledgmentsAcknowledgments

Here you acknowledge the sponsors of Here you acknowledge the sponsors of your work (e.g., your work (e.g., YES ProgramYES Program))

You also acknowledge other people that You also acknowledge other people that helped you produce this paper and they helped you produce this paper and they are not included in the author listare not included in the author list


Questions?Questions?

the yes program technical writing guidelines & tips michael georgiopoulos michael georgiopoulos

Documents

performance of gfam

art categories

program paper

program abstract

competitive artmap classifiers

competitive classifiers

celebrated art architectures

similar problem