Research Methods 1

Research Methods

Scientific Publications

• Types of Publications

• Citations

• Impact Factor

• Refereed Conference/Journal Papers

• Review Papers

• Paper Review

Research Methods 2

Scientific Publications:• Scientific Publications report original empirical and theoretical work

within a scientific field.• Why Publish?

– Academics and researchers• Ensure that you get the credit of your work• Publications and research grants main criteria for getting a position and

being promoted• Secure more research grants• Make your work known among researchers in the same area• Get connected with other researchers and form research teams

(consortiums) – Graduate Students

• Show that your work is original, novel, technically sound, and accepted by the academics and researchers in the area.

• Not easy for a PhD committee to question your work if already published

• How to value publications– Peer review– Citations– Impact factor

Research Methods 3

Where to Publish:• Primary Scientific Publications:

– Scientific research on original work

– Authors present their own research work

– Published in

• Scientific journals

• Conference Proceedings

• Technical Reports

• Monographs

• Book chapters (compilations of articles)

• Secondary Scientific Publications:

– Surveys on research work

– Authors present a synthesis of research articles on a topic to highlight advances of research,

– published in

• Review journals

• Books or compilations of articles.

Research Methods 4

Scientific Journals

• Scientific journals intend to further the progress of science by reporting the latest theoretical research and experimental results in the specific field of science– Thousands of scientific journal titles in publication– Highly specialized, and highly technical

• Incomprehensible to anyone except for researchers in the field and advanced students

• Articles are Peer Reviewed to ensure that articles meet – the journal's standards of quality, and scientific validity

• Published 2 to 6 times per year– No Call-for-Papers required– No submission deadlines

• Except in the case of “Special Issues” on hot topics • Selection standards can vary widely from journal to journal.

– The most prestigious journal in a field tends to be the most selective – Some journals have a reputation of publishing articles that mark a

fundamental breakthrough in their respective fields.• If an article describes experiments or calculations, it must supply

enough details that an independent researcher could repeat the experiment or calculation to verify the results.

Research Methods 5

Types of Scientific Journals - Articles• Normal Articles

– Complete descriptions of current original research findings– Most of the journal publish this type of articles– Usually between six to twenty pages (Typical 12 or 16)

• Letters (also called communications) – short descriptions of important current research findings – fast-tracked for immediate publication because they are considered urgent– Four to six pages in the form of extended abstracts

• Research Notes– Short descriptions of current research findings that are considered less

urgent or important than Letters.• Supplemental Articles

– Large volume of tabular data which is the result of current research– Dozens or hundreds of pages with mostly numerical data. – Some journals now only publish this data electronically on the internet.

• Review Articles– Coherent narrative report about the state of the art in that field– Do not cover original research but rather accumulate the results of many

different articles on a particular topic– Provide information on the topic and also references to the original work – Twelve to thirty pages

Research Methods 6

Conference Proceedings• Conference proceedings are the collection of research articles that are published

in the context of an academic conference. – Proceedings contain the contributions made by researchers at the conference. – They are the written record of the work that is presented to fellow researchers– They are usually distributed as printed books (or sometimes in electronic form – CDs,

memory sticks or posted on the web) • either before the conference opens or after the conference has closed.

• Researchers are invited to submit articles on their research work with a Call-for-Papers announcement

– There is usually a 2 to 6 month period to the deadline– Often 1 to 2 week extensions

• For most conferences all submissions must go through a peer review process– Handled by the Technical Committee of the conference– Use of external reviewers experts on the topic of the article

• Proceedings are published in-house, by the organizing institution of the conference, or

– via an academic publisher • For example, the Lecture Notes in Computer Science by Springer Verlag

– A number of academic journals also use this name in their title, for example, Proceedings of SPIE

– Many conferences publish selected articles in post-proceedings or Journal Special Issues

Research Methods 7

Technical Reports• A technical report is a document that describes the process, progress, or

results of technical or scientific research or the state of a technical or scientific research problem

– Technical reports are often prepared for sponsors of research projects.

– Provide a technical report with more information that produced for a journal or conference paper

• Examples of this include in-depth experimental details, additional results, or the architecture of a computer model.

– Researchers may also publish work in early form as a technical report to establish novelty, without having to wait for the often long production schedules of academic journals.

• technical reports rarely undergo comprehensive independent peer review before publication.

• Technical reports are commonly published electronically– on the Internet or on the originating organization's intranet.

• Many organizations collect their technical reports into a formal series where they are assigned an identifier (report number, volume number) and share a common cover-page layout. The entire series might be uniquely identified by an ISSN.

Research Methods 8

Monographs and Book Chapters

• A monograph is a writing on a single subject, usually by a single author. – released in the manner of a book or an extended journal article

– It is by definition a single document that forms a complete text in itself.

– Normally the term is used for a work intended to be a complete and detailed exposition of a substantial subject at a level more advanced than that of a textbook

– Monographs form a component of the review of literature in science and engineering.

Research Methods 9

References - Citations• Citation is a reference to a published or unpublished source• A citation is an abbreviated alphanumeric expression (e.g. [Kyr98]) embedded in

the text that denotes an entry in the bibliographic references section of the work for the purpose

– of showing that the arguments we are making is based on the work of others – of acknowledging the relevance of the works of others to the topic of discussion, and– to spot where the citation appears.

• Generally the combination of both the in-body citation and the bibliographic entry constitutes what is commonly thought of as a citation

• Citation has several important purposes: – to uphold intellectual honesty – to give credit to the work and ideas of others, – to allow the reader to determine independently whether the referenced material

supports the author's argument in the claimed way, and – to help the reader locate the original source of information

• Citations are used as a measure of the impact of the work of a researcher or the impact of a conference or journal

• There are many accepted citations systems, such as the Oxford, Harvard, MLA, American Sociological Association (ASA), American Psychological Association (APA)

– Most journals and conferences employ a specific citation system

Research Methods 10

Citation Formats (IEEE_Tr)1 Introduction

According to the handbook of van Leunen [1], this paragraph—and certainly this section—should be longer than one sentence.

2 More referencesHere we see if the reference [2] to the Narendra article comes out OK, in particular, with volume, number & pages. The necessary information for those who would use BibTeX is available in the 1988 document of Prof. Patashnik [3]. Interested readers who can read French may also want to read Poussin’s proof[4], though it has nothing at all to do with BibTeX.

3 ConclusionThis is the concluding paragraph. Here I cite another of Oren Patashnik’s books[5] and, again, van Leunen’s and Poussin’s [1, 4].

References[1] M.-C. van Leunen, A Handbook for Scholars. Knopf, 1979.[2] K.S.Narendra and K.Parthsarathy, “Identification and control of dynamical

system using neural networks,” IEENN, vol. 1, no. 1, pp. 4–27, 1990.[3] O. Patashnik, “Using BibTeX.” Documentation for BibTeX users, Jan. 1988.[4] C. L. X. J. de la Vall´ee Poussin, 1879. A strong form of the prime number

theorem, 19th century.[5] O. Patashnik, “Designing bibtex styles.” The part of BibTeX’s documentation

that’s not meant for general users, Jan. 1988.

Research Methods 11

Citation Formats (cj)1 Introduction

According to the handbook of van Leunen [1], this paragraph—and certainly this section—should be longer than one sentence.

2 More referencesHere we see if the reference [2] to the Narendra article comes out OK, in particular, with volume, number & pages. The necessary information for those who would use BibTeX is available in the 1988 document of Prof. Patashnik [3]. Interested readers who can read French may also want to read Poussin’s proof[4], though it has nothing at all to do with BibTeX.

3 ConclusionThis is the concluding paragraph. Here I cite another of Oren Patashnik’s books[5] and, again, van Leunen’s and Poussin’s [1, 4].

References[1] van Leunen, M.-C. (1979) A Handbook for Scholars. Knopf.[2] K.S.Narendra and K.Parthsarathy (1990) Identification and control of dynamical

system using neural networks. IEENN, 1, 4–27.[3] Patashnik, O. (1988) Using BibTeX, documentation for general BibTeX users.[4] de la Vall´ee Poussin, C. L. X. J. (1879). A strong form of the prime number

theorem, 19th century.[5] Patashnik, O. (1988) Designing bibtex styles, the part of BibTeX’s documentation

that’s not meant for general users.

Research Methods 12

Citation Formats (decsci)1 Introduction

According to the handbook of van Leunen [van Leunen(1979)], this paragraph—and certainly this section—should be longer than one sentence.

2 More references Here we see if the reference [K.S.Narendra & K.Parthsarathy(1990)] to the Narendra article comes out OK, in particular, with volume, number & pages. The necessary information for those who would use BibTeX is available in the 1988 document of Prof. Patashnik [Patashnik(1988b)]. Interested readers may also want to read Poussin’s proof [de la Vall´ee Poussin(1879].

3 ConclusionThis is the concluding paragraph. Here I cite another of Oren Patashnik’s books[Patashnik(1988a)] and, again, van Leunen’s and Poussin’s [van Leunen(1979), de la Vall´ee Poussin(1879)].

References[de la Vall´ee Poussin(1879)] De la Vall´ee Poussin, C. L. X. J. (1879). A strong form of the

prime number theorem, 19th century.[K.S.Narendra & K.Parthsarathy(1990)] K.S.Narendra & K.Parthsarathy (1990).

Identification of dynamical system using neural networks. IEENN, 1 (1), 4–27.[Patashnik(1988a)] Patashnik, O. (1988a). Designing bibtex styles. The part of BibTeX’s

documentation that’s not meant for general users.[Patashnik(1988b)] Patashnik, O. (1988b). Using BibTeX. Documentation for general

BibTeX users.

Research Methods 13

Citation Formats (paren)1 Introduction

According to the handbook of van Leunen [Leun79], this paragraph—and certainly this section—should be longer than one sentence.

2 More references Here we see if the reference [Nare90] to the Narendra article comes out OK, in particular, with volume, number & pages. The necessary information for those who would use BibTeX is available in the 1988 document of Prof. Patashnik [Pata88b]. Interested readers who can read French may also want to read Poussin’s proof [Pous79], though it has nothing at all to do with BibTeX.

3 ConclusionThis is the concluding paragraph. Here I cite another of Oren Patashnik’s books [Pata88a] and, again, van Leunen’s and Poussin’s [Leun79, Pous79].

References[Nare90] K.S.Narendra & K.Parthsarathy (1990). Identification and control of dynamical

system using neural networks. IEENN, 1 (1), 4–27.[Pata88a] Patashnik, O. (1988a). Designing bibtex styles. The part of BibTeX’s documentation

that’s not meant for general users.[Pata88b] Patashnik, O. (1988b). Using BibTeX. Documentation for general BibTeX users.[Pous79] de la Vall´ee Poussin, C. L. X. J. (1879). A strong form of the prime number

theorem, 19th century.

Research Methods 14

Citation Indexing• CiteSeer was a public search engine and digital library for scientific and

academic papers, primarily in the fields of computer and information science. – It was considered a predecessor of academic search tools such as – Google Scholar and Microsoft Academic Search.

• CiteSeer freely provided open access to all indexed documents and links indexed documents when possible to other sources of metadata such as DBLP and the ACM Portal

• IEEE Xplore is a scholarly research database that indexes, abstracts, and provides full-text for articles and papers on computer science, electrical engineering and electronics.

• The ACM Portal is an online service of the Association for Computer Machinery. Its core are two main sections: ACM Digital Library (ACM DL), and the Guide to Computing Literature.

• http://citeseer.ist.psu.edu/ • http://www.pubzone.org/index.do

Research Methods 15

Scientific Journal or Conference Articles: Review ProcessA scientific paper has a significant value if it appears in a refereed journal or conference

– It should be reviewed by peers before it is decided whether it is suitable for publication

– Three to five reviewers comment on the• relevance of the work of the paper with the areas of interest of the journal

or conference• originality of the work presented

– Work presented in conferences can also be presented in journals or post conference proceeding if there is significant new work (usually 30% of new work)

• correctness or soundness of the work presented• soundness of the evaluation methodology• presentation and analysis of the results achieved• organization of the paper and the syntax/grammar.

– It is the responsibility of the authors to convince the reviewers.– Journal papers can be accepted as they are, accepted with minor changes

(no need for a second review cycle), accepted with significant changes (need a second review cycle), or rejected.

– Conference papers can be accepted as they are, accepted with minor changes, or rejected.

Research Methods 16

Scientific Paper Structure: Abstract• Abstract

– Quick introduction to the content of the paper.

• It is available for anyone - provided to the sites of journals or conferences

• Researchers usually read the abstract and decide whether they should read the rest of the paper (or buy it)

– One or two paragraphs

• State the motivation for the work presented,

• the contributions of the paper,

• the evaluation methodology and

• a summary of the results achieved.

– Often limitation to the number of words (eg up to 300 words)

– No citations appear in the abstract

Research Methods 17

Scientific Paper Structure: Introduction and Related Work• Introduction:

– Opening Part (one or two paragraphs)• What is the problem and the motivation for the work presented • Which are the contribution of your work

– Present a quick review of the state-of-the-art in the area leading to the motivation for your work

– Overview of the methodology used– Summary of the evaluation methodology and the main results achieved– Overview of the structure of your Thesis (Last / Optional)

• E.g. { Chapter 1 presents an overview of the ….. Chapter 2 describes the ????. etc.

– About 1 to 2 pages• Related Work

– Summary and in some cases just reference to related work. – Should related it to your work to put your work into content– Shows that you acknowledge and give credit to the work of others

• Some authors put the related work at the end of the paper and compare their results with the results of others

• In short papers (less than 6 pages) the related work can be part of the Introduction.

Research Methods 18

Scientific Paper Structure: Methodology and Results• Methodology

– Describe your approach in solving the problem

– Justify your design choices

– Use of diagrams and flowcharts and provide a clear and technically sound description of your work

• Evaluation Methodology– Describe in detail the methodology used to evaluate your work– Use of experiments, simulations, surveys, analytical modelling – Give the technical specification and settings of equipment used for your experiments

or describe the sample used for your surveys and statistical analysis – Justify your evaluation methodology

• Results achieved• Use mostly graphs and rarely tables• Discussion of results

– Critically discuss your results– Avoid describing in words what is given by the tables or figures– If possible relate your results with the results obtain by others

» Make sure you compare similar and compatible things

About 50% of the paper allocated for the introduction, related work, and methodology. The rest 50% must be allocated for the presentation and the analysis of the results achieved

Research Methods 19

Scientific Paper Structure: Conclusions and References• Conclusions

– Overview and interpretation of the work presented • Emphasize pros and acknowledge drawbacks

– Refer to figures and results– Suggestions for Future work

• Acknowledgements – Needed in the case of funded research projects.– If it is omitted, then is possible to loose funding for the paper publication or

conference participation

• References– Use a coherent way for presenting the references– In most cases the format for citations is specified in the journal or conference

guidelines for the authors – Make sure that your references are updated with the most recent work

Research Methods 20

Research Methods

Survey Papers and Paper Reviews

• Review Papers (Surveys)

• Paper Review

Research Methods 21

Survey Papers:• Surveys or review papers serve to discuss and synthesize key findings

on a particular research area.

• Graduate students often produce survey papers as part of their coursework

– Done at the early stages of their graduate studies

– Show that they have extensive knowledge in the area of their research

• Established researches produce review papers that includes their work in the field as well as the most prominent work in the area produced by others.

– Helpful to the writer and their colleagues in gaining critical awareness in specialized fields.

• Surveys are published in

– Special survey journals

– Special issues of scientific journals

• Survey papers undergo a review process.

Research Methods 22

Types of Survey Papers:• State of the art review

– Presents and critically compares mainly the most current research in a given area. – It may offer new perspectives on an issue or point out an area in need of further

research.

• A Historical review– A survey of the development of a particular research area. – Examines

• the early stages of the field, • key findings to present, • key theoretical models, and • their evolution.

• A Comparison of perspectives review– Compare various ways of looking at a certain issue in a research field. – Debate between different approaches. – It may introduce a new perspective as a conclusion of comparing existing

approaches. (Theoretical model building review)

• A Synthesis of more than one fields review– Often researchers in different fields may be working on similar problems.– A synthesis of two fields review provides insights into a given topic based on a review

of the literature from two or more disciplines.

Research Methods 23

Sources for Survey Papers

• Established researchers have a knowledge of the developments in the area by: – Reading papers from scientific journals in the field

– Attending conferences in the field

– Reviewing papers for conferences and journals

– Participating in research networks and forums • Who-is-Who in ….• Networks of Excellence (such as the HiPEAC) – EU funded• COST - EU funded

– Visits to/from other universities and research centers

• New graduate students:– Get advise on what to search for from their advisors

– Track the work (through personal sites) of the most prominent researchers in the field

• Persons listed in steering or technical committees of conferences• Persons or research projects appearing frequently in the related work of scientific

papers • Number of citation appearing in citation index sites

• Work presented in other relevant surveys or review papers

Research Methods 24

Main Sections of a Survey Paper: Abstract and Introduction• Abstract

– A brief summary of your review.

– Include only the main points of your review.

– This is a chance for the readers to preview the paper and decide if they want to read it or not.

• Introduction– Introduce your topic

• Introduce the reader to the topic of the survey

• Provide any necessary background information specific to the topic

• Define terms that may be unfamiliar to the reader

• Explain the scope of the discussion, and your purpose for writing the review.

• State your topic's relevance

– Outline the topics to be discussed in the survey

• Give a brief overview of the main topics/ideas/approaches to be discussed.

– Reveal your thesis to the reader

• Your position (thesis) is the main idea that you want to get across to your reader.

• It should be a clear statement of what you intend to prove or illustrate by your review.

• By revealing your thesis in the introduction the reader knows what to expect in the rest of the paper.

Research Methods 25

Main Sections of a Survey Paper: Main Body of the Survey• Discussion

– Main body of your paper.

– Information that develops and supports your research work or thesis.

• Main considerations.– Don't just summarize other papers, but Analyze, Synthesize, Interpret

• The review must not be a pure summary of the information you read. You must analyze, synthesize, and interpret the information you read in a meaningful way.

– Keep your discussion focused.

• Often many issues are related in a chain form. Your discussion should be focused on the issues related to your thesis.

– Relate the discussion to your thesis.

• Identify and emphasize issues that will help you in developing and supporting the ideas in your research work or thesis.

Research Methods 26

Main Sections of a Survey Paper: Conclusions and References• Conclusions

– A good conclusion should illustrate the key connections between your major points and your thesis as well as they key connections between your thesis and the broader discussion—what is the significance of your research work in a larger context?

– Make some conclusions—where have you arrived as a result of writing this review paper?

– Be careful not to present any new information in the conclusion section.

• References – Here you report all the works you have cited in your paper. The format for a

references page varies by discipline as does how you should cite your references within the paper.