CPSC 699

Summary

Refereeing is the foundation of academic word:

it promotes equity, diversity, openness, free exchange of ideas, and drives the progress.

Lecture planWhat is refereeingJournal and Conference refereeing structureRefereeing:

Why refereeHow to refereeOutcomes of the process

Summary

QualityHume emphasized education and experience: men of taste acquire certain abilities that lead to agreement about which authors and artworks are the best. Such people, he felt, eventually will reach consensus, and in doing so, they set a‘standard of taste’ which is universal. These experts can differentiate works of high quality from less good works.

Cynthia Freeland, But is it Art, Oxford University Press, 2001.

David Hume - wikipedia.org

Refereeingrefereeing: what your peer does in peer

reviewSaul’s recommended sources:

google parberry forscher refereeJ. Boyd recommends to search for

referees guidelines for conference/journalSee also extensive list of links on CPSC 699

web site

Journal organizationEditor

aka managing editor, editor-in-chief

Associate Editor

referee

referee

referee

• submit paper to editor• editor assigns to associate editor

• assign paper to referees• make decision

• review paper• recommend to associate editor

conference organizationconference chair

program chair

program committee

member

referee

referee

referee

• organize conference

• submit paper to conference program• assign papers to committee members

• assign paper to referees

• review paper• recommend to program committee

• make decision

Blind reviewblind review

authors do not know identity of refereesavoids pressure on referees

double-blind reviewreferees do not know identity of authorseliminates reputation as factorcreates unnecessary complications/extends

refereeing time

Why peer reviewquality control

allocates scarce space resources to best papersfilter to eliminate bad papers for readers

as a side effectuseful feedback to authors

Author

responsibilities:write an submit paperassures that paper meets venue’s requirements

Editor/associate edresponsibilities:

first quality filterassign associate editor (if necessary)choose referees (if necessary)generalist refereemake decision base on referee reports

Refereeresponsibilities:

critical review of paper

justify comments in review

suggest changessuggest action

(accept/reject)

usually three reviews per paper

typesexpertsgeneralists

Why refereeservice to communityestablish your participationgood way to see new researchLearnImprove your CVDownside ??

more work

How to referee: Things to look forwhen refereeing look

for1. correctness2. significance3. innovation4. interest5. replication

re-invention plagiarism self-plagiarism

6. timeliness

7. quality of writingclarityconcisenessgrammar

andspellingexcessive jargonunsupported work

Ethicsdo unto otherstreat others fairlydo not use derogatory languagerespect confidentiality

submission to conference or journal is not a public disclosure

Ethics (continued)

are you working on a similar problemconsider turning down requesttalk to editor honestly

Self plagiarismjournal papers can be reasonable

expansions of conference papers (Saul) his attitude may be changing

in generalcan re-publish if original forum was obscure

Saul’s generic template1. title, authors (if

known), manuscript no.

2. summarize the contributionnot what they did or

howno judgement

3. qualitysound analysis,

proofs, equationsare methods valid?

Reasonable interpretation of results

relation to existing work

4. can it be duplicatedsufficient detail for

expert to reproduce results

template (continued)5. writing

clarityorganizationgrammarspellingfigures/tablesstylelogicESL (suggest

improvements)6. relevance

domaindepth

specializationall must be

appropriate for readers

7. other feedbacktyposmissing connections

to other workConstructive

suggestions

Outcomesconference

definitely rejectprobably rejectborderlineprobably acceptdefinitely accept

journalrejectreject and resubmitmajor revisionsminor revisionsaccept

other feedbackyou are usually

asked to rate your confidence in a reviewextremely confident

toknow nothing

it is accepted that referee is expert in general area

can usually submit comments to editor that will not be seen by authorsgood place to

disclose your concerns/conflicts

Summary

Refereeing is the foundation of academic word:

it promotes equity, diversity, openness, free exchange of ideas, and drives the progress.

SourcesWeb links on refereeingChapter 1 web siteJeff Boyd presentation on refereeing (with

permission)