hit4010-9010 modules 03-04

© Swinburne University of Technology 1

Modules 03/04

HIT4010/9010 - Research Methods

2 HIT4010/9010, © Swinburne

Outline� Literature review

- library training/tutorial

� Research design� Ethical conduct of research

Reading: Textbook Chapters 4 and 5Extra reading: Booth, Colomb & Williams (1995)

SUT Guidelines for ethics approval


Literature review� Literature

� What other researches have done

� Review� To look again (re + view)

� A literature review sets out the major themes in the paper, showing how they relate to previously published results.� e.g., a precise analysis of previous work could explain what methods have

been proposed and tested, how they are similar and where they differ, and compare their relative strengths and weaknesses.

� “Critical” when used to describe reading literature does not mean “criticising” or “looking for faults”. It comes from the practice of “critiquing” or giving insightful comments.


Purpose of review� Increase your confidence� Provide you with new ideas and approaches� Inform you about other researchers working in this area� Show you how others handling methodological and design issues� Reveal sources of data you may not have considered� Introduce you to measurement tools developed/used� Reveal methods of dealing with similar problems/difficulties� Help you interpret and make sense of your findings


Questions to ask yourself to helpdevelop your sense of judgement� As you read, actively try to identify contributions and

shortcomings of a paper.� “What are the main results?� How precise are the claims?� How could the outcomes be used?� What is the evidence?� How was the evidence gathered?� How were measurements taken?� How carefully are the algorithms and experiments described?� Why is the paper trustworthy?� Has the right background literature been discussed?� What would reproduction of the results involve?


Sources and strategies (1)� Librarians

�Subject liaison librarian� General reference works

�Encyclopaedias: general or specialised� Text books: understanding a new area � Index references

� Related associations (in ICT)�ACM� IEEE (and IEEE Computer Society etc.)� IFIP

� Problem driven



Sources and strategies (2)� Library catalogue

� Individual University libraries�Coolcat – nation wide catalogue�Key words, author, title…

� Online databases�Specialised areas�Provide searches of journal articles and books in a field

�ACM digital library, IEEE digital library�Usually provide you with an abstract and more often these

days – a full copy of journal articles�Citation index

� Avoid World Wide Web (e.g. wikipedia, ‘google’ search engine)


Using sources (Booth et al. 1995)� Read critically� Evaluate your sources

� Identify the sources most valuable to your research problem �Skim read many articles and decide which deserve more

careful attention�Read critical references slowly and carefully

� Take full notes�Keep a history of what you have read�Authors, title, journal, volume, page numbers…�Key words�Summary of main points�Consider using bibliographic package

e.g. EndNote, ProCite


Taking notes while reading literature (cont)

� Notes & Words� Spell technical words and authors’ names correctly from the

beginning� Learn to pronounce the names� Learn the right format et al. (not etal)

� Quotes and ideas� In your notes, distinguish between quotes and paraphrases� Keep records of all sources of ideas and also page numbers

for quotes.


Speedy reading (1) (Booth et al. 1995)1. Become familiar with geography of source� Books

�Read first few sentences in each paragraph of the preface� Look at table of contents for summary chapters�Skim index for topics with the most page numbers�Skim bibliography, noting dates and sources cited most often

� Articles�Read the abstract� Flip through pages to see if there are section headings�Skim the bibliography


Speedy reading (2) (Booth et al. 1995)2. Locate the point of the argument� Read the introduction and conclusion� Identify the evidence being used to support main claims3. Identify key-subpoints� Look for places where the writer wraps up a topic (“Finally…”) or

uses transitional works (“First.. Second… Third…”, “Now we have to consider…”

4. Identify key themes� Use subject headings, key points and claims to identify the main

themes � Decide whether these are of relevance to your topic and

deserving of further attention


Library training/tutorial� Hands on introduction to library data bases and

electronic journals

� EndNote/ProCite training at SwinburneCheck library website for detailsRegister for a session



Discussion items (1)

1. Describe three benefits of conducting a thorough review of existing literature prior to writing a research proposal.

2. Kirby is conducting a literature review in preparation for his study of “expectations regarding the sharing of financial and practical responsibilities among married and cohabiting couples in which both partners are between the ages of 20 and 29”.Conducting a keyword search on “couples” and “responsibility”, Kirby has generated a lengthy list of research articles. He decides he will shorten the list of potential articles by eliminate all articles from the list that were not published in prestigious research journals and include all of the remaining articles in his literature review. What is your opinion of Kirby’s approach to locating articles to be included in the literature review? Explain.

3. You are reading a literature review written by a novice researcher in your field. You notice that the researcher seems to have relied heavily on the Web in preparing the literature review. A glance at the reference page supports your hunch – a long list of Internet addresses. What goes through your mind regarding the quality and adequacy of the literature review in this situation?

Research Design


Planning Research Design� Format of research process (diagram)

� Research planning vs Research methodology

� General criteria for a research project

� Validation


Criteria for research project� Universality

�Research plan could be carried out by any competent person� Replication

�Another competent researcher should be able to achieve the same results as yourself

� Control� Factors central to the research project must be isolated and

controlled by the researcher � Measurement

�Data should be measurable


Nature and Role of Data� Manifestations of reality

- Not absolute reality (i.e., Truth)

� Transient and ever changing

� Primary data and secondary data


Admissibility of data� Defective data� Levels of precision� (Limits of acceptability)� Standards� (Specification in research proposal)

�What counts as admissible and inadmissible data



Measurement� Use of existing measurement tools� Development of customised measurement tools� Faulty measurement tool

� Undermine research effort

� Validity� Reliability


Validity of measurement instruments

Extent to which measurement tool measures what it is supposed to measure

� Face validity� Looks like its measuring the right thing

� Content validity�Representative sample of content area

� Criterion validity�Correlation with some other related measure

� Construct validity�Measurement of characteristic

that can’t be directly observed


Reliability of measurement instrumentsExtent to which the measurement tool yields consistent

results when the characteristic hasn’t changed� Interrater reliability

�Extent to which two or more people rating the same thing� Internal consistency reliability

�All items in a tool yield similar results� Test-retest reliability

�Extent to which same tool yields same result on two different occasions


Linking Data and MethodologyMethodology is used to extract meaning from the data� Quantitative

�Measuring relationships between variables�Prediction, explanation, control

(traditional, experimental, positivist)� Qualitative

�Complex nature of phenomena�Description and understanding of phenomena from peoples

point of view(interpretive, constructivist, postpositivist)


Validity of the Method

The accuracy, meaningfulness, and credibility of research� Internal validity (sufficient control)

� The extent to which the design and data allow accurate conclusions about relationships within the data

� External validity (generalisation)�Extent to which the results apply to situations beyond the

original (specific) study


“A Research Checklist” [1]� 1. Are the ideas clear and consistent?� 2. Is the problem worthy of investigation?� 3. Does the project have appropriate scope?� 4. What are the specific research questions?� 5. Is there a hypothesis?� 6. What would disprove the hypothesis? Does it have any

improbable consequences?� 7. Are the premises sensible?� 8. Has the work been critically questioned? Have you satisfied

yourself that it is sound science?[1] Zobel, J. (2004) Writing for Computer Science, Springer Verlag. Pp. 182-183



“A Research Checklist” cont. [1]� 9. How are the outcomes to be evaluated? Why are the

chosen methods of evaluation appropriate or reasonable?� 10. Are the roles of the participants clear? What are your

responsibilities?� 11. What activities will others undertake?� 12. What are the likely weaknesses in your solution?� 13. Is there a written research plan?� 14. What forms of evidence are to be used?� [1] Zobel, J. (2004) Writing for Computer Science,Springer Verlag. Pp. 182-183


“A Research Checklist” cont. [1]� 15. Have milestones, timelines, deadlines been identified?� 16. Do the deadlines leave enough room for your advisor to

provide feedback on your drafts, or for your colleagues to contribute to the material?

� 17. Has the literature been explored in appropriate depth? Once the work is largely done – and your perspective has changed –does it need to be explored again?

� Zobel, J. (2004) Writing for Computer Science, Springer Verlag. Pp. 182-183


Ethical Issues in Research� Protection from harm� Informed consent� Right to privacy� Honesty with professional colleagues

� Ethics approval


Ethics� Ethics: Scientific integrity

�Professional standards (fabrication, falsification, plagiarism etc)

� Ethics: responsibility to participants (animal and human)� 1946 Nuremberg code� Risk/benefit analysis� Idea of informed consent


Informed consent� “a person who, in possession of suitable information,

grants authority to someone else to take actions affecting that person” p 22 Beach (1996)

� Not just signing a form� Investigator must ensure that the participant understands:

� The procedure including potential risks�Participation is voluntary, and they may withdraw at any time


Ethics approval form� Project title� Applicant details� Date of activities� Type of activity� Benefits (to participant & more generally)� Potential problems� Future use of data� External involvement� External approvals



Discussion items (2)

1. Professor Parsnip wants to know more about the mental process students go through as they learn about research methods. She also wants to know which instructional methods are most effective for teaching research methods to her students. Assume that Professor Parsnip decides to take a qualitative approach to studying these questions. State a specific research question to her general research problem that lends itself well to a qualitative research approach. Then explain why your question is better addressed using a qualitative, rather than a quantitative, methodology.

2. Swap ‘qualitative’ with ‘quantitative’ above and do it again.3. Sometimes students believe that taking the time to prepare a complete and

carefully designed research proposal is something to be preached, but not really practiced. Identify at least two different problems related to research design that are commonly encountered when doing research. For each problem, explain how a complete and detailed research proposal would help the researcher avoid the problem.

hit4010-9010 modules 03-04

Documents