itec6310 research methods in information technology instructor: prof. z. yang course website:...

ITEC6310Research Methods in Information Technology

Instructor: Prof. Z. YangCourse Website: http://people.math.yorku.ca/~zyang/itec6310.htmOffice: Tel 3049

2

What is Research?

There are many accepted definitions for the term “research”, for example:

Research is an active, diligent and systematic process of inquiry in order to discover, interpret or revise facts, events, behaviours, or theories, or to make practical applications with the help of such facts, laws or theories. The term ‘research’ is also used to describe the collection of information about a particular subject

3

What is Research?

Other insightful definitions of “research” are: Systematic, intensive, patient study and

investigation in some field of knowledge, usually employing the techniques of hypothesis and experiment, whose purpose is to reveal new facts, theories, or principles

Means a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge

Systematic study directed toward more complete scientific knowledge or understanding of the subject studied

4

Basic and Applied Research

• Basic research– Conducted to evaluate theories

or empirical positions– Goal is to acquire information

about a phenomenon– Little emphasis on application to

real-world problems

5

• Applied Research– Evaluates real-world problem– Hypotheses may come from

theory, but goal is to apply results to real-world problem

• Substantial overlap between basic and applied research– Each type of research has

implications for the other

Basic and Applied Research (cont’d)

6

The Research Process

IdeaCasualand/or

systematicobservation

LibraryResearch

Choose an appropriate research design(experimental, correlational,etc.)

Develop idea into a testable hypothesisDeductiveReasoning

Choose an apporpriate subjectpopulation (animals, humans, etc.)

Decide on what to observe and theappropriate measures

Conduct study (pretesting, piolt work,acutal study)

Analyze data (descriptive andinferential statistics)

Report results (write paper or makepresentation)

7

Criteria for Identification of a Suitable

Research Problem1. The research problem should

address an important question so that the

answer will make a difference.

2. The research problem should advance the frontiers of knowledge by leading to new ways of thinking, suggesting possible applications, or paving the way for further research in the field.

8

State the Assumptions

• Assumptions are basic to the research problem.

• All assumptions that have a material bearing on the problem should be openly and unreservedly set forth.

• A statement of the assumptions is necessary for others to evaluate the conclusions of the study.

• A statement of the assumptions reveals what the researcher may be taking for granted with respect to the problem.

9

Benefits of Conducting a Literature Review

1. It can offer new ideas, perspectives, and approaches that may not have occurred to you.

2. It can inform you about other researchers who conduct work in the same area.

3. It can show you how others have handled methodological and design issues in studies similar to your own.

4. It can reveal sources of data that you may not have known existed.

5. It can introduce you to measurement tools that other researchers have developed and used effectively.

10

Benefits of Conducting a Literature Review (con’t)

6. It can reveal methods of dealing with problem situations that may be similar to difficulties you are facing.7. It can help you interpret and make sense of your findings and, ultimately, help you tie your results to the work of those who have preceded you.8. It will bolster your confidence that your topic is one worth studying, because you will find that others have invested considerable time, effort, and resources in studying it.

In short: the more you know about investigations and perspectives related to your topic, the more effectively you can tackle your own research problem.

11

Case study

• You are reading a literature review written by a novice researcher in your field. You notice that the researcher seems to have relied heavily on the World Wide Web in preparing the literature review. Your hunch is supported when you glance at the reference page and find a long list of Internet addresses. What goes through your mind regarding the quality and adequacy of the literature review in this situation?

12

Organization of a Research Project

1. Title Page 2. Abstract 3. Statement of the Problem 4. Background and Literature Review 5. Research hypothesis and Null hypothesis 6. Definition of research variables and operationalization 7. Research design and strategy 8. Threats to internal and external validity 9. Sampling 10. Data collection11. Data analysis and statistical tests 12. Conclusions 13. References

13

Preliminary Research Proposal

• in one or two pages• states the research question • states the purpose of the

research • sketched the initial model • discusses (explains) the initial

model • identifies pertinent background

literature

Choosing Variables for Your Study

• Research Tradition– Your variables may be similar to those

included in previous studies.– You may use previously used dependent

variables while manipulating new independent variables.

• Theory– A theory on which you are relying may

suggest certain variables to be included.

14

• Availability of New Techniques– Sometimes a new technique is

developed, allowing you to observe a variable that previously could not be observed.

• Availability of Equipment– The variables you manipulate or

observe may be limited by the equipment available to you.

Choosing Variables for Your Study

15

The RELIABILITY of a Measure

• A reliable measure produces similar results when repeated measurements are made under identical conditions.

16

• Reliability can be established in several ways– Test-retest reliability: Administer

the same test twice– Parallel-forms reliability: Alternate

forms of the same test used– Split-half reliability: Parallel forms

are included on one test and later separated for comparison

The RELIABILITY of a Measure

17

Example

Indicate for each of the types of reliability listed below whether it measures stability over time, equivalency of items, or both of these things:

– Test-retest reliability– Parallel-forms reliability– Split-half reliability

18

• An accurate measure produces results that agree with a known standard.

• A measurement instrument can be inaccurate but reliable.– The reverse cannot be true

ACCURACY of a Measure

19

• A valid measure measures what you intend it to measure

• Validity can be established in a variety of ways– Face validity: Assessment of adequacy of content.

Least powerful method– Content validity: How adequately does a test

sample behavior it is intended to measure?– Criterion-related validity: How adequately does a

test score match some criterion score? Takes two forms

• Concurrent validity: Does test score correlate highly with score from a measure with known validity?

• Predictive validity: Does test predict behavior known to be associated with the behavior being measured?

– Construct validity: Do the results of a test correlate with what is theoretically known about the construct being evaluated?

The VALIDITY of a Measure

20

Example

You have developed a comprehensive test for research methods course that covers all aspects of the course. What type(s) of validity would you recommend establishing for this measure? Why is face validity not considered a true measure of validity? How is it possible for a test to be reliable but not valid?

21

• Using Established Versus New Measures– You need not spend time assessing

validity and reliability of established measures. • However, they may not meet your research

needs.– New measures must be evaluated for

validity and reliability, which takes time and effort.• However, they may better meet your

research needs.

Choosing Your Measures

22

Scales of Measurement

• Nominal Scale– Lowest scale of measurement involving

variables whose values differ by category (e.g., male/female)

– Values of variables have different names, but no ordering of values is implied

• Ordinal Scale– Higher scale of measurement than

nominal scale– Different values of a variable can be

ranked according to quantity (e.g., high, moderate, or low self-esteem)

23

• Interval Scale– Scale of measurement on which the

spacing between values is known (e.g., rating a book on a scale ranging from 0 to 10)

– No true zero point

• Ratio Scale– Similar to interval scale, but with a

true zero point (e.g., running time of a certain algorithm)

Scales of Measurement

24

Factors Affecting Your Choice of a Scale of

Measurement• Information Yielded

– A nominal scale yields the least information. – An ordinal scale adds some crude information. – Interval and ratio scales yield the most

information.

• Statistical Tests Available– The statistical tests available for nominal and

ordinal data (nonparametric) are less powerful than those available for interval and ratio data (parametric).

– Use the scale that allows you to use the most powerful statistical test.

25

Examples

Identify the scale of measurement for each of the following variables:

• ZIP code• Software cost• SAT score• Class rank• Letter grade• Correction rate

26