Subject : Research Methodology

Group 5

QUANTITATIVE RESEARCH METHODS

Outline

1. An introduction to Quantitative research method

2. 4 concepts of Quantitative Method :

Population Sampling of Quantitative research Samples of Quantitative research Qualitative Scale EXPERIMENTAL DESIGN

An introduction to Quantitative research

methodPresenter: Minh Sang

What is Quantitative research ? Quantitative research is an inquiry into

an identified problem, based on testing a theory, measured with numbers, and analyzed using statistical techniques.

The goal of quantitative methods is to determine whether the predictive generalizations of a theory hold true.

Assumptions Underlying Quantitative Methods

Reality is objective, “out there,” and independent of the researcher -- therefore reality is something that can be studied objectively.

The researcher should remain distant and independent of what is being researched.

The values of the researcher do not interfere with, or become part of, the research --research is value-free.

Assumptions Underlying Quantitative Methods

Research is based primarily on deductive forms of logic and theories and hypotheses are tested in a cause-effect order.

And the goal is to develop generalizations that contribute to theory that enable the researcher to predict, explain, and understand some phenomenon.

Three general types of quantitative methods: 1. Experiments True experiments are

characterized by random assignment of subjects to experimental conditions and the use of experimental controls.

2. Quasi-Experiments Quasi-experimental studies share almost all the features of experimental designs except that they involve non-randomized assignment of subjects to experimental conditions.

Three general types of quantitative methods: 3. Surveys Surveys include cross-

sectional and longitudinal studies using questionnaires or interviews for data collection with the intent of estimating the characteristics of a large population of interest based on a smaller sample from that population.

Comparison of quantitative and qualitative research approaches

General framework

Qualitative Seek to explore phenomena

Instruments use more flexible,iterative style of eliciting andcategorizing responses to questions

Use semi-structured methods such as in-depth interviews, focusgroups, and participant observation

QuantitativeSeek to confirm hypotheses about Phenomena

Instruments use more rigid styleof eliciting and categorizingresponses to questions

Use highly structured methodssuch as questionnaires, surveys,and structured observation

Comparison of quantitative and qualitative research approaches

Analytical objectives

QualitativeTo describe variation

To describe and explain relationships

To describe individual experiences

To describe group norms

QuantitativeTo quantify variation

To predict causal relationships

To describe characteristics of apopulation

Comparison of quantitative and qualitative research approaches

Question format

Qualitative

Open-ended

Quantitative

Closed-ended

Comparison of quantitative and qualitative research approaches

Data format

Quantitative

Numerical (obtained by assigningnumerical values to responses)

Quanlitative

Textual (obtained from audiotapes,videotapes, and field notes)

Comparison of quantitative and qualitative research approaches

Flexibility in study design

Qualitative Some aspects of the study areflexible (for example, the addition,exclusion, or wording of particularinterview questions)

Participant responses affect howand which questions researchersask next

Study design is iterative, that is,data collection and researchquestions are adjusted accordingto what is learned

QuantitativeStudy design is stable frombeginning to end

Participant responses do notinfluence or determine how andwhich questions researchers asknextStudy design is subject tostatistical assumptions andconditions

VARIABLES

Outline

• Definitions• Relating notions• Types

– Independence– Dependence– Control– Moderator – Extraneous – Correlation

• Cause

Definition

• A variable is something that can change, such as 'gender' and are typically the focus of a study

Relating notions

• Attributes: sub-values of a variable, such as 'male' and 'female‘

• Mutually exclusive attributes are those that cannot occur at the same time.

• Quantitative data is numeric. This is useful for mathematical and statistical analysis predictive formula.

• Qualitative data is based on human judgement. You can also turn qualitative data into quantitative data

• Units are the ways that variables are classified. These include: individuals, groups, social interactions and objects.

Types

1. Independence

2. Dependence

3. Control

4. Moderator

5. Extraneous

6. Correlation

Independent (Experimental, Manipulated, Treatment,

Grouping) Variable

• That factor which is measured, manipulated, or selected by the experimenter to determine its relationship to an observed phenomenon.

• In a research study, independent variables are antecedent conditions that are presumed to affect a dependent variable. They are either manipulated by the researcher or are observed by the researcher so that their values can be related to that of the dependent variable.

Dependent (Outcome) Variable• That factor which is observed and measured to

determine the effect of the independent variable, i.e., that factor that appears, disappears, or varies as the experimenter introduces, removes, or varies the independent variable.

• In a research study, the independent variable defines a principal focus of research interest. It is the consequent variable that is presumably affected by one or more independent variables that are either manipulated by the researcher or observed by the researcher and regarded as antecedent conditions that determine the value of the dependent variable.

• The dependent variable is the outcome.

Control

• In an experiment there may be many additional variables beyond the manipulated independent variable and the measured dependent variables. It is critical in experiments that these variables do not vary and hence bias or otherwise distort the results. There is control variable to struggle to manage this.

Moderator

• That factor which is measured, manipulated, or selected by the experimenter to discover whether it modifies the relationship of the independent variable to an observed phenomenon. It is a special type of independent variable

Extraneous

• Those factors which cannot be controlled.

• They may or may not influence the results. One way to control an extraneous variable which might influence the results is to make it a constant (keep everyone in the study alike on that characteristic).

Correlation• With perfect correlation, the X-Y graph of

points (as a scatter diagram) will give a straight line.

• Correlation can be positive (increasing X increases Y), negative (increasing X decreases Y) or non-linear (increasing X makes Y increase or decrease, depending on the value of X).

• Correlation can also be partial, that is across only a range of values X. As all possible values of X can seldom be tested, most correlations found are at best partial.

Cause

• When correlation is determined, a further question is whether varying the independent variable caused the independent variable to change. This adds complexity and debate to the situation.

• Sometimes a third variable is the cause, such as when a correlation between ice-cream sales and drowning is actually due to the fact that both are caused by warm weather.

POPULATION

Presenter: Ngoc Cam

Population

1. What is a population?2. When is a population identified?3. Collecting data about a population

What is a population?

• A population is any complete group with at least one characteristic in common.

• Populations are not just people. Populations may consist of, but are not limited to, people, animals, businesses, buildings, motor vehicles, farms, objects or events.

Populations

Identify the population

• When looking at data, it is important to clearly identify the population being studied or referred to, so that you can understand who or what are included in the data.

• For example, if you were looking at some Australian farming data, you would need to understand whether the population the data refers to is all farms in Australia, just farms that grow crops, those that only have livestock, or some other type of farm.

When is a population identified?

• The population needs to be clearly identified at the beginning of a study.

• The study should be based on a clear understanding of who or what is of interest, as well as the type of information required from that population.

Collecting data about a population

SAMPLing of Quantitative reseach

Presenter: Huu Loc

Definition of 'Sampling'• A process used in statistical analysis in which a

predetermined number of observations will be taken from a larger population. The methodology used to sample from a larger population will depend on the type of analysis being performed, but will include simple random sampling, systematic sampling and observational sampling.

• The sample should be a representation of the general population.

Sampling Methods

• In most surveys, access to the entire population is near on impossible, however, the results from a survey with a carefully selected sample will reflect extremely closely those that would have been obtained had the population provided the data.

• There are essentiality two types of samplingo probability samplingo non-probability sampling

Probability Sampling Methods

• Probability or random sampling gives all members of the population a known chance of being selected for inclusion in the sample and this does not depend upon previous events in the selection process. In other words, the selection of individuals does not affect the chance of anyone else in the population being selected.

• Many statistical techniques assume that a sample was selected on a random basis. There are four basic types of random sampling techniques:

1) Simple Random Sampling2) Systematic Sampling3) Stratified Sampling

4) Cluster or Multi-stage Sampling

1) Simple Random Sampling

• This is the ideal choice as it is a ‘perfect’ random method. Using this method, individuals are randomly selected from a list of the population and every single individual has an equal chance of selection.

• This method is ideal, but if it cannot be adopted, one of the following alternatives may be chosen if any shortfall in accuracy.

2) Systematic Sampling• Systematic sampling is a frequently used variant of simple random

sampling. When performing systematic sampling, every element from the list is selected (this is referred to as the sample interval) from a randomly selected starting point. For example, if we have a listed population of 6000 members and wish to draw a sample of 2000, we would select every 30th (6000 divided by 200) person from the list. In practice, we would randomly select a number between 1 and 30 to act as our starting point.

• The one potential problem with this method of sampling concerns the arrangement of elements in the list. If the list is arranged in any kind of order e.g. if every 30th house is smaller than the others from which the sample is being recruited, there is a possibility that the sample produced could be seriously biased.

3) Stratified Sampling• Stratified sampling is a variant on simple random and systematic

methods and is used when there are a number of distinct subgroups, within each of which it is required that there is full representation. A stratified sample is constructed by classifying the population in sub-populations (or strata), base on some well-known characteristics of the population, such as age, gender or socio-economic status. The selection of elements is then made separately from within each strata, usually by random or systematic sampling methods.

• Stratified sampling methods also come in two types – proportionate and disproportionate.

• In proportionate sampling, the strata sample sizes are made proportional to the strata population size. For example if the first strata is made up of males, then as there are around 50% of males in the UK population, the male strata will need to represent around 50% of the total sample.

4) Cluster or Multi-stage Sampling• Cluster sampling is a frequently-used, and usually more practical,

random sampling method. It is particularly useful in situations for which no list of the elements within a population is available and therefore cannot be selected directly. As this form of sampling is conducted by randomly selecting subgroups of the population, possibly in several stages, it should produce results equivalent to a simple random sample.

• The sample is generally done by first sampling at the higher level(s) e.g. randomly sampled countries, then sampling from subsequent levels in turn e.g. within the selected countries sample counties, then within these postcodes, the within these households, until the final stage is reached, at which point the sampling is done in a simple random manner e.g. sampling people within the selected households. The ‘levels’ in question are defined by subgroups into which it is appropriate to subdivide your population.

Non-probability Sampling Methods

• Non-probability sampling procedures are much less desirable, as they will almost certainly contain sampling biases. Unfortunately, in some circumstances such methods are unavoidable.

• If you are forced into using a non-random method, you must be extremely careful when drawing conclusions. You should always be honest about the sampling technique used and that a non-random approach will probably mean that biases are present within the data. In order to convert the sample to be representative of the true population, you may want to use weighting techniques.

• The importance of sampling should not be underestimated, as it determines to whom the results of your research will be applicable. It is important, therefore to give full consideration to the sampling strategy to be used and to select the most appropriate. Your most important consideration should be whether you could adopt a simple random sample. If not, could one of the other random methods be used? Only when you have no choice should a non-random method be used.

• All to often, researchers succumb to the temptation of generalizing their results to a much broader range of people than those from whom the data was originally gathered. This is poor practice and you should always aim to adopt an appropriate sampling technique. The key is not to guess, but take some advice.

SAMPLES of Quantitative reseach

Presenter: Minh Dang

Definition

• A sample is a finite part of a statistical population whose properties are studied to gain information about the whole.

• When dealing with people, it can be defined as a set of correspomdents (people) selected from a larger population for the purpose of the survey.

Sample size• Sample size is important must be

large enough• Too big sample increases costs, too

small sample causes insufficient of data to reach any meaningful conclusions

• Have as large a sample as possible• Larger sample more accurate

results• Take advice from a statistician who

will help you decide the numbers required to give validity to your results.

Purpose of sampling (choosing a sample)

1. Save time2. Save money3. Unable to survey some large

population4. Maybe only some parts of population

are accessible5. Just observation is never enough

Qualitative Scale

Presenter: Ngan Giang

Measurement scales of quantitative research

• There are four types : • nominal• ordinal• interval • ratio

• Nominal (categories): represents the lowest level of measurement.

• When a nominal scale is used, the data simply indicate how many subjects are in each category.

• Category 4 and category 1 are not different base on the number 4 and 1;  4 is not higher than 1 or more than 1.  

• Example:  Categories for IQ, types of school…

• Ordinal (ranks): puts the subjects in order from the highest to lowest, form the most to least.  

• Although ordinal scales indicate that some subjects are higher, or better, than other, they do not indicate how much higher or better. 

• Interval (scores): has all the characteristics of a nominal and ordinal scale, in addition it is based upon predetermined equal intervals.

• Examples: achievement tests, aptitude tests, and intelligence tests …

• Interval scale, however, do not have a true zero point.

• If an IQ test produces scores ranging from 0 to 200, a score of 0 does not indicate the absence of intelligence, nor does a score of 200 dedicate possession of the ultimate intelligence.

• We cannot say that a person scoring 90 knows twice as much as a person scoring 45

•   Ratio: A ratio scale represents the highest, most precise, level of measurement. 

•  It has a meaningful, true zero point.  

• Examples: height, weight, time, distance, and speed …

EXPERIMENTAL DESIGN

Outline

1. Definition of Experimental Design

2. Considerations in Design Selection

3. Experimental Design Terminology

4. Describing Experimental Designs

5. Basic Experimental Designs

1. What is Experimental Design?

• Experimental design is a planned interference in the natural order of events by the researcher. He does something more than carefully observe what is occurring. This emphasis on experiment reflects the higher regard generally given to information so derived

• The importance of experimental design also stems from the quest for inference about causes or relationships as opposed to simply description. Researchers are rarely satisfied to simply describe the events they observe some form of experimental design is ordinarily required

• The kinds of planned manipulation and observation called experimental design entail:– selecting or assigning subjects to experimental

units – selecting or assigning units for specific

treatments or conditions of the experiment (experimental manipulation

– specifying the order or arrangement of the treatment or treatments

– specifying the sequence of observations or measurements to be taken

2. Considerations in Design Selection

• The selection of a specific type of design depends primarily on both the nature and the extent of the information we want to obtain

• Two ways of checking potential designs:

ask yourself

– What questions will this design answer

– What is the relative information gain/cost

picture

3. Experimental Design Terminology

• The group in an experiment which receives the specified treatment is called the Treatment Group or the experimental group. However, the term Control Group refers to another group assigned to the experiment, but not for the purpose of being exposed to the treatment.

• A variable refers to almost anything under the sun. Two types: constants + variables

• Extraneous variables (external to the experiment) are variables that may influence or affect the results of the treatment on the subject.

• Level refers to the degree or intensity of a factor

• Randomness refers to the property of completely chance events that are not predictable (except in the sense that they are random).

• Ex post factor refers to causal inferences drawn “after the fact”

• Variance refers to the variability of any event

• The inside logic of an experiment is referred to as internal validity

• External validity, on the other hand, refers to the proposed interpretation of the results of the study

• Blocks usually refers to categories of subjects with a treatment group

• The Hawthorne Effect refers to the behavior of interest being caused by subject being in the center of the experimental stage, e.g., having a great deal of attention focused on them

• The study is termed a blind experiment when the subject does not know whether he or she is receiving the treatment or a placebo

• The study is termed double blind when neither the subject nor the person administering the treatment/placebo knows what is being administered knows either.

• Six major classes of information with which an experimental designer must cope:

– post -treatment behavior or physical measurement [P1]

– pre-treatment behavior or physical measurement [P2]

– internal threats to validity [I]

– comparable groups [C]

– experiment errors [E]

– relationship to treatment [R]

• Post-Treatment Behavior or Physical Measurement :

• In a typical experiment, this is the data, the class of information of primary interest

• Pre-Treatment Behavior or Physical Measurement:

• Information concerning pre-treatment behavior or condition requires come observation, a test, or measurement, to be administered before the experimental manipulation. Without such observations, the design itself will not answer any questions about the subjects before the experimental conditions have been introduced.

• Such information, however, may be accrued from general knowledge or other studies. Direct acquisition of this information adds to the cost of an experiment. Furthermore, it may have a confounding effect, that is, sometimes the pre-treatment observation or measurement influences the subsequent behavior of the subject. When it is over, it may not be clear whether the behavior was due to the treatment, the pre-treatment observation or measurement, or both.

• Internal Threats to Validity:• This class of information refers to some

rival hypothesis that threatens clear interpretation of the experiment.

• Comparable Groups:• This class of information, available only

when two or more experimental units or groups of subjects are used, deals with whether the subjects in the different units were about the same in relevant attributes before the treatment, and during the treatment, except for the treatment condition itself

• Experiment Errors:• Experiment error refers to some unwanted

side effect of the experiment itself which may be producing effect rather than the treatment

• Relationship to Treatment:

• This class of information deals with the possible interaction of the treatment effects with: different kinds of subjects, other treatments, different factors within a complicated treatment, different degrees of intensity, repeated applications or continuation of the treatment, and different sequences or orders of the treatment or several treatments

4. Describing Experimental Designs

ACTIVITY LETTER(S)Selection of the group or experimental unit GP

Random assignment to a group R

Blocking subjects, or other variables, into sets BK

Administering a treatment to a group T

Observing (measuring) results O

The following letters will be used to describe the various experimental design activities:

5. Basic Experimental Designs

• Eleven commonly used experimental designs:1. One-Shot 2. One-Group, Pre-Post 3. Static Group 4. Random Group 5. Pre-Post Randomized Group 6. Solomon Four Group 7. Randomized Block 8. Factorial 9. One-Shot Repeated Measures 10. Randomized Groups Repeated Measures 11. Latin Square

• One-Shot • The One-Shot is a design in which a group

of subjects are administered a treatment and then measured (or observed).

• Example

Ridgeway, G., Pierce, G.L., Braga, A.A., Tita, G., Wintemute, G., and Roberts, W. (2008).Strategies for Disrupting Illegal Firearms Markets: A Case Study of Los Angeles. Santa Monica, CA: Rand.

This report details research and a program development effort to understand the nature of illegal gun markets operating in Los Angeles, California. The primary goal of this project was to determine whether a data-driven, problem-solving approach could yield new interventions aimed at disrupting the workings of local, illegal gun markets serving criminals, gang members, and juveniles in Los Angeles.

• One-Group, Pre-Post • In this design, one group is given a pre-

treatment measurement or observation, the experimental treatment, and a post-treatment measurement or observation. The post-treatment measures are compared with their pre-treatment measures.

• Static Group • In this design, two intact groups are used,

but only one of them is given the experimental treatment. At the end of the treatment, both groups are observed or measured to see if there is a difference between them as a result of the treatment

• Random Group • This design is similar to the Static Group

design except than an attempt is made to insure similarity of the groups before treatment begins. Since it is difficult to have exactly similar subjects in each of two groups (unless you separate identical twins), the design works toward a guarantee of comparability between groups by assigning subjects to groups at random. If the researcher does this there is likely to be reasonable comparability between the two groups

• Pre-Post Randomized Group • This design adds a pre-test to the previous

design as a check on the degree of comparability of the control and experimental groups before the treatment is given

• Solomon Four Group • The Solomon Four Group design attempts

to control for the possible "sensitizing" effects of the pre-test or measurement by adding two groups who have not been a part of the pre-test or pre-measurement process.

• Randomized Block • This design is of particular value when the

experimenter wishes to determine the effect of a treatment on different types of subjects within a group

• Factorial • As you saw above in the blocking design,

the subjects were assigned to different groups on the basis of some of their own characteristics such as age, weight, or some other physical characteristic. Sometimes we wish to assign different variations of the treatment as well, and the procedure is similar

• One-Shot Repeated Measures • This design, or variations of it, is used to

assess the effects of a treatment with the same group or the same individual over a period of time. A measure, or observation is made more than once to assess the effects of the treatment

• Randomized Groups Repeated Measures

• The Randomized Groups Repeated Measures design is a variant of the previous design in which two or more experimental methods are compared and repeatedly measured or observed.

• Latin Square • A researcher may wish to use several

different treatments in the same experiment, for example the relative effects of an assortment of perhaps three or more drugs in combination in which the sequence of administration may produce different results

The Question of External Validity

• Questions of a different sort than we have faced arise from our need to generalize from a limited set of observations. No one is interested in observations than in no way extend beyond this particular restricted set of data. Generalizability depends on whether the observed behavior measurement [O] is representative of the people, the surrounding conditions and the treatments to which we now wish to extend it

• Classes of questions include:1. Did some of the early procedure in the

research affect the subjects so that their later measurements were, in part a result of that?

2. Were the subjects themselves a representative sample of the general population of people to which it is desired to extend the research findings?

3. Was there something in the research or setting that would cause or influence the measurement of the variable of interest?

4. Was the treatment accompanied by any personal interaction that may be somewhat peculiar to the research or to the subjects or the experimenter involved?

• The important thing is to clarify where the results of your observations may be legitimately extended and where they can not yet be legitimately extended. Helpful in this regard is a comprehensive description of the demographic characteristics of the subjects of the research and a complete and comprehensive description of the methodology used so that the reader of the research can judge for himself or herself whether the results can be generalized to his or her situation.