Data CollectionPopulationSampling

More reminders ..

Get your variables sorted by now Chapters 1, 2, 3,4 are not yours Strong remark Use of phrases from another article Citing secondary sources Block quotation and Page reference In text citation Colloquial – “In my honest opinion….” Connecting ideas – on the other hand,

moreover, in support of, This is supported by.., in contrast to, in addition, nonetheless….

Instructor assigned task (25%)

TASK : Design a research

Survey

Correlational

Experimental

Causal Comparative

Case study – Mixed method

Ethnography

TOPIC : 21st century learners

Group work

Group of 3 (max)

Study task description on i-Learn

Assignment/Projects

Instructor Assigned Task

Use Blendspace

Due 30th May 2015

Marks given for product and comments/discussions

Where we are now…Observation Background survey

Broad area of research

Literature review

Problem definition

Problem statement

Research questions

Theoretical framework

Variables clearly identified

Hypothesis generation

Research design

Methods

Data collection

Analysis

Interpretation

Deduction

Hypothesis substantiated?

RQ answered?

Lesson Objectives

Describe procedures

Obtaining permissions for data collection

Selecting participants for data collection

Identifying data options

Recording and administering data collection

Let’s move on to data collectionWhat is data?Collection of facts from which conclusions may be drawn

Key ideas

Who will you study (unit, sampling, sample size)?

What permissions will you need? (levels,MOE)

What information will you collect? (types of data, links to questions/variables)

What instrument(s) will you use? (selecting an instrument, scales of measurement, validity, reliability)

How will you administer the data collection? (standardization, ethical issues)

Who will you study?

Unit of analysis is the level (e.g. individual, family, school, school district) the data will be gathered.

There may be different units of analysis

one for the dependent variable

one for the independent variable

Procedures for Collecting Quantitative Data

(1) Obtain permissions

secure permissions obtain informed consent

from participants

Obtaining Permissions

Institutional or organizational (e.g. school district)

Site-specific (e.g. secondary school)

Individual participants or parents

Campus approval (e.g. university or college)

Obtaining Informed Consent from EPRD,MOE

Obtain Approval via State Education Department

Have participants sign an informed consent form

Procedures for Collecting Quantitative Data

(2) Select participants

specify a population and sample

use probability and non-probability sampling

choose a sample size

Procedures for Collecting Quantitative Data

(3) Identify data options

specify variables operationalize variables select scales of

measurement choose types of data

measures

Procedures for Collecting Quantitative Data

(4) Record and administer data collection

locate or develop an instrument

obtain reliable and valid data develop administrative

procedures for data collection

Population and Sampling

POPULATION and SAMPLING

A population is a group of individuals that comprise the same characteristics

A sample is a sub-group of the target population that the researcher plans to study

Select Participants: Specify a Population and Sample

Samples

for the purpose about making generalizations about the target population (quantitative research).

samples are only estimates the difference between the

sample estimate and the true population is the “sampling error.”

Manageable research population

Populations and Samples

Sample

TargetPopulation

Population• All science teachers

in secondary schools in Kuantan

• College students in all community colleges

• Adult educators in all faculties of education

Better?All sec. school biology teachers in KuantanStudents in one community collegeAdult educators in faculties of education in the East Malaysia

Probability and Non-Probability Sampling

Probability sampling is the selection of individuals from the population so that they are representative of the population

Non-probability sampling is the selection of participants because they are available, convenient, or represent some characteristic the investigator wants to study.

Types of Sampling

Sampling Strategies

Probability/Random Sampling

Non-Probability/Purposeful Sampling

Simple Stratified ClusterRandom Sampling Sampling Sampling

Convenience SnowballSampling Sampling

Differences Between Random and Purposeful Sampling

Random “Quantitative” SamplingSelect Representative individuals

To generalize from sample to population

To make claims about the populationTo build/test “theories” that explain the population

Purposeful “Qualitative” SamplingSelect people/sites who can best help us understand our phenomenon

To develop detailed understanding

That might be “useful: informationThat might help people “learn” about the phenomenonThat might give voice to “silenced” people

Types of Probability Sampling

Simple Random: selecting a sample from the population so all in the population have an equal chance of being selected

Systematic: choosing every “nth” individual or site in the population until the desired sample size is achieved

Types of Probability Sampling Stratified sampling: stratifying the population on a characteristic (e.g. gender), then sampling from each stratum.

BoysN=6000

GirlsN=3000

Population(N=9000)

.66 of pop. 200

.33 of pop 100

Sample = 300

Types of Probability Sampling

Cluster Sampling:

Selects groups, not individuals

All members in the groups have similar characteristics

Useful when the population is large or spread over a wide geographical area

Example: Cluster Sampling

Population : All primary school teachers in Klang valley (5000)

The desired sample : 400 Cluster: School No. of primary schools in Klang valley: 150 Average number of teachers per school : 40 Number of cluster : 400/ 40 10 out of 150 schools are randomly selected All teachers in the selected schools make up

the sample

Try one

You want to study the resilience of UiTM students.

Population?

Sampling?

Select Participants: Choose a Sample Size

Select a sample size as large as possible from the individuals available

Select a sufficient number of participants for the statistical tests you will use

Calculate the sample size using a sample size formula

Select Participants: Choose a Sample Size

A rough estimate:

15 participants in each grp in an expt

30 participants for a correlational study

350 individuals for a survey study but depend of several factors

Calculating sample size

Krejcie and Morgan (1970)

EXAMPLE

Gender difference in vocational interest of post matriculation at UiTM

Independent variable: Dependent variable: Research Design: Population: Sample size: Instrument:

EXAMPLE

Gender difference in vocational interest of post matriculation at UiTMIndependent variable: GenderDependent variable: Vocational interestResearch Design: SurveyPopulation: Post matriculation studentsSample size: 350Instrument: Self developed

Calculating sample size

Use web calculator

http://www.raosoft.com/samplesize.html

http://survey.pearsonncs.com/sample-calc.htm

Non Probability Sampling

Types of Purposeful SamplingWhen Does Sampling Occur?

After Data Collection has startedWhat is the intent?

To developmanyperspectives

ExtremeCase

Sampling

To describeparticularlytroublesomeor enlighteningcases

TypicalSampling

To describe whatis “typical” to those unfamiliarwith the case

What is the intent?

To take advantageof whatever caseunfolds

OpportunisticSampling

To locatepeople orsites to study

SnowballSampling

To exploreconfirming ordisconfirmingcases

Confirming/Disconfirming

Sampling

MaximalVariationSampling

To generate a theory or concept

Theory or ConceptSampling

To describe somesub-group in depth

HomogenousSampling

Before Data Collection

One may sample..

Maximal variation

Most hardworking/ Highest achiever

Lowest achiever

Extreme case

??

Examples of Non-Probability Samples

Convenience Sampling: participants are selected because they are willing and available to be studied

Snowball Sampling: the researcher asks participants to identify other participants to become members of the sample.

Non Probability sampling example

Study delinquent behaviour during recess. I selected 1 school out of 4 - Interviewed all 4

principals and toured all 4 schools. I chose school to which I was given most access with fewest restrictions. Also school that reported the widest variations in delinquent behaviour during recess (very high to virtually no display).

Then sampled different locations with camera to find most varied activity and least self-conscious/guarded behavior. Where? Turned out to be behind the surau.

Later used snowball approach in choosing children to interview.

Your sample?

Proceed to collecting quantitative data

Collecting quantitative data

Flow of Activities in Collecting Data

1.Identify the variable2.Operationally define the

variable3.Locate data (measures,

observations, documents with questions and scales)

4.Collect data on instruments yielding numeric scores

Identify Data Options: Specify Variables

Independent Variables

Dependent Variables

Intervening

Moderating

Identify Data Options: Operationalize Variables

Operational Definition: The specification of how the variable will be defined and measured

typically based on the literature

often found in reports under “definition of terms”

Sometimes the researcher must construct it

Flow of Activities in Collecting Data

Identify the variable

Operationally define the variable

Locate data (measures, observations, documents with questions and scales)

Collect data on instruments yielding numeric scores

Self-efficacy for learning from others

Level of confidence that an individual can learn something by being taught by others

13 items on a self-efficacy attitudinal scale from Bergin (1989)

Scores of each item ranged from 0-10 with 10 being “completely confident.”

Flow of Activities Example

Flow of Activities in Collecting Data

Identify the variable

Operationally define the variable

Locate data (measures, observations, documents with questions and scales)

Collect data on instruments yielding numeric scores

Learning motivation of adult learners

Interest and level of engagement of an individual

----- items on a questionnaire developed

Scores of each item ranged from 1-5 with 5 being “most interested”

Flow of Activities Example

Scales of measurement

Identify Data Options: Select Scales of Measurement

Nominal (Categorical): categories that describe traits or characteristics

participants can check

Female Male

Ordinal: participants rank the order of a characteristic, trait or attribute

Identify Data Options: Select Scales of Measurement

Interval: provides “continuous” response possibilities to questions with assumed equal distance ; scale with no true zero

Discrete (SD ---------------------SA)

Metric (oC)

Ratio: a scale with a true zero and equal distances among units

PracticeIdentify the level of measurement

Measurement level? Age Religion Gender Income bracket Test scores CGPA Frequency of asking

questions Time spent on task Level of acceptance

(0 – never, 5 – all the time)

Activity ( 0 –not active, 5 – very

active)

Ratio ?

Nominal?

Ordinal?

Interval?

Measurement level? Age Religion Gender Income bracket Test scores CGPA Frequency of asking

questions Time spent on task Level of acceptance

(0 – never, 5 – all the time)

Activity ( 0 –not active, 5 – very

active)

Ratio Nominal Nominal Ordinal Ratio Ordinal Ratio Ratio Interval/

Ordinal

Interval/Ordinal

Identify Data Options: Choose Types of Data Measures

An instrument is a tool for measuring, observing, or documenting quantitative data

Types of Instruments Performance Measures (e.g. test performance) Attitudinal Measures (measures feelings

toward educational topics) Behavioral Measures (observations of

behavior) Factual Measures (documents, records)

Record and Administer Data Collection: Locate or Develop an Instrument

Develop your own instrument

Locate an existing instrument

Modify an existing instrument

Record and Administer Data Collection: Locate or Develop an Instrument

Strategies to use

Look in published journal articles Run an ERIC search and use the

term “instruments” and the topic of the study

Go to ERIC web site for Evaluation and Assessment

Examine guides to commercially available tests

Developing a questionnaire

Obtain Reliable and Valid Data

Reliability: individual scores from an instrument should be nearly the same or stable on repeated administrations of the instrument Bathroom scale

Reliability

Types of reliability

Test-retest (scores are stable over time)

Internal consistency (consistent scores across the instrument)

Cronbach coefficient alpha if items are scored as continuous variables (SA—SD)

Inter-rater reliability (similarity in observation of a behavior by two or more individuals)

Validity

Validity: the ability to draw meaningful and justifiable inferences from the scores about a sample or a population

Types of validity

Content (representative of all possible questions that could be asked)

Criterion-referenced (scores are a predictor of an outcome or criterion they are expected to predict

Construct (determination of the significance, meaning, purpose and use of the scores)

Let’ s look at a Rosenberg Self Esteem Scale

Collecting Qualitative Data

Key Ideas

Gaining site permission

Purposive sampling

Types of qualitative data

Protocols and Issues regarding administering and recording qualitative

Gaining Permission

Gain permission from Institutional Review Board

Gain permission from “gatekeepers” at the research site

Gatekeepers: individuals at the site who provide site access, helps researcher locate people and identifies places to study

The gatekeeper may require written permission about the project

Information for the gatekeeper Why their site was chosen

What time and resources are required What will be accomplished at the site What potential there is for your

presence to be disruptive What individuals at the site will gain

from the study How will you use and report the results

Types of data to Collect

Observations

Interviews

Documents

Audio-Visual Materials

Sources of Qualitative Data From People:

Interviews Surveys Focus Groups Participant

Observation (field notes)

From Things: Agency case records Miscellaneous

documents Historical Artifacts Media Published materials

ObservationInterviewsDocumentsAudio Visuals

Types of Data to Collect: Observations

An observation is the process if gathering first-hand information by observing people and places at a research site.

Observational roles

Participant observer

Non-Participant observer

Observational roles can be changed

Types of Data to Collect: Observations

Conduct multiple observations Record both descriptive and reflective

field notes during the observation Descriptive field notes describe the

events, activities and people Reflective field notes record

personal reflections that relate to their insights, hunches or broad themes that emerge

Administering and Recording Data: Observational Protocols

The header: essential information about the observation

Left column to record descriptive notes

Right column to record reflective notes

A picture of the site may be sketched

Types of Data to Collect: Interviews

Types of Interviews

Individual

Focus group

Telephone

e-mail

Structured, Unstructured, and Semi-Structured Interviews

Approach toData

Collection

Type ofResponse Options toQuestions

Types of Interviews

Leading to

Data

Quantitative Close-Ended

Structured/semi-structuredInterviews

Scores toanswers

Qualitative Open-Ended

UnstructuredInterviews

Transcriptionof words

Types of Data to Collect: Interviews

General open-ended questions are asked

allows the participant to create options for responding

participants can voice their experiences and perspectives

Information is recorded then transcribed for analysis

Administering and Recording Data: Interview Protocols

The header: essential information about the interview

Open-ended questions include

“ice-breaker”

ones that address major research questions

probes that clarify and elaborate

Closing comments thanking the participant

EXAMPLE: Semi structured Interview protocols

Name: Date: Time: Venue: Experience – please describe Questions

1. Is the method a valid approach to addressing the learning needs of adult learners? Please describe your thoughts of the method

2. What do you think are the barriers to the utilization of the method in UiTM?

3. How would you rate the quality of learning using the method?

4. What overall suggestions do you have for improving the method?

Types of Data to Collect: Documents

Public and private records Good source for text data You must obtain permission before

using documents Scan documents when possible

Types of Data to Collect: Audio-Visual Materials

Determine the material that can provide evidence to address your research questions

Determine if the material is available and obtain permission to use it

Check the accuracy and authenticity of the material if you do not record it yourself

Collect the data and organize it

Administering and Recording Data: Field Issues Time needed for data collection

Limit initial collection or one or two observations or interviews

Time is needed to establish a substantial data base

Administering and Recording Data: Field Issues Obtaining permission to use materials

Ethical issues Anonymity of participants Convey true purpose of study without

deception

Let’s gather data

Fill in the Qnaire Tabulate results Answer research Qs Qualitative data

??

NEXTQuantitative Data Analysis