ETEC 668 Quantitative Research in Educational Technology

Dr. Seungoh PaekJanuary 15, 2014

Instructors

Seungoh Paek (spaek@hawaii.edu) Yahna Kawaa (yahna@hawaii.edu) Youxin Zhang (youxin@hawaii.edu)

Tonight’s Agenda

Introductions Course Overview

Review the Syllabus and Schedule

Review Content, Grading, and Objectives

Introduction to Quantitative Research in Educational Technology

Ice Breaker

ETEC 668

Activity

Find a partner- Someone you don’t know well

Ask them questions

Introduce them to the rest of the class

Questions

Partners Name? Where is your partner from? What is your partner’s educational

background?- Where did they go to school?- What did they study?

Why is your partner taking this class? What is something special about your

partner?

Course Overview

Overview of ETEC 668

What are we doing this semester?- Explore quantitative research in educational

technology/Education How are we doing this?

- Research Process- Statistics (Theory)- SPSS (Practice)

Course Description

This course will provide– the foundational concepts and techniques of

quantitative research analysis– with an emphasis on educational technology

research. The course will cover – descriptive statistics– inferential statistics

Statistical Analysis

Frequency distributions Mean and variance Linear correlation Simple regression Chi-square test t-test Analysis of Variance (ANOVA)

Course Objectives

At the end of the course, students will be able to1. describe the different types of quantitative ed-

tech research methodologies;2. identify an area of interest that the student

wants to gather and use data for research;3. locate and critically evaluate and/or build on

previous research in area of interest;

Course Objectives

At the end of the course, students will be able to4. determine the relationship between the

research question or problem and the selection of a methodology;

5. determine data needs for research;6. access secondary data that is available about

an issue or identify an appropriate method or methods of data gathering to secure primary data;

Course Objectives

At the end of the course, students will be able to

6. conduct and interpret basic statistical analyses using SPSS;

7. produce a research paper for submission to a conference or journal publication

Course StructureSTATISTICS ETEC Research

Introduction to Statistics Introduction to Quantitative Research

Statistical Procedures Research Problem

Descriptive Statistics Literature Review

Frequency, Mean, Variance, etc. Research Methods

Inferential Statistics Research Design

Correlations Data Collection

Linear Regression Data Analysis

t-test Report Results

Analysis of Variance (ANOVA) Discussion & Presentation

Course Assignments and Grading

The course has one major research paper as well as three smaller assignments to build skills.

Course Assignments and Grading

The course has one major research paper as well as three smaller assignments to build skills.

1. Research Paper (250pt)2. Group Review (50pt) 3. Research Presentation (100pt)

Total 400 pt

Course Assignments and Grading

The course has one major research paper as well as three smaller assignments to build skills.

1. Research article critique workshop (100pt)2. Review of professional journal for paper submission

(100pt)3. Akamai consulting scenario (200pt)

Total 400 pt

ETEC 668 – The Big Picture

1

• Readings• Additional resources• In-Class activities

2

• Research article critique• Review of a professional journal for paper submission• Akamai Consulting Scenario (4 Tasks)

3

• Research Paper (literature review & research proposal)• Group Reviews• Group Presentation

Individual & Group WorkIndividual Work (40%) Points

Class attendance & ParticipationNote: Participation will be assessed according to contribution to group discussion, meeting deadlines for postings, and quality of contributions and postings in Laulima site)

200

Research article critique workshop 100

Review of a professional journal for paper submission 100

Group Work (60%)  

Group Project – Akamai Consulting Scenario (4 Tasks) 200

Group Project – Research Paper (literature review & research proposal) 250

Group Reviews 50

Group Presentation 100

Total points 1000

Individual & Group WorkIndividual Work (40%) Points

Class attendance & ParticipationNote: Participation will be assessed according to contribution to group discussion, meeting deadlines for postings, and quality of contributions and postings in Laulima site)

200

Research article critique workshop 100

Review of a professional journal for paper submission 100

Group Work (60%)  

Group Project – Akamai Consulting Scenario (4 Tasks) 200

Group Project – Research Paper (literature review & research proposal) 250

Group Reviews 50

Group Presentation 100

Total points 1000

Grading

Grade A B C D F

+ 100+ 86-89 76-79 66-69

59 & below0 95-100 83-85 73-75 63-65

- 90-94 80-82 70-72 60-62

Grades should not be the focus of your assignments/projects!

Required TextbookStatistics for People Who (Think They) Hate Statistics

4th Edition(Paperback or Kindle)

5th Edition

Required TextStatistics for People Who (Think They) Hate Statistics

4th Edition(Paperback or Kindle)

5th Edition

Recommended Textbook

IBM SPSS for introductory statistics: Use and interpretation

IBM SPSS for intermediate statistics: Use and interpretation

Statistical Software

PSPP – A free alternative for IBM SPSS Statistics

IBM SPSS Statistics

IBM SPSS Statistics Grad Pack 21.0 BASE ($68.99)

IBM SPSS Statistics Grad Pack 21.0 STANDARD ($87.99)

Questions?

Introduction to Quantitative Research

What is Research?

Studious inquiry or examination; especially :  investigation or experimentation aimed at the discovery and interpretation of facts, revision of accepted theories or laws in the light of new facts, or practical application of such new or revised theories or laws

~ Merriam-Webster

The systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions

~ Google

What is Research?

Studious inquiry or examination; especially :  investigation or experimentation aimed at the discovery and interpretation of facts, revision of accepted theories or laws in the light of new facts, or practical application of such new or revised theories or laws

~ Merriam-Webster

The systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions

~ Google

Research = Problem Solving

Suppose when you flip the light switch and the lamp does not come on.

FACTPREDICTION

(HYPOTHESIS)

Suppose when you flip the light switch and the lamp does not come on.

FACTPREDICTION

(HYPOTHESIS)

Lamp does not work.

Suppose when you flip the light switch and the lamp does not come on.

FACTPREDICTION

(HYPOTHESIS)

Lamp does not work. New bulb will cause lamp to come on.

Suppose when you flip the light switch and the lamp does not come on.

FACTPREDICTION

(HYPOTHESIS)

Lamp does not work. New bulb will cause lamp to come on.

Replace bulb and light still does not come on.

Suppose when you flip the light switch and the lamp does not come on.

FACTPREDICTION

(HYPOTHESIS)

Lamp does not work. New bulb will cause lamp to come on.

Replace bulb and light still does not come on.

Notice lamp cord is not plugged fully into wall socket.

Suppose when you flip the light switch and the lamp does not come on.

FACTPREDICTION

(HYPOTHESIS)

Lamp does not work. New bulb will cause lamp to come on.

Replace bulb and light still does not come on.

Notice lamp cord is not plugged fully into wall socket.

Pushing in plug will make light work.

Suppose when you flip the light switch and the lamp does not come on.

FACTPREDICTION

(HYPOTHESIS)

Lamp does not work. New bulb will cause lamp to come on.

Replace bulb and light still does not come on.

Notice lamp cord is not plugged fully into wall socket.

Pushing in plug will make light work.

Lamp comes on.

What is Quantitative Research?

Quantitative research refers to the systematic empirical investigation of social phenomena via statistical, mathematical or numerical data or computational techniques.

~ Given (2008)

What is Quantitative Research?

Quantitative research refers to the systematic empirical investigation of social phenomena via statistical, mathematical or numerical data or computational techniques.

~ Given (2008)

Statistics is boring?

Sebastian Wernicke: Lies, damned lies and statistics (about TEDTalks)

Quantitative vs. Qualitative Research

Deductive vs. Inductive

Contrasting positivist and interpretive approaches

Positivist Interpretive Reality is “real” – exists independent

of human consciousness Human beings are rational creatures

governed by social laws Science is based on strict rules

based on universal causal laws Science is value free

Reality is in the minds of people Human beings are actors who create

social reality by assigning meaning systems to events

Science represents reality symbolically in a descriptive way

Science is not value free, value neutrality is neither necessary or possible

Contrasting positivist and interpretive approaches

Positivist Interpretive Reality is “real” – exists independent

of human consciousness Human beings are rational creatures

governed by social laws Science is based on strict rules

based on universal causal laws Science is value free

Reality is in the minds of people Human beings are actors who create

social reality by assigning meaning systems to events

Science represents reality symbolically in a descriptive way

Science is not value free, value neutrality is neither necessary or possible

Contrasting positivist and interpretive approaches

Positivist Interpretive Controlled setting for research Subject is an object in the research Research design fixed Researcher ‘outside’ Emphasis on reliability

Complex, real world setting for research

Subject is a participant in the research process

Research design evolving Researcher ‘inside’ Emphasis on validity

Theory building in quantitative and qualitative research

Quantitative QualitativeLogic of theory Deductive Inductive

Direction of theory building

Begins from theory Begins from observation of reality

Verification Takes place after theory building is complete

Data generation, analysis and theory verification concurrent

Concepts Designed before research begins

Begins with flexible concepts

Generalizations Inductive sample-to-population

Analytical or exemplar

Quantitative Research

Quant: what, where, and when of natural phenomena

– develop and employ mathematical models, theories and hypotheses pertaining to natural phenomena

– Involve large samples of subjects; deal with cause/effect– Associated with positivism: that objective truth can be

known with certainty, that it can be gained through rational methods

Qualitative Research

Qual: why and how of human behaviour– Work with a range of models, theories, pertaining to

human phenomena– Involve small groups of participants; interpretation &

reflection– Speech and texts, and their interpretation are very

important– People's accounts of their actions significant– Not Positivist: no objective truth; different

interpretations; no final certainty in knowledge

Group Activity

In groups of two, please spend 10 minutes discussing the pros and cons of Quantitative Research?

Advantages of Quantitative Research

Tradition and history implies rigor Indispensable in areas like user demographics,

issues of equity, patterns of use Numbers and statistics allows precise and exact

comparisons Generalization of findings

Limitations of Quantitative Research

Cannot capture complexity of communication over time

Reality cannot be defined objectively, it is interpreted by social interaction

Takes the subject outside of natural setting/tasks Difficult to apply outside of controlled

environments The quantitative model is the natural sciences, this

model may not be relevant for the social sciences

Advantages of qualitative research

Purpose is to understand rather than measure

Is holistic and accommodates the emergent properties of real-world situations

Places findings in context – historical, cultural, political

Limitations of Qualitative Research

Qual: Requires a different way of thinking to address issues like:– Reliability: repeatable with same/comparable

results– Validity: relationship between conditions and

results– Generalizability: historical and cultural

limitations

When is Qualitative Useful?

Qualitative research relies on imprecise and everyday notions of what is valid, etc.

But it does so reflexively; in a self-aware and theoretically-mediated manner.

Qualitative is useful in relationship to quantitative if:– The topic has been researched for a long time in the

same way– The topic is new to research– You would like in-depth information that may be

difficult to convey quantitatively

Types of Quantitative Educational Technology Research

Non-experimental– numeric description of trends– attitudes– Opinions

Experimental (treatment outcome)

Types of Non-Experimental Educational Technology Research

Correlational Research Survey Research Descriptive Research (qual & quant)

Types of Experimental Educational Technology Research

Random assignment of subjects (“Experimental”)

Nonrandomized & single-subject design (Quasi-experimental)

“Experimental” Designs

R X1 O (R = randomly assigned subjects; X = treatment)

R X2 O (O = observation/outcome)

Should be:– Replicable: repeat with the same results in another setting– Generalizable, representative– Cumulative: observations from earlier experiment used as a

basis for new one– Causal: establishes cause and effect (predictive)

Quasi-Experimental

X1 O (X = treatment)

X2 O (O = observation/outcome)

Randomness is approximated through pre-tests to ensure “equivalence”

Research Methodology Used in Distance Education Research

74.83%

5.96%6.63%

12.58%

(Berge & Mrozowski, 2001)

Questions/Comments?

The Language of Research Ethics

No harm may be done to participants Voluntary participation “Informed consent” Right to privacy:– Confidentiality – no identifying info– Anonymity - even to researcher

Institutional Review Board (IRB) aka Committee on Human Studies (CHS) http://www.hawaii.edu/irb/index.htm

What To do in Week 01

Post individual response to course expectations/research interest questions by Tuesday, January 21st.

Do the required readings for Week 02.Note: Readings are available for download on Laulima (Modules Week 02)