2007 yngve lindvig, 2007 yngve lindvig, the learninglab norway, overview 15/01 13-15 –...
TRANSCRIPT
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Overview
15/01 13-15 – Introduction, Validity and Reliability.16/01 14-16 – What is science 1 Facts, observation,
induction and falsification.19/01 09-13 – What is science 2 Paradigms, programs,
against method, realism and anti-realism. (Rom 234) 22/01 13-15 – Introduction, qualitative research23/01 14-16 – Conducting qualitative research25/01 09-13 – Introduction Quantitative research29/01 13-15 – Quantitative research30/01 14-16 – Quantitative research01/02 09-13 – Quantitative measurement
Seminarrom U31 Helga Engs hus
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Data collectionDevelopment of:-Observation log-Interview guide-QuestionnaireImplementation
The process of empirical research
Generalised knowledge/theory/the Generalised knowledge/theory/the typical/constructed constructstypical/constructed constructs
Pre-understanding- Experience- Scientific theory- Subject theory- Method comprehension
Writing-Developing categories-Summary -Test-Back to the outset
Analysis-Confirming-Exploring-Structuring- 1 d- 2 d- N d
Reflection- Validity- Reliability- Researcher - object
Operationalising- Planning- Confronting the domain- Concept orientation- Test dialogue/interviews- Design
CommunicationCommunication
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Reliability and validity
• Reliability – random errors
• Validity - systematic errors
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Validity in social sciences
1. Internal validity
2. External validity
3. Construct validity
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Internal validity
To what extent does the research design permit us to reach casual conclusions about the effect of the independent variable on the dependent variable?
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Internal validity
The independent variable: Causes effect
The dependent variable: Is affected
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
External validity
To what extent can we generalize from the research sample and setting to the populations and settings specified in the research hypothesis?
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Construct validity
To what extent are the constructs of theoretical interest successfully operationalized in the research?
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Construct validity• Coheres between the theoretical construct
and the operationalization of the construct
Theoretical construct
O1 O3O2 systematic errors
random errors
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Construct validity
Construct Theoretical level
Empirical level
Operationalization of construct
Measurement of constructEmpirical domain
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Sampling
1. Random samples
2. Stratified random samples
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Qualitative research
1. Research question
2. Selecting subjects
3. Collecting of data
4. Analyze
5. Conceptual and theoretical work
6. Writing
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Participant observation
1. Active or passive
2. Complete participant to complete observer
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Interviewing
1. Unstructured or semi-structured
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Focus groups
1. Size
2. Organisation
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Structuring data in qualitative analyze
1. Research question 2. Theoretical sampling3. Collection of data4. Coding – looking to constructs and concepts5. Comparison – looking to categories6. Exploring relationships - looking to the hypothesis and the research
question7. Exploring new hypothesis 8. New sampling9. New collection10. Test of hypothesis
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Main models of administration of a survey
Survey
Structured interview Self-completion questionaire
Face-to face
TelephonePaper
CAPI Paper
CATI
Supervised
Postal
Internett
WebE-mail
Embedded Attached
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Structured interviewing
1. Standardized
2. Pre-coded and fixed choice
3. Goal to give each respondent the same interview stimulus
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Possible errors in Structured interviewing
1. Poorly worded question2. The way the question is asked by the interviewer3. Misunderstanding on the part of the interviewee4. Memory problems on the part of the interviewee5. The way the information is recorded by the
interviewer6. The way the information is processed, either
when the answers are coded or when data are entered into the computer
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Important in Structured interviewing
1. Clear instructions
2. Question order
3. The way we ask questions
4. The way we recording answers
5. Probing
6. Leaving the interview
7. Training
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Problems with Structured interviewing
1. Pre understanding
2. Characteristics of interviewers
3. Yea-saying or NO-saying
4. Culture
5. Meaning
6. Asymmetrical relation
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Self-completion questionnaire
1. No interviewer
2. Postal or web-based
3. Fewer open questions
4. Easy to follow design
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Advantages related to Self-completion questionnaires
1. Cheaper
2. Quicker
3. Absence og interviewer effects
4. No interviewer variability
5. Convenience for respondents
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Disadvantages related to Self-completion questionnaires
1. Cannot prompt2. Cannot probe3. Cannot ask possible boring questions4. Cannot ask to many open questions5. Questionnaire can be read as a whole6. Do not control who answers7. Cannot collect additional data8. Reading skills may influence the result9. Risk of missing data10. Lower response rate
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Improve response rate on Self-completion questionnaires
1. Good covering letter
2. Return postage
3. Follow up
4. Not to long
5. Clear instructions and design
6. Start with the most interesting
7. Money
8. Constructs that not provoke
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Asking questions
1. Open
2. Closed
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
General rules for Asking questions
1. Don’t forget your research questions
2. Focus on structures
3. Test the questions on your self and others
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Specific rules for Asking questions1. Avoid word as often or regularly, be specific
2. Avoid long questions
3. Avoid double-barred questions
4. Avoid very general questions
5. Avoid two questions in one
6. Avoid questions that include negatives
7. Avoid technical terms
8. Avoid questions the respondent don’t know something about
9. Avoid unbalanced categories
10. Don’t rely on the respondent memmory
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Ethic
1. What are the beneficial consequences of the study?
2. How can the informed consent of the participating subjects be obtained?
3. How can the confidentiality of the subjects be protected?
4. What is the structure behind the study? Financing, who want the study and who want what result?
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
What to know? Validity and Reliability ScalesDesignSamplingStarting an analyse in SPSSOrganizing and Graphing dataDistributions and Central TendenciesCrosstabulationCorrelationRegression analyzeSignificanceEffect sizeFactor analysisSum variablesHypothesis testingT-testStructural Equation Modeling
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Quasi-experimental and surveys designs
Design 1:
Group 1 x O1
------------------------------
Group 2 not-X O2
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Quasi-experimental and surveys designs
Design 2: Pretest-posttest
Group 1 O1 x O2
-------------------------------------------------
Group 2 O3 not-X O4
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Quasi-experimental and surveys designs
Example
High
Low
Post-test
Pre-test
Electric consumption for hot water
O1
O4
O3
O2 People who did not install solar panels
People who install solar panels
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Scales or levels of measurement
1. Nominal
2. Ordinal
3. Interval
4. Ratio
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Nominal
1. Different categories
2. No numerical meaning
Example:
Gender? Male female
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Ordinal
1. Different categories that can be ordered by rank
2. Arithmetic meaning (more or less)
Example:
Do you have autonomy?
Little moderate much
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Interval
1. Scale and equal quantities2. Numerical meaning
Example:
Is it Correct of USA to go to war against Irak?
Totally disagree 1 2 3 4 5 Totally agree
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Ratio
1. Same as interval an a true zero
Example:
How old are you? _______ years
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Starting an analyse in SPSS
Designing a questionnaire
Coding of questions
Entering data
Cleaning data
Describing variables
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Null and research hypotheses
Ho (null hypothesis) The population mean of group x is equal to the population mean of group y.
H1 (research hypothesis) The population mean of group x is not equal to the population mean of group y.
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Type 1 and type 2 errors
Type 1 Correct
Decision
Correct
Decision
Type 2
Population
Ho is true Ho is false
Reject Ho
Accept HoDec
isio
n
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
q3976 Do you have a good time with your fellow students?
5 ,5 ,5 ,5
36 3,6 3,6 4,1
240 23,9 24,0 28,1
720 71,6 71,9 100,0
1001 99,6 100,0
4 ,4
1005 100,0
1 Not in any class
2 In some classes
3 In many classes
4 In all casses
Total
Valid
SystemMissing
Total
Frequency Percent Valid PercentCumulative
Percent
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
q3971 Are you interested in learning at school?
4 ,4 ,4 ,4
216 21,5 21,5 21,9
478 47,6 47,7 69,6
305 30,3 30,4 100,0
1003 99,8 100,0
2 ,2
1005 100,0
1 Not in any class
2 In some classes
3 In many classes
4 In all casses
Total
Valid
SystemMissing
Total
Frequency Percent Valid PercentCumulative
Percent
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Statistics
1003 1001
2 4
3,08 3,67
3,00 4,00
3 4
,728 ,568
,529 ,322
Valid
Missing
N
Mean
Median
Mode
Std. Deviation
Variance
q3971 Areyou interestedin learning at
school?
q3976 Doyou have agood timewith your
fellowstudents?
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Correlations
1 ,395**
,000
1003 995
,395** 1
,000
995 996
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
q3971 Are youinterested inlearning at school?
q3974 Do youwork as well as youcan at school?
q3971 Areyou interestedin learning at
school?
q3974 Doyou workas well asyou can atschool?
Correlation is significant at the 0.01 level (2-tailed).**.
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Correlations
1 ,136**
,000
1003 996
,136** 1
,000
996 998
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
q3971 Are youinterested in learningat school?
q3978 Do you have agood time in breaksbetween classes?
q3971 Areyou interestedin learning at
school?
q3978 Doyou have agood timein breaksbetweenclasses?
Correlation is significant at the 0.01 level (2-tailed).**.
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Rotated Component Matrixa
,054 ,778
,034 ,779
,126 ,780
,789 ,136
,786 ,118
,744 ,022
-,441 -,010
q3971 Are youinterested in learning atschool?
q3974 Do you work aswell as you can atschool?
q3975 Do you enjoydoing schoolwork?
q3976 Do you have agood time with yourfellow students?
q3978 Do you have agood time in breaksbetween classes?
q3979 Do you havemany friends at school?
q3983 Do fellowstudents often harassyou?
1 2
Component
Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.
Rotation converged in 3 iterations.a.
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Chi-square = 121,121Degrees of freedom = 8Probability level = ,000
Model fit:
CFI=0,983
IFI=0,983
TLI=0,968
RMSEA=0,052
Model - students learning culture
Confirmatory Factor Analysis
Miljø
Motivasjon
sosial trivsel
tilhørighet
mobbing
interesse for læring
motiverende lærere
innsats og utholdenhet
f1
f2
f3
f4
f5
f6
,69
,89
-,30
,87
,53
,62
,73
-,45
,96
,49
,85
-,79
,32
Friends at school
School culture
School environment Harassment
School environment
Interest in learning
Motivating teachers
Work effort
Motivation
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
•Formulate a hypothesis, discuss the possibilities of errors and discuss the following T-test output:
Assume that your pre-understanding was that
there is a relation between gender and factor 2.
Group Statistics
672 2,8046 ,57332 ,02212
309 2,9385 ,52790 ,03003
q4051 SEX1 Male
2 Female
factor2N Mean Std. Deviation
Std. ErrorMean
t-test for Equality of Means
t dfSig. (2-tailed)
Mean Difference
Std. Error Differenc
e
95% Confidence Interval of the
Difference
Lower Upper
factor2 Equal variances assumed
-3,484 979 ,001 -,13395 ,03845 -,20940 -,05849
Equal variances not assumed
-3,591 645,532 ,000 -,13395 ,03730 -,20718 -,06071
2007 Yngve Lindvig, 2007 Yngve Lindvig, The Learninglab Norway, www.laeringslaben.no
Coefficientsa
2,996 ,064 46,988 ,000
-,126 ,054 -,075 -2,330 ,020
(Constant)
q3983 Do fellowstudents oftenharass you?
Model1
B Std. Error
UnstandardizedCoefficients
Beta
StandardizedCoefficients
t Sig.
Dependent Variable: factor2 Motivationa.
Coefficientsa
4,029 ,051 78,565 ,000
-,326 ,043 -,236 -7,512 ,000
(Constant)
q3983 Do fellowstudents oftenharass you?
Model1
B Std. Error
UnstandardizedCoefficients
Beta
StandardizedCoefficients
t Sig.
Dependent Variable: fector1 Friendsa.