2011 an introduction to confirmatory factor analysis cfa

8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA

1/23

8/02/2011

1

An Introduction to Confirmatory Factor

Analysis (CFA) and Structural

Equation Modeling (SEM)Gavin T L Brown, PhD

Presentation to Research Development Office, Continuing Professional DevelopmentProgramme, HKIEd, 10 February 2011

What is a CFA or SEM model?

A theoretically informed simplification of

the com lexities of realit created to testor generate hypotheses about how

various constructs are related


2/23

8/02/2011

2

Theory We have theories that explain the way things are (not just

descriptions)

eory an a a are n er- w ne

We see phenomena and seek to explain them with theories

We have theories and seek to test them with phenomena

Theories Knowledge

but theories that do not explain phenomena are certainly false[Knowledge--Popper]

CFA/SEM is situated in hypothetico-deductive orabductive approaches to meaning

Models

Everything is connected to everything in the real world

Its messy and hard to make sense of

in a model we select for theoretical reasons the important

connections that we THINK explain most of what is going on

in the phenomenon of interest

It is not the real thing, but a simplification

The arrangement of the connections between and among

variables of interest constitute testable expressions ofour theories about how things go together


3/23

8/02/2011

3

Prediction, Causation, Association CFA/SEM models assume linear (i.e., correlations and

regressions) relationships (paths) exist among constructs.

or examp e:

(AB) [2 things are correlated]

(AB) C [2 correlated things jointly influence a 3rd

thing]

(A + B) C [2 things separately and/or jointly influence a 3rd

thing]

A B C [1 thing influences a 2nd which influences a 3rd] And so on. [moderation, mediation, complex inter-

relationships]

CFA/SEM Involves Mathematical Testing of

Models

A sophisticated correlational-causal mathematical testingof a model against a data set

ow c ose are t ey oes t e mo e t t e ata

Models are rejected if they do NOT have close fit to the data

the data cant be wrongits the reality we are trying to model

Models are NOT accepted if they have close fit to the data

They are NOT YET DISCONFIRMEDPopper

Multiple models can fit equally well the same data

Fit could be attributable to chance factors in the data we collected


4/23

8/02/2011

4

CFA/SEM: Extending Latent Trait Theory Observed manifest behaviours

e.g., test scores, attitude item responses, observed frequenciesof behaviours, etc.

Are shaped and influenced by invisible (LATENT) sharedcauses. For example, Answers to items [manifest observed] on a test are caused (in

part) by INTELLIGENCE [latent unobserved] traits

Student responses to Browns Conceptions of Assessmentinventory are shaped in part by the hypothesised beliefs that:

ASSESSMENT IS FOR IMPROVEMENT; ASSESSMENT IS IRRRELEVANT;

ASSESSMENT HAS AFFECTIVE/SOCIAL BENEFITS

ASSESSMENT REFLECTS EXTERNAL CAUSES

Latent Trait Theory Multiple manifest indicators are required to have stable

estimation of the latent traits existence, strength, anddirection

ence, ac or ana ys s expec s o ems per ac or Hence, test scores rely on 5 to 30 test questions

WHY?

CHANCE.ERROR.DEFICIENCIES IN STIMULI Observed behaviour is not perfectly controlled or reflective of our

TRUE intelligence, attitude, etc. I chose B but I meant A; I chose response 3 but I meant 4

I want 3.4 but I had to choose 3 or 4

Hence, all values are ESTIMATES A range of most likely values exists Multiple indicators reduces error/chance effects


5/23

8/02/2011

5

Interpreting Output Values in CFA/SEM

CFA and SEM modelsuse conce ts of ecausation and prediction linear regression

Changes in XXX cause alinear change (increase ordecrease) in YYY

Formula: Y= m*X + b

Yv

ariabl

X

bintercept

m=slope [standardisedbeta = a proportion ofstandard deviation]

b=intercept [starting pointof equation; represents allthe unknown stuff]

var a e

Interpretations:1. For every 1 SD change in X, youwill get m*SD change in Y.2. This relationship explainsx% ofvariance in Y

Looking Under the Hood: Components of

CFA and SEM models

Variables

Manifest [observed behaviours,,

Latent [unobserved, explanatory, ovals]

Residual [unobserved, unexplained, ovals]

Manifest variables are predicted by both Latent traits andresiduals

Goal to have large proportion of variance in manifest explainedby latent rather than residual disturbances


6/23

8/02/2011

6

Looking Under the Hood: Components of

CFA and SEM models Paths

Fixed: equations require SEED values to solve; 1 is the.

predicted manifest variables with a fixed value. All other valuesare estimated relative to the seed value.

Free: All other paths are allowed to be estimated freely based

on the data provided to the model; they may be stronger thanthe fixed path, but better to make the strongest path in a

factor the fixed path.

Zero: Paths not required by the model are forced to be non-existent. This contrasts to EFA where all paths have somefreely estimated value.

Example of Path Values

EFA indicated Grades wasthe strongest value

Grades e12

1

1

Thus, seed value on path

Residual terms exist and

have seed value of 1 becausethey are equal to each other

Note: manifest variablesONLY have paths from the

Well-being

Evaluative

Ticks e13

Praise e14

Stickers e15

Answers e16

1

1

1

conceptua tra t Zero between each other

If 2 or more factors, itemsshould have ZERO paths toother factors


7/23

8/02/2011

7

Estimation Maximum likelihood

The parameter values in the data set (a sample) are the most likelyvalues in the population (not present, but to which we wish togeneralise)

Hence, procedure attempts to maximise the input values (means,standard deviations, covariances) when estimating the solution

Hence, it matters that the sample reflects the population and issufficiently large that parameters are likely to apply to population

N500; if 100) and large

number of manifest variables

o s a poor es , no w s an ng ve emen o ec ons y

some researchers


8/23

8/02/2011

8

Evaluating Results: Which Fit indices &

What Values?

Goodness of Fit Badness of fit

Decision p of2/df CFI

RMSEA SRMR*gamma a

Good >.05 >.95 .90


9/23

8/02/2011

9

Distinguishing CFA from SEM CFA = measurement model of a construct

CFA models can have multiple dimensions and complex

An achievement score can be hierarchical

total consists of surface AND deep cognitive processes

An attitude or opinion can be multi-correlated

Total consists of correlations between 3 or more related dimensions

SEM = structural model of paths between constructs

Attitudes towards X influence performance on Y

Attitude towards X is related to attitude towards Y

Example: CFA + SEM(Brown & Hirschfeld, 2008)

CFA: Measurement Model-4 correlated factorsNote. Accurate measurement models are also needed for

reading score, year, sex, & ethnicity

Structural model:multiple predictorsof performance

Note.If measurements of each construct are NOT robust, do NOTuse them for anything!!!


10/23

8/02/2011

10

Linear Models are Recursive(Brown et al., 2009)

CFA/SEM assume models are recursive

origins

ave a eg nn ng an an en w c are no e same

NOT circular

endings

How to Test Reciprocal Models?

Make it longitudinal

Time 1 Time 2

1 1 1 2 2 2

Use 2 different methods of measuring construct A

AM1BCA

M2

These approaches honour the reciprocal effects in theorywithout invalidating the linear regression equations

beyond todays talk


11/23

8/02/2011

11

Interpreting a Model Statistical significance of paths

The weights & directions of each path

The proportion of variance explained (the effect size)

Evaluating Results

Statistically significant paths

The strength of the path should exceed what might occur by

option to remove such paths or indicate as ns

If p>.05 pathnot stat sig

Note. Fixedpaths have no

probability.


12/23

8/02/2011

12

Evaluating Results Variance explained (SMC)

Equivalent to R2

.19MQQ44e42 .44

f2 =.19/.81=.23 (medium)

e ec s ze -

Small: .02 to .14

Medium: .15 to .34

Large: >.35

(Cohen, 1992)

.08

Evaluation

.34MQQ23e37

.29MQQ8e38

.17MQQ25e39

.23MQQ5e40

.MQQ63e41

.58

-.54

.41

.48.37

-.28

Note. SMC = Beta squaredBalanced not explained is in theresidual (goal small residuals, sotarget >.50)

Developing a Model

Evidence from theory

Evidence from Exploratory Factor Analysis

Evidence from Regression Analysis


13/23

8/02/2011

13

The Role of Theory in Designing Models to

Test My research questions:

Do conceptions of assessment influence performance?

eoret ca ramewor :

Icek Ajzen: Reasoned or Planned Behaviour

Beliefs & Intentions influence Behaviour & Outcomes

Beliefs are inter-correlated

Outcomes

Criterion of effectiveness

EFA to CFAStatement 1 2 3 4 5 6 7

29.Assessmentfostersstudents'character. 0.556 0.023 0.11 0.154 0.097 0.047 0.072

22.Assessmentcultivatesstudents'positiveattitudestowardslife. 0.685 0.049 0.02 0.074 0.065 0.059 0.008

20.Assessmentisusedtoprovokestudentstobeinterestedinlearning. 0.591 0.04 0.084 0.066 0.059 0.02 0.048

14.Assessmenthelpsstudentssucceedinauthentic/realworldexperiences. 0.446 0.085 0.105 0.216 0.092 0.14 0.124

13.Assessmentensuresstudentspayattentionduringclass. 0.533 0.066 0.131 0.012 0.007 0.22 0.224

34.Assessmentmeasuresstudents'higherorderthinkingskills. 0.509 0.167 0.007 0.03 0.176 0.11 0.077

Note. Non-zero values on otherfactors, but all weak.

27.Assessmentallowsdifferentstudentstogetdifferentinstruction. 0.487 0.017 0.102 0.128 0.011 0.15 0.213

24.Assessmentstimulatesstudentstothink. 0.678 0.061 0.074 0.008 0.001 0.12 0.105

49.Assessmentforcesteacherstoteachinawayagainsttheirbeliefs. 0.083 0.458 0.03 0.121 0.071 0.19 0.106

31.Assessmentinterfereswithteaching. 0.102 0.54 0.08 0.06 0.086 0.13 0.066

10. Assessmenthaslittleimpactonteaching. 0.134 0.384 0.19 0.034 0.062 0.01 0.067

26.Assessmentisanimpreciseprocess. 0.004 0.629 0.034 0.008 0.021 0.057 0.09423.Assessmentresultsarefiled&ignored. 0.017 0.646 0.01 0.057 0.02 0.022 0.056

45.Teachersconductassessmentsbutmakelittleuseoftheresults. 0.019 0.493 0.045 0.003 0.193 0.008 0.012

EFA steps

1. Run MLE, oblimin allowing eigenvalues>1.00

NB. This is the SPSSpattern matrix of

2. Remove items with cross-loadings >.303. Remove items with no loading >.304. Remove items which did not logically fit their factor5. Remove items that seem literally repetitive in content6. Remove factors that are repetitive in meaning to earlier factorsRESULT

Items kept fit conceptually and have strong unique loadings on 1 factor

regress ons


14/23


15/23

8/02/2011

15

Additional Causes of Broken Models To be examined in the next session

negative error variances

covar ance ma r ces a are no pos ve e n e

Recommended solutions to be discussed as well

Testing Multiple Models

analyst job is to identify which model fits best and makessense in terms of what we already know and believe

Instrument: Teachers Conceptions of Feedback

Theoretically expected 10 factors

Data: independent samples from Louisiana and NewZealand

comparison of 2 groups, re-analysis of NZ sample

Results: multiple structures and many possible validmodels could fit; better model found in a series of studies


16/23


17/23

8/02/2011

17

What is Confirmation in CFA? Most studies follow this process

An inventory is developed using theory

The validit of the uestionnaire ma be ex lored

EFA identifies a plausible model within a data set

CFA tests the fit of the EFA model to the data

CFA refines the EFA model with the same data

This process is better considered Restrictive analysis not CFA

True confirmation comes when an existing model is

No EFA needed

Just run the model, does it fit?

If NOT, then EFA must begin again

True Confirmatory Study

TCoA: 9 factors in 4 factor

New Sample: Cyprus primary &secondary teachers

Tested:

CFA NZ Model (original &

simplified);

EFA Cyprus Model;

joint hierarchical model

Result: Model D fits bothgroups satisfactorily Brown & Michaelides, 2011


18/23

8/02/2011

18

Developing a structural model (SEM) Identify possible structural paths between important

variables in measurement models

Regression analysis

If theory suggests causal relations use regressions

If no idea, look at correlations

Note. In SEM, a correlation and a regression will have the

.theoretically if there is cause or temporal precedent

Why Use SEM instead of Multiple

Regressions?

Limitations of multiple regressions

only 1 construct can be predicted at a time; its not

The joint correlations among predictor constructs is not takeninto account

The paths from origin to terminus cannot be accounted for

Thus, SEM is better able to test for statistical significance

of regressions

Provided N is large enough


19/23

8/02/2011

19

Example: SCoA to Definitions of Assessment(Brown, Irving, et al., 2009)

Hypothesis

beliefs about the nature and purpose of assessment predict the

Multiple regression analysis

2 latent traits were predicted by 8 latent traits in 2 separateanalyses; only 4 were statistically significant

Interactive-Informal assessment practices (R2=.02):

Class Environment, = .12, p = .01,

Assessment is ignored (Ignore), = .10, p = .06. The Teacher-Controlled assessment practices (R2=.08):

Teacher Improves Student Learning, = .14, p = .02, and

Personal Enjoyment, = -.14, p = .003.

Example: SCoA to Definitions of Assessment

SEM

Beta values much

regression values

Proportionvarianceexplained much

higher thanregression


20/23

8/02/2011

20

CFA/SEM: Belief

to Belief(Brown, 2009)

CFA: A change in each latent trait predicts a large SEM: Only statistically

change in responses for each contributing variable.Range = .38 to .88; proportion variance explained = 2

, hence 13% to 77%. Relatively low proportion ofunexplained. This is required for good measurement in

CFA.Conclusion: Latent Traits predict responses on

Observed Variables.

significant paths kept inmodel.

Consider This Model

Theory Self-regulation involves increasing adaptive beliefs & practices

and decreasin malada tive ones

Inventory development Multiple studies, multiple versions, multiple samples

Include measure of academic performance N=520; #manifest variables=46; 9 factors; 3 measurement

models; 2 models are hierarchical.

Fit: 2= 2146.58 d=970 2/d=2.21 =.13gamma hat=.91; RMSEA=.048; SRMR=.064; SMC=.20

What beliefs are adaptive or maladaptive to performancein mathematics? Does it matter?


21/23

8/02/2011

21

Summary

Theories are used to devise models that attempt toexplain how changes occur in various constructs and in

CFA/SEM mathematical equations are based on linearregressions to identify the strength of relationships

among Latent, Manifest, and Unexplained variables

CFA/SEM models are used to establish validity ofmeasurements and answer substantive questions

CFA/SEM are powerful because of simultaneousproperties and tighter specification of model


22/23

8/02/2011

22

Summary Same techniques used to validate measurement models

and explore relations between constructs

equ res arge an sop s ca e ma ema ca ormu ae

Is powerful to test and generate hypotheses

Logically depends on the notion of causation andprediction

Can be done relatively easily with modern software but nd

2 weeks

References: Studies used Brown, G. T. L., Harris, L. R., & Harnett, J. (2010, July). Teachers conceptions of feedback:

Results from a national sample of New Zealand teachers. Paper presented at theInternational Test Commission biannual conference, Hong Kong.

Brown, G. T. L., Harris, L. R., OQuinn, C., & Lane, K. E. (2011, April). New Zealand andous ana rac cng eac ers conce ons o ee ac : m ac o ssessmen o earnng

versus Assessment for Learning policies?Paper accepted for presentation to theClassroom Assessment SIG at the annual meeting of the American EducationalResearch Association, New Orleans, LA.

Brown, G. T. L., & Hirschfeld, G. H. F. (2008). Students conceptions of assessment:Links to outcomes.Assessment in Education: Principles, Policy and Practice, 15(1), 3-17.

Brown, G. T. L., Irving, S. E., Peterson, E. R., & Hirschfeld, G. H. F. (2009). Use ofinteractive-informal assessment practices: New Zealand secondary studentsconceptions of assessment. Learning & Instruction, 19(2), 97-111.

Brown G. T. L. & Michaelides M. 2011 . Ecolo ical rationalit in teachers. . . . .conceptions of assessment across samples from Cyprus and New Zealand. EuropeanJournal of Psychology of Education. doi:10.1007/s10212-010-0052-3

Brown, G. T. L., Peterson, E. R., & Irving, S. E. (2009). Self-regulatory beliefs aboutassessment predict mathematics achievement. In D. M. McInerney, G. T. L. Brown, &G. A. D. Liem (Eds.) Student perspectives on assessment: What students can tell us aboutassessment for learning(pp. 159-186). Charlotte, NC: Information Age Publishing.


23/23

8/02/2011

References: Authorities Ajzen, I. (2005).Attitudes, personality and behavior(2nd ed.). New York:

Open University Press.

Byrne, B. M. (2001). Structural Equation Modeling with AMOS: BasicConcepts, Applications, and Programming. Mahwah, NJ: LEA.

Fan, X., & Sivo, S. A. (2007). Sensitivity of fit indices to modelmisspecification and model types.Multivariate Behavioral Research,42(3), 509529.

Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules:Comment on hypothesis-testing approaches to setting cutoff valuesfor fit indexes and dangers in overgeneralizing Hu and Bentler's

(1999) findings. Structural Equation Modeling, 11(3), 320-341. Marsh, H. W., Hau, K.-T., Balla, J. R., & Grayson, D. (1998). Is more

ever too much? The number of indicators per factor in confirmatoryfactor analysis.Multivariate Behavioral Research, 33(2), 181-220.

Basic Readings on CFA/AMOS . (2007).AMOS. Taipei, Taiwan:.

Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratoryfactor analysis: Four recommendations for getting the most from

. , ,Available online: http://www.pareonline.net/pdf/v10n17.pdf.

Klem, L. (2000). Structural equation modeling. In L. G. Grimm & P. R.Yarnold (Eds.), Reading and Understanding More Multivariate Statistics

(pp. 227-260). Washington, DC: APA. Kline, P. (1994).An easy guide to factor analysis. London: Routledge.

Kim, J.-O., & Mueller, C. W. (1978). Factor Analysis: Statistical methodsand practical issues (Vol. 14). Thousand Oaks, CA: Sage

Thompson, B. (2000). Ten commandments of structural equationmodeling. In L. G. Grimm & P. R. Yarnold (Eds.), Reading andUnderstanding More Multivariate Statistics (pp. 261-283). Washington,DC: APA.

2011 an introduction to confirmatory factor analysis cfa

Documents