factor analysis and structural equation modelling · factor analysis and structural equation...

Factor Analysis and Structural equation

modelling

Herman Adèr

Previously:Department Clinical Epidemiology and Biostatistics,

VU University medical center, Amsterdam

Stavanger July 4–13, 2006

Herman Adèr 1

Overview

1 E ←→ C

2 EFA

3 CFA

4 SEM

Exploratory ←→ Confirmatory methods

Exploratory Factor Analysis

Confirmatory Factor Analysis

Structural Equation Modelling

Herman Adèr 2

Exploratory and confirmatory methods

Exploratory factor analysis

Confirmatory factor analysis

Structural equation modelling

References

Part VI

Factor analysis and Structural equation modelling

Herman Adèr Factor Analysis and SEM 3





References

Detective ←→ Judge






References

Characterization of Tukey (1977)

Exploratory data analysis is detective in character.

Confirmatory data analysis is judicial or quasi-judicial in

character. . . Unless the detective finds the clues, judge or jury

has nothing to consider. Unless exploratory data analysis

uncovers indications, usually quantitative ones, there is likely to

be nothing for confirmatory data analysis to consider.

On the other hand:

Exploratory data analysis can never be the whole story, but

nothing else can serve as the foundation stone – the first step.






References

Questions and answers

Three steps

Determining the meaningful factors

Rotation

Interpretation of the factor structure

F = M∪U ∪ E ,

of which M indicate the meaningful factors, U so-called

‘unique’ factors (factors on which only one item loads) and E

error factors.

Questions we try to settle using EFA

1 How many meaningful dimensions are present?

2 What is the structure of those dimensions?






References


Three steps


Rotation


�

�

determinemeaningful factors

�� rotate

��

��determine factor meaning

�� unrotated

��

��Exploratory Factor Analysis

Items

k meaningful factors

final factor solution

factor structure

factor structure

Subject

Data set:

D =

{ }

?

?

?

Three steps

1 Determine the meaningful factors

2 Rotate

3 Interpret the factor structure






References


Three steps


Rotation


determine meaningful factors:��

��scree plot

��

��total variance explained more than 50%�

�

�

�

select factors withintelligible

loading patternsand name them�

��communalities ≈ reliabilities

��

��eigenvalues larger than 1

�� doubtful�

�

�

GoF changesignificant

between solutions






References


Three steps


Rotation


CES-D: A multidimensional scale to assess Depression

FACTOR

/VARIABLES cesd1 cesd2 cesd3 cesd4 cesd5

cesd6 cesd7 cesd8 cesd9 cesd10



/MISSING LISTWISE

/ANALYSIS cesd1 cesd2 cesd3 cesd4 cesd5




/PRINT all

/FORMAT BLANK(.30)

/PLOT EIGEN

/CRITERIA MINEIGEN(0) ITERATE(100) factors(3)

/EXTRACTION pc

/ROTATION norotate.






References


Three steps


Rotation


Scree Plot

Component Number

20

19

18

17

16

15

14

13

12

11

10

9

8

7

6

5

4

3

2

1

Eig

envalu

e

10

8

6

4

2

0






References


Three steps


Rotation


8.516 42.578 42.578

1.775 8.876 51.454

1.229 6.147 57.600

1.040 5.201 62.802

.944 4.719 67.520

.767 3.836 71.356

.738 3.689 75.045

.684 3.419 78.465

.606 3.030 81.495

.577 2.886 84.381

.508 2.542 86.923

.442 2.211 89.134

.426 2.129 91.263

.339 1.697 92.960

.318 1.592 94.552

.286 1.429 95.981

.274 1.371 97.352

.202 1.010 98.362

.178 .891 99.253

.149 .747 100.000

Component1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Total % of Variance Cumulative %

Initial Eigenvalues

Extraction Method: Principal Component Analysis.






References


Three steps


Rotation


Interpretation of the unrotated factor structure

.525 .372 -.373

.757 -.318

.802 .389

-.573 .538

.687 .336

.689

.726 .349

-.635 .531

.721 -.353

.672 .398

.623 .415

-.690 .489

.599 -.513

.710

.432 .490 -.627

-.758 .383

.657 .500

.763 .349

.482 -.447 -.376

.784 .359

CESD1

CESD2

CESD3

CESD4

CESD5

CESD6

CESD7

CESD8

CESD9

CESD10

CESD11

CESD12

CESD13

CESD14

CESD15

CESD16

CESD17

CESD18

CESD19

CESD20

1 2 3 4 5 6

Component







References


Three steps


Rotation


Communalities: four factors and all factors

Communalities

1.000 .418

1.000 .751

1.000 .798

1.000 .683

1.000 .492

1.000 .526

1.000 .533

1.000 .763

1.000 .695

1.000 .543

1.000 .597

1.000 .734

1.000 .655

1.000 .546

1.000 .442

1.000 .721

1.000 .699

1.000 .709

1.000 .631

1.000 .624

CESD1

CESD2

CESD3

CESD4

CESD5

CESD6

CESD7

CESD8

CESD9

CESD10

CESD11

CESD12

CESD13

CESD14

CESD15

CESD16

CESD17

CESD18

CESD19

CESD20

Initial Extraction


Communalities

1.000 .966

1.000 .970

1.000 .868

1.000 .817

1.000 .919

1.000 .912

1.000 .830

1.000 .834

1.000 .812

1.000 .883

1.000 .896

1.000 .841

1.000 .861

1.000 .829

1.000 .961

1.000 .829

1.000 .828

1.000 .793

1.000 .963

1.000 .773

CESD1

CESD2

CESD3

CESD4

CESD5

CESD6

CESD7

CESD8

CESD9

CESD10

CESD11

CESD12

CESD13

CESD14

CESD15

CESD16

CESD17

CESD18

CESD19

CESD20

Initial Extraction







References


Three steps


Rotation


Communalities: two and three factors

Communalities

1.000 .414

1.000 7.631E-02

1.000 .795

1.000 .617

1.000 .481

1.000 .489

1.000 .531

1.000 .686

1.000 .544

1.000 .539

1.000 .399

1.000 .716

1.000 .389

1.000 .540

1.000 .427

1.000 .721

1.000 .434

1.000 .587

1.000 .291

1.000 .615

CESD1

CESD2

CESD3

CESD4

CESD5

CESD6

CESD7

CESD8

CESD9

CESD10

CESD11

CESD12

CESD13

CESD14

CESD15

CESD16

CESD17

CESD18

CESD19

CESD20

Initial Extraction


Communalities

1.000 .418

1.000 .650

1.000 .796

1.000 .639

1.000 .491

1.000 .517

1.000 .532

1.000 .720

1.000 .669

1.000 .541

1.000 .571

1.000 .733

1.000 .392

1.000 .546

1.000 .437

1.000 .721

1.000 .449

1.000 .587

1.000 .490

1.000 .621

CESD1

CESD2

CESD3

CESD4

CESD5

CESD6

CESD7

CESD8

CESD9

CESD10

CESD11

CESD12

CESD13

CESD14

CESD15

CESD16

CESD17

CESD18

CESD19

CESD20

Initial Extraction







References


Three steps


Rotation


Goodness-of-fit criterium

# Facts GoF χ2 Df χ

2 change Df χ2

α=.05

1 444.399 1702 278.177 151 166.222 19 30.1443 218.422 133 59.755 18 28.8694 170.845 116 47.577 17 27.5875 126.361 100 44.484 16 26.296

As one can see by comparing the 0.05 threshold χ2 in the last column






References


Three steps


Rotation


Unrotated Rotated

Factor Factor

Item I II III I II III

CESD01 + + +

CESD02 + +

CESD03 + + +

CESD04 − + −

CESD05 + + +

CESD06 + + +

CESD07 + + +

CESD08 − + −

CESD09 + − + +

CESD10 + +

CESD11 + + + +

CESD12 − + −

CESD13 + +

CESD14 + +

CESD15 + + +

CESD16 − + −

CESD17 + + +

CESD18 + + +

CESD19 + +

CESD20 + + +






References


Three steps


Rotation


Interpretation of table I

Unrotated Rotated

Factor Factor


CESD01 + + +

CESD02 + +

CESD03 + + +

CESD04 − + −

CESD05 + + +

CESD06 + + +

CESD07 + + +

CESD08 − + −

CESD09 + − + +

CESD10 + +

CESD11 + + + +

CESD12 − + −

CESD13 + +

CESD14 + +

CESD15 + + +

CESD16 − + −

CESD17 + + +

CESD18 + + +

CESD19 + +

CESD20 + + +

All items with high positive loadings on

the unrotated Factor I still load on this

factor after rotation, but some of them

(5, 6, 7, 9, 17, 18 and 20) also load on

factor II. This makes that Factor I is

now separated into two sublists, of

which the above could be called the

pure depression scale while (1, 3, 10,

11, 13, 14, 15, 19) includes a sublist

indicating less severe symptoms of

depression.






References


Three steps


Rotation


Interpretation of table II

Unrotated Rotated

Factor Factor


CESD01 + + +

CESD02 + +

CESD03 + + +

CESD04 − + −

CESD05 + + +

CESD06 + + +

CESD07 + + +

CESD08 − + −

CESD09 + − + +

CESD10 + +

CESD11 + + + +

CESD12 − + −

CESD13 + +

CESD14 + +

CESD15 + + +

CESD16 − + −

CESD17 + + +

CESD18 + + +

CESD19 + +

CESD20 + + +

Factor II now also contains all the

items that loaded negatively on Factor

I before (4, 8, 12 and 16). We called

this a factor that measures:

general attitude towards life.






References


Three steps


Rotation


Interpretation of table III

Unrotated Rotated

Factor Factor


CESD01 + + +

CESD02 + +

CESD03 + + +

CESD04 − + −

CESD05 + + +

CESD06 + + +

CESD07 + + +

CESD08 − + −

CESD09 + − + +

CESD10 + +

CESD11 + + + +

CESD12 − + −

CESD13 + +

CESD14 + +

CESD15 + + +

CESD16 − + −

CESD17 + + +

CESD18 + + +

CESD19 + +

CESD20 + + +

After rotation, Factor III now contains

only item 2 (‘I did not feel like eating:

my appetite was poor’) and item 11

(‘My sleep was restless’), suggesting

that it is a factor that has to do with

physical aspects of depression.






References

Example of EQS input


Conclusions of EFA and CFA combined


/TITLE Cesd;

/SPECIFICATIONS

VARIABLES=6; CASES= 200;

DATAFILE=’cesd.ess’; MATRIX=RAW;

ME = ML;

/EQUATIONS

V1 = 2 * F1 + E1;

V2 = 2 * F1 + E2;

V3 = 2 * F2 + E3;

V4 = 2 * F2 + E4;

V5 = 2 * F3 + E5;

V6 = 2 * F3 + E6;

F1 = 2 * F2 + 2*F3+ D1;

/VARIANCES

E1 to E10 = 0.2*;

D1, D2, D3 = 0.2*;

/COVARIANCES

F2,F3 = .5*;

/END






References




CES-D: Confirmatory factor analysis

Conclusions of the CFA

The varimax solution is not fully confirmed: item 5, 6, 7, 18

and 20 have negligible coefficients on the second factor.

The assumption of a orthogonal factor structure is

unfounded: The correlations between the factors are

(F1, F2) = 0.241; (F1, F3) = 0.382 and (F2, F3) = −0.025.






References





1 The first factor is a general depression factor (item 1, 3, 5,

6, 7, 10, 13, 14, 15, 18, 19 and 20)

2 The second factor contains the positively formulated items

(4, 8, 9, 12, 16, 17). It represents a general attitude

towards life.

3 The third factor contains items that have to do with

physical aspects of depression.






References

Example: Post-traumatic stress disorder

Example of a postulated structure

Draw pictures!






References

Postulated structure

Structure found






References

Recommended literature

Factor analysis: Friendly (1995)

(http://www.psych.yorku.ca/lab/psy6140/fa/factorbi.htm)

Principal component analysis: Jackson (1991)

Structural equation modelling: Bollen (1989), Kaplan (2000)


http://www.psych.yorku.ca/lab/psy6140/fa/factorbi.htm





References

Summary






References

Summary

In factor analysis the contrast between exploratory and

confirmatory approaches is quite clear: the available

techniques are different although related.

Exploratory factor analysis can well be performed with

principal component analysis, possibly combined with

Maximum likelihood factor analysis. PCA is first used to

determine the meaningful factors, MLFA for the final factor

structure.

Structural Equation Modelling makes it possible to analyze

a research problem that has been represented as a

diagram.






References

Bollen, K. A. (1989). Structural equations with latent variables. New

York: John Wiley and Sons.

Friendly, M. (1995). Annotated Factor Analysis Bibliography.

Jackson, J. E. (1991). A user’s guide to principal components. New

York: Wiley.

Kaplan, D. (2000). Structural Equation Modeling. Foundations and

Extensions. Thousand Oaks London New Delhi: Sage

Publications.

Tukey, J. W. (1977). Exploratory data analysis. Reading, MA:

Addison Wesley.


factor analysis and structural equation modelling · factor analysis and structural equation...

Documents