principal component and exploratory factor analysisprincipal component and exploratory factor...

Principal component and exploratory factor analysis

Hans Baumgartner

Penn State University


x1 x2 x3 x4 x5 x6 x7 x8

What’s the structure underlying

28 distinct covariances between

8 observed variables?


The principal component model

x1 x2 x3 x4 x5 x6 x7 x8

x1 xn. . . . . . . . . . . . . . .


How many components to retain?

▪ Eigenvalue greater than one rule

(Kaiser’s rule)

▪ Scree test (Cattell)

▪ Parallel analysis (Horn)

▪ Minimum Average Partial (MAP) test

(Velicer)

▪ Theoretical considerations


Factor rotation

▪ Orthogonal rotation

e.g., Varimax

▪ Oblique rotation

e.g., Promax


SAS specification

DATA coupon;

INFILE 'd:\ipss\factor.dat';

INPUT id aa1t1 aa2t1 aa3t1 aa4t1 aa1t2 aa2t2 aa3t2 aa4t2;

ods graphics on;

TITLE 'PRINCIPAL COMPONENT ANALYSIS, TWO COMPONENTS'; run;

PROC FACTOR NFACTORS=2 ROTATE=PROMAX PLOTS=(scree preloadings loadings);

VAR aa1t1 aa2t1 aa3t1 aa4t1 aa1t2 aa2t2 aa3t2 aa4t2;

run;

ods graphics off;


The FACTOR Procedure

Initial Factor Method: Principal Components

Prior Communality Estimates: ONE

Eigenvalues of the Correlation Matrix: Total = 8 Average = 1

Eigenvalue Difference Proportion Cumulative

1 5.56628614 4.99193128 0.6958 0.6958

2 0.57435486 0.05222193 0.0718 0.7676

3 0.52213293 0.14851325 0.0653 0.8328

4 0.37361969 0.06185008 0.0467 0.8795

5 0.31176961 0.05897629 0.0390 0.9185

6 0.25279331 0.03126256 0.0316 0.9501

7 0.22153076 0.04401806 0.0277 0.9778

8 0.17751270 0.0222 1.0000

2 factors will be retained by the NFACTOR criterion.


Parallel Analysis:

Principal Components

Specifications for this Run:Ncases 250Nvars 8Ndatsets 100Percent 95

Random Data Eigenvalues

Root Means Prcntyle

1.000000 1.263893 1.3444632.000000 1.163970 1.2300473.000000 1.088389 1.1332354.000000 1.026321 1.0601315.000000 0.967248 1.0070766.000000 0.904000 0.9466257.000000 0.834617 0.8784278.000000 0.751562 0.807438


Velicer's Minimum Average Partial (MAP) Test:

Eigenvalues

5.56600.57820.52100.37270.30760.25700.22050.1772

Average Partial Correlations

squared power4

0.0000 0.4280 0.18921.0000 0.0440 0.00412.0000 0.0662 0.01503.0000 0.1058 0.03974.0000 0.1692 0.09125.0000 0.2582 0.12246.0000 0.4869 0.36537.0000 1.0000 1.0000

The smallest average squared partial correlation is0.0440

The smallest average 4rth power partial correlation is0.0041

The Number of Components According to the Original (1976) MAP Test is1.0000


Factor PatternFactor1 Factor2

aa1t1 0.81547 0.35121aa2t1 0.83341 0.34182aa3t1 0.77536 0.01958aa4t1 0.84339 0.23736aa1t2 0.86051 -0.03867aa2t2 0.86274 -0.23905aa3t2 0.82149 -0.37671aa4t2 0.85709 -0.27731

Variance Explained by Each Factor

Factor1 Factor2

5.5662861 0.5743549

Final Communality Estimates: Total = 6.140641

aa1t1 aa2t1 aa3t1 aa4t10.78833603 0.81141733 0.60156808 0.76764683

aa1t2 aa2t2 aa3t2 aa4t20.74197084 0.80145580 0.81674966 0.81149643



Prerotation Method: Varimax

Rotated Factor Pattern

Factor1 Factor2

aa1t1 0.33675 0.82154

aa2t1 0.35614 0.82739

aa3t1 0.54017 0.55658

aa4t1 0.43637 0.75976

aa1t2 0.64176 0.57455

aa2t2 0.78357 0.43298

aa3t2 0.85044 0.30577

aa4t2 0.80631 0.40169


The FACTOR ProcedureRotation Method: Promax (power = 3)

Inter-Factor Correlations

Factor1 Factor2

Factor1 1.00000 0.67589Factor2 0.67589 1.00000

Rotated Factor Pattern (Standardized Regression Coefficients)

Factor1 Factor2

aa1t1 0.02338 0.87192aa2t1 0.04504 0.86973aa3t1 0.41062 0.43660aa4t1 0.17970 0.74463aa1t2 0.53035 0.40913aa2t2 0.77916 0.16017aa3t2 0.92612 -0.03361aa4t2 0.82327 0.10941


The exploratory factor model

d1 d2 d3 d4 d5 d6 d7 d8

x1 x2 x3 x4 x5 x6 x7 x8

x1 xn

q11d q22d q33d q44d q55d q66d q77d q88d

11

. . . . . . . . . . . . . . .


+

=

8

7

6

5

4

3

2

1

2

1

8281

7271

6261

5251

4241

3231

2221

1211

8

7

6

5

4

3

2

1

δ

δ

δ

δ

δ

δ

δ

δ

ξ

ξ

λλ

λλ

λλ

λλ

λλ

λλ

λλ

λλ

x

x

x

x

x

x

x

x

=

1

1

21 8811 .... qq

d Diag=

Two-factor model


SAS specification

TITLE 'PRINCIPAL FACTOR ANALYSIS, TWO FACTORS'; run;PROC FACTOR NFACTORS=2 ROTATE=PROMAX PRIORS=SMC;VAR aa1t1 aa2t1 aa3t1 aa4t1 aa1t2 aa2t2 aa3t2 aa4t2;

run;

TITLE 'EXPLORATORY MAXIMUM LIKELIHOOD FACTOR ANALYSIS, TWO FACTORS'; run;PROC FACTOR METHOD=ML NFACTORS=2 ROTATE=PROMAX SE COVER=.40 ALPHA=.1;VAR aa1t1 aa2t1 aa3t1 aa4t1 aa1t2 aa2t2 aa3t2 aa4t2;

run;


The FACTOR ProcedureInitial Factor Method: Principal Factors

Prior Communality Estimates: SMC

aa1t1 aa2t1 aa3t1 aa4t1

0.61896228 0.66565385 0.55622099 0.66929517

aa1t2 aa2t2 aa3t2 aa4t2

0.69849794 0.70624991 0.64787954 0.70324282

Eigenvalues of the Reduced Correlation Matrix:Total = 5.26600251 Average = 0.65825031

Eigenvalue Difference Proportion Cumulative

1 5.22802782 4.98990454 0.9928 0.99282 0.23812328 0.10148438 0.0452 1.03803 0.13663890 0.10807443 0.0259 1.06404 0.02856447 0.05139663 0.0054 1.06945 -.02283216 0.05684943 -0.0043 1.06506 -.07968159 0.02020619 -0.0151 1.04997 -.09988779 0.06306265 -0.0190 1.03098 -.16295043 -0.0309 1.0000


Parallel Analysis:

Principal Axis / Common Factor Analysis

Specifications for this Run:Ncases 250Nvars 8Ndatsets 100Percent 95

Random Data Eigenvalues

Root Means Prcntyle

1.000000 0.297504 0.3896522.000000 0.193068 0.2688263.000000 0.113040 0.1589434.000000 0.048229 0.0872085.000000 -0.011975 0.0373076.000000 -0.073090 -0.0345227.000000 -0.137988 -0.1000638.000000 -0.213994 -0.163157



Rotation Method: Promax (power = 3)

Inter-Factor Correlations

Factor1 Factor2

Factor1 1.00000 0.74679Factor2 0.74679 1.00000

Rotated Factor Pattern (Standardized Regression Coefficients)

Factor1 Factor2

aa1t1 0.15925 0.68215aa2t1 0.12325 0.74482aa3t1 0.35117 0.43689aa4t1 0.21434 0.66378aa1t2 0.51948 0.37873aa2t2 0.71140 0.18860aa3t2 0.72635 0.12017aa4t2 0.74590 0.14729


EXPLORATORY MAXIMUM LIKELIHOOD FACTOR ANALYSIS, TWO FACTORS

Significance Tests Based on 250 Observations

Pr >Test DF Chi-Square ChiSq

H0: No common factors 28 1534.0584



Rotation Method: Promax (power = 3)

Inter-Factor CorrelationsWith 90% confidence limits

Estimate/StdErr/LowerCL/UpperCL

Factor1 Factor2

Factor1 1.00000 0.728480.00000 0.01570. 0.70161. 0.75328

Factor2 0.72848 1.000000.01570 0.000000.70161 .0.75328 .


Rotated Factor Pattern (Standardized Regression Coefficients)With 90% confidence limits; Cover |*| = 0.4?

Estimate/StdErr/LowerCL/UpperCL/Coverage Display

Factor1 Factor2

aa1t1 0.19857 0.646280.08017 0.077720.06389 0.500040.32616 0.75668

0[]* 0*[]

aa2t1 0.06195 0.827830.05207 0.05846-0.02394 0.704230.14693 0.90272

[0]* 0*[]

principal component and exploratory factor analysisprincipal component and exploratory factor...

Documents