1

Factor Analysis

• There are two main types of factor analysis:

1.Confirmatory Analysis, and

2.Exploratory Factor Analysis

– In here, we only consider the second type

(to p2)

2

Factor Analysis (cont.)

1. Introduction

2. Model representation

3. Data type

4. Steps to perform (to p12)

(to p11)

(to p8)

(to p3)

3

Introduction

Exploratory Factor Analysis

– Definition– Objective– Assumption

(to p2)

(to p7)

(to p6)

(to p4)

4

Definition

• Analyze the structure of the interrelationship (correction) among a large set of decision variables to determine whether the information can be summarized into smaller set of factors

• that is decision variables that are corrected with one another but largely independent to others are combined into factors

• example (to p5)

(to p3)

5

example• For example, a retail store manager wants to know the

store performance in order to develop an action plan. He identifies 80 variables to identify the store performance. However, developing an action plan with 80 specific variables will becoming troublesome.

• So, can the manager summarize those 80 specific variables into smaller set of factors?

• Yes! He/she can adopt the exploration factor analysis to identify the underlying dimension of those variables, and then group those variables that are interrelated to each others --- statistically, that is

(to p4)

6

Objective

• To summarize the information contained in the number of decision variables into smaller set of factors subjected to its minimum loss of information

• it provides as a tool to better interpret the results of observations when a large number of decision variables is grouped into smaller set of factors

(to p3)

7

Assumption

• Given that we have a correction matrix for all decision variable, a general guideline is that a correction which has a value greater than 0.3 in a correlation matrix among some decision variables is considered as significant

• Note that: later, we will examine different criteria are to be applied to control to those corrections

(to p3)

8

Model representation

– The Basic model representation of exploratory factor analysis is:

R=FFTwhere,

– R = Correlation Matrix

– F = Factor Matrix

– FT = Factor Matrix Transpose

• example(to p9)

(to p3)

9

example• For Example,

Correlation Matrix A B C D

A -.953 -.055 -.130

B -.953 -.091 -.036

C -.055 -.091 .990

D -.130 -.036 .990

Factor MatrixFactor I Factor II

A -.400 .900

B .251 .947

C .932 .348

D .956 .286

Eigenvalue 2 1.91(to p10)

10

R= -.400 .900 -.400 .251 .932 .956

.251 -.947 .900 -.947 .348 .286

.932 .348

.956 .286

• Because the factor matrix multiply the factor matrix transpose, it detects the one to one correspondent correlation. Then the correlation matrix is formed which is a critical element of factor anlaysis.

(to p8)

11

Data type

• Variables for exploratory factor analysis should be represented in a metric form. When the variables in non-metric form, data transformation should be adopted (Hair et al, 1998).

(to p2)

12

Steps to perform

• Five Simple steps to follow:1. Testing assumption

2. Selecting proper sample sizes

3. Extracting factors

4. Rotating factors

5. Refining and labeling factors

Example,

Other reference: see attached files of 3620factor and 3620factor21 and

Vincent S. Lai, “Intraorganizational Communication with Intranets”, Communications of the ACM, July 2001/ Vol 44. No. 7, pp 95-100

(to p13)

(to p23)

(to p14)

(to p15)

(to p18)

(to p22)

13

1. Testing assumption

• Two tests:a. Barlett test of sphericity– A statistical test for the presence of correlations among

the variables. It determines if the correlation matrix has significant correlation to some of decision variables

b. K.M.O. measure of sampling adequacy– measure calculated both for entire correlation matrix evaluating

the appropriateness of applying exploratory factor analysis. This value should be greater than 0.5

– (Kaiser-Meyer-Olkin)

(to p12)

14

Selecting proper sample sizes

• Regarding the sample size, Hair et al (1998) indicate that the sample fewer than 50 observation is questionable. As a general rule, the sample size should be 10 times or greater than number of variables, for example, there are 10 variables, the sample size should be at least 100 observations.

• While Tabachnick and Fidell (1996) show the general guideline of sample size regardless to the number of variables when adopting factor analysis. 50 observations as very poor, 100 as poor, 200 as fair, 300 as good, 500 as very good, and 1000 as excellent.

Tabachnick, B.G. and Fidell, L.S. (1996), Using Multivariate Statistics, Third edition, HarperCollins College Publishers

(to p12)

15

3. Extracting factors

• Two common methods – 1. Principal component analysis

• transform the original set of variables into a smaller set of linear combinations that account for most of the variance of the original set

– 2. Common factor analysis• transforms the original set of variables into a smaller set of

factors to which their variances are common among the original factors

– (to p16)

16

The result is the same for both methods, thus in practice we just randomlypick one and Principal component analysis

(to p17)

17

• Criteria for extraction (ie to determine No. of factors)

– there are several criteria to extract factors: Latent Root Criterion, A Prior Criterion, Percentage of Variance Criterion, Scree Test Criterion.

– The most commonly used technique is the latent root criterion. The rationale for the latent root criterion is that any individual factor should account for the variance of at least a single variable if it is to be retained for interpretation. Each variable contributes a value of one to the total Eigenvalue. Thus, only the factor having Eigenvalue greater than one is considered significant

(to p12)

18

Rotating factors• When the factors are extracted, factor loading is obtained.

Factor loadings are the correlation of each variable and the factor. When rotating the factors, the variance has been redistributed so that the factor loading pattern and percentage of variance for each of the factors is difference.

• The objectives of rotating is to redistribute the variance from earlier factors to later ones to achieve a simple, theoretically more meaningful factor pattern, and make the result easily to be interpreted. There are three rotation methods (Hair et al, 1998): 1) Quartimax, 2) Varimax, 3) Equimax.

(to p19)

19

1.The quartimax rotation is to simplify the rows of a factor matrix, i.e. focus on rotating the initial factor so that a variable loads high on one factor and as low as possible on all other factor.

2. The varimax rotation is to simplify the columns of the factor matrix. With this approach, the maximum possible simplification is reached of there are only 1s and 0s in a column.

3.The equimax rotation is a compromise between the qurtimax and varimax.

In practice, the first two are the most common one to apply.(to p20)

20

Factor Loadings

• After the rotation, redistribution of factor loading is obtained. Hair et al (1998) provide a guidelines for indentifying significant factor loadings based on sample size which shows in the Table 2.

(to p21)

21(to p12)

22

Refining and labeling factors• Refining processes:

– it can be based on reliability, construct validity, convergent validity that we discussed

• Labeling factors– Finally, when a factor solution has been obtained, ie all

variables have significantly loaded onto a factor, label or give a name to each factor that makes up by their variables Here, variables with higher loadings are considered more important and have greater influence on the name or label the factor

(to p12)

23

Example

(to p24)

24

Step 1

(to p25)

25

Step 2

• Assumed that the sample size is 103 observations which is 10 times greater than the number of variables (Hair et al, 1998).

• The sample size is thus quite good for the exploratory factor analysis.

(to p26)

26

Step 3

(to p27)

27

Step 4

(to p28)

28

Step 5

(to p12)