multidimensionality multi-dimensionality & factor analysis
TRANSCRIPT
Multidimensionality
Multi-dimensionality &Factor analysis
Multidimensionality
Multidimensionality
• Adding dimensions: Multidimensionality
• Compressing dimensions: Factor analysis
• Intelligence testing and g
Multidimensionality
Adding new dimensions
• Most tests measure single dimensional constructs– We have already seen one case of dual-
dimensionality in IQ– It is easy to imagine extending into a third
dimension• e.g. Gould’s ‘football’, correlating growth of three
body parts
Multidimensionality
Adding new dimensions
• Mathematically, there is no need to stop at three dimensions
– Correlations, Euclidean distance, orthogonality [to be explained] etc. are all well-defined in higher dimensional spaces
• Informally, it is easy to imagine high dimensional spaces
– e.g. Think of the qualities that make up a good car, or a good apartment, or a good mate
Multidimensionality
Adding new dimensions
• Visually, it is not so difficult either…at least for a few more dimensions
– Imagine using color for the 4th dimension, and size of point for the fifth dimension [etc. etc.]
D1
D2 D3 = SizeD4 = Colour
Multidimensionality
What is a dimension?
• Dimensions are information = they are differences that make a difference
Multidimensionality
What is a factor?
• A factor is a dimension
Multidimensionality
Correlation and information
• What is the relation between a correlation and information/probability?– A significant correlation between x and y tells
us that x contains information about y– Another way of saying this is that x and y are
not independent, in the sense we used the term in discussing probability
Multidimensionality
Correlation and information
• However, note that the dependence may be roundabout or even spurious if it depends on relations between features that are very common or just accidental
• Examples:– In American towns and cities, the correlation between
the number of churches and the number of violent crimes is about 0.85: Why?
– Income and homelessness are positively correlated: Why?
– Outside temperature and rape rates are positively correlated: Why?
Multidimensionality
Relations between dimensions
• Any relation between any two dimensions (which means: any correlation) can be expressed in terms of two orthogonal (=right-angle) components– By definition, these two components that have nothing
to do with each other, each containing no ‘amount’ of the other dimension
– If you know something about one dimension, you know nothing about an orthogonal dimension
– If knowing something about one dimension does give you information about another: they are not orthogonal = they are correlated
Multidimensionality
Relating two dimensions
Dimension 1
Dim
ensi
on 2
Pure vertical component
Pure horizontal component
Multidimensionality
A concrete example
Picture arrangement score
Dig
it s
pan
scor
e Pure vertical component
Pure horizontal component
Multidimensionality
Pure dimensions
• When two dimensions are orthogonal, they are ‘pure’ dimensions, wholly separable from one another – No information about one is contained in the other
• By the same token, when two dimensions are not orthogonal, they are ‘contaminated’ by at least one common component– Information about one is contained in the other– i.e. Gould: 14 independent bone measurements reduce
to the single dimension of 'size'
Multidimensionality
Relating two dimensions
Dimension 1
Dim
ensi
on 2
Dimensions 1 and 2 here both contain a little bit of ‘horizontalness’ and a little bit of ‘verticality’
Multidimensionality
The need for orthogonality• Dimensions which are orthogonal are independent =
whatever happens on one dimension has no effect on what happens on the other dimension (or, equivalently, knowing the value on one dimension provides no information about value along the other dimension or they correlate with r = 0).– Example: Beauty and intelligence are orthogonal (if not,
the world sure is unfair!); height and weight are not orthogonal
• For this reason, the theoretical ‘true’ dimensionality of a thing (the number of things we need to know to know ~everything that thing) is the value of ~all orthogonal dimensions relevant to that thing
Multidimensionality
Information and orthogonality
Dimension 1
Dim
ensi
on 2
Here we know something about Dimension2 (the value of B) when we know the value along Dimension 1(the value of A)- precisely because the dimensions are not orthogonal.
A
B
Multidimensionality
Information and orthogonality
Dimension 1
Dim
ensi
on 2
A
B
This is mathematically equivalent to saying that the dimensions are correlated. For the curious, the cosine of the angle = r
Multidimensionality
Factor A
Fac
tor
B
Example: Four correlations of test results with factors
Multidimensionality
We rotate the dimensionshere to get 2 orthogonal dimensions.
Factor A'
Fact
or B
'
MultidimensionalityFactor A
Fac
tor
B
F1: General intelligence (g)
F2: Verbal load
Non-verbal subtests Verbal subtests
Multidimensionality
Test results as factors• Note (to repeat) that this eliminates any hard distinction
between the identified dimensions (say, the labels on a graph of one subtest against another) and some alleged ‘real’ factors– Beware of falling in love with the axes labels on a graph– these are just factors that you have chosen (or have other reason) to
graph– they are no more (or less) true than a rotated graph in which
‘abstract’ factors now defined the axes, and text scores are vectors (regression lines) on the graph
• One hard idea to grasp is that the differences here are merely notational = no information is gained or lost through rotation, and the labels are just a convenience
Multidimensionality
Correlation matrix
• We can represent the relationship between many dimensions with a correlation matrix
Dimensions 1 2 3 4 51 12 0.3 13 0.8 0.6 14 0.2 0.7 0.1 15 0.8 0.3 0.9 0.5 1
Multidimensionality
Factor analysis• Factor analysis is a method for reducing a correlation
matrix with a large number of dimensions to (usually) their orthogonal dimensions, called principal components– In other words, it is a method of dimensional reduction – It looks for ‘contamination’ of one dimension by others
(= inter-correlations), and tries to re-present the matrix without that ‘contamination’
– Preference expression is an everyday example of such dimensional reduction, as when we ask ‘If you had pick just three qualities to look for in your next car, what would you pick?’
Multidimensionality
What is a factor?• Factors are not Platonic 'true' objects gifted to man from
the benevolent inhabitants of the Heavens– A factor is a mathematical construct, containing
abstracted information about the inter-relations between other information/dimensions
– it is no better than those measures: GIGO
"The 'factors', in short, are to be regarded as convenient mathematical abstractions, not as concrete mental 'faculties', lodged in separate 'organs' of the brain.
Cyril Burt, 1937
Multidimensionality
What is a factor?– mathematically, a factor is a (usually but not
necessarily) linear combination of correlations between the items in the analysis
– What does it mean to be a ‘linear combination’?• The factors are produced produced by adding in
portions of the variance accounted for by some dimensions (for example, subtests) and subtracting out portion of the variance accounted for by others
• e.g. F1 = aX + bY + cZ, if X,Y,Z are correlations in the input table, and a,b,c are (positive or negative) real number coefficients
Multidimensionality
The factor table• The principal components are taken out in an ordered fashion,
from those accounting for the most variance (strong loading or weighting) to those accounting for the least (weak loading or weighting)
• They are presented in a factor table
FACTORS A B C D
Item1 0.81 0.21 -0.11 -0.12
Item2 -0.13 0.76 0.72 -0.04
Item3 0.08 0.12 0.34 0.88
...
Multidimensionality
Are factors ‘real’?• The danger of reification: Just because a factor can
be extracted does not necessarily mean it has any relevant reality– Some principal component can always be extracted so
long as most correlations are in the same direction• In modern factor analysis, the axes are also rotated to see if
one rotation is better able to account for the data than another.– Because different rotations lead to different
descriptions, Gould warns against reifying any particular description
Multidimensionality
" 'When I use a word', Humpty Dumpty said, in rather a scornful tone, 'it means just what I choose it to mean - neither more nor less.' "
Lewis CarrollThrough the Looking-Glass
Multidimensionality
Spearman's g
• Spearman (1904) invented factor analysis as a way of studying correlations between mental test scores
• He called the first principal component (accounting for about 50-60% of variance) g = general intelligence
• People have argued ever since whether g is real
MultidimensionalityFactor A
Fac
tor
B
F1: General intelligence (g)
F2: Verbal load
Non-verbal subtests Verbal subtests
MultidimensionalityFactor A
Fac
tor
B
F1: Performance IQ
F2: Verbal IQ
Non-verbal subtests Verbal subtests
Multidimensionality
Cyril Burt's Error
• Intelligence is an abused concept
• Burt cut off 80% of people tested from access to higher education, entirely on the basis of a single test
Multidimensionality
Questions• Is a factor more or less 'real' than a 'traditional'
construct? – How can we evaluate the ‘reality’ of any given
factor/dimension?
• What advantage is there is discussing psychometric results in terms of non-orthogonal factors?
• Should people's lives be planned around abstract mathematical constructs?
• Should there be legal limits on what constructs can be defined?