Common mistake 1 with Factor analysis

Confusing it with PCA. PCA repackages all the variance; factor analysis models only the shared part and explicitly sets the unique part aside. They answer different questions and often give different answers.

Common mistake 2 with Factor analysis

Rotation changes the loadings but not the fit. It cannot make a bad solution good, and choosing the rotation that best supports your story is not analysis.

Common mistake 3 with Factor analysis

Deciding the number of factors from the eigenvalue-above-1 rule alone. Use the scree plot, interpretability and, if you can, parallel analysis.

Common mistake 4 with Factor analysis

Treating factor scores as though they were measured without error. They are estimates and carry uncertainty into whatever you do next.

Common mistake 5 with Factor analysis

Running exploratory factor analysis and then reporting it as though you had confirmed a hypothesised structure.

EconStatsCraft › Factor analysis

Factor analysis

PCA repackages the variance you have; factor analysis asks a different question - what unobserved thing could be causing these variables to move together? The right tool for survey data, where several questions are noisy measures of one underlying attitude or ability.

What it needs

Observed variables - The measured items you think share a common cause.

The intuition

Factor analysis and PCA look alike and answer different questions. PCA asks what combinations of your variables carry the most variance, treating every variable as measured perfectly. Factor analysis asks whether a few unobserved traits could have GENERATED the correlations you see, and explicitly allows each variable its own measurement error. If you believe there is a latent construct - ability, sentiment, quality of governance - factor analysis is the model of that belief; PCA is just a rotation.

When to use it

With survey and psychometric data, where several questions are noisy measures of one underlying attitude, ability or attribute, and you want to recover that attribute rather than just compress the questions.

Where the estimator comes from

The model is x=\Lambda f+\varepsilon with f\sim(0,I), \varepsilon\sim(0,\Psi) diagonal, and f\perp\varepsilon. This implies \Sigma=\Lambda\Lambda'+\Psi, so the off-diagonal covariances come entirely from the common factors while the diagonal splits into COMMUNALITY h_i^2=\sum_k\lambda_{ik}^2 and UNIQUENESS \psi_i. That diagonal split is the whole difference from PCA, which puts all of the variance into components. Estimation is by maximum likelihood under normality, or by principal-axis factoring. Any orthogonal rotation \Lambda Q with QQ'=I fits identically, which is why rotation is free and why varimax - maximising the variance of squared loadings - is used to push loadings towards zero or one.

The model

x_i=\Lambda f_i+u_i,\qquad \operatorname{Var}(x)=\Lambda\Lambda'+\Psi,\qquad \Psi \text{ diagonal}

What it assumes

A linear factor structure with a modest number of factors.
The unique parts are uncorrelated with each other and with the factors - this is the real content of the model, and it is what makes factor analysis a MODEL rather than a transformation.
Enough correlation among the observed variables for common factors to exist. Test this first.
The sample is large enough for the correlation matrix to be stable. Factor solutions on fewer than about five observations per variable are driven by sampling noise in the correlations.

Statistical properties

The factor solution is only identified up to rotation. There is no such thing as THE loadings; there is a subspace, and rotation picks a readable basis within it. This is a feature, not an indeterminacy to be apologised for.
Communality plus uniqueness equals one for standardised variables. A variable with communality below about 0.3 shares little with the others and is a candidate for removal.
PCA components are exact linear combinations of the variables; factor scores are ESTIMATES of unobservable quantities and carry their own uncertainty. This is why factor scores should not be fed into a second regression as though observed.
A Heywood case - a negative uniqueness, meaning communality above one - is impossible in the population and signals too many factors, too few variables per factor, or a sample too small.
Bartlett's test of sphericity and the KMO measure check whether the correlation matrix has enough structure to factor at all. KMO below 0.6 means it probably does not.
Oblique rotations such as promax allow factors to correlate, which is usually more plausible for psychological or economic constructs than the orthogonality varimax imposes.

How to read the output

Loadings are the correlations between each variable and each factor. Above about 0.4 is usually taken as meaningful.
Communality is the share of a variable explained by the common factors; uniqueness is one minus that. A variable with uniqueness near 1 shares nothing and probably does not belong.
After varimax rotation, look for simple structure - each variable loading strongly on one factor and weakly on the rest. That is what makes a solution interpretable.
Rotation changes the loadings but not the fit. Varimax makes them easier to name, and naming them is interpretation - two analysts rotating the same solution can reach different stories with identical arithmetic.

A worked reading

This demo is deliberately the unhappy case, and the suitability tests are the reason to run it. Six macroeconomic indicators across 110 countries give a Kaiser-Meyer-Olkin statistic of 0.5641 - below the 0.6 usually treated as a floor - and Bartlett's test of sphericity fails to reject at p = 0.3626, meaning the correlation matrix is not distinguishable from the identity. There is, in short, nothing shared to factor. The loadings then show exactly what that looks like: communalities between 0.04 and 0.27, so every variable is between 73% and 96% its own unique variance, and the two "factors" are fitting almost nothing. A rotation will still produce a tidy table, and reporting it would be the error. The lesson is that factor analysis, unlike PCA, has suitability tests at all - PCA would have happily reported two components explaining a respectable share of variance on this same data, because it has nowhere to put measurement error and so cannot tell you that there is nothing to find.

Symptom → diagnosis → what to do

If you see this	It means	Do this
A negative uniqueness	Heywood case	Reduce the number of factors
Every variable loading on one factor	A general factor, or unstandardised input	Check the scaling first
Loadings unreadable after extraction	Rotation not applied	Rotate; the unrotated solution is rarely interpretable
Communality below 0.3 for a variable	It shares little with the others	Consider dropping it and refitting
KMO below 0.6	The matrix has too little common structure	Factor analysis is not appropriate here
Factors correlated after varimax	Impossible: varimax forces orthogonality	You are looking at an oblique solution

Common mistakes

Confusing it with PCA. PCA repackages all the variance; factor analysis models only the shared part and explicitly sets the unique part aside. They answer different questions and often give different answers.
Rotation changes the loadings but not the fit. It cannot make a bad solution good, and choosing the rotation that best supports your story is not analysis.
Deciding the number of factors from the eigenvalue-above-1 rule alone. Use the scree plot, interpretability and, if you can, parallel analysis.
Treating factor scores as though they were measured without error. They are estimates and carry uncertainty into whatever you do next.
Running exploratory factor analysis and then reporting it as though you had confirmed a hypothesised structure.

Where to read this properly

Spearman (1904), "General Intelligence, Objectively Determined and Measured", American Journal of Psychology 15(2).
Bartholomew, Knott and Moustaki, Latent Variable Models and Factor Analysis (2011).
Kaiser (1958), "The Varimax Criterion for Analytic Rotation", Psychometrika 23(3).
Fabrigar et al. (1999), "Evaluating the Use of Exploratory Factor Analysis", Psychological Methods 4(3).

Related Multivariate statistics

Principal components (PCA) · Cluster analysis

All analyses

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Factor analysis

What it needs

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Multivariate statistics

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.