Factor analysis — latent factors plus measurement noise
Factor analysis writes each column as a linear mix of a few latent factors plus a noise term that belongs to that column alone. PCA has no per-column noise term: every direction is signal, ranked by variance. The noise term is why two columns can correlate without either being a factor by itself, and why the factors are not the same vectors as the principal components.
Rotation is part of the interpretation, not of the likelihood. Varimax seeks factors that each load on a few columns, which is what a person can name. The unrotated solution can have the same fit and be unreadable. Quartimax concentrates the loadings differently. Try the rotation before you name a factor.
The number of factors is not a variance cutoff. Asking for too many factors makes the noise variances hit zero and the model becomes an ordinary decomposition with a warning attached.
Settings that change this fit
- n factors
- Latent dimension. Increase it only while the column-wise noise stays positive and the factors stay nameable.
- rotation
- Varimax spreads loadings toward a simple structure. None leaves the statistical solution untouched.