Definition
Plain language
A way of finding the few directions that explain most of the variation in a bunch of high-dimensional data.
As stated in the literature
Principal Component Analysis, a linear technique that finds orthogonal directions of maximum variance in a dataset.
Why it matters: Reducing high-dimensional data to a few meaningful axes is the workhorse first step for visualization, denoising, and feature analysis.
For example, PCA on faces might reveal that the largest source of variation across images is overall lighting.
Heard on the show
“The image the paper offers — they project the model's internal state during inference onto two dimensions using PCA, and watch it evolve over sixteen iteration steps.”Episode 041 — When the Iteration Teaches the Model to Skip the Iteration