Definition
Plain language
A way of breaking a big table of numbers down into its handful of most important underlying patterns, ranked by how much each one matters.
As stated in the literature
A matrix factorization expressing a matrix via orthogonal directions scaled by singular values; used to find a weight matrix's dominant high-energy directions, e.g. to initialize a LoRA adapter in those directions for fast few-step adaptation.
Also called: SVD
Why it matters: It pinpoints the few directions that carry most of a matrix's information, which lets methods focus their effort where it counts instead of spreading it everywhere.
For example, it can take a giant table of movie ratings and reveal a few underlying tastes — like preference for comedies or action — that explain most of the data.
Heard on the show
“What they do instead is use singular value decomposition — and you don't need the linear algebra, you just need this picture.”Episode 114 — Agents That Rewrite Their Own Weights Instead of Just Taking Notes