Definition
Plain language
A cheap way to fine-tune a big model by training small added pieces instead of all its weights.
As stated in the literature
Low-Rank Adaptation, a parameter-efficient fine-tuning method that trains small low-rank update matrices while keeping the base weights frozen.
Why it matters: It made domain-specific fine-tuning of large models feasible for groups without huge GPU budgets.
For example, instead of updating all 70 billion parameters of a base model, LoRA might train a few million parameters in small inserted matrices, getting most of the benefit at a fraction of the cost.
Heard on the show
“The standard tool for cheap weight edits is LoRA — a thin patch you add on top of the existing weights instead of rewriting them, like a transparent overlay on a huge painting.”Episode 199 — Finding a Model's Hidden Behaviors Without Knowing What You're Looking For