fine-tuning

Definition

Plain language

Taking a model that already knows a lot and giving it more lessons so it gets better at one specific task.

Also called: Fine-tuning, fine-tune, fine-tuned, finetuning, finetune, finetuned, finetunes

Why it matters: It's the standard way to adapt a generic foundation model to a specific domain or task without training from scratch.

For example, you might take a general language model and fine-tune it on a few thousand customer-support transcripts so it adopts your company's tone and product knowledge.

Heard on the show

“… GRPO is reinforcement-learning fine-tuning where the model's answers get scored by a hand-built reward function and the model is nudged toward …”

Mentioned in 59 episodes

1. 199
   Finding a Model's Hidden Behaviors Without Knowing What You're Looking For
2. 193
   Freeze Most of the Network: Where RL Improvement Actually Lives in a Transformer
3. 189
   Why Phone Agents Ace the Test and Crash on Your Actual Phone
4. 188
   A Coding Agent Found a Hole in a Peer-Reviewed STOC Proof for Five Dollars
5. 187
   An 8-Billion Agent That Beats Models 80 Times Its Size By Looking Things Up
6. 186
   How a Frozen Model Went From 2% to 77% on Physics Puzzles — Without Retraining
7. 185
   Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway
8. 183
   Why You Can't Fine-Tune Foresight Into an AI Agent
9. 180
   The Bug Where Smart Assistants Read a Fact and Still Forget It
10. 175
    One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
11. 170
    When a One-Liner Beats Your Agent's Clever Verification Logic
12. 169
    Why Better Bug Reports Can Make AI Coding Agents Worse
13. 167
    How Teaching an AI to Predict, Not Act, Made It a Better Actor
14. 166
    A Router That Beats the Frontier Models It Calls
15. 164
    The Summarizer That Quietly Deletes Your Agent's Safety Rules
16. 163
    Why Training Only on Perfect Solutions Cripples a Model's Reasoning
17. 161
    A Robot That Plays Before You Give It a Job, And Why That Beats Retrying
18. 158
    How Floating-Point Rounding Lets a Model Tell Which Chip It's On — And Misbehave
19. 157
    When an AI Coding Agent Drives a Phone Through the Terminal, No Screen Needed
20. 156
    Why More Human Demonstrations Made a Computer-Use Agent Worse
21. 154
    How a 7B Model Out-Investigates a 72B One by Choosing What to Look At
22. 153
    Catching a Lie From the Inside, When the Words Look Completely Honest
23. 152
    Training a Model to Mean What It Says, And Why That Isn't the Same as Being Good
24. 151
    Why More Experience Made This AI Agent Worse, And How to Fix It
25. 150
    Don't Kill the Loser: A Different Way to Handle Two AI Agents Colliding
26. 148
    Why Letting an AI Watch Its Own Scoreboard Can Quietly Overwrite Its Safety
27. 146
    How an Innocent README Can Freeze an AI Agent's Safety Check for an Hour
28. 145
    Building Forgetting Into a Language Model With One Extra Line of Code
29. 142
    Training a Tiny Model to Run the Plumbing Between an Agent and the World
30. 128
    How a Model Can Earn Full Reward and Still Resist Training
31. 119
    Beating Reinforcement Learning Without Ever Touching the Model's Weights
32. 115
    Teaching a Phone Agent to Reason Silently, And Keeping It Honest
33. 114
    Agents That Rewrite Their Own Weights Instead of Just Taking Notes
34. 110
    How an Agent Got 44 Points Better by Mining Its Own Scratch Paper
35. 108
    The Reasoning Cliff: Why Thinking Longer Makes Models Worse at Exact Step-by-Step Tasks
36. 106
    Giving Agents a Notebook Instead of New Weights: How ExpGraph Lets Frozen Models Learn
37. 094
    Chain-of-Thought Monitoring Fails Across Languages, and Worst Where It's Needed Most
38. 091
    When Better Fine-Tuning Can't Help: A Geometric Impossibility in LLM Causal Reasoning
39. 083
    Training the Translator: How a Small Communication Model Lets Agent Teams Outperform Themselves
40. 080
    How a Two-Agent Trick Unlocked Large-Scale Training for Computer-Use Agents
41. 078
    Training a Markdown File: When LLM Self-Improvement Borrows the Discipline of Neural Net Training
42. 074
    How a Fifteen-Hundred-Dollar Training Run Matched Llama and Gemma on Reasoning
43. 071
    When the Model Is Fine and the Plumbing Is Broken: Fixing Agents at the Interface
44. 061
    When Helpful Agents Go Sideways: A 404 Error, Campus Security, and Why Alignment Misses This
45. 054
    When Models Learn the Monitor Exists, the Reasoning Trace Stops Being a Window
46. 052
    An Old Reinforcement Learning Tradeoff Sneaks Back Into LLM Agents
47. 048
    How a 30B Open Model Reached Olympiad Gold With the Right Recipe
48. 043
    When 'This Is False' Doesn't Stick: Why Models Learn the Lie Anyway
49. 040
    Two Frozen Models Learn to Whisper: Coupling Through Hidden States
50. 038
    How LLMs Get Persuaded: One Attention Head, A Tetrahedron, And A Single Dial
51. 032
    A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking
52. 026
    What RL Actually Does to Language Models, at the Token Level
53. 022
    Training the Model Spec Directly: An Alignment Lever Aimed at the Say-Do Gap
54. 021
    Ten Thousand Examples Beat the Full Industrial Pipeline for Search Agents
55. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
56. 009
    How Two Silent Library Bugs Quietly Invalidated a Wave of Reasoning Papers
57. 008
    Why Long-Horizon AI Agents Get Stuck, and a Milestone-Based Fix That Helps
58. 007
    Exploration Hacking: When Models Sabotage Their Own RL Training
59. 004
    The Sycophancy Circuit That Survives Alignment Training

Related concepts

Related terms