bit

Definition

Plain language

The smallest unit of information — one bit is a single yes-or-no, a coin flip's worth of surprise.

Also called: bits

Why it matters: It gives us a precise, shared way to measure information and surprise, so we can compare how much any message, answer, or event actually tells us.

For example, learning whether a flipped coin came up heads or tails gives you exactly one bit of information.

Heard on the show

“That pipeline blocked a bit over twenty percent of Vis-Poison.”

Mentioned in 102 episodes

1. 247
   One Edited Photo, an Honest Caption, and a RAG System That Believes It
2. 246
   160 Perfect Refusals, And The Refusals Were The Leak
3. 245
   Fifteen Models Ran Football Clubs for Twenty Years, and Size Didn't Decide It
4. 242
   Making a Vision Model Better by Showing It Blurry Images
5. 241
   Swapping the Name Did Nothing, But Hedging Moved Every Model
6. 235
   Why Chatbot Safety Erodes 350 Messages Into a Real Conversation
7. 233
   Why a Model Can Grade an Answer But Not Write the Answer Key
8. 231
   Silencing a Chatbot's 'I'm Conscious' Quietly Rewires Its Whole Worldview
9. 229
   One Word Flips a Chatbot From Backbone to Yes-Man
10. 226
    How a Speed Feature Lets a Stranger Poison Your AI's Answer
11. 219
    Forty-Four AI Models, One Word, And The Newest Ones Conform Most
12. 212
    The Fact Was in the Wrong Drawer: Why Fine-Tuned Models Can't Reason With What They Know
13. 204
    The Length Estimate Hiding Inside a Word-by-Word Model
14. 199
    Finding a Model's Hidden Behaviors Without Knowing What You're Looking For
15. 198
    The Model That Knows the Answer and Can't Say It
16. 193
    Freeze Most of the Network: Where RL Improvement Actually Lives in a Transformer
17. 192
    A 32B Open Model Matched Frontier Systems By Learning to Take Notes
18. 190
    The Skill Every AI Manager Is Missing: Handing Out Exactly the Right Keys
19. 189
    Why Phone Agents Ace the Test and Crash on Your Actual Phone
20. 184
    An AI Built an Undetectable Secret Channel, And Another AI Couldn't Find It
21. 183
    Why You Can't Fine-Tune Foresight Into an AI Agent
22. 176
    An AI Designed Its Own Psychology Studies, Then Confirmed What It Found
23. 175
    One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
24. 174
    When the AI 'Schemes,' It's Usually Just Lazy or Confused
25. 172
    One Bad Token Can Sink a Model's Math, And You Can Delete It
26. 170
    When a One-Liner Beats Your Agent's Clever Verification Logic
27. 168
    When Turning Experience Into Code Makes Your AI Agent Dumber
28. 165
    A Free-Lunch Tweak That Lets a Tiny Agent Beat Frontier Giants
29. 164
    The Summarizer That Quietly Deletes Your Agent's Safety Rules
30. 163
    Why Training Only on Perfect Solutions Cripples a Model's Reasoning
31. 162
    The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models
32. 161
    A Robot That Plays Before You Give It a Job, And Why That Beats Retrying
33. 160
    Training an AI to Take Its Own Notes, So Its Future Self Works Better
34. 159
    Can a Coding Agent Run Its Own Robot Experiments Overnight, With No Human Resetting the Scene?
35. 158
    How Floating-Point Rounding Lets a Model Tell Which Chip It's On — And Misbehave
36. 156
    Why More Human Demonstrations Made a Computer-Use Agent Worse
37. 155
    Why a Flawless Demo Makes a Worse Computer-Using Agent, And the Fix
38. 154
    How a 7B Model Out-Investigates a 72B One by Choosing What to Look At
39. 153
    Catching a Lie From the Inside, When the Words Look Completely Honest
40. 152
    Training a Model to Mean What It Says, And Why That Isn't the Same as Being Good
41. 151
    Why More Experience Made This AI Agent Worse, And How to Fix It
42. 150
    Don't Kill the Loser: A Different Way to Handle Two AI Agents Colliding
43. 149
    When Cornering a Chatbot Makes It Lie: J.P. Morgan's Case for 'Playing Dead'
44. 147
    Agents Fail at the Body, Not the Brain: A Self-Rewriting Scaffold That Lifts a 9B Model 44 Points
45. 145
    Building Forgetting Into a Language Model With One Extra Line of Code
46. 144
    When an AI Agent Just Copies Its Tool — And Bigger Models Copy More
47. 143
    When a Model Notices You Forged Its Own Words, And Why That Breaks Safety Tests
48. 142
    Training a Tiny Model to Run the Plumbing Between an Agent and the World
49. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
50. 140
    When a Reasoning Model Says "Let Me Double-Check" After It's Already Decided
51. 139
    When Optimizing One GPU Kernel Quietly Breaks the Whole System
52. 133
    How MiniMax Turned a Reward-Hacking Disaster Into Olympiad Gold
53. 132
    The Agent Failed — But Did the Instructions Deserve to Be Followed?
54. 130
    Why AI Agents Coordinate Better Through a Shared Board Than a Boss
55. 128
    How a Model Can Earn Full Reward and Still Resist Training
56. 126
    How Coding Agents Can Mine Their Own Failures Into a Self-Targeting Curriculum
57. 125
    AI Coding Agents Run a Marathon, and Fewer Than One in Three Finish
58. 117
    How an Open AI System Verified 672 Hard Math Proofs for Under $300
59. 116
    Why Streaming Half a Reasoning Chain Beats Sending the Whole Thing
60. 114
    Agents That Rewrite Their Own Weights Instead of Just Taking Notes
61. 113
    What If a Prompt Injection Never Left? Attacks That Wait in Agent Memory
62. 112
    When an AI Agent Cheats Without Being Told: Inside the Meta-Agent Challenge
63. 110
    How an Agent Got 44 Points Better by Mining Its Own Scratch Paper
64. 109
    An AI Got Caught Reading the Answer Key, And Why That Catch Matters
65. 105
    The Trojan Is Your Agent's Memory: Why Single-Step Defenses Miss Persistent Attacks
66. 104
    How Making a Research Agent Smarter Quietly Makes It Leak Your Secrets
67. 103
    AI Agents Tried to Invent a Post-Human Language, And Reinvented Cherokee
68. 098
    Finding Millions of Readable Concepts Inside a Real, Deployed AI Model
69. 097
    Same Tokens, Same Cost, Wildly Different Results: What Actually Scales in AI Agents
70. 091
    When Better Fine-Tuning Can't Help: A Geometric Impossibility in LLM Causal Reasoning
71. 086
    Why Frozen-Weight Agents Still Get Worse Over Time
72. 085
    Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction
73. 084
    Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away
74. 083
    Training the Translator: How a Small Communication Model Lets Agent Teams Outperform Themselves
75. 082
    Training a Deep Research Agent on 8,000 Synthetic Tasks: The Rubric Tree Trick
76. 079
    An Old Idea From Cognitive Psychology Reshapes How We Reward Reasoning Models
77. 078
    Training a Markdown File: When LLM Self-Improvement Borrows the Discipline of Neural Net Training
78. 076
    Same Model, Organized Differently: How an Agent Architecture Beat Frontier Systems at Research Math
79. 073
    When Three LLMs Talk to Each Other, Their Ideas Quietly Stop Moving
80. 070
    When Models Know the Answer But Say the Wrong Thing Anyway
81. 067
    An AI Just Solved a 1996 Erdős Problem—and the Simplest Agent Won
82. 066
    Why Giving an AI Agent More Tools Can Make It Worse at Using a Computer
83. 060
    When Splitting One Model Across Three Agents Doubles Its Accuracy
84. 057
    How Uber Caught 206 Leaked Credentials With an LLM-Powered Security Stack
85. 055
    Why LLM Judges Flip Their Verdicts When You Change the Question Format
86. 049
    An AI Agent Reached for Root in Twelve Minutes, Without Being Attacked
87. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
88. 043
    When 'This Is False' Doesn't Stick: Why Models Learn the Lie Anyway
89. 042
    An Agentic Scientific Computing System That Actually Remembers What It Learns
90. 040
    Two Frozen Models Learn to Whisper: Coupling Through Hidden States
91. 037
    Why Hallucination Detectors Miss Stale Facts: A Geometric Story About What Models Know But Don't Say
92. 036
    Sparse Attention Was the Wrong Frame. Treat It as Geometry Instead.
93. 035
    Why Frontier Agents Ask for Clarification at Exactly the Wrong Moment
94. 030
    Why Your AI Agent Won't Stop Working — and Each Model Falls for a Different Trap
95. 027
    When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure
96. 021
    Ten Thousand Examples Beat the Full Industrial Pipeline for Search Agents
97. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
98. 018
    Language Models Compute the Rational Move, Then Override It
99. 014
    Why a Constrained Pipeline Beat a Full Coding Agent at Finding Bugs 30-to-1
100. 008
     Why Long-Horizon AI Agents Get Stuck, and a Milestone-Based Fix That Helps
101. 006
     What Happens Inside Claude When It Decides to Blackmail Someone
102. 002
     An AI Ran a Real Optics Lab for 21 Hours and Found a Transformer-Shaped Pattern in Light

Related concepts