Glossary · Term

neural network

← all terms

Definition

Plain language

A computing system, loosely inspired by the brain, made of layers of simple numbers that get tuned until the whole thing can spot patterns or produce answers.

As stated in the literature

A parameterized function composed of layers of weighted connections and nonlinearities, trained by gradient descent to map inputs to outputs; transformers, MLPs, and state-space models are all instances.

Also called: neural networks, neural net, neural nets

Why it matters: It matters because this trainable, pattern-finding structure underlies nearly all modern AI, from image recognition to language models.

For example, a neural network can be trained on many photos of cats and dogs until it reliably tells a new picture apart.

Heard on the show

“The trained neural network, call it the engine, is only one layer.”
Episode 228 — Same Chatbot, Two Doors: Why 'Grok's Opinion' Doesn't Exist

Mentioned in 29 episodes

  1. 228
    Same Chatbot, Two Doors: Why 'Grok's Opinion' Doesn't Exist
  2. 209
    How 2.6 Billion Doodles Exposed the Culture Words Quietly Delete
  3. 206
    How Four-Second Clips Become Hours of Playable AI Soccer
  4. 194
    How a Robot Builds a Debugging Notebook It Can Read, Edit, and Hand to Another Robot
  5. 185
    Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway
  6. 182
    How a Tiny Model Too Weak to Plan Cuts a Bigger Agent's Hallucinations by 80%
  7. 179
    How DeepSeek Made One User Faster Without Slowing Down the Crowd
  8. 165
    A Free-Lunch Tweak That Lets a Tiny Agent Beat Frontier Giants
  9. 162
    The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models
  10. 161
    A Robot That Plays Before You Give It a Job, And Why That Beats Retrying
  11. 160
    Training an AI to Take Its Own Notes, So Its Future Self Works Better
  12. 133
    How MiniMax Turned a Reward-Hacking Disaster Into Olympiad Gold
  13. 108
    The Reasoning Cliff: Why Thinking Longer Makes Models Worse at Exact Step-by-Step Tasks
  14. 095
    Seven Wins to Zero: How Organizing AI Agents Like a Lab Changes the Search
  15. 085
    Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction
  16. 078
    Training a Markdown File: When LLM Self-Improvement Borrows the Discipline of Neural Net Training
  17. 077
    Reading a Model's Confidence Curve to Decide When Chain-of-Thought Is Worth It
  18. 073
    When Three LLMs Talk to Each Other, Their Ideas Quietly Stop Moving
  19. 072
    A Robot Made Graphene Without Help, And Caught Itself Hallucinating
  20. 067
    An AI Just Solved a 1996 Erdős Problem—and the Simplest Agent Won
  21. 060
    When Splitting One Model Across Three Agents Doubles Its Accuracy
  22. 053
    An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script
  23. 042
    An Agentic Scientific Computing System That Actually Remembers What It Learns
  24. 041
    When the Iteration Teaches the Model to Skip the Iteration
  25. 037
    Why Hallucination Detectors Miss Stale Facts: A Geometric Story About What Models Know But Don't Say
  26. 025
    The Missing Gradient Term That Predicts Sycophancy in RLHF
  27. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
  28. 013
    Why Search Keeps Rediscovering the Same Workflow, and What That Means
  29. 008
    Why Long-Horizon AI Agents Get Stuck, and a Milestone-Based Fix That Helps

Related concepts

Amortized Inference Causal Intervention Circuit Analysis Global Workspace Linear Representation Mechanistic Interpretability Superposition Hypothesis World Models

Related terms

gradient layer state-space model transformer