Glossary · Term

embedding

← all terms

Definition

Plain language

A list of numbers that represents the meaning of a word or sentence, so the computer can compare meanings mathematically.

As stated in the literature

A dense vector representation of a discrete input (token, sentence, image patch), learned so that semantic similarity corresponds to vector closeness.

Also called: embeddings

Why it matters: Embeddings are the bridge between symbolic data and the geometric world where neural networks actually do their work, underlying everything from search to retrieval-augmented generation.

For example, the words 'king' and 'queen' end up with very similar embeddings, while 'king' and 'broccoli' point in very different directions.

Heard on the show

“… Under the shared-embedding pipeline — where pictures and text queries live in one numeric space, CLIP-style — ninety-nine …”
Episode 247 — One Edited Photo, an Honest Caption, and a RAG System That Believes It

Mentioned in 22 episodes

  1. 247
    One Edited Photo, an Honest Caption, and a RAG System That Believes It
  2. 246
    160 Perfect Refusals, And The Refusals Were The Leak
  3. 229
    One Word Flips a Chatbot From Backbone to Yes-Man
  4. 219
    Forty-Four AI Models, One Word, And The Newest Ones Conform Most
  5. 213
    A Model Learned to Control a Robot by Watching Video It Never Acted On
  6. 204
    The Length Estimate Hiding Inside a Word-by-Word Model
  7. 198
    The Model That Knows the Answer and Can't Say It
  8. 157
    When an AI Coding Agent Drives a Phone Through the Terminal, No Screen Needed
  9. 151
    Why More Experience Made This AI Agent Worse, And How to Fix It
  10. 132
    The Agent Failed — But Did the Instructions Deserve to Be Followed?
  11. 127
    What Diffusion Language Models Were Missing: A Map, Not an Algorithm
  12. 114
    Agents That Rewrite Their Own Weights Instead of Just Taking Notes
  13. 110
    How an Agent Got 44 Points Better by Mining Its Own Scratch Paper
  14. 106
    Giving Agents a Notebook Instead of New Weights: How ExpGraph Lets Frozen Models Learn
  15. 102
    How to Catch an AI Attack That No Single Conversation Reveals
  16. 085
    Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction
  17. 073
    When Three LLMs Talk to Each Other, Their Ideas Quietly Stop Moving
  18. 067
    An AI Just Solved a 1996 Erdős Problem—and the Simplest Agent Won
  19. 041
    When the Iteration Teaches the Model to Skip the Iteration
  20. 039
    When Smarter Agents Get Fooled by Three Extra Nodes in a Database
  21. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
  22. 002
    An AI Ran a Real Optics Lab for 21 Hours and Found a Transformer-Shaped Pattern in Light

Related concepts

Contrastive Loss Dense Retrieval Greedy Coordinate Gradient LIMIT Benchmark

Related terms

token