Definition
Plain language
A list of numbers that represents the meaning of a word or sentence, so the computer can compare meanings mathematically.
As stated in the literature
A dense vector representation of a discrete input (token, sentence, image patch), learned so that semantic similarity corresponds to vector closeness.
Also called: embeddings
Why it matters: Embeddings are the bridge between symbolic data and the geometric world where neural networks actually do their work, underlying everything from search to retrieval-augmented generation.
For example, the words 'king' and 'queen' end up with very similar embeddings, while 'king' and 'broccoli' point in very different directions.
Heard on the show
“The first is a known approach — using the scattering medium as a single-token embedding engine, basically optical reservoir computing.”Episode 002 — An AI Ran a Real Optics Lab for 21 Hours and Found a Transformer-Shaped Pattern in Light