Definition
Plain language
A way of multiplying two lists of numbers together to get a single number measuring how much they line up.
As stated in the literature
The sum of elementwise products of two vectors; the core relevance operation in attention, where a query's dot product against each key sets the attention weights, and the projection test in geometric attention sparsification.
Also called: dot products
Why it matters: It is the basic measure of how aligned two sets of numbers are, and inside attention it decides which earlier words a model pays attention to.
For example, the dot product of [1, 2] and [3, 4] is 1×3 plus 2×4, which equals 11.
Heard on the show
“A point in eleven-dimensional space is just a list of eleven numbers, and "spheres touching" becomes a condition on the dot products between those lists.”Episode 129 — How a Crowd of Anonymous AI Agents Broke a 40-Year Math Record