Definition
Plain language
A way of counting how much raw arithmetic a computation takes.
As stated in the literature
Floating-point operations — a measure of computational work, used to compare training and inference cost across architectures independent of wall-clock time, as in isoFLOP analysis.
Also called: FLOP
Why it matters: It offers a hardware-independent way to compare the cost of different models, so researchers can judge efficiency on a level playing field.
For example, comparing two model designs by their FLOPs shows which one needs more raw arithmetic to answer the same question, regardless of which is faster on a given machine.
Heard on the show
“"System design narrows the gap to scale" would be a much stronger claim with a dollars-or-FLOPs axis on the chart, and there isn't one.”Episode 133 — How MiniMax Turned a Reward-Hacking Disaster Into Olympiad Gold