Definition
Plain language
An early, much smaller language model from OpenAI, now mostly used as a historical comparison point.
As stated in the literature
OpenAI's 2019 transformer language model, with variants up to 1.5B parameters; used as a low baseline in architecture comparisons, where beating it is necessary but unimpressive for a modern model.
Also called: GPT-two
Why it matters: It serves as a low historical baseline, useful for sanity checks even though surpassing it no longer signals real strength.
For example, researchers sometimes compare a new model against GPT-2 to confirm it clears a basic bar, since simply beating it is expected nowadays.
Heard on the show
“Two sizes of GPT-2, three random seeds, three different off-the-shelf instruct models.”Episode 153 — Catching a Lie From the Inside, When the Words Look Completely Honest