Definition
Plain language
A high-end NVIDIA data-center chip used to train and run large AI models, a step up from the H100.
As stated in the literature
NVIDIA's Hopper-architecture data-center GPU with expanded high-bandwidth memory relative to the H100, used in large-scale training and serving.
Why it matters: Its expanded memory lets bigger models be trained and served more efficiently, directly affecting what's practical to build.
For example, a lab might train a large language model on a cluster of H200 chips because each one holds more high-speed memory than the previous generation.
Heard on the show
“And the only place it dips to ninety-two is the hardest case — two same-generation NVIDIA chips, an H100 versus an H200, which are numerically very close cousins.”Episode 158 — How Floating-Point Rounding Lets a Model Tell Which Chip It's On — And Misbehave