Definition
Plain language
The large but slow pool of memory on a chip — plentiful, but reaching it makes a program wait.
As stated in the literature
Dynamic Random-Access Memory — the high-capacity, high-latency off-chip memory tier in a GPU's hierarchy, far slower than on-chip registers and shared memory; a kernel that repeatedly waits on DRAM can run orders of magnitude slower than one arranged to keep data on-chip.
Why it matters: How often code reaches into this slow memory often decides whether a GPU program is fast or painfully sluggish.
For example, a program that keeps fetching the same data from this slow memory pool over and over can run far slower than one that keeps that data close to the chip's fast on-chip storage.
Heard on the show
“The big off-chip pool, DRAM, is slow.”Episode 177 — Why Raw Profiler Data Made an AI Worse at Writing GPU Code