Definition
Plain language
A widely used recipe for adjusting a neural network's settings during training, named after a famous paper.
As stated in the literature
Adaptive Moment Estimation, an optimizer that scales each parameter's update using running averages of recent gradients and their squares; its introducing paper 'Adam: A Method for Stochastic Optimization' by Kingma and Ba is among the most-cited works in machine learning.
Why it matters: It lets networks learn efficiently by tuning each setting at its own pace, so training converges faster and needs less hand-holding.
For example, when someone trains a neural network to recognize photos, they often pick Adam to steer how the model's internal numbers get nudged after each batch.
Heard on the show
“Adam Smith versus Adam Lambert.”Episode 070 — When Models Know the Answer But Say the Wrong Thing Anyway