Definition
Plain language
A widely used recipe for adjusting a neural network's settings during training, named after a famous paper.
As stated in the literature
Adaptive Moment Estimation, an optimizer that scales each parameter's update using running averages of recent gradients and their squares; its introducing paper 'Adam: A Method for Stochastic Optimization' by Kingma and Ba is among the most-cited works in machine learning.
Why it matters: It lets networks learn efficiently by tuning each setting at its own pace, so training converges faster and needs less hand-holding.
For example, when someone trains a neural network to recognize photos, they often pick Adam to steer how the model's internal numbers get nudged after each batch.
Heard on the show
“The best case in the paper: RefChecker flagged "Adam: A Method for Stochastic Optimization" by Kingma and Ba, one of the most-cited papers in the history of machine learning.”Episode 201 — One in Four NeurIPS Papers Cites a Reference That Doesn't Exist