Definition
Plain language
The basic training method that nudges a model's settings a little at a time, using small random batches of data.
As stated in the literature
Stochastic Gradient Descent, the foundational optimization algorithm for neural networks; its dynamics determine which of several equally-loss-fitting solutions a model settles into, e.g., the negation-neglect bias.
Also called: stochastic gradient descent
Why it matters: It's the engine behind training neural networks, and the way it nudges weights shapes which of many possible solutions a model ends up with.
For example, it adjusts a model's settings a tiny step at a time using a small random batch of examples, gradually reducing its errors.