Definition
Iterative training alternates between training a model and generating new training data — from the model itself, from its environment, or from improved labels — rather than training once on a fixed dataset. It’s how a lot of modern reasoning models are made.
Episodes covering this
Worth reading next
Papers we haven't done a deep dive on yet, but would recommend on this topic.