Glossary · Term

entropy phase transition

← all terms

Definition

Plain language

A sudden change in how uncertain a model is, as if it freezes from exploring into committing.

As stated in the literature

In the EDRM framework, the visual clustering of early-decoding entropy trajectories into distinct families — declining versus oscillating — analogized to phase transitions in dynamical systems.

Why it matters: Spotting which trajectory a generation is on early can let you intervene, resample, or route compute before a model commits to a bad answer.

For example, you might plot a model's uncertainty over the first few decoded tokens and see two clear groups of curves — one smoothly dropping toward a confident answer and another bouncing around before locking in.

Related terms