Definition
A way of telling a language model where each word sits in a sequence by rotating its internal vectors.
Rotary position embeddings (RoPE), a positional-encoding scheme that applies rotation matrices to query/key vectors as a function of position, now standard across most modern transformers.
Also called: RoPE