Concept · 1 episode(s)

Greedy Coordinate Gradient

← all concepts

Definition

Greedy Coordinate Gradient is a discrete optimization algorithm for crafting adversarial text against language models, which uses gradients with respect to token embeddings to rank candidate token substitutions at each position and then greedily accepts the swap that most improves an attack objective. It underlies many jailbreak and prompt-injection attacks, letting an attacker search token-by-token for a seemingly meaningless string that reliably forces a target behavior, such as an affirmative response or a hidden answer baked into a cached context.

Episodes covering this

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.