Definition
Plain language
When a system loses track of who originally said something and ends up trusting it more than it should.
As stated in the literature
A failure mode in which downstream components make correct decisions given their inputs but the origin metadata attached to those inputs no longer reflects the true source of the content.
Also called: provenance failures
Why it matters: Every component can behave correctly and the system still gets compromised, because the labels the components rely on no longer match reality.
For example, a note copied from a stranger's email into your own to-do list later looks like something you wrote yourself.
Heard on the show
“This is a provenance failure, not a judgment failure.”Episode 250 — The Same Model Refused a Backdoor, Then Its Own Sub-Agent Ran It