Glossary · Term

safety property

← all terms

Definition

Plain language

A guarantee that something bad never happens in a system, no matter how events unfold.

As stated in the literature

In formal methods, a property asserting that a bad state is never reached under any execution or interleaving; contrasted with liveness, which asserts that something good eventually happens. Model checkers like TLC verify safety properties such as deadlock-freedom and mutual exclusion.

Also called: safety properties

Why it matters: It matters because guaranteeing that a bad state can never happen is essential wherever a single failure could be catastrophic.

For example, an elevator's safety property might be that its doors never open while the car is moving between floors.

Heard on the show

“That's the intellectual move: taking a safety property and making it a measurable capability the field can optimize against.”
Episode 190 — The Skill Every AI Manager Is Missing: Handing Out Exactly the Right Keys

Mentioned in 6 episodes

  1. 190
    The Skill Every AI Manager Is Missing: Handing Out Exactly the Right Keys
  2. 153
    Catching a Lie From the Inside, When the Words Look Completely Honest
  3. 094
    Chain-of-Thought Monitoring Fails Across Languages, and Worst Where It's Needed Most
  4. 054
    When Models Learn the Monitor Exists, the Reasoning Trace Stops Being a Window
  5. 039
    When Smarter Agents Get Fooled by Three Extra Nodes in a Database
  6. 034
    Catching Multi-Agent Deadlocks Before Deployment With a 40-Year-Old Tool

Related concepts

Related terms