Definition
Plain language
Software that runs many programs across many computers and automatically restarts, scales, and load-balances them.
As stated in the literature
An open-source container orchestration system for automating deployment, scaling, and management of containerized applications across clusters.
Also called: kubernetes, k8s
Why it matters: It's the dominant substrate for running ML inference and agent workloads in production, shaping how serving systems are designed.
For example, a web service deployed on Kubernetes can have ten copies running across a cluster, with new ones spun up automatically when traffic spikes.
Heard on the show
“The ten-x cost gap is Orchard run by experts with their own Kubernetes cluster on spot instances, versus a managed service that handles everything for you.”Episode 047 — When Agent Benchmarks Lie: The Harness Problem in Open-Source AI