Definition
Plain language
A classical search method that picks the next experiment based on a probabilistic model of past results.
As stated in the literature
A black-box optimization technique that fits a surrogate (often a Gaussian process) over an objective and uses an acquisition function to choose new query points; a traditional baseline for neural architecture search.
Why it matters: For decades it was the standard way to tune expensive black-box systems, and it remains a strong baseline against which newer agent-based search methods are measured.
For example, Bayesian optimization might try a few network architectures, fit a probabilistic model over the scores, and use it to pick the next architecture most likely to improve.
Heard on the show
“Traditional Neural Architecture Search uses Bayesian optimization or evolutionary algorithms — but those are rigid, mechanical procedures.”Episode 053 — An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script