Definition
Plain language
A long-running U.S. survey that records how real Democrats and Republicans actually answer political questions.
As stated in the literature
Pew Research Center's nationally representative online survey panel; its item-level partisan response distributions serve as the empirical ground truth against which LLM political-bias outputs are compared in audit studies.
Why it matters: It provides a real-world yardstick of how actual partisans answer, so claims about an AI model's political bias can be checked against people instead of guesses.
For example, when researchers want to know how a typical Republican or Democrat would answer a question about taxes, they can check the actual responses people gave in this survey.
Heard on the show
“… And the killer addition — fifteen-hundred-and-forty items from the American Trends Panel, real Pew survey questions where they have empirical distributions of how actual Democrats …”Episode 015 — The Audit Number Isn't What You Think: Sycophancy and the Case Against Single-Prompt Bias Tests