Spec-Driven Testing for Agents With A Brain the Size of A Planet — Steven Willmott, SafeIntelligence

Key Concepts

• Spec-Driven Validation: A methodology for testing AI agents by defining explicit behavioral requirements, rules, and constraints rather than relying solely on static datasets.
• Formal Verification: A technique used to analyze entire regions of an input space to identify potential failure points or "edge cases" under perturbation.
• Agent Cards: Documentation (inspired by the A2A spec) that outlines an agent's purpose, capabilities, and operational boundaries.
• Robustness Requirements: The ability of an agent to maintain performance under stress, such as typos, rephrasing, or environmental changes.
• Surface Area: The total range of inputs, tools, and infrastructure access an agent has, which correlates to its risk profile.

---

1. The Challenge of Agent Specification

Steve, CEO of Safe Intelligence, argues that relying on traditional ML evaluation—using a static dataset to measure F1 scores or accuracy—is insufficient for deploying autonomous agents.

• The "Bigger is Better" Fallacy: Larger models are not inherently safer. In fact, their increased reasoning capabilities can make them more susceptible to sophisticated jailbreaks (e.g., hiding malicious instructions within a poem).
• The Risk-Capability Trade-off: There is a tension between building a highly capable agent and ensuring it cannot perform arbitrary harm. The risk is defined by two factors: the flexibility of the prompts it accepts and the level of access it has to critical infrastructure (e.g., financial systems).

2. Framework for Spec-Driven Testing

To move beyond simple input-output testing, Steve proposes a comprehensive specification framework that includes:

• Ground Truth Data: A baseline set of examples defining "good" performance.
• Business Rules: Explicit constraints (e.g., "no discounts over 10%," "no refunds after 30 days").
• Ontologies and Dictionaries: Defining the "universe" of valid terms and destinations relevant to the specific domain (e.g., airline routes or financial terminology).
• Domain Knowledge: Ensuring the agent understands context-specific definitions (e.g., distinguishing between "gross profit" and "gross sales").
• Rights and Roles: Testing how the agent behaves based on user permissions and authentication levels.
• Robustness Requirements: Stress-testing the agent against real-world noise like typos, rephrasing, and varying input conditions.

3. Implementation and Methodology

The speaker emphasizes that testing should be implementation-agnostic. Whether using LangSmith, Vertex AI, or other frameworks, the tests should exist independently in a repository (similar to how API specs like OpenAPI function).

• Security Integration: By defining the "spec" of what an agent is supposed to do, developers can identify the boundaries where the agent is most vulnerable to exploitation.
• Integration Testing: Treat agent evaluations like software integration tests. This allows for a "closed-loop" system where results from automated tests are used to iterate on the agent’s design and fill robustness gaps.
• Version Control: The speaker advocates for treating these specifications as versioned code within a GitHub repository, allowing for consistent testing across different infrastructure iterations.

4. Key Arguments and Perspectives

• "Think Harder": The speaker uses the metaphor of Marvin the Paranoid Android to illustrate that high intelligence without clear, constrained goals leads to inefficiency and potential failure.
• Beyond the Eval: Steve asserts that while "evals" (test sets) are useful, they are only one component. A robust system requires a holistic view of the agent's role, context, and operational environment.
• Standardization: Drawing on his experience with the OpenAPI spec, Steve suggests that the industry needs a standardized way to express agent specifications so they can be shared and utilized across different testing platforms.

5. Synthesis and Conclusion

The primary takeaway is that as AI agents move toward full automation, developers must shift from "data-centric" testing to "spec-centric" validation. By explicitly defining the rules, domain knowledge, and robustness requirements of an agent, organizations can create a safer, more predictable deployment environment. The goal is to build agents that are "good enough" to perform their tasks effectively without possessing the capability to cause unintended harm, effectively closing the gap between raw model intelligence and reliable, production-ready software.