Harness Engineering: How to Build Software When Humans Steer, Agents Execute — Ryan Lopopolo, OpenAI

Key Concepts

• Harness Engineering: The practice of building software systems, structures, and processes that enable AI agents to execute the full software development lifecycle (SDLC) autonomously.
• Token Billionaire: A developer who utilizes massive amounts of model tokens to delegate the entirety of software engineering tasks to AI agents.
• Code as a Disposable Artifact: The perspective that code is free to produce, refactor, and delete; therefore, human focus should shift from implementation to defining guardrails and non-functional requirements.
• Context Engineering: The strategic surfacing of instructions, documentation, and constraints to agents at the exact moment they are needed to ensure high-quality output.
• Skills: Modular, reusable tools or instructions given to agents to perform specific actions (e.g., launching an app, running observability stacks, or interacting with Chrome DevTools).
• Non-Functional Requirements (NFRs): The "hidden" quality standards (reliability, security, maintainability) that agents must be explicitly prompted to follow to avoid "slop."

1. The Shift in Software Engineering

Ryan Leopo argues that implementation is no longer the scarce resource in software engineering. With the advent of advanced models (e.g., GPT-5.2+), code is abundant and free.

• New Role of the Engineer: The engineer’s role has shifted from "hands on keyboard" to systems thinking, system design, and delegation.
• Infinite Capacity: Engineers now have access to the equivalent of thousands of developers 24/7, constrained only by GPU capacity and token budgets.
• The "AGI Pill": Accepting that models are capable of producing, refactoring, and deleting every line of code required, provided the human provides the correct structure and guardrails.

2. Operationalizing "Harness Engineering"

To move from manual coding to agent-driven development, teams must build "harnesses" that make the codebase legible and navigable for agents.

• Outside-In Development: The development process should start with the agent (e.g., Codex) as the entry point, not the human environment.
• Skills-Based Architecture: Instead of building complex shells, provide agents with 5–10 high-leverage "skills" (e.g., a skill to boot Chrome DevTools or attach to a local observability stack).
• Standardization: To minimize context window usage, enforce a single, canonical way to perform common tasks (e.g., one way to handle async helpers, one ORM, one way to write CI scripts). This makes agent output predictable.

3. Managing Quality and "Slop"

A major challenge is preventing agents from producing low-quality code ("slop").

• Garbage Collection Days: Dedicate time (e.g., Fridays) to identifying recurring "slop" and systematically eliminating it by updating documentation or adding linting rules.
• Reviewer Agents: Deploy specialized agents that run during CI/CD to check for NFRs (e.g., ensuring every network call has a timeout and retry).
• Prompt Injection via Linting: Use linting error messages as a mechanism to provide "remediation steps" to the agent, effectively "prompting" it to fix its own mistakes.
• Test-Driven Constraints: Write tests that enforce structural rules (e.g., files must be under 350 lines) to keep the codebase context-efficient for the models.

4. Collaboration and Workflow

• Hub and Spoke Model: Use GitHub PRs as the primary collaboration hub where humans and agents interact.
• Delegation over Micromanagement: Avoid being a bottleneck. If an agent rejects feedback or chooses a different path, trust the agent’s reasoning unless it violates defined guardrails.
• The "Continue" Failure: Leopo notes: "Every time I have to type 'continue' to the agent, it is a failure of the harness to provide enough context." The goal is for the agent to reach completion without human intervention.

5. Synthesis and Conclusion

The future of software engineering is meta-programming: defining the processes, acceptance criteria, and guardrails that allow agents to operate autonomously. By treating the LLM as a "fuzzy compiler," engineers can focus on high-level product goals, reliability, and user experience, while the agents handle the heavy lifting of implementation. The ultimate goal is to reach a state where an engineer can provide a set of success metrics and a token budget, and the agents handle the entire lifecycle of the product from development to production monitoring.

Notable Quote: "The important thing is not the code, but the prompt and the guardrails that got you there."