Developer Experience in the Age of AI Coding Agents – Max Kanat-Alexander, Capital One

Developer Experience in the Age of AI Agents

Key Concepts:

• Developer Experience (DevEx): The holistic experience a developer has while interacting with tools, systems, and processes.
• AI Agents (Coding Agents): AI systems capable of autonomously writing, modifying, and testing code.
• No-Regrets Investments: Investments in areas that will provide value regardless of the future trajectory of AI technology.
• Standardization: Adopting industry-standard tools and practices to align with AI training data.
• Validation: Implementing robust, objective, and deterministic validation mechanisms for code.
• Testability & Code Structure: Designing codebases that are easily testable and understandable.
• Code Review Velocity: The speed and efficiency of the code review process.
• SLOs (Service Level Objectives): Specific, measurable goals for service performance, including code review turnaround times.

The Current Landscape & Future Uncertainty

The speaker begins by highlighting the rapid pace of change in the software development landscape over the past 12 months, characterized by a constant influx of new tools and technologies. This creates uncertainty for CTOs and DevEx professionals, leading to the question of whether current investments will be worthwhile by 2026. The initial reaction is often to focus solely on coding agents, but the speaker argues this is insufficient. A more nuanced approach is needed, focusing on investments that will benefit both human developers and AI agents. The core challenge is predicting the future, but focusing on foundational principles can mitigate risk.

Two Key Questions to Guide Investment

To navigate this uncertainty, the speaker proposes focusing on two key questions:

1. How can we leverage our understanding of DevEx principles to ensure lasting value, regardless of technological advancements?
2. What must we fix at all levels to maximize the effectiveness of AI agents, enabling both agents and developers to perform optimally?

Addressing these questions is critical for long-term success as a software business.

Inputs to Agents: Foundational Investments

The speaker frames the necessary investments around “inputs to agents” – things that enhance their effectiveness. The first and most crucial area is the development environment.

• Industry Standard Tools: Utilizing industry-standard package managers, linters, and other tools is paramount. This is because AI agents are trained on vast datasets of publicly available code, which predominantly uses these standard tools. Deviating from these standards requires fighting the training data and creates unnecessary complexity. The speaker explicitly advises against inventing custom package managers.
• Programming Language Choices: The speaker discourages the use of obscure programming languages in production code intended for agentic development. While personal enthusiasm for niche languages is acknowledged, practical work should focus on mainstream languages present in the training data.
• The Persistence of New Tool Evaluation: The speaker acknowledges the perennial challenge of developers wanting to adopt the latest tools. The core principle remains the same: premature adoption of unvetted technologies is risky, even with the advent of AI agents.

Enabling Agent Action: CLI & API Access

For agents to be effective, they require a means of taking action. The speaker strongly advocates for providing Command Line Interfaces (CLIs) or Application Programming Interfaces (APIs) for all relevant tasks.

• Text-Based Interaction: CLIs are preferred because they align with the agent’s native mode of interaction – text.
• Accuracy & Reliability: Using CLIs and APIs is particularly important in areas where accuracy is critical, as they offer more reliable execution than alternatives like browser automation (e.g., Playwright).

The Critical Role of Validation

Validation is identified as a key investment area.

• Objective & Deterministic Validation: High-quality, objective, and deterministic validation mechanisms are essential for increasing agent capabilities. Clear error messages are crucial, as agents cannot interpret vague errors like “500 Internal Error.”
• Addressing Legacy Codebases: A significant challenge is the prevalence of legacy codebases lacking adequate testing infrastructure. Agents attempting to write tests for such codebases often produce trivial tests (“I pushed the button and it worked”).
• Prioritizing Testability: Codebases must be designed for testability, allowing agents to iteratively run tests and receive actionable feedback.

Code Structure & Reasonability

The structure of systems and codebases is another critical factor.

• Agent & Human Comprehension: Well-structured code is easier for both agents and humans to understand and reason about. Poorly structured legacy code can be incomprehensible, hindering both.
• Testability Dependency: Code structure directly impacts testability. If the codebase is too complex to understand, it’s impossible to write meaningful tests.

Documentation: Shifting Priorities

The speaker revisits the perennial debate around documentation.

• Agent-Specific Needs: Agents cannot rely on undocumented tribal knowledge or information conveyed in verbal meetings.
• Focus on External Context: Documentation should focus on information not present in the code itself, such as the shape of external data or the rationale behind design decisions. Code itself should be self-documenting where possible.

Code Review in the Age of Agents

The speaker highlights a significant shift: writing code is now largely reading code. This elevates the importance of code review.

• Increased PR Volume: AI agents generate significantly more Pull Requests (PRs), increasing the burden on code reviewers.
• Improving Velocity: The goal is to improve code review velocity – the speed at which reviews are completed – without sacrificing quality.
• Addressing Bottlenecks: The common practice of broadcasting PR requests to a team channel creates a bottleneck, with one or two individuals handling the majority of reviews.
• SLOs & Assignment: Implementing Service Level Objectives (SLOs) and assigning reviews to specific individuals is crucial for distributing the workload.
• GitHub Limitations: Current tools like GitHub lack clear indicators of who is responsible for taking action on a PR.
• Maintaining Quality: Despite the need for speed, maintaining a high bar for code review quality is essential to prevent technical debt and ensure long-term maintainability.
• Investing in Junior Engineers: Experienced code reviewers should mentor junior engineers through code reviews, fostering their growth and distributing expertise.

The Virtuous vs. Vicious Cycle

The speaker concludes by outlining the potential outcomes of these investments.

• Vicious Cycle: Ignoring these principles leads to a vicious cycle of bad code, frustrated developers, and decreasing agent productivity.
• Virtuous Cycle: Investing in these areas creates a virtuous cycle of increasing productivity and accelerating software development velocity.

Final Thoughts: The speaker emphasizes that these investments are not merely about enabling AI agents; they are about improving the developer experience for all developers, regardless of whether they are human or artificial. Investing in these areas is a “no-regrets” strategy that will yield benefits regardless of the future of AI. The key takeaway is that what’s good for humans is good for AI.