How Building with AI Can Double the Throughput of Your Engineering Team — Brian Scanlan, Intercom

Key Concepts

• 2x Project: An internal Intercom initiative aimed at doubling engineering throughput without doubling team size.
• Claude Code: The primary AI platform adopted by Intercom for all technical tasks.
• Agent-First Development: A methodology where AI agents are the primary drivers of the software development life cycle (SDLC), moving engineers "up the stack."
• Skills: Modular, testable, and durable AI-driven components designed to perform specific technical tasks (e.g., fixing flaky tests, security incident response).
• Flywheel Effect: The continuous improvement loop where AI performance is refined through feedback, data mining of session transcripts, and backtesting.
• Developer Productivity: Measured primarily through code changes per R&D person, supported by developer surveys and DX tools.

---

1. Main Topics and Key Points

• Organizational Transformation: Intercom, a 15-year-old B2B SaaS company, successfully pivoted to an AI-first organization. The company emphasizes "shipping" as its heartbeat and has integrated AI into every facet of R&D.
• The 2x Goal: Launched in mid-2023, the project aimed to double engineering throughput. By standardizing on a single platform (Claude Code) rather than allowing fragmented tool usage, Intercom achieved this goal in under a year.
• Platform Strategy: Intercom mandates the use of a single AI platform to avoid "model anxiety" and to maximize the compounding benefits of a well-optimized, unified environment.
• Data-Driven Engineering: The company uses internal dashboards, Honeycomb for observability, and S3-stored session transcripts to analyze skill effectiveness and refine AI performance.

2. Real-World Applications

• Security Incident Response: Brian shared a case study where an AI agent automatically identified a data leak (snowflake table metadata), analyzed the risk, and determined it was innocuous, completing a 20-minute manual task in two minutes.
• Flaky Test Resolution: An internal "skill" was developed to identify and fix flaky specs. The agent, guided by human feedback, created a robust, organized solution that performed at the level of a senior Rails engineer.
• Automated Code Review: Intercom has implemented automated code approvals for safe, simple pull requests, achieving a 17.6% approval rate. This process is fully compliant with SOC 2, ISO 27001, and HIPAA.

3. Methodologies and Frameworks

• The "Problem, Not Task" Approach: Engineers are encouraged to provide agents with high-level problems rather than granular, step-by-step instructions. This allows the agent to determine the necessary skills and workflows.
• Continuous Improvement Flywheel:
  1. Capture: Log all technical work and session transcripts.
  2. Analyze: Use backtesting and human labeling to evaluate output quality.
  3. Refine: Update internal guidance, plugins, and "skills" based on performance data.
• Maturity Model: Intercom tracks engineer maturity based on their ability to:
  1. Use the AI tool for basic tasks.
  2. Automate workflows into reusable "skills."
  3. Optimize the architecture and environment to make agents more effective.

4. Key Arguments and Perspectives

• AI as a Career Multiplier: Brian argues that just as the cloud moved sysadmins to SRE roles, AI is moving engineers "up the stack." The work becomes more automation-oriented, impactful, and higher-value.
• Binary Expectations: Intercom enforces a strict policy: if an employee (designer, PM, or engineer) is not adopting AI, they are not meeting expectations.
• Risk Mitigation: Contrary to the belief that AI increases risk, Brian argues that well-defined, tested agents actually remove risk by eliminating human error in repetitive or complex tasks.

5. Notable Quotes

• "Shipping fast and iteratively is the best way to build high-quality products that customers love to use."
• "If you're not adopting AI in Intercom... you are not meeting expectations. Binary."
• "Our job is moving up the stack as engineers, product builders, whatever."

6. Technical Terms

• Majestic Monolith: A large, unified codebase (typically Ruby on Rails at Intercom) that is maintained as a single, cohesive unit rather than microservices.
• Backtesting: Using historical data and past code changes to validate that new AI skills or automated processes perform correctly and safely.
• Progressive Disclosure: A design pattern used in the AI's generated code/logic to manage complexity by revealing information only when necessary.

7. Synthesis and Conclusion

Intercom’s success in doubling engineering throughput is attributed to a combination of executive mandate, platform consolidation, and a culture of continuous improvement. By treating AI agents as first-class citizens in the SDLC and focusing on building durable, testable "skills," the company has moved beyond simple code completion to agent-driven problem solving. The primary takeaway is that organizations must stop treating AI as a "wrapper" and start integrating it into the core of their operational and architectural workflows to achieve significant productivity gains.