Vibes won't cut it — Chris Kelly, Augment Code

Key Concepts

AI coding, production systems, code generation, software engineering, code review, LLMs (Large Language Models), context, software development lifecycle, adoption of AI, vibe coding, four nines (99.99% availability), monolith vs. microservices, emergent behavior, changing software safely, documented standards, reproducible environments, code review, AI agents.

AI Coding: Hype vs. Reality

The speaker addresses the hype surrounding AI coding, particularly the notion that it will replace software engineers. While acknowledging the intelligence behind these claims, the speaker argues that they are likely overstated because the proponents haven't worked on production systems recently. The core argument is that AI-generated code is still code and must function within existing, complex systems.

• 30% Code Generation: The speaker questions the significance of AI generating 30% of code, especially within large codebases where architectural decisions are already established.
• Meta Example: The speaker uses the example of Meta engineers spending six months building a button to illustrate the limited scope for AI influence in highly defined tasks.
• Code Still Needs to Run: The speaker emphasizes that AI-generated code must still run in production environments, where complex systems exhibit emergent behavior and failures occur. This necessitates human expertise for debugging and maintenance.
• History Repeats Itself: The speaker draws a parallel to the DevOps transformation, where system administrators adapted and thrived by working on more valuable tasks. The analogy of tractors replacing farmhands, not farms, is used to illustrate that AI will change the industry but not eliminate it.

Vibe Coding vs. Production Software

The speaker contrasts "vibe coding," where AI writes code without thorough examination, with the requirements of production software.

• Four Nines: Production software requires "four nines" (99.99%) availability, handling thousands of users and gigabytes of data. Vibe coding is insufficient for such demanding environments.
• Code is Not the Job: The speaker argues that generating code is not the primary function of a software engineer. The job involves making decisions about software architecture, package selection, and overall design.
• Jeff Atwood Quote: The speaker quotes Jeff Atwood (founder of Stack Overflow): "The best code is no code at all." Every line of code introduces maintenance and debugging burdens. The goal should be to minimize code while maximizing functionality.
• Monolith vs. Microservices: The speaker uses the example of building a flight booking system using monolith, microservices, and adventure system architectures to illustrate the thousands of decisions that go into software design, decisions that LLMs cannot make.
• Snowflake Software: The speaker points out that many production systems have unique idiosyncrasies that cannot be pattern-matched by AI. When these systems fail, human expertise is required for diagnosis and resolution.

Changing Software Safely

The speaker defines the core work of software engineering as "changing software safely," which involves adding new functionality or modifying existing code without causing system failures.

• Methods for Safe Changes: The speaker lists methods used to ensure safe changes, including personal knowledge, version control, testing, type systems, and deployment strategies.
• Context is Key: The speaker's company, Augment, believes that context is the most important factor in AI code generation. AI can augment these processes, but it doesn't eliminate the need for human oversight.

Adoption of AI by Software Engineers

The speaker notes that professional software engineers are the last to adopt AI coding tools, a trend they find unusual.

• Evolution of AI Coding: The speaker traces the evolution of AI coding from a "pile of bricks" to the significant improvement with GPT-3.5 and the recent emergence of AI agents.
• Building Software for AI: The speaker provides tips for building software that is easier for AI to write:
  • Documented Standards and Practices: Maintain clear documentation of coding standards and package usage.
  • Reproducible Environments: Ensure that developer environments can be easily spun up.
  • Easy Testing: Enable fast and local test execution.
  • Clear Boundaries: Define specific tasks for AI, avoiding vague instructions.
  • Clearly Defined Tasks and Work: Break down complex tasks into smaller, manageable units.

Code Review: A Critical Skill

The speaker emphasizes the importance of code review as a critical skill in the age of AI-generated code.

• Code Review Tools: Current code review tools are inadequate, often presenting changes in a lexicographical order rather than a logical flow.
• Interviewing for Code Review: The speaker suggests that companies should prioritize code review skills in their hiring processes.

Tips for Software Engineers Using AI

The speaker provides several tips for software engineers who are hesitant to use AI:

• AI Talks Like a Human, But Is a Machine: Be aware that AI may generate text that mimics human communication but doesn't necessarily reflect its actual processes.
• Code is Just Different: Accept that AI-generated code may differ from human-written code. Focus on functionality and adherence to style guides rather than personal preferences.
• Write a Rules File: Provide AI with a file outlining the project's stack, guidelines, and desired coding style.
• Create, Refine Loop: Use a workflow that involves creating a plan, having AI generate code based on the plan, and then refining the code through human edits.

Conclusion

The speaker concludes by reiterating their belief that software engineering jobs are not going away. AI will change the industry, but human expertise remains essential for building, maintaining, and ensuring the safety of production software. The key is to adapt to the new tools and focus on skills like code review and system design.