Vibe coding in prod | Code w/ Claude

Key Concepts

• Vibe Coding: Giving into the vibes, embracing exponentials, and forgetting the code exists.
• Leaf Nodes: Parts of the codebase where nothing depends on them.
• Core Architecture: The underlying branches and trunks of the system that need to be deeply understood.
• Verifiability: Knowing whether a change is correct without needing to read the code.
• Exponential Growth: The length of tasks that AI can do is doubling every seven months.
• Abstraction Layer: A layer that can be verified without knowing the implementation underneath it.

1. Defining Vibe Coding

• Vibe coding is not just extensive use of AI to generate code, like with Cursor or Copilot, where there's a tight feedback loop.
• Andre Carpathy's definition: Fully giving into the vibes, embracing exponentials, and forgetting the code even exists. The key is "forget the code even exists."
• Vibe coding became popular because it allowed non-engineers to code entire apps, which was a significant unlock.

2. Downsides and Upsides of Vibe Coding

• Downsides: Random things happening, maxed-out API usage, bypassed subscriptions, and random database entries.
• Upsides: Building video games and fun side projects where bugs are acceptable.
• The speaker poses the question of why we should care about vibe coding if it seems high-stakes for real products and successful cases are toy examples.

3. The Exponential and the Need for Responsible Vibe Coding

• The length of tasks AI can do is doubling every seven months. Currently, AI can handle about an hour's worth of work.
• Next year, AI might generate an entire day's or week's worth of work. It will be impossible to keep up in lockstep.
• To leverage this exponential growth, we must find a way to responsibly give in and leverage the task.
• Analogy: Compilers. Early developers didn't trust them and read the assembly output. But that doesn't scale. We must trust the system.
• The challenge is how to vibe code in production safely. The answer is to forget that the code exists but not that the product exists.
• We should still be able to build good software without understanding the assembly under the hood.

4. Managing Implementations You Don't Understand

• This is not a new problem. Examples: How does a CTO manage an expert they aren't an expert in? How does a PM review an engineering feature they can't read the code for? How does a CEO check the accountant's work?
• Solutions exist: A CTO can write acceptance tests, a PM can use the product, and a CEO can spot-check key facts.
• Managing implementations you don't understand is as old as civilization. Every manager deals with this.
• Software engineers are used to being individual contributors who understand the full depth down to the stack.
• To be most productive, we need to let go of some details, like understanding the assembly.
• The way to do this safely is to find an abstraction layer that you can verify without knowing the implementation underneath it.

5. Tech Debt Caveat and Focus on Leaf Nodes

• Currently, there is no good way to measure or validate tech debt without reading the code.
• Most other systems have ways to verify things you care about without knowing the implementation.
• Tech debt is a rare thing where there isn't a good way to validate it other than being an expert in the implementation.
• Focus on leaf nodes in the codebase: parts of the code where nothing depends on them.
• It's okay if there is tech debt in leaf nodes because nothing else depends on them. They are unlikely to change.
• Protect the core architecture (trunks and underlying branches) because that's what will change and what other things will be built on.
• Models are getting better, and we might trust them more to write extensible code without tech debt.
• The speaker has given Claude 4 models much more trust than 3.7.

6. How to Succeed at Vibe Coding: Be Claude's PM

• Ask not what Claude can do for you, but what you can do for Claude.
• Think like a product manager. What guidance or context would a new employee need to succeed at this task?
• Don't just do quick back-and-forth chats. Give Claude a tour of the codebase, requirements, specifications, and constraints.
• Spend 15-20 minutes collecting guidance into a single prompt and then let Claude cook.
• This involves a separate conversation with Claude, exploring the codebase, looking for files, and building a plan together.
• Once you have all that information, give it to Claude.
• You need to be able to ask the right questions.
• Vibe coding in production is not for everyone. Non-technical people shouldn't try to build a business fully from scratch because they can't be effective product managers for Claude.

7. Case Study: 22,000-Line Change to Production Reinforcement Learning Codebase

• Merged a 22,000-line change to the production reinforcement learning codebase that was heavily written by Claude.
• Days of human work went into coming up with the requirements and guiding Claude.
• The change was largely concentrated in leaf nodes.
• Heavy human review of the parts that needed to be extensible.
• Carefully designed stress tests for stability.
• Designed the system to have easily human-verifiable inputs and outputs.
• Created verifiable checkpoints to ensure correctness without reading the full implementation.
• Biggest concern was stability, which was measured through stress tests.
• Verified correctness based on the input and outputs of the system.
• The system was designed to be understandable and verifiable without reading all the code.
• This allowed them to be just as confident in the change as any other change but deliver it in a tiny fraction of the time and effort.
• Knowing they could do this made them think differently about their engineering and what they could do.
• Marginal cost of software is lower, letting you consume and build more software.

8. Closing Thoughts

• Be Claude's PM.
• Focus vibe coding on leaf nodes, not the core architecture.
• Think about verifiability.
• Remember the exponential. It's okay today if you don't vibe code, but in a year or two, it will be a huge disadvantage if you demand to read or write every line of code.
• You will not be able to take advantage of the newest wave of models and will become the bottleneck.

9. Q&A Highlights

• How do we learn now? If you take the time and want to learn, there are amazing resources, and Claude will help you understand what it vibe coded for you. We'll be able to take so many more shots on goal.
• Balance between too much and too little information? It depends on what you care about. If you don't care how it does it, just give the requirements. If you know the codebase well, go into more depth. Models do best when you don't over-constrain them.
• Balance effectiveness and cybersecurity? All comes down to being Claude's PM and understanding enough about the context to know what is dangerous and safe.
• How do products need to change to make it easier for people to vibe code and build software while avoiding API key leaks? Build provably correct systems where the important parts are built for you, and you just fill in the UI layer.
• Tips for test-driven development? Test-driven development is very useful in vibe coding. Encourage Claude to write minimalist end-to-end tests.
• How to embrace exponentials? Assume the models are going to get better faster than we can possibly imagine.
• Workflow for vibe coding? The speaker uses both terminal and VS Code. Claude Code is doing most of the editing, and the speaker is reviewing the code in VS Code. Compact whenever you get Claude to a good stopping point.
• How to approach a part of the codebase you're not familiar with? Use Claude Code to help explore the codebase. Ask it to tell you where things happen and similar features.

10. Synthesis/Conclusion

The presentation advocates for a strategic and responsible approach to "vibe coding" in production environments, leveraging the exponential growth of AI capabilities. It emphasizes the importance of understanding the product requirements, acting as a product manager for AI models like Claude, focusing on leaf nodes in the codebase, and ensuring verifiability through well-designed tests and abstraction layers. The key takeaway is that while the models are rapidly improving, human oversight and strategic guidance remain crucial for successful and safe implementation of AI-generated code in production systems. The future of software engineering will involve a shift from writing every line of code to managing and verifying AI-generated code, requiring a new set of skills and a different mindset.