Everything You Thought About Building AI Agents is Wrong

Key Concepts

• AI Agent Frameworks: Libraries like Pydantic AI, LangChain, and N8N used for building custom, scalable AI agents from the ground up.
• Batteries-Included SDKs: Developer kits like the Claude Agent SDK or Codeium SDK that provide pre-built tools, memory management, and prompting capabilities.
• MCP (Model Context Protocol): A standard for connecting AI assistants to systems, allowing for reusable, modular tools.
• Skills: A modular approach to giving agents capabilities (e.g., generating PowerPoints) via external instruction files.
• RAG (Retrieval-Augmented Generation): A technique for providing LLMs with external data; evolving from simple semantic search to "Agentic RAG" and "Graph RAG."
• Non-determinism: The degree to which an agent’s behavior is managed by the underlying SDK versus being explicitly defined by the developer.

1. The Evolution of Agent Development

The landscape of AI agent development has shifted from building everything from scratch using frameworks to leveraging "batteries-included" SDKs.

• The "Old Way" (2024–2025): Developers manually defined agent loops, state/memory management, and RAG pipelines. This required significant "glue code" to handle chunking, embedding, and database interactions (e.g., using PostgreSQL/Neon).
• The "New Way" (SDK-first): Modern SDKs (like the Claude Agent SDK) handle conversation history, tool integration, and prompting out of the box. This allows for rapid prototyping with significantly less code.

2. Decision Framework: SDKs vs. Frameworks

The speaker proposes a simple two-question framework to decide which path to take:

1. Who is the user? If it is a personal tool (e.g., a "second brain"), use an SDK. If it is a multi-user production application, use a framework.
2. What is the tolerance for speed and scale? SDKs are often "bloated" with reasoning overhead, making them slower and more token-heavy. Frameworks allow for sub-second response times and granular control over costs.

Key Trade-offs:

• SDKs: High convenience, built-in tools, but higher latency, higher token costs, and potential Terms of Service (ToS) violations if used for multi-user production apps (due to subscription-based API usage).
• Frameworks: High performance, cost-effective at scale, full observability, and total control over the agent’s logic.

3. Modernizing Agent Capabilities: Skills and MCP

Instead of hard-coding tools, developers are moving toward modular architectures:

• MCP Servers: Allow tools to be defined once and reused across different agents.
• Skills: By using a skills/ directory containing .md instruction files, agents can dynamically load capabilities. The agent reads the instructions and executes the necessary API calls, mimicking the power of built-in SDK tools while maintaining the flexibility of a custom framework.

4. The State of RAG (Retrieval-Augmented Generation)

There is a common narrative that "RAG is dead" in favor of simple file search. The speaker clarifies this nuance:

• File Search (Grep/Keyword): Highly effective for small codebases and smaller knowledge sets. It often outperforms semantic search in these specific contexts.
• Semantic Search: Remains essential for large-scale enterprise knowledge bases (thousands of documents) where keyword matching is insufficient.
• Agentic/Graph RAG: The current industry standard involves a hybrid approach—using file search for immediate context and semantic/graph-based search for deeper, broader knowledge retrieval.

5. Notable Quotes

• "There’s a lot of nuance that we have to cover here... everything you thought about building AI agents needs to be thrown out of the window [for some use cases], but then for other use cases, this isn’t true at all."
• "If you have multiple people using your agent in production, you have to use your API key. You have to pay an obscene amount for API costs. So, that’s really another limitation for these SDKs."

6. Synthesis and Conclusion

The choice between an SDK and a framework is not binary; it is a matter of use-case alignment.

• Use SDKs for rapid development, personal productivity tools, and when the "batteries-included" features outweigh the need for extreme speed.
• Use Frameworks for production-grade applications where cost-efficiency, low latency, and observability are non-negotiable.

The most effective modern strategy is to start with an SDK to test functionality (using Skills and MCP) and transition to a custom framework (like Pydantic AI) once the agent is ready for production scaling.