The Agent Factory - Episode 1: Agents, their frameworks and when to use them

Key Concepts:

Agentic AI, Domain-Specific Agents, LLMs, Memory, Tools, LangGraph, LangChain, CrewAI, ADK (Agent Development Kit), Multi-Agent Systems, MCP (Model Context Protocol), Evaluation, Production-Ready Agents, Asynchronous Operations, Token Usage, Code Executor, Agent Architecture, Sub-Agents, GKE, Cloud Run, Vertex AI Agent Engine.

Agent Industry Pulse and Framework Updates:

The hosts, Shu and Ivan, discuss the challenges of finding relevant AI agent news for developers, leading to the development of an agent to source information from Google search and Reddit. The agent identifies top developer conversations, model launches, framework updates, and emerging trends. Key open-source framework updates include:

• LangGraph: New features for Python and JavaScript, including node-level caching for faster development and deferred nodes to support map-reduce and multi-agent collaboration patterns.
• CrewAI: Focus on optimizing agents for synchronous tool and task execution, improving workflow efficiency and scalability for multi-agent systems. A key benchmark highlighted the performance degradation of single-agent systems as the number of tools increases, suggesting multi-agent architectures like supervisors or swarms maintain better performance and cost-efficiency.
• LlamaIndex: Launched a production-ready spreadsheet agent that allows users to interact with spreadsheets and ask questions in other languages.
• DataBricks: Agent bricks from databricks automates the process of building evaluating and optimizing production ready AI agent using your own enterprise data.
• Anthropic & Mistral: Anthropic released the Cloud Code SDK, enabling developers to integrate Claude's capabilities into their applications and CI/CD processes. Mistral launched its AI coding assistant, integrating directly into IDEs with options for on-premise deployment and fine-tuning on private codebases.
• ADK (Agent Development Kit): Hit its v1 release, now officially production-ready. Key improvements focus on developer experience and control, including the fully open-sourced ADK web UI with tracing and evaluation components, a more flexible MCP toolset for filtering tools per agent, and a URL context tool for directly extracting content from URLs. Token usage tracking was also introduced to help manage LLM costs.

ADK v1 Breaking Changes:

Ivan outlines three key breaking changes in ADK v1 that developers should be aware of:

1. Asynchronous Services: All main services are now asynchronous, requiring refactoring of older synchronous code.
2. Built-in Code Executor: The code executor is now a dedicated parameter of the agent, not an external tool, requiring changes to agent definitions using the code tool.
3. Evaluation Data Schema: The evaluation data schema has been redesigned for better understanding, necessitating updates to existing evaluation datasets.

ADK is now available for Java developers as well.

Factory Floor: What is an Agent?

Shu defines an agent as an application comprising three elements:

1. LLM (Large Language Model): Powers the brain for reasoning and planning.
2. Memory: Tracks the state and history.
3. Tools: Act as the hands to operate.

These elements work together to achieve a user-specified goal.

A key distinction is made between an agent and a simple FAQ chatbot, which performs keyword searches for pre-written articles in a single, reactive step. An agent, conversely, uses reasoning and dynamic tools to form a plan and adapt to new information.

Example: A customer support agent resolving a shipping issue.

• Plan: 1) Identify the customer and issue, 2) Investigate order status, 3) Communicate findings and provide a solution.
• Adaptation: If the internal order database doesn't provide new information, the agent reasons and calls an external shipping carrier API to determine if there is a weather delay.

The agent follows a loop of plan, act, observe, and replan until the goal is met. A logic map illustrating this process is provided in the episode resources.

Why Use a Framework?

Frameworks like LangGraph and ADK provide state management and observability tools, crucial for ensuring LLMs receive the right context at each step of a process. This helps prevent agents from forgetting their original goal. The frameworks enables developers to debug agents more efficiently.

Framework Comparison: LangChain, LangGraph, CrewAI, and ADK:

• LangChain: Foundational toolkit suitable for predictable workflows like RAG (Retrieval-Augmented Generation), but its flexibility can lead to complex, hard-to-debug chains.
• LangGraph: Provides more control over state and cycles, ideal for applications requiring observation and replanning. It has a steeper learning curve due to concepts like nodes and edges.
• CrewAI: Built for multi-agent collaboration with a role-based structure, but it's highly opinionated, making it less suitable for systems that don't fit the crew model.
• ADK: Aims to help developers not only build agents but also deploy them to production by overcoming the limitations of other frameworks.

ADK for Production Agents (with Julia Vizinger from Google):

Julia explains that ADK is designed to address the challenge of turning core agent logic into a secure, integrated, scalable, and production-ready application. It serves as an "opinionated assembly line" for production agents.

• Integration: ADK offers native support for protocols like MCP and seamless integrations with Google Cloud services like BigQuery.
• ADK Web UI: Provides a pre-built, customizable web interface for testing and demoing agents instantly.
• Multimodal Live API: Enables building agents that interact with more than just text, streaming audio and video in real-time.

Julia's three tips for developers:

1. Start with a clear, well-defined problem.
2. Invest heavily in tools and data integrations.
3. Use frameworks with good evaluation and observability capabilities.

Community Questions:

Question 1: When to use a sub-agent vs. an agent as a tool?

• Use an agent as a tool for single, well-defined service calls requiring intelligent, specialized output (e.g., a product description tailored and refined by multiple agents).
• Use a sub-agent for broader goals requiring handoffs and user interaction to gather additional information and reason (e.g., a personalized teacher designing a custom curriculum).

Question 2: When to use GKE vs. Cloud Run vs. Vertex AI Agent Engine?

• GKE (Google Kubernetes Engine): For maximum control or long-running tasks (DIY).
• Cloud Run: For stateless containerized agents due to fast deployment and scaling to zero (DIY).
• Vertex AI Agent Engine: A fully managed AI agent engine on Vertex AI built on Cloud Run, providing serverless benefits and integrated features like session management, tracing, and evaluation. It is the most "agent-native" solution on Google Cloud.

Question 3: How do you evaluate agents?

Evaluation requires a multi-faceted approach, considering not only the answer generated but also the tools used, how they were used, the reasoning process, and operational metrics like response time. Evaluation tools are being integrated into frameworks like ADK.

Conclusion:

The episode provides a comprehensive overview of agentic AI, from defining what an agent is to exploring different frameworks and deployment options. It emphasizes the importance of production readiness and offers practical advice for developers looking to build and deploy AI agents. The ADK is positioned as a solution for building production-ready agents with integrated features and a focus on developer experience. The key takeaway is to start with a clear problem, invest in tools and integrations, and iterate with robust evaluation.