Building Agents (the hard parts!) - Rita Kozlov, Cloudflare

Key Concepts:

• Cloudflare Workers and Durable Objects
• AI Agents: Client, AI (Reasoning), Workflows, Tools
• MCP (Machine Communication Protocol): A standard for LLMs to interact with tools.
• Agents SDK: Cloudflare's SDK for building AI agents.
• Durable Objects: Serverless functions with state attached.
• Human-in-the-loop workflows

1. Introduction and Cloudflare's Mission

• Rita, VP of Product for Cloudflare's developer platform (Workers and Durable Objects), introduces Cloudflare's mission to simplify the development process from code to production.
• Cloudflare handles approximately 20% of internet traffic, meaning most internet users have indirectly used Cloudflare services.
• Cloudflare offers services for developers, including functions, storage, compute, and AI inference.

2. The Rapid Evolution of AI

• AI is recognized as a major technological paradigm shift, comparable to cloud, mobile, and social.
• A year ago, around 44% of developers used AI daily for coding. Gartner predicted 50% of knowledge workers would use AI by 2030.
• Currently, over 75% of knowledge workers and 76% of developers use AI, surpassing previous estimates.
• AI workloads are shifting from training to inference, exemplified by OpenAI's 01 model and Deepseek's training optimization.

3. The Rise of AI Agents and Automation

• The next step after training and inference is automation through AI agents.
• AI agents can automate complex tasks, such as customer campaign management, email drafting, and response tracking.
• Examples of agentic workflows: "I have a campaign I want to run grab me a full list of the customers that I talked to this week at the conference uh then draft me up the email then actually I do want to review it before it goes to a customer so do send it to me for approval and then ping me when the customer responds"
• Businesses adopting AI agents are seeing significant benefits:
  • 20% revenue increases in sales automation.
  • 90% faster response times in support.
  • 50-75% time savings in general tasks.

4. Components of an AI Agent

• An AI agent consists of four key components:
  • Client: The interface for human interaction (e.g., voice, chat UI).
  • AI (Reasoning): The "thinking" part that determines the next action.
  • Workflows: Manages the execution of actions.
  • Tools: Provides access to resources and services (e.g., web browser, APIs, databases).

5. Example: CRM Agent

• A CRM agent example is provided to illustrate the components:
  • Client: WebRTC for voice interaction or a chat UI.
  • AI: An LLM to plan and execute actions.
  • Workflows: Tracks executed and pending actions.
  • Tools: Access to a web browser, APIs, internal services, or a vector database.
  • Human-in-the-loop verification may be required.

6. Building Agents: Tools and MCP

• The discussion starts with the "tools" component and MCP (Machine Communication Protocol).
• Anthropic introduced MCP, which facilitates LLMs' interaction with APIs through natural language.
• MCP enables LLMs to effectively use tools, a capability that has significantly improved recently.
• MCP follows a client-server architecture, allowing multiple clients to connect to an MCP server.
• MCP servers include resources (files, databases), prompts (instructions for the agent), tooling (API connections), and sampling (shorthand for LLM completion).
• Challenges in building MCP servers include transport protocol (SSE, websockets), authentication (Oauth), and memory management.

7. Cloudflare's Agents SDK: A Cheat Code

• Cloudflare offers the Agents SDK to simplify MCP server development.
• The Agents SDK provides:
  • MCP agent class for hosting remote MCP servers with built-in Oauth, transport, and HTTP streaming.
  • State management using Durable Objects (serverless functions with attached state).
  • Real-time websocket communication.
  • React integration hooks.
  • Basic chat capabilities.

8. Deploying an MCP Server on Cloudflare

• An example of deploying a "Goodreads" MCP server is provided:
  • Define an MCP class extending MCP Agent.
  • Add tools like add genre to save user preferences.
  • Create a get recommendations tool with personalized prompts.
  • The memory of user preferences persists across interactions.
• Benefits of using MCP Agent:
  • No need to set up a database or manage connections.
  • Automatic scaling and low latency.
  • Easy deployment with a "Deploy to Cloudflare" button.
• Companies like Atlassian, Asana, Stripe, and Intercom are building MCP servers using this approach.

9. Workflows and Human-in-the-Loop

• Workflows are essential for maintaining state across tool interactions, especially with human-in-the-loop processes.
• Human-in-the-loop workflows require long-running tasks that can handle delays in LLM reasoning or human responses.
• A use case with Knock (notification management) is presented:
  • An agent provisions credit card requests with approval from a manager.
  • Users request a card through a chat interface using the use agent hook from the Agents SDK.
  • The issue card action requires human input delegated to Knock.
  • Approval notifications are sent via email, Slack, or in-app notifications.
  • The tool call to issue the card is stalled until approval is received.
  • Durable Objects automatically route messages to the correct agent.
  • Duplicate actions are prevented through state management.

10. Reasoning and Choosing the Right Model

• Choosing the right AI model for reasoning is crucial.
• The speaker defers to other experts on model selection and evaluation.

11. Client and User Interface

• MCP allows users to interact with agents through various clients.
• Developers can use existing tools like Cursor, Claude, and ChatGPT, which support remote MCP servers.
• Custom apps can be built for more control over the client and server interaction.
• Voice interaction is possible using Cloudflare tools that translate WebRTC to websockets.

12. Conclusion

• Building an AI agent requires careful consideration of the client, AI (reasoning), workflows, and tools.
• The Agents SDK is recommended as a starting point for building AI agents.

Notable Quotes:

• "Our vision for developers is to make it as easy as possible for someone to bring their idea to life from the moment that they write their first line of code to deploying it to production to making it live for the first user to the millions that come after that." - Rita, VP of Product, Cloudflare
• "I think that the uh the real missed headlines of MCPs was actually that LLMs became really really good at tool calling." - Rita, VP of Product, Cloudflare

Technical Terms:

• Cloudflare Workers: Serverless execution environment that allows you to create scalable and event-driven applications.
• Durable Objects: Serverless functions with state attached, providing a way to persist data without managing a database.
• MCP (Machine Communication Protocol): A standard for LLMs to interact with tools and APIs.
• Agents SDK: Cloudflare's software development kit for building AI agents.
• LLM (Large Language Model): A type of AI model that can understand and generate human language.
• WebRTC: A technology that enables real-time communication over the web.
• SSE (Server-Sent Events): A server push technology that enables a server to send updates to a client over HTTP.
• Oauth: An open standard for access delegation, commonly used to grant websites or applications access to their information on other websites without giving them their passwords.

Synthesis/Conclusion:

The presentation outlines the key components and steps involved in building AI agents, emphasizing the importance of tools, workflows, reasoning, and client interfaces. Cloudflare's Agents SDK and Durable Objects are presented as valuable resources for simplifying agent development, particularly in managing state and integrating with various tools and services. The rise of MCP as a standard for LLM tool interaction is highlighted, and practical examples are provided to illustrate the concepts and benefits of using AI agents in real-world applications. The main takeaway is that building effective AI agents requires a holistic approach, combining the right technologies and tools to create seamless and automated workflows.