Every API Is a Tool for Agents - Matt Carey, Cloudflare

Key Concepts

• MCP (Model Context Protocol): A standardized protocol for connecting AI agents to external tools and data sources.
• Tool/Function Calling: The process where an LLM identifies a function to execute to retrieve data or perform an action.
• Context Window Explosion: The issue where providing too many tool definitions to an LLM consumes its available token limit, rendering it ineffective.
• Progressive Discovery: A methodology where tools are loaded into an agent's context only when needed, rather than all at once.
• Code Mode: A paradigm where agents are provided with typed SDKs and write executable code to interact with APIs, rather than relying on pre-defined tool calls.
• Programmable Sandboxes: Secure, isolated environments (e.g., Cloudflare Workers, Deno, Pydantic Monty) used to execute untrusted code generated by AI agents.

1. The Evolution of Agent-API Interaction

The speaker, Matt from Cloudflare, outlines the progression of how agents interact with the world:

• Bundled Tools: Early agents had tools hard-coded into them, leading to redundant development.
• Shared Tools (MCP): The rise of MCP allowed for standardized, reusable tools. However, as API surfaces grew (e.g., Cloudflare’s 2,600+ endpoints), the "naive" approach of loading all tools into the context window caused it to "explode," exceeding token limits (e.g., 1.1 million tokens for tool definitions).
• The "Split" Strategy: Initially, companies split APIs into multiple smaller MCP servers. This proved inefficient, as it required manual user selection and often resulted in incomplete API coverage.

2. Methodologies for Scaling API Access

To solve the context window problem, the speaker proposes three primary approaches:

1. CLI-based Interaction: Agents use shell access to run CLI commands (e.g., Wrangler). While popular, it requires shell access, which is often restricted or insecure.
2. Tool Search: A keyword-matching system that dynamically loads a subset of relevant tools into the context. This is effective but still limited by the static nature of the tools.
3. Code Mode (The Preferred Approach): Instead of defining thousands of tools, the API provides TypeScript types. The agent writes code against these types. This is more flexible, compact, and leverages the LLM's ability to reason through complex logic.

3. Security and Execution: The "Untrusted Code" Problem

A major barrier to "Code Mode" is the risk of executing untrusted code. The speaker argues that modern infrastructure primitives have solved this:

• Dynamic Workers: Using isolated V8 environments (like Cloudflare’s WorkerD), code can be executed with strict guardrails.
• Programmable Guardrails: Developers can toggle permissions (e.g., internet access, environment variable access) via simple booleans or functions, ensuring the agent cannot exfiltrate secrets or consume excessive resources.
• Comparison: The speaker notes that similar primitives are emerging in other ecosystems, such as Deno and Pydantic Monty, signaling a broader industry shift toward safe, agent-driven code execution.

4. Future Outlook: Client-Side Innovation

The speaker predicts significant shifts in how agents are built:

• Programmatic Tool Calling: Clients will increasingly move toward generating and running code rather than simple tool calls.
• Saved Mini-Scripts: Agents will generate scripts for recurring tasks (e.g., web scraping, cron jobs). If the script breaks, the agent will self-heal and resave the script.
• MCP as Middleware: MCP will become a native, lightweight feature in full-stack frameworks (like Next.js). Developers will simply set MCP = true to expose their entire API surface to agents.

5. Notable Quotes

• "We shouldn't be dumping loads of tools into context. That's like the main thing."
• "Code is actually a very compact plan. Instead of doing tool calls, you can have one tool called 'code' where the model generates the code of your choice and then you run it."
• "In the 1950s... you printed out some punch cards... that was kind of like running untrusted code. And now I think we're going to go much more back to that where your users can write code, 'cause your users are AI."

Synthesis

The core takeaway is that the industry is moving away from static, pre-defined tool sets toward dynamic, code-based interaction. By treating APIs as typed SDKs and utilizing secure, isolated sandboxes for execution, developers can provide agents with full access to massive API surfaces without overwhelming context windows. The future of agentic workflows lies in "Code Mode," where agents act as developers, writing and maintaining their own scripts to interact with infrastructure.