Is This the End of MCP for AI Agents?

Key Concepts

• Model Context Protocol (MCP)
• Context Rot
• Tool Definitions
• Intermediate Tool Results
• Code Agents
• TypeScript API
• Directory Structure for Tools
• Bash and Grip Commands
• Code Execution
• Progressive Disclosure
• Privacy-Preserving Operations
• State Persistence
• Skills (Anthropic)
• RPC Interface
• Uniform API Interface
• Sandbox Execution

Problems with Traditional MCP for Agents

The video transcript highlights significant practical limitations of the Model Context Protocol (MCP) when used for AI agents, primarily stemming from context management issues.

• Context Rot: MCPs contribute heavily to "context rot," where the context window of an LLM or agent becomes overloaded with unnecessary information, diminishing its effectiveness.
• Tool Definition Bloat: Even before any messages are sent, the mere presence of tool definitions within the MCP can consume a substantial portion of the context window. For instance, the transcript mentions that with only two MCPs connected, the context already uses nearly 20,000 tokens, representing about 10% of the context, solely for tool definitions.
• Intermediate Tool Result Consumption: When a tool is executed, its intermediate results are also fed into the LLM's context. This further exacerbates context bloat, as these results may not always be relevant or necessary for the agent's immediate task.
• Example of Context Overload: The transcript cites an example from an Anthropic blog post where a two-hour sales meeting transcript, returned by a tool, could add approximately 50,000 tokens to the context, which is likely not needed.

Proposed Solution: Code Agents Interacting with Tools Independently

The core argument presented is that the solution lies not in directly using tools through MCP but in building code agents that interact with these tools independently through code.

• Inspiration from Existing Research: This approach is inspired by research such as the paper "Executable Code Actions Elicit Better LLM Agents" and Cloudflare's "Code Mode."
• Cloudflare's "Code Mode" Approach: Cloudflare's strategy involves converting MCP tools into TypeScript APIs and then instructing an LLM to write code that calls these APIs.
  • Mechanism: MCP tools are essentially wrappers around APIs. The idea is to create "tool agents" that directly call these APIs or tools.
  • Benefits: According to the Cloudflare team, agents can handle a greater number and complexity of tools when presented as TypeScript APIs rather than being directly invoked via MCP. This is particularly beneficial for chaining multiple tools together.

Step-by-Step Process for Code Agent Interaction

The transcript outlines a methodology for implementing this code agent approach:

1. Treat Tools as a Directory Structure: Each tool is implemented in a separate file, creating a directory-like structure for available tools.
2. Agent Generates Code to Call Tools: The AI agent is responsible for generating code to invoke different tools.
3. Tool Discovery via Code Execution: Instead of loading all tool definitions into the LLM's context, the agent uses simple bash tools and grep commands to search for relevant tools within the directory structure. This is akin to retrieval on code.
4. Code Execution in a Sandbox: The generated code is executed in a separate, sandboxed environment (e.g., a dynamic isolate sandbox) that is isolated from the main agent's context. This prevents context rot.
5. Data Retrieval and Processing: If tools need to access external data, this can be handled through code execution, allowing for further processing of the retrieved data.
6. Concatenation of Tool Calls: Multiple tool calls can be chained together.
7. Final Results Sent to LLM: Once all necessary operations are complete, the final results, along with the tasks performed, are passed to the LLM or AI agent. This preserves the LLM's context without introducing excessive noise.

Architecture Comparison: Traditional MCP vs. Cloudflare's Code Mode

The transcript contrasts the traditional MCP server architecture with Cloudflare's recommended "Code Mode" architecture:

• Traditional MCP Server:

  • MCP server provides tool schema and definitions.
  • Agent receives a list of tools.
  • Agent decides which tool to use and provides instructions for execution.
  • This process directly pollutes the agent's context.

• Cloudflare's Code Mode Architecture:

  • MCP server provides tool schema.
  • Instead of passing tool definitions directly, a list of API calls is created.
  • The agent searches for relevant tools based on its needs.
  • The agent writes code to make API calls.
  • An "agent worker" (a separate sandbox) executes these API calls.
  • Results are sent back to the LLM.
  • Execution happens in a dynamic isolate sandbox, preventing context rot.

Key Arguments and Supporting Evidence

• Argument: MCPs are inefficient for context management in agents.
  • Evidence: The transcript cites the significant token usage for tool definitions and intermediate results, leading to "context rot."
• Argument: Code agents interacting with tools via APIs are a more efficient and robust approach.
  • Evidence: Cloudflare's experience shows agents can handle more complex tools when presented as TypeScript APIs. The sandbox execution prevents context pollution.
• Argument: This approach offers benefits like progressive disclosure and privacy preservation.
  • Evidence: Tools are loaded on demand, and data can be anonymized before being sent to the LLM.
• Argument: MCPs still hold value for their uniform interface.
  • Evidence: MCPs provide a standardized way to connect to and learn about APIs, enabling interoperability between agents and servers that may not know each other. This uniform interface is beneficial even when using code agents.

Notable Quotes and Significant Statements

• "MCP or model context protocol is great in theory, but in practice, well, not so much." (Attributed implicitly to the speaker's observation)
• "MCPS are pretty bad at it [context management]." (Attributed implicitly to the speaker's observation)
• "So using MCP servers are contributing a lot to context rot." (Attributed implicitly to the speaker's observation)
• "The solution is not to directly use tools through MCP but rather build code agents that are going to interact with these tools independently through code." (Attributed implicitly to the speaker's recommendation)
• "The idea is that we are going to be providing a unified or uniform API interface." (Attributed implicitly to the speaker's explanation of MCP's value)
• "MCP makes it possible for agentic frameworks to implement this [sandbox execution] by handling connectivity and authorization in the standard way independent of the AI code." (Attributed implicitly to the speaker's explanation of MCP's value)

Technical Terms and Concepts Explained

• Model Context Protocol (MCP): A protocol designed for agents to interact with tools, typically by exposing tool definitions and schemas.
• Context Rot: The degradation of an LLM's performance due to an overloaded context window with irrelevant or excessive information.
• Tool Definitions: Descriptions of the functionalities and parameters of tools that an agent can use.
• Intermediate Tool Results: The outputs or partial results generated by a tool during its execution.
• Code Agents: AI agents that can write and execute code to interact with external systems or tools.
• TypeScript API: An application programming interface defined using the TypeScript programming language, offering structured access to functionalities.
• Directory Structure for Tools: Organizing tools as files within a hierarchical directory, allowing for programmatic discovery and access.
• Bash and Grip Commands: Command-line utilities used for scripting and text pattern matching, employed here for searching tool definitions.
• Code Execution: The process of running computer code, in this context, to invoke tools or APIs.
• Progressive Disclosure: Revealing information or functionalities only when they are needed, rather than presenting everything upfront.
• Privacy-Preserving Operations: Techniques that protect sensitive data, such as anonymization, before it is processed or sent to an LLM.
• State Persistence: The ability of a system or agent to retain its state and information across different interactions or sessions.
• Skills (Anthropic): A new concept introduced by Anthropic that aims to progressively load information, similar to progressive disclosure.
• RPC Interface (Remote Procedure Call): A protocol that allows a program to cause a procedure (subroutine) to execute in another address space (commonly on another computer on a shared network) without the programmer explicitly coding the details for this remote interaction.
• Uniform API Interface: A standardized way for different systems or agents to interact with APIs, promoting interoperability.
• Sandbox Execution: Running code in an isolated environment to prevent it from affecting the host system or other processes.

Logical Connections Between Sections

The transcript logically progresses from identifying a problem with a current technology (MCP) to proposing and detailing a solution.

1. Problem Identification: The initial sections clearly articulate the shortcomings of MCPs, specifically focusing on context management issues like context rot, tool definition bloat, and intermediate result consumption.
2. Solution Proposal: The transcript then introduces the concept of code agents as an alternative to direct MCP usage.
3. Methodology and Examples: The explanation of how code agents work, including the directory structure, code generation, and sandbox execution, provides a practical framework. Examples from Cloudflare and discussions of bash/grep commands illustrate the methodology.
4. Architectural Comparison: The direct comparison between traditional MCP and Cloudflare's Code Mode visually and conceptually highlights the differences and advantages of the proposed solution.
5. Benefits and Value Proposition: The discussion of progressive disclosure, privacy, and the continued utility of MCPs for their uniform interface reinforces the arguments for the new approach.
6. Conclusion: The summary concludes by reiterating the need to re-evaluate MCP usage and the potential of the code agent approach.

Data, Research Findings, and Statistics

• Token Usage Example: With two MCPs connected, context uses ~20,000 tokens (10% of context) for tool definitions alone.
• Sales Meeting Example: A two-hour sales meeting transcript returned by a tool could add ~50,000 tokens.

Synthesis/Conclusion

The core takeaway is that while MCPs offer a theoretical framework for tool interaction, their practical implementation for AI agents suffers from severe context management issues, leading to "context rot." The proposed and increasingly adopted solution involves shifting from direct MCP tool invocation to building code agents. These agents generate code to interact with tools as APIs, leveraging a directory structure for tool discovery and executing this code in isolated sandboxes. This approach, exemplified by Cloudflare's "Code Mode," offers significant advantages in terms of efficiency, scalability, progressive disclosure, and privacy. While MCPs may still retain value for their uniform API interface, their direct use for tool calling in agents is being superseded by more robust code-centric methodologies. The speaker personally advocates for reduced MCP usage due to context rot and finds the code agent approach promising for mitigating these issues.