MCP has a BIG problem

Key Concepts:

• mCP (Machine Communication Protocol): A protocol for AI agents to interact with tools.
• Stateful vs. Stateless: Stateful protocols require a persistent connection, while stateless protocols do not.
• Serverless Functions: Cloud computing execution model where the cloud provider dynamically manages the allocation of machine resources.
• REST API: An architectural style for building web services.
• SSE (Server-Sent Events): A server push technology enabling a server to send updates to a client over HTTP.
• Sampling: An mCP feature allowing servers to request LLM completions from the client.
• Context Window: The amount of text an AI model can consider at once.
• agents.json: A lightweight alternative to mCP by Wildcard AI, using JSON on top of OpenAPI spec.

1. The Problem with mCP's Stateful Nature

• The video addresses a significant limitation discovered after the previous video: mCP's stateful nature.
• mCP requires a long-lived server connection using Server-Sent Events (SSE), making it incompatible with serverless environments commonly used for REST APIs.
• AI agent tool calling is argued to be an ideal use case for serverless functions due to its quick, transactional nature and unpredictable scaling needs.
• The stateful requirement forces developers to use more complex and expensive infrastructure like managed Kubernetes clusters, which is a barrier to entry.

2. The Unnecessary Burden on API Developers

• The video argues that mCP places an undue burden on existing API developers.
• Most existing tools already have stateless REST APIs.
• mCP requires developers to create new endpoints and change their deployment methods, which is a significant ask without a clear incentive.
• The video suggests that a simple description layer detailing API capabilities would be sufficient for AI agents to understand and use existing APIs.

3. Sampling: A Problematic Feature

• Sampling, a key feature driving mCP's stateful requirement, is described as allowing servers to request LLM completions from the client (AI agent).
• "Sampling is a powerful mCP feature that allows servers to request llm completions through the client enabling sophisticated agentic behaviors while maintaining security and privacy."
• This is interpreted as the server being able to make the AI agent execute commands, raising security concerns.
• If the AI agent has access to API keys, a malicious server could potentially exploit this to access those keys under the guise of an LLM task request.
• The video argues that communication should be one-way: the AI agent making requests to tools, not the other way around.
• The video also points out that no existing mCP clients support sampling, rendering it a useless feature.

4. Alternative Solutions to Statefulness

• The video suggests that the other reason for statefulness, allowing the server to notify the client of changes in functions or resources, can be solved by the client periodically polling the server.
• If the client makes a request based on outdated information, the server can simply return a 400-level error.
• The video advocates for a simplified, stateless mCP where the client makes a request, the server runs a function, and returns data.
• More complex agentic flows can be achieved by piecing together multiple tool calls.

5. Wildcard AI's agents.json: A Lightweight Alternative

• The video introduces Wildcard AI's agents.json as a lightweight alternative to mCP.
• This approach uses a JSON file on top of an OpenAPI specification to standardize inputs for AI agents.
• It leverages existing technologies and doesn't require changes to server deployment.
• The video argues that this approach is more aligned with the idea that AI agents should be able to interact with existing APIs with minimal changes to the server.

6. Addressing Context Window Overload

• The video addresses a comment from the previous video regarding context window overload.
• Registering too many tools with an agent can overwhelm its context window, leading to confusion and poor performance.
• The video suggests a routing component where the AI agent first requests a list of relevant tools from a tool, and then calls the specific tool.
• This avoids passing the full list of tools in every API call.

7. Conclusion: A Call for Statelessness

• The video concludes that the current stateful implementation of mCP is not ideal for AI agent tool calling.
• The speaker expresses a desire for a stateless version of mCP that can be used with serverless functions.
• Until then, the speaker will be looking for alternative solutions.
• The video ends with a call for viewers to share their thoughts and experiences with similar tools.