Remote MCPs: What we learned from shipping — John Welsh, Anthropic

Key Concepts

• MCP (Model Context Protocol): A standard for communication between models and context providers.
• Tool Calling: The ability of models to interact with external tools and services.
• Integration Chaos: The proliferation of custom endpoints and duplicated functionality in tool integrations.
• JSON RPC: A lightweight remote procedure call protocol encoded in JSON.
• Global Transport Standard: The part of MCP dealing with streamable HTTP, OAuth 2.1, and session management.
• MCP Gateway: A shared infrastructure service providing a single point of entry for MCP connections.
• Pit of Success: Designing systems where the easiest path is also the correct path.
• Read Stream/Write Stream: Abstractions for handling data flow, used to integrate various transport mechanisms with the MCP SDK.

Implementing MCP Clients at Scale: Anthropic's Approach

The Problem: Integration Chaos and Duplication

The rapid advancement of tool calling capabilities in AI models has led to an explosion of custom endpoints and integrations. This results in:

• Duplication of Functionality: Similar features (authentication, billing, etc.) are implemented repeatedly across different services.
• Inconsistent Integrations: Integrations that work well in one service may be difficult or impossible to reuse in another.
• Increased Complexity: Managing numerous custom endpoints and integrations becomes challenging.

The Solution: Standardizing on MCP

Anthropic realized that these custom endpoints were converging towards the functionality provided by MCP. They decided to standardize on MCP for all interactions involving providing model context to models.

• MCP as a JSON RPC Specification: The core value of MCP lies in its message specification, providing a standard way for models and context providers to communicate.
• Transport Layer Flexibility: The actual transport mechanism (WebSockets, gRPC, etc.) is less critical and can be adapted to the specific needs of the organization.

Why Standardize on MCP?

• Avoid Reinventing the Wheel: Integrating with common services (e.g., Google Drive) is not a competitive advantage and should be standardized.
• Ecosystem Demand: MCP is becoming an industry standard, with major AI labs contributing to its development.
• Future-Proofing: MCP solves problems that organizations may not have encountered yet, providing solutions for future needs.
• Example: Sampling Primitives: MCP includes sampling primitives that can be used to handle different billing models, token limits, and usage tracking across various products.

Anthropic's MCP Gateway: A Pit of Success

To encourage adoption of MCP, Anthropic built a shared infrastructure service called the MCP Gateway.

• Single Point of Entry: The gateway provides a single connect to MCP call that returns an MCP SDK client session.
• Simplified Authentication: The gateway handles credential management and OAuth flows, relieving developers of this burden.
• Centralized Rate Limiting and Observability: The gateway provides a central point for managing rate limits and monitoring MCP traffic.
• URL-Based Routing: The gateway uses URL-based routing to connect to both internal and external MCP servers.

Technical Implementation Details

• Client Libraries: Anthropic provides client libraries in multiple languages that simplify connecting to the MCP Gateway.
• Transport Flexibility: The internal transport mechanism can be chosen based on the organization's needs (WebSockets, gRPC, Unix sockets, etc.).
• Unified Authentication: The gateway handles OAuth authentication, allowing users to access resources without re-authenticating for each service.
• Credential Portability: Credentials can be easily passed to batch jobs and other internal services.

Benefits of Centralized Context Management

• Security: The gateway provides a central point for enforcing security policies, such as banning malicious servers and classifying content.
• Auditing: All MCP messages are in a standardized format, making it easier to audit and monitor model interactions.
• Policy Enforcement: The gateway can be used to enforce policies related to data access and usage.

Key Takeaways

• MCP is Primarily About JSON Streams: The core of MCP is the standardized JSON message format. The transport mechanism is a secondary implementation detail.
• Standardize on Something: Even if MCP is not the right choice, it is important to standardize on some protocol for model context exchange.
• Build Pits of Success: Make the correct way to do things the easiest way to do things.
• Centralize Shared Problems: Solve common problems like authentication and external connectivity in a centralized location.

Notable Quotes

• "It's not a competitive advantage to be really good at making Google Drive talk to your app, it's just a thing that you need to do."
• "MCP is really just JSON streams and how you pipe those streams around your infrastructure is a small implementation detail."
• "You really want to make the right way to do a thing the easiest way to do a thing and then everyone just falls into doing the right thing naturally."

Conclusion

By standardizing on MCP and building a shared infrastructure service (the MCP Gateway), Anthropic has been able to reduce integration complexity, improve security, and enable developers to focus on building features rather than plumbing. The key is to treat MCP as a standardized message format and to build "pits of success" that encourage adoption of the standard.