The State of MCP observability: Observable.tools — Alex Volkov and Benjamin Eckel, W&B and Dylibso

Key Concepts

• MCP (Modular Computation Protocol): A protocol for AI agents to use tools.
• Observability: The ability to understand the internal state of a system from its external outputs.
• OpenTelemetry (OTel): An open-source observability framework for generating, collecting, and exporting telemetry data (traces, metrics, logs).
• Traces: A representation of a single request or operation as it flows through a system, composed of spans.
• Spans: A named, timed operation representing a piece of the workflow in a trace.
• Sync: A centralized database or platform where telemetry data is stored and analyzed.
• Distributed Tracing: The ability to track a single request as it flows through multiple services or components of a distributed system.
• Context Propagation: The mechanism by which trace context (e.g., trace ID, span ID) is passed between services.
• Semantic Conventions: Standardized naming and structure for telemetry data attributes, enabling consistent interpretation across different observability tools.
• Weave: Weights & Biases' observability tool.

The Observability Blind Spot in MCP Ecosystems

The rise of MCP is creating an observability blind spot as AI agents become more prevalent. As AI agents use more and more tools via MCP, developers have less visibility into the end-to-end happenings within their agent.

• Problem: Current observability solutions don't seamlessly integrate with MCP-powered agents, making it difficult to understand why things go wrong in production.
• Impact: Diminished ability to quickly respond to issues, hindering reliability and security.
• Enterprise Requirement: Enterprise engineering teams require robust observability before deploying systems to production.

Weights & Biases (W&B) Weave's Initial MCP Support

W&B Weave offers initial support for MCP observability.

• Implementation: Developers of both the client and server can set the MCP_TRACE_LIST_OPERATION environment variable.
• Functionality: Weave displays the list of tool calls and their durations.
• Limitation: This is a bespoke integration within Weave's Python SDK and doesn't provide visibility into the internal workings of the MCP server itself.

The Need for a Vendor-Neutral and Standardized Approach

The current bespoke integrations are not scalable or vendor-neutral.

• Observable Tools Manifesto: Alex Volov advocates for a vendor-neutral and standardized approach to MCP observability.
• Goal: To enable observability providers and tool builders to collaborate on a common solution.

Leveraging OpenTelemetry (OTel) for MCP Observability

OpenTelemetry offers a vendor-neutral and scalable way to add observability to MCP-powered agents.

• Analogy: MCP-powered agents are essentially distributed systems, and OpenTelemetry is a well-established standard for observing distributed systems.
• Key Primitives:
  • Traces: Represent an atomic operation in the system.
  • Spans: Represent a step within a trace, including duration and metadata.
  • Syncs: Centralized databases or platforms for storing and analyzing telemetry data.
• Distributed Tracing with OTel: By sending spans from different services to the same sync, OpenTelemetry can stitch together traces and provide a complete view of a request's journey.

Handling Different Domains: Client and Server

There are two scenarios:

1. Different Domains (e.g., Client and Third-Party Server): The server appears as a black box to the client.
2. Same Domain (e.g., Client and Server in Different Data Centers): Distributed tracing and context propagation can be used to gain visibility into the server's internal operations.

Context Propagation in MCP with OpenTelemetry

To connect traces between MCP clients and servers, trace context needs to be propagated.

• Implementation: Ben Eckle used the protocol's meta payload to shuttle the data.
• Process:
  1. Extract the current span from the client.
  2. Pass the trace context to the server via the meta payload.
  3. In the server, extract the trace context and inherit it as the current span.
  4. Send spans from the server to the same sync as the client.
• Result: The sync can stitch together the traces, providing end-to-end visibility.
• GitHub Demo: A working demo of this implementation is available on GitHub.

Weave's OpenTelemetry Integration

Weave supports OpenTelemetry, allowing it to receive traces from MCP agents written in different languages.

• Example: Alex Volov built an agent in TypeScript and sent traces to Weave without significant code changes.
• Configuration: Define W&B Weave as the OTLP endpoint and provide authorization headers.

A Magic Moment: MCP Observability and Agent Interaction

Alex Volov used Quadopus 4 to weify Ben Eckle's agent and add MCP observability.

• Scenario: The agent interacted with Weave's MCP server (which provides access to traces) to identify and fix an issue with the trace data.
• Meta Interaction: The coding agent discovered and interacted with a support bot (another agent) via MCP to learn how to fix the issue.

MCP.Run's OpenTelemetry Support

MCP.Run will also export telemetry data to OpenTelemetry-compatible syncs.

• Scope: Includes both MCP servers (profiles) and clients (tasks).
• Benefit: Users will be able to send traces from MCP.Run directly to W&B Weave.

Recap and Call to Action

• Current State: Observability in MCP is available but not evenly distributed.
• Community Involvement: The community needs to collaborate on tooling and conventions to make it smoother.
• AI Engineers: Start thinking about observability and ensuring end-to-end visibility.
• Tool Builders and Platform Providers: Join and work on higher-level SDKs and instrumentation.
• Semantic Conventions: Contribute to defining semantic conventions for agents.
• Platform Builders: Add OpenTelemetry support and participate in RFC reviews.

Semantic Conventions Explained

Semantic conventions are standardized naming and structure for telemetry data attributes.

• Purpose: To ensure that different observability tools can consistently interpret the meaning of spans and their attributes.
• Example: Defining conventions for HTTP requests (e.g., status code) or MCP tool calls (e.g., error status).
• GenAI Semantic Conventions Effort: The OpenTelemetry team is working on defining semantic conventions for GenAI applications.

Conclusion

The speakers emphasize the importance of observability in the emerging MCP ecosystem. While initial solutions exist, a vendor-neutral and standardized approach based on OpenTelemetry is crucial for enabling comprehensive and scalable observability. They encourage community involvement in developing tooling, defining semantic conventions, and contributing to the OpenTelemetry project.