MCP at Sourcegraph

Key Concepts

• Model Context Protocol (MCP): A protocol for providing relevant context to language models (LLMs).
• Tool Calling Models: LLMs that can use external tools to perform tasks.
• Agentic AI: AI systems that can autonomously plan and execute tasks using tools.
• Retrieval Augmented Generation (RAG): A technique for improving LLM performance by providing relevant context.
• AMP: A coding agent built by SourceCap that leverages tool calling models and MCP.
• Sub-agents: Agents that are used as tools by other agents.
• Tool Use LLM: A language model specifically designed to effectively utilize external tools.
• Feedback Loops: Mechanisms for providing agents with information about their progress and allowing them to correct errors.
• Tool Maggedon: The practice of providing an LLM with too many tools, which can confuse the model and reduce its performance.

SourceCap's Journey with MCP and Tool Calling Models

Byang, CTO and co-founder of SourceCap, discusses their journey integrating the Model Context Protocol (MCP) into their architecture and building a new coding agent called AMP. Their journey began in the summer of the previous year when David from Anthropic introduced them to MCP, describing it as "like LSP but for model context." SourceCap became an early design partner, providing feedback on the protocol's evolution.

The Paradigm Shift: From Co-pilots to Agents

Byang argues that AI application architecture has gone through three waves:

1. The Co-pilot Wave: Early AI applications were limited by the capabilities of text completion models. The interaction paradigm was human typing and the model completing tokens.
2. The RAG Chat Era: The advent of chat-based models like ChatGPT allowed for more explicit asks. Copying and pasting relevant code snippets into the context window improved performance.
3. The Agent Era: This era is dictated by the capabilities of tool calling models. SourceCap realized that existing assumptions about building on top of LLMs had changed, necessitating a complete rethink of their application architecture.

AMP: A Coding Agent Built from the Ground Up

SourceCap built AMP, a coding agent, from the ground up to leverage tool calling models and MCP.

Live Demo of AMP

Byang demonstrates AMP by having it make a live change to its own code: changing the background panel to red.

• AMP uses a linear tool (provided through an MCP server) to fetch the contents of a linear issue describing the desired change.
• The linear tool is not a first-party tool but the official linear MCP server.
• AMP uses a Playwright MCP server to interact with the browser, take screenshots, and verify that the change has been made.
• While AMP is working on the change, Byang uses AMP CLI to explain the AMP architecture.
• AMP generates an architectural diagram showing how MCP is integrated into its components.
• In parallel, AMP is tasked with coding a 3D Flappy Bird game.
• AMP successfully completes all three tasks: changing the background color, explaining its architecture, and coding the game.
• AMP also marks the linear issue as done, using the linear MCP server again.

AMP Architecture and MCP Integration

Byang explains that MCP is deeply integrated into AMP's architecture.

• AMP has an AMP client, an AMP server, and external services/local tools.
• MCP is used to connect to all these different types of tools and services.
• Local tools like Playwright and Postgres communicate over MCP or standard IO.
• External services, such as Sourcegraph, issue trackers, and observability tools, also communicate over MCP.
• SourceCap has implemented a secure MCP proxy to handle secrets and forward user identity to external services.

The Recipe for AI Agents

Byang outlines a recipe for building AI agents:

1. A Strong Tool Use LLM: The latest cloud models, like Claude 4, are well-suited for this.
2. A Way to Provide Tools: MCP is the perfect solution for this.
3. Focus on Feedback Loops: Agents need to be able to validate their progress and correct errors.
4. Imperative UX: The user experience should be focused on asking the agent to do something and then refining the feedback loops.

Top Tools Used in AMP

• Playwright (local)
• Postgres (local)
• Web search (Anthropic's web search API or Brave web search)
• Context 7 (documentation)
• Linear (issue tracking)
• Sentry (error tracking)

Pitfalls of MCP Integration: Tool Maggedon

Byang warns against "tool maggedon," the practice of plugging in too many MCP servers and tools. This can confuse the model and reduce its performance. SourceCap limits the set of tools that a particular MCP server provides to a smaller subset that are essential to the workflows they want to enable.

They find three buckets of tools particularly useful:

1. Tools for finding relevant context.
2. Tools for providing high-quality feedback (e.g., invoking unit tests or the compiler).
3. Tools for submitting done or declaring success (e.g., marking an issue as done).

Securing MCP

Securing MCP is a high priority for SourceCap. They have implemented a secure secret store where the AMP server takes care of the OAuth handshake and proxies the MCP connection from the client to external services, ensuring that no secret is ever stored unencrypted on the local machine.

The Future of Tool Calling and MCP

Byang speculates about the future of tool calling and MCP:

• Sub-agents: Sub-agents can be used to implement tools. AMP uses a sub-agent to gather context about the codebase.
• Dynamic Tool Synthesis: The model can plan out how to invoke tools and compose them in different ways. This could lead to a revisiting of code interpreters.
• Stateful Sessions and Two-Way Communication: The MCP protocol has features like stateful session management and two-way communication that are not being widely used.

Conclusion

Byang concludes by emphasizing the potential of tool calling LLMs and MCP. He believes that the next year will be "really weird" and that there is still much to be discovered about how to use these technologies effectively. SourceCap is building AMP as a tool calling native coding agent and is excited to see what others build on top of it.