Anthropic Just Fixed MCP’s Biggest Problem

Entropic’s MCP Tool Search: A Detailed Overview

Key Concepts:

• MCP (Model Context Protocol): A protocol for connecting Large Language Models (LLMs) like Claude to external tools and data sources.
• Tool Definition: The metadata describing a tool’s functionality, input parameters, and expected output.
• Context Window: The limited amount of text an LLM can process at once. Efficient context management is crucial for performance.
• Deferred Loading: A technique to load tool definitions only when they are needed, rather than all at once upon connecting to an MCP server.
• Tool Search: Entropic’s new feature that dynamically loads tool definitions based on user queries, reducing initial context usage.
• Regular Expression (Regex) Search: A pattern-matching search method for tool discovery.
• BM25: A keyword-based search algorithm used for semantic search and relevance ranking of tools.
• Server Instructions: System prompts that guide the LLM on how to use the available tools.

1. The Problem: Context Window Pollution by Tool Definitions

The primary issue addressed by Entropic’s new tool search feature is the significant consumption of the LLM’s context window by pre-loading tool definitions when connecting to an MCP server. This is particularly problematic for servers with a large number of tools. The example of the GitHub MCP server illustrates this: it contains 91 tools, consuming approximately 46,000 tokens – nearly 22% of Opus 45’s 200k token context window – before any conversational input is even processed. This limits the space available for actual user queries and responses. This issue has been previously highlighted in prior videos by the speaker.

2. Entropic’s Solution: MCP Tool Search

Entropic has introduced “MCP Tool Search” to mitigate this problem. Instead of loading all tool definitions upfront, the system now dynamically loads tools into the context only when they are needed. This builds upon previous work with code execution, which treated the MCP as a file system, but utilizes two distinct search mechanisms: Regular Expression (Regex) search and BM25.

3. How Tool Search Works: A Step-by-Step Process

The process unfolds as follows:

1. Initially, only the “tool search tool” itself is loaded into the context window.
2. When the LLM (e.g., Claude) receives a query, the system searches a tool catalog.
3. The search identifies three to five relevant tools based on the query.
4. Only the full definitions of these three to five tools are then loaded into the context.

This dynamic loading can potentially reduce token usage by up to 85%.

4. Search Mechanisms: Regex vs. BM25

Two search approaches are available:

• Regular Expression (Regex) Search: This method relies on pattern matching. The LLM generates patterns (e.g., “weather,” “get star data”). It’s most effective when tool names follow consistent naming conventions.
• BM25: This is a keyword-based search algorithm that performs semantic search with relevance ranking. The LLM uses natural language queries (e.g., “tool for weather,” “database operations”). BM25 is better suited for tools with varying names and descriptions. The choice between these depends on the consistency of tool naming. Importantly, no embeddings are used in either approach; it’s purely keyword-based.

5. Implementation for Developers: Server-Side & Client-Side

a) Server-Side (Making Tools Findable):

• Optimize Tool Descriptions: This is the most impactful step. Descriptions should be concise and prioritize functionality. A “before and after” example was provided, demonstrating how a verbose, repetitive description can be streamlined to “get current weather, forecast or historical data” followed by relevant keywords.
• Best Practices for Descriptions:
  • Lead with the function (1-2 sentences).
  • Include searchable keywords (e.g., “fetch,” “get,” “retrieve”).
  • Synonyms are helpful.
  • Place constraints in the input schema, not the description. Descriptions are for discovery; schemas are for validation.
  • Recognize that every word consumes tokens – be ruthless in optimization.
• Server Instructions: Utilize the “server instructions” field (a system prompt) to guide the LLM on how to use the tools, specifying the desired workflow and order of operations. The order of instructions is significant when tool search is enabled.

b) Client-Side (Implementing Tool Search in an MCP Client):

1. Enable the Beta: Include a specific header in your API request to activate the tool search feature.
2. Add the Tool Search Tool: Add either the Regex or BM25-based tool search tool to your tools list. Do not set deferred loading on this tool; it must be loaded immediately.
3. Mark Tools for Deferred Loading: Add a specific keyword to the tool definitions of tools you don’t want to load upfront. This signals the system to load them only when searched for.
4. Keep Essential Tools Loaded: Maintain three to five frequently used tools without deferred loading for immediate access.

6. Implementation Checklist & Best Practices

• Enable the beta header.
• Add the tool search tool (Regex or BM25).
• Mark non-essential tools with deferred_loading: true.
• Keep 3-5 essential tools with deferred_loading: false.
• Optimize tool descriptions (function first, keywords, concise).
• Add server instructions.
• Test with 30+ tools to see significant improvements.
• Monitor context usage before and after implementation.

7. When to Use Tool Search & Common Pitfalls

Use Tool Search When:

• You have 10 or more MCP tools.
• These tools occupy more than 10% of the context window.

Skip Tool Search When:

• You have only 3-5 tools.
• All tools are frequently used.
• Latency is absolutely critical (search adds some latency).

Common Pitfalls:

• Don’t defer load the tool search tool itself.
• Don’t make descriptions too short – keywords matter. ("get weather" is less effective than "get current weather forecast or historical data").
• Don’t defer load all tools – you’ll gain no benefit.

8. Data & Statistics

• GitHub MCP server: 91 tools, consuming 46,000 tokens (22% of a 200k context window).
• Potential token reduction: up to 85%.
• Dynamic loading: 3-5 tools loaded at a time.

9. Conclusion

Entropic’s MCP Tool Search represents a significant improvement in managing context window usage when working with MCP servers containing numerous tools. By dynamically loading tool definitions only when needed, developers can substantially reduce initial token consumption, leaving more space for user queries and responses. Successful implementation requires careful optimization of tool descriptions, strategic use of deferred loading, and consideration of the trade-off between latency and context efficiency. The speaker encourages viewers to experiment with the feature and share implementation questions in the comments.