Claude with MCP is not better than ChatGPT. Not by a long shot. Even for technical users. Live demo

Key Concepts

• Model Context Protocol (mCP): A standard for connecting AI assistants to data sources.
• mCP Servers: Providers of APIs that AI assistants can use.
• AI Assistants: Consumers of mCP server APIs.
• OpenAPI Schema: A specification for describing APIs, used by OpenAI.
• Zod Schemas: Used by mCP servers to define their APIs, then converted to JSON schema.
• Local mCP Servers: Servers that run on the user's local machine.
• Custom GPT Actions: OpenAI's equivalent of mCP, allowing GPTs to interact with external APIs.

Model Context Protocol (mCP) Overview

• Anthropic announced and open-sourced the Model Context Protocol (mCP) as a new standard for connecting AI assistants to systems where data resides.
• The goal of mCP is to provide a universal, open standard for connecting AI systems with data sources, replacing fragmented integrations with a single protocol.
• mCP servers are providers of APIs that AI applications or assistants can use to access information on behalf of the user.
• Anthropic allows local mCP servers to be connected to the Claude desktop app, a feature not available with OpenAI's custom GPTs, which require globally hosted servers.

Comparison with OpenAI's Custom GPT Actions

• The speaker initially questioned how mCP differs from OpenAI's custom GPT actions, released in October of the previous year.
• OpenAI uses the OpenAPI schema for describing APIs, which serves a similar purpose to mCP.
• The speaker notes that mCP feels more disconnected from Anthropic, potentially allowing non-Anthropic models to communicate with non-Anthropic services.
• There seems to be more adoption of mCP servers compared to custom GPT actions, as the speaker had to create their own custom GPT action server boilerplate due to a lack of examples.

mCP Servers and Ecosystem

• Anthropic provides a repository of example mCP servers that can be downloaded and used with the Claude desktop app.
• Reference servers include integrations with Brave Search, file systems, GitHub, GitLab, Google Drive, Google Maps, Knowledge Graph-based persistent memory, PostgreSQL, Puppeteer, Sentry, Slack, and SQLite.
• Third-party companies are providing official mCP servers for managing Cloudflare, Raygun, e2b, and various databases.
• Community servers are also available, including those for installing servers, manipulating Spotify, and connecting to Apache BigQuery and Obsidian.
• Pull requests are being added to expand existing servers, such as adding search and issue management functionalities to the GitHub server.

Testing the File System Server

• The speaker attempts to test the file system server from Anthropic's list, aiming to replicate a similar tool created for ChatGPT that can manipulate the local machine through the terminal.
• The speaker uses a custom GPT to check out the mCP repository and read the README file, demonstrating what ChatGPT can already do.
• The speaker then tries to set up the file system server for use with the Claude desktop app, encountering some initial confusion.
• The process involves installing dependencies, navigating to the server folder, and configuring the server by editing the Claude config file.
• The speaker asks Claude to locate the config file and open the folder in Finder, but encounters errors.
• After some troubleshooting, the speaker creates the config file and allows access to the work folder.
• The speaker then runs the file system server and restarts Claude, eventually seeing the available tools in the Claude interface.
• The speaker tests the server by checking for a directory and listing files, noting that it takes some time to execute.
• The speaker observes that the server is not logging anything, making it difficult to debug.
• The speaker notes that the setup process is a bit confusing and requires creating a config file.
• The speaker also observes that Claude asks for permission each time, even after allowing it for the current chat, although this may be due to running multiple commands.
• The speaker concludes that working with a local server is faster and more reliable than using a local tunnel with ChatGPT.

Schema Comparison

• The speaker compares the schemas used by OpenAI and mCP to describe their APIs.
• OpenAI uses the OpenAPI schema, while mCP uses Zod schemas that are converted to JSON schema.
• The speaker provides an example of an OpenAPI schema used for a custom GPT action server.
• Claude explains that OpenAPI is designed for HTTP REST APIs, while mCP is designed for tool-based interactions where each tool is a callable function rather than an HTTP endpoint.
• mCP schemas focus on tool definitions with input and response handling, using a simpler success/error model with content and an error flag.
• The speaker notes that mCP seems to work only with local servers, while custom GPTs can connect to servers on the internet.

Attempting to Create a New Server (Terminal)

• The speaker attempts to create a new mCP server called "terminal" that gives Claude the ability to execute terminal commands.
• The speaker asks Claude to create a new server folder in the src directory and generate the necessary files.
• Claude creates a directory, a README file, and a main index.ts file with code for executing terminal commands.
• The code includes a whitelist of allowed commands for security reasons.
• The speaker then tries to install the dependencies for the new server, but encounters an error due to an incorrect version of a dependency.
• The speaker asks Claude to fix the package.json file, and this time it works.
• The speaker then asks Claude to provide the config needed to give Claude access to the new server tool.
• The speaker then asks Claude to add a command that builds and runs all it start to package.
• The speaker adds the new server to the Claude config file and restarts Claude, but the new tool does not appear in the Claude interface.
• The speaker tries various approaches to run the server, including using npm start and other commands, but none of them work.
• The speaker checks the Claude logs, but they do not provide any clear indication of what is wrong.
• The speaker concludes that it is not straightforward to run a custom mCP server and that Claude is not picking up the new server.

Debugging and Troubleshooting

• The speaker researches how to debug why Claude is not picking up the custom server.
• The speaker finds that there are logs in the Library folder that can be used for debugging.
• The logs show that Claude is connecting to the mCP server, but there are no other errors.
• The speaker observes that even though Claude does not have access to the file system, it still tries to access them.
• The speaker removes the terminal tool from the config file and restarts Claude, but the tools are still not loading.
• The speaker validates the JSON config file to ensure that it is valid.
• The speaker tries various combinations of starting and restarting the server and Claude, but nothing seems to work.
• The speaker is running out of ideas and decides to wrap up the stream.

Conclusion

• The speaker concludes that mCP is different from OpenAI's custom GPT actions in that it works with local servers and has some very opinionated requirements.
• The speaker failed to get the custom terminal server working during the stream.
• The speaker plans to make another video when they figure out why the server is not working.
• The speaker thanks the viewers for watching and says goodbye.