AI Code That Fixes Itself (An MCP You Can Try Now)

Key Concepts

• AI Coding Assistants
• Hallucination Detection
• Knowledge Graphs
• Retrieval-Augmented Generation (RAG)
• MCP (Management, Control, and Planning) Server
• Pyantic AI
• Archon V2
• Claude Code
• Self-Correction
• Neo4j

Knowledge Graphs for AI Coding Assistance

The Problem: AI Coding Assistant Hallucinations

• AI coding assistants, while speeding up development, often make frustrating and subtle mistakes, including hallucinating functions or attributes.
• Example: When using Windsurf to build an agent with Pyantic AI, the AI hallucinated the invoke_async function, which doesn't exist in the Pyantic AI library.

Solution: Combining Knowledge Graphs with AI Coding Assistance

• The presenter proposes using knowledge graphs to provide AI coding assistants with a reliable source of information about libraries and tools.
• A knowledge graph is created from the Pyantic AI GitHub repository, representing files, classes, functions, and attributes as nodes and relationships.
• This allows the AI to query the graph and verify the existence and correct usage of functions and parameters.

Knowledge Graph Structure

• Nodes:
  • Repositories (tagged as "repository")
  • Files (green nodes)
  • Functions (dark blue nodes)
  • Classes (teal nodes)
  • Attributes (beige nodes)
• Relationships: Represent connections between these elements (e.g., a class contains functions, a function belongs to a file).

Hallucination Detection Process

1. An AI coding assistant generates code.
2. A script (hallucination detector) analyzes the code.
3. The script queries the knowledge graph to verify the existence and correct usage of functions and parameters.
4. If a hallucination is detected (e.g., an invalid function call or parameter), the script reports the error.
5. The AI coding assistant receives the feedback and self-corrects the code.

Example: Detecting Invalid Function Parameters

• If the AI uses the correct function (run_stream) but adds an invalid parameter (test), the hallucination detector identifies this by checking the function's parameter list in the knowledge graph.

Benefits of Using Knowledge Graphs

• Improved Accuracy: Reduces hallucinations and ensures the AI uses correct syntax and parameters.
• Self-Correction: Enables AI coding assistants to identify and fix their own mistakes.
• Granular Information: Provides access to details not always found in documentation, such as function parameters.
• Speed: Hallucination detection is fast because it relies on deterministic code and graph queries, not LLM analysis.

Crawl for AI RAG MCP Server

Overview

• The presenter has added the knowledge graph capabilities to their "Crawl for AI RAG MCP server."
• This server provides AI coding assistants with external knowledge for libraries and tools, including the ability to query knowledge graphs.

New Tools Added to the MCP Server

1. Hallucination Check Script: Runs the hallucination detection script and returns the output to the AI coding assistant for self-correction.
2. Knowledge Graph Query Tool: Allows the AI to explore the knowledge graph by listing methods, attributes, and other information. It can execute commands to explore the knowledge graph.
3. GitHub Repository Parser: Parses a GitHub repository and adds its contents to the knowledge graph.

Setting Up the MCP Server with Knowledge Graph Support

1. Set the use_knowledge_graph environment variable to true.
2. Enter credentials for a Neo4j instance (can be hosted locally using the presenter's local AI package or Neo4j Desktop).

Using the MCP Server with Claude Code

• The presenter demonstrates using the MCP server with Claude Code to build a Pyantic AI agent.
• A detailed scope of work is defined in a claw.md file, outlining the steps for Claude to follow, including:
  • Checking for sources in the knowledge graph and knowledge base.
  • Performing RAG to explore Pyantic AI documentation.
  • Querying the knowledge graph for specific methods and attributes.
  • Generating code and validating it for hallucinations.

Results

• Claude Code, guided by the MCP server and the knowledge graph, successfully generates a working Pyantic AI agent on the first try.
• The agent has minimal to zero hallucinations.
• The hallucination detector confirms that the generated code has zero hallucinations.

Alternative Solutions and Browserbase Director

Front-End Development

• Tools like Bolt.new, Bolt.DIY, and Lovable already provide self-feedback and correction mechanisms for building front-end applications.

Browser Automation

• Browserbase Director is a platform for creating browser automations with self-correction capabilities.
• It uses an open-source framework called Stage Hand, built on top of Playwright.
• Director allows users to create automations by describing the desired outcome, and the AI agent intelligently navigates websites and generates code.

Archon V2

Overview

• Archon V2 is an upcoming overhaul of the presenter's AI agent builder.
• It will combine:
  • Project management (similar to Claude Taskmaster).
  • Knowledge for RAG (from the Crawl for AI RAG MCP server).
  • Knowledge graph capabilities.
  • MCP server management.
• Archon V2 will remain free and open source.

Conclusion

• Combining knowledge graphs with AI coding assistance is a promising approach for reducing hallucinations and improving the accuracy of AI-generated code.
• The Crawl for AI RAG MCP server provides the tools and infrastructure for implementing this approach.
• Archon V2 will integrate these capabilities into a comprehensive AI agent building platform.