Introducing RAG 2.0: Agentic RAG + Knowledge Graphs (FREE Template)

Key Concepts

• Agentic RAG: Retrieval Augmented Generation where the AI agent can reason about how it explores the knowledge base.
• Knowledge Graph: A graph database representing entities and their relationships, used for relational knowledge retrieval.
• Vector Database: A database storing vector embeddings of text chunks for semantic similarity search.
• PG Vector: A PostgreSQL extension enabling vector storage and similarity search within a SQL database.
• Neo4j: A graph database management system used for storing and querying knowledge graphs.
• Podantic AI: An AI agent framework used as the core of the agent.
• Graffiti: A knowledge graph library working alongside Neo4j.
• Claude Code: An AI coding assistant used to help build the agent.
• MCP (Managed Compute Provider) Server: A server that allows the AI coding assistant to interact with external services.

Agentic RAG with Knowledge Graphs: A Deep Dive

1. Introduction

The video focuses on combining Agentic RAG and knowledge graphs to create powerful knowledge retrieval systems for AI agents. The presenter shares a neatly packaged agent template that leverages both vector databases and knowledge graphs.

2. Demonstration

A command-line interface (CLI) demonstrates the agent's capabilities. The agent has access to both a vector database (Postgres with PG vector) and a knowledge graph (Neo4j) through agent tools.

• Vector Database: Stores document chunks about AI initiatives of big tech companies (OpenAI, Microsoft, Google).
• Knowledge Graph: Represents relationships between companies (e.g., Amazon invests in Anthropic, Microsoft partners with OpenAI).

The demo showcases three types of queries:

• Vector Search: "What are the AI initiatives for Google?" - The agent uses the vector database to find relevant information.
• Graph Search: "How are OpenAI and Microsoft related?" - The agent uses the knowledge graph to find the relationship between the two companies.
• Combined Search: "What are the initiatives for Microsoft, and how does that relate to Anthropic?" - The agent uses both the vector database and the knowledge graph to answer the question.

3. Tech Stack

The tech stack used for building the agent includes:

• Podantic AI: For the AI agent framework.
• Graffiti: For the knowledge graph library.
• Neo4j: As the underlying knowledge graph engine.
• Postgres with PG vector: As the vector database.
• FastAPI: For building the agent API in Python.
• Claude Code: As the AI coding assistant.

4. Why Agentic RAG and Knowledge Graphs?

The video explains the evolution from traditional RAG to Agentic RAG, highlighting the limitations of vanilla RAG.

• Vanilla RAG (Naive RAG): A simple process where documents are chunked, embedded, and stored in a vector database. User queries are embedded, and relevant chunks are retrieved and fed into the LLM.

  • Limitation: Inflexible; the agent must use the retrieved context, even if it's not optimal.

• Agentic RAG: Gives the agent the ability to reason about how it explores the knowledge base.

  • Benefits: Allows the agent to refine searches, explore different knowledge sources, and choose the best retrieval strategy.

Knowledge graphs are particularly useful for relational lookups, allowing the agent to understand how entities are connected.

5. Setting up the Agent Template

The video provides a step-by-step guide to setting up the agent template:

1. Prerequisites: Python, Postgres database (Neon), Neo4j database, LLM provider API key.
2. Virtual Environment: Create a virtual environment and install dependencies using pip.
3. Postgres Setup:
   • Copy SQL code from the SQL folder.
   • Update vector dimensions if using a different embedding model.
   • Paste the SQL code into a Postgres database (e.g., Neon) to create tables.
4. Neo4j Setup:
   • Option A: Use the presenter's local AI package (Neo4j is included).
   • Option B: Install Neo4j Desktop.
   • Obtain connection details (username, password).
5. .env Configuration:
   • Copy .env.example to .env.
   • Set database URL for Postgres.
   • Set Neo4j connection details (bolt URL, username, password).
   • Configure LLM provider (OpenAI, Open Router, Olama, Gemini) and API key.
   • Configure embedding provider and model.
   • Configure LLM for knowledge graph transformation.
6. Knowledge Base Setup:
   • Create a documents folder.
   • Add markdown documents to the folder.
7. Basic Ingestion:
   • Run python -m ingestion.py -d clean to ingest documents into both the knowledge graph and vector database.
   • The -d clean flag wipes the knowledge graph and vector database tables.
8. Configure Agent Behavior:
   • Edit agent/prompts.py to define the system prompt.
   • Specify when the agent should use the vector database, knowledge graph, or both.
9. Start API Server:
   • Run python -m agent.api.
10. Run CLI:
    • Open a second terminal.
    • Run python CLI.py.

6. Configuring Agent Behavior

The agent/prompts.py file contains the system prompt that dictates how the agent uses its tools. The prompt should be tailored to the specific data in the knowledge base. For example, the default prompt instructs the agent to use the knowledge graph when the user asks about two companies in the same question.

7. Using Cloud Code for Development

The presenter shares how they used Claude Code, an AI coding assistant, to build the agent.

• MCP Servers:
  • Crawl for RAG MCP server: Provides external documentation to the AI coding assistant.
  • Neon MCP server: Allows the AI coding assistant to create projects, run SQL queries, and manage tables in Neon.
• Planning Mode:
  • Use Shift + Tab twice to enter plan mode.
  • Create claw.md (global rules), planning.md (project architecture), and task.md (list of tasks).
• Execution:
  • Exit planning mode using Shift + Tab again.
  • Provide a simple prompt to kick off the build (e.g., "Take a look at planning and task markdown files and execute that plan").
• Examples:
  • Provide example Python scripts in the examples folder for the AI coding assistant to reference.

8. Conclusion

The video provides a comprehensive guide to building an Agentic RAG system with knowledge graphs. By combining these two technologies, AI agents can reason about how they explore knowledge and provide more accurate and relevant answers. The presenter also shares their experience using Claude Code to accelerate the development process.