You HAVE to Try Agentic RAG with DeepSeek R1 (Insane Results)

DeepSeek R1 Agentic RAG Setup with Small Agents: A Detailed Summary

Key Concepts:

• Agentic RAG: Combining reasoning LLMs (like DeepSeek R1) with faster, lightweight LLMs to create more intelligent and efficient retrieval-augmented generation workflows.
• Small Agents: A simple Python framework by Hugging Face for building agentic workflows with LLMs.
• Reasoning LLM: A large language model specifically designed for complex reasoning tasks (e.g., DeepSeek R1).
• Primary LLM: A faster, more lightweight LLM that drives the overall conversation and calls upon the reasoning LLM when needed (e.g., Llama, Qwen).
• Tool Calling: The ability of an LLM to invoke external functions or tools to perform specific tasks (e.g., RAG).
• Ollama: A tool for running LLMs locally on your machine.
• Context Limit: The maximum number of tokens an LLM can process in a single input.
• Chroma DB: An open-source embedding database.

1. Overview of Agentic RAG with R1

The video focuses on building a simple yet powerful agentic RAG system using DeepSeek R1 (a reasoning LLM) and Small Agents (a lightweight agent framework). The core idea is to leverage the reasoning capabilities of R1 for in-depth insights while maintaining a nimble conversation flow with a faster primary LLM.

• Problem: Reasoning models like R1 are powerful but slow.
• Solution: Combine R1 with a faster LLM that directs the conversation and uses R1 as a tool for complex reasoning tasks.
• Workflow: The primary LLM drives the conversation, and when it needs to extract in-depth insights from a knowledge base, it calls upon an R1-driven RAG tool.

2. System Architecture

The agentic RAG system consists of the following components:

• Primary LLM (Fast Agent): A fast LLM (e.g., Llama, Qwen) that manages the overall conversation flow. It uses Small Agents to break down problems into steps.
• RAG Tool: A tool that allows the primary LLM to access and reason over a knowledge base.
  • Vector Database (Chroma DB): Stores the knowledge base as vector embeddings.
  • Embedding Model: Used to embed user queries and knowledge base chunks.
  • Reasoning LLM (R1): Extracts insights from the retrieved context using its reasoning capabilities.

Workflow Diagram:

1. User query is received by the Primary LLM.
2. The Primary LLM decides if it needs to look up knowledge.
3. If knowledge lookup is needed, the Primary LLM calls the RAG Tool.
4. The RAG Tool retrieves relevant context from the Vector Database.
5. The retrieved context and user query are passed to the Reasoning LLM (R1).
6. R1 extracts insights from the context.
7. The insights are returned to the Primary LLM.
8. The Primary LLM continues the conversation with the user.

3. Implementation with Small Agents

Small Agents simplifies the process of building agentic workflows. The video demonstrates how to set up the agentic RAG system using Small Agents and local models.

Steps:

1. Install Dependencies:
   • pip install small-agents
   • Install Ollama from https://ollama.com/
2. Import Libraries: Import necessary classes from Small Agents (e.g., Agent, Tool, UI).
3. Load Environment Variables: Load API keys and model IDs from a .env file.
4. Define get_model Function: This function determines how to load the models, either from Hugging Face or locally using Ollama.
   • Hugging Face: Uses HF_API_MODEL to load models from the Hugging Face Inference API.
   • Ollama: Uses OpenAIServerModel with a modified base_url to point to the local Ollama instance.
5. Create Reasoning Model (R1):
   • Get the model using get_model.
   • Create a CodeAgent instance with the reasoning model.
   • Set max_steps=2 to limit the reasoning loop.
6. Create RAG Tool:
   • Define an embedding model using HuggingFaceEmbeddings.
   • Get an instance of the Vector Database (Chroma DB).
   • Use the @tool decorator to define the rag_with_reasoner function.
     • This function takes the user query as input.
     • It performs a similarity search in the Vector Database.
     • It formats the context and user query into a prompt for R1.
     • It calls R1 using model.run to extract insights.
     • It returns the response to the primary LLM.
7. Create Primary LLM (Tool Calling Agent):
   • Get the model using get_model.
   • Create a ToolCallingAgent instance with the primary LLM.
   • Pass the rag_with_reasoner tool to the agent.
   • Set max_steps=3.
8. Create User Interface:
   • Use UI(agent).launch() to create a Gradio UI for interacting with the agent.

4. Knowledge Base Setup

The video uses Chroma DB as a local vector database for storing the knowledge base.

Steps:

1. Load PDFs: Use Langchain's PyPDFLoader to load PDF documents from a data directory.
2. Split into Chunks: Split the documents into smaller chunks using RecursiveCharacterTextSplitter.
3. Clear Vector Database: Define a function to delete the existing Chroma DB instance.
4. Create Embeddings: Use the same embedding model used in the agent to create embeddings for the chunks.
5. Insert into Chroma DB: Insert the chunks and their embeddings into the Chroma DB instance.
6. Run the script: Execute the script to load the PDFs and create the vector database.

5. Ollama Setup and Model Configuration

The video provides instructions on how to set up Ollama and configure the context limit for the models.

Steps:

1. Install Ollama: Download and install Ollama from https://ollama.com/
2. Increase Context Limit:
   • Navigate to the Ollama models folder.
   • Create a new model file (e.g., DeepSeek-R1-7B-8k).
   • Add the following lines to the file:

     FROM deepseek-r1-7b
     PARAMETER num_ctx 8192
     

   • Run the following command in the terminal:

     ollama create DeepSeek-R1-7B-8k -f DeepSeek-R1-7B-8k
     

   • This will create a new version of the model with an increased context limit of 8192 tokens.
3. Set Environment Variables: Update the .env file with the correct Ollama model IDs for the reasoning model and the tool model.

6. Running the Agent

After setting up the agent, knowledge base, and Ollama, you can run the agent using the following command:

python R1_small_agent_rag.py

This will launch a Gradio UI in your browser, allowing you to interact with the agent.

7. Example Conversation

The video demonstrates an example conversation where the user asks the agent to compare and contrast the services offered by companies in the knowledge base. The agent uses the RAG tool to retrieve relevant context, and R1 extracts insights from the context to provide a comprehensive answer.

8. Key Takeaways

• Agentic RAG is a powerful approach for combining the strengths of different LLMs.
• Small Agents provides a simple framework for building agentic workflows.
• DeepSeek R1 is a capable reasoning LLM that can be used to extract in-depth insights from a knowledge base.
• Ollama allows you to run LLMs locally on your machine.
• Increasing the context limit of Ollama models is crucial for many use cases.

9. Conclusion

The video provides a practical guide to building an agentic RAG system with DeepSeek R1 and Small Agents. By combining the reasoning capabilities of R1 with the speed and flexibility of a primary LLM, you can create a more intelligent and efficient system for accessing and reasoning over knowledge bases. The video encourages viewers to explore the possibilities of agentic RAG and to experiment with different LLMs and tools to create even more powerful workflows.