This One Fix Made Our RAG Agents 10x Better (n8n)

Key Concepts

• RAG (Retrieval-Augmented Generation) Agents
• Vector Databases
• Chunking
• Markdown Splitting
• Recursive Character Text Splitter
• HTML to Markdown Conversion
• OCR (Optical Character Recognition)
• Contextual Retrieval
• Re-ranking

1. Introduction: The Tiny Fix for RAG Agents

• The video introduces a simple, 10-second fix that can significantly improve the effectiveness of RAG agents.
• The core of a RAG system involves breaking documents into chunks and storing them as vectors in a vector database.
• In NN, this chunking process is typically handled by the data loader in the RAG pipeline.

2. The Problem: Out-of-Context Chunks

• By default, the NSN data loader chunks documents into segments of approximately 1,000 characters with overlap.
• The loader uses a sliding window approach, looking ahead and back for ideal separation points like new lines or paragraphs.
• This method can lead to chunks that lack context, reducing the effectiveness of the RAG agent.

3. The Solution: Markdown Splitting

• A better approach is to split documents based on markdown formatting, which works well across different document types.
• Markdown splitting prioritizes headings, resulting in chunks that are better organized and provide more context.
• Instead of arbitrary character-based chunks, the document is split at meaningful structural elements.
• Example: Converting HTML to markdown and then splitting by headings.

4. Step-by-Step: Implementing Markdown Splitting in NN

• Default Behavior: The recursive character splitter in NN does not default to markdown splitting, despite the UI suggesting it might.
• How to Enable:
  1. Go to the text splitter node.
  2. Add the "split code by markdown" option.
  3. Re-execute the workflow.
• Failing to explicitly select the markdown option results in basic text splitting (paragraphs, new lines, spaces).
• Example: The video demonstrates the difference in chunking with and without the markdown option enabled.

5. Ensuring Correct Markdown Conversion

• To fully leverage markdown splitting, convert unstructured data to markdown before passing it to the data loader.
• The video showcases an adapted blueprint from their RAG Masterclass which handles file ingestion from Google Drive and web pages.
• The blueprint uses a switch to differentiate between file types (Google Docs, PDF, HTML).

6. Converting Different File Types to Markdown

• Google Docs:
  • Use the "Get a Document" node.
  • Set "simplify output" to "false" to get structured output.
  • Use a code node to convert the structured output to markdown (the presenter surprisingly used Chat GPT to generate the code that worked immediately).
• PDFs:
  • The default "Extract from File" node in NN only provides plain text.
  • Use the Mistral OCR API for clean markdown conversion from PDFs.
• HTML:
  • Use the HTML to Markdown node or services like firecraw.dev (as discussed in the RAG Masterclass) for website scraping and markdown conversion.

7. Potential Issues: Markdown Formatting

• If markdown isn't being chunked correctly, ensure the markdown is in the correct format.
• Extra processing on the markdown can alter its format, causing the splitter to fail.
• Example: Duplicating the markdown node and re-executing the workflow can corrupt the markdown format if you aren't careful about what is getting passed between nodes.

8. Real-World Application: AI Agent and Vector Database

• The video demonstrates an AI agent accessing a vector database.
• The agent is asked about long-term memory in NN, and it retrieves relevant, context-rich chunks from the database.
• The vector database tool confirms that the retrieved chunks contain the answer within context.
• Techniques like re-ranking and contextual retrieval can further improve results.

9. Community and Resources

• The video encourages viewers to join their community for access to blueprints, discussion boards, and live workshops.

10. Conclusion

• Enabling markdown splitting in the recursive character text splitter is a simple but powerful way to improve the performance of RAG agents.
• Ensuring correct markdown formatting and conversion is crucial for effective chunking.
• By implementing this fix, users can retrieve more contextually relevant information, leading to better AI agent responses.