The One RAG Method for Incredibly Accurate Responses (n8n)

Key Concepts

• RAG (Retrieval-Augmented Generation) Agents: AI agents that retrieve information from a vector store to answer questions.
• Metadata Filtering: Dynamically filtering data in a vector store based on metadata (e.g., document date, department) to improve the accuracy of RAG agents.
• Vector Store: A database that stores data as vectors, allowing for semantic similarity searches. (Superbase, Pinecone)
• Hybrid Search: Combining semantic (vector-based) and keyword-based searches for more comprehensive results.
• Contextual Retrieval: Adding context to data chunks to improve the accuracy of RAG agents.
• Re-ranking: Reordering search results using a separate neural network to prioritize the most relevant results.
• JSON Schema: A structured format for defining the structure and constraints of JSON data, used to guide the AI in creating metadata filters.
• Unix Timestamp: A numerical representation of a point in time, used for date filtering in Pinecone.
• Edge Functions: Serverless functions that run closer to the user, providing faster response times.
• Sparse and Dense Embeddings: Numerical representations of text used for semantic search. Dense embeddings capture semantic meaning, while sparse embeddings focus on keyword presence.

Metadata Filtering for RAG Agents: Enhancing Accuracy and Relevance

Introduction

The video addresses the problem of inaccurate or irrelevant results from RAG agents due to unfiltered data in vector stores. It demonstrates how to implement dynamic metadata filtering to ensure agents retrieve the most relevant information. The video covers implementations for both Superbase and Pinecone vector databases.

The Importance of Metadata in RAG Pipelines

• During the ingestion phase, data (documents, web pages) is chunked and stored in a vector database.
• AI can enrich this data by adding metadata such as document date, department, and category.
• Metadata filtering allows agents to retrieve data based on specific criteria, improving accuracy.

Superbase Implementation

Dynamic Metadata Definition

• A separate table in Superbase is used to define metadata fields dynamically.
• These fields are automatically picked up in the agent's template.

Example Scenario

• Question: "How much annual leave is there per year as per the 2025 HR policy?"
• The agent dynamically constructs a metadata filter to retrieve documents relevant to the HR department and the year 2025.
• The filter includes date ranges and logical operators.

Advanced RAG Techniques

• The Superbase template uses contextual retrieval, hybrid search, and re-ranking.

Step-by-Step Process

1. Data Ingestion: HR policy manuals are ingested into the RAG pipeline.
2. Metadata Field Definition: Document date and department fields are defined in the Superbase metadata table.
3. Metadata Population: The AI infers and populates the metadata fields based on the document content.
4. Query Processing: The agent receives a question and retrieves metadata field information from the Superbase table.
5. Prompt Preparation: A system prompt is dynamically populated with the available metadata fields and values.
6. JSON Schema Definition: A JSON schema is used to guide the AI in creating complex metadata filters.
   • Instead of using "generate from JSON example", the video uses "define using JSON schema"
   • The schema defines the types of operators and constraints the AI to ensure the correct format.
7. Hybrid Search: The Superbase vector store is queried using both semantic and keyword-based searches.
8. Re-ranking (Optional): The results are re-ranked using Cohere to prioritize the most relevant documents.

Example Metadata Fields

• Product categories
• Tags
• Document dates
• Customers
• Suppliers
• Product types
• Numerical amounts

Superbase Ingestion Pipeline

• Adapted from Daniel's RAG masterclass template.
• Uses a Google Drive node to pick up new files.
• A switch node alters the flow based on the document type (e.g., Google Doc, HTML file).
• A record manager tracks documents loaded into the vector database.
• An LLM chain generates metadata for the file based on its name and content.

Superbase Agent Setup

• The agent uses an OpenAI chat model with GPT 4.1 and a low sampling temperature.
• It calls a separate workflow to query the vector store.
• The workflow retrieves metadata field information from the Superbase table.
• A system prompt is used to guide the AI in constructing metadata filters.

Superbase Edge Function

• Handles hybrid search and advanced metadata filtering.
• Receives the query embeddings and metadata filter from the agent.
• Triggers a database function to perform the search and filtering.

Superbase Database Function

• Implements hybrid search and metadata filtering.
• Supports logical operators (AND, OR) and lists of values.
• Parses and processes complex filters.

Pinecone Implementation

Limitations of the Native Pinecone Node

• The native Pinecone vector store node in n8n is limited in functionality.

Alternative Approach

• The video uses Daniel's hybrid search blueprint as a basis and extends it to add advanced metadata filtering.

Step-by-Step Process

1. Data Ingestion: An example HR policy manual is downloaded.
2. Metadata Enrichment: An LLM chain extracts metadata from the file content.
   • The prompt is manually updated with the available metadata fields and values.
3. Metadata Formatting: The document date is transformed into a Unix timestamp.
4. Chunking: The document is chunked into smaller pieces.
5. Embedding Generation: Dense and sparse embeddings are generated for each chunk.
6. Metadata Mapping: The metadata fields are mapped to the Pinecone vector store.
7. Upsert: The chunks and metadata are upserted to Pinecone.

Pinecone Agent Setup

• Similar to the Superbase setup, the agent calls a separate workflow to query the vector store.
• The workflow prepares the metadata filter and transforms the date into a Unix timestamp.
• The Pinecone vector database is queried directly using the filter.

Pinecone Metadata Filtering

• Pinecone natively supports metadata filtering using a specific schema.
• The video uses a JSON schema to guide the AI in creating filters that conform to the Pinecone schema.

Pinecone Query

• The query is transformed into dense and sparse embeddings.
• The filter value is passed to the Pinecone HTTP request.
• The chunks are filtered based on the query.

Best Practices for Metadata Filtering

• Keep metadata fields as high-level as possible.
• Use metadata filtering to segregate data by department, product type, or document date.
• Combine metadata filtering with other advanced RAG strategies.

Conclusion

Metadata filtering is a powerful technique for improving the accuracy and relevance of RAG agents. By dynamically filtering data based on metadata, agents can retrieve the most relevant information and avoid incorrect or irrelevant results. The video provides detailed instructions and templates for implementing metadata filtering in both Superbase and Pinecone.

Main Takeaways

• Metadata filtering significantly improves the accuracy of RAG agents.
• Dynamic metadata definition allows for flexible filtering criteria.
• JSON schema definition is a powerful tool for guiding AI in creating complex filters.
• Hybrid search and re-ranking further enhance the quality of search results.
• Superbase and Pinecone offer different approaches to metadata filtering, each with its own advantages and limitations.