LangExtract + RAG: Smarter Retrieval with Metadata Filtering

Key Concepts

• Retrieval Augmented Generation (RAG)
• Metadata Extraction
• Lang Extract
• Unstructured Data to Structured Data Conversion
• LLMs (Large Language Models)
• Vector Store
• Metadata Filtering
• Gemini 2.5 Flash
• Few-Shot Learning
• Data Normalization
• Dense Embedding Search

1. The Problem: RAG and Document Versions

• Main Point: In RAG systems, having multiple versions of the same document can confuse the LLM during retrieval, as it retrieves chunks from all versions without differentiation.
• Specific Detail: The LLM lacks the context to distinguish between different document versions, leading to potentially irrelevant information being used for generation.

2. Solution: Metadata Enhancement with Lang Extract

• Main Point: Adding reliable metadata to each chunk is a solution. Lang Extract is used to extract metadata directly from unstructured text documents.
• Specific Detail: Lang Extract converts unstructured data into structured data using LLMs.
• Benefit: Custom schema definition allows control over metadata quality.

3. Lang Extract Overview

• Description: An open-source project from Google (not an official product) that uses LLMs like the Gemini series to extract structured data from unstructured documents.
• Feature: Includes a visualization tool.
• Enhancement: Supports other API providers, including local models via Ollama.

4. RAG System with Lang Extract: A Concrete Example

• Goal: Build a RAG system that uses Lang Extract to enhance metadata and filter documents used by the LLM.
• Process: Convert unstructured text files into structured metadata using Lang Extract, then use this metadata to filter results for the RAG system.

5. Code Implementation: Key Packages and API Keys

• Requirement: Import necessary packages (details in video description and GitHub link).
• API Key: Requires a Gemini API key (using Gemini 2.5 Flash).

6. Sample Data: Multiple Document Versions

• Data Structure: Unstructured files with ID, title, and raw text (representing chunks or documents).
• Examples:
  • API reference documentation version 2.0
  • Legacy documentation
  • Storage services guide
  • Troubleshooting guide for authentication errors

7. Fixed Lang Extract Processor: The Core Pipeline

• Class: fixed_lang_extract_processor is the base for creating the Lang Extract pipeline.
• Importance: Metadata quality depends on the extraction pipeline's effectiveness.
• Components:
  • Raw text
  • LLM call with a schema
  • Few-shot examples

8. Prompt Engineering and Few-Shot Examples

• Prompt Example: "Extract these specific fields and technical documentation: service name, version number, document category, rate limits of any duplicated items."
• Customization: The prompt is highly dependent on the specific use case.
• Few-Shot Learning: Providing examples helps the LLM extract better metadata.
• Example Structure: Example text/document with corresponding extracted values for each field (e.g., service name, version name).

9. Lang Extract Object Creation and Configuration

• Parameters:
  • Prompt
  • Examples
  • LLM (default: Gemini 2.5 Flash, can use other APIs or local models)
  • Extraction passes (default: 1 or 2, higher values increase accuracy but also cost)
• Process: Pass the document, prompt, and extract metadata.

10. Data Normalization and Regular Expression Extraction

• Normalization: Ensures default values for missing fields.
• Regular Expression: Used as a fallback if LLM extraction fails.
• Best Practice: Run metadata extraction at the document level and enhance chunks with the metadata.

11. Vector Store Creation and Metadata Addition

• Class: Handles vector store creation and document addition with metadata.
• Metadata Filters: Added to each document.
• Hierarchical Approach: Initial retrieval based on metadata filters, followed by dense embedding search on filtered documents. This reduces the search space.

12. Comparison Loop: With and Without Metadata Filtering

• Process:
  1. Load sample documents.
  2. Create an extractor.
  3. Normalize metadata.
  4. Create a vector store.
  5. Add documents to the vector store.
  6. Run example queries with and without metadata filtering.
• Focus: Demonstrates retrieval, but the retrieved documents can be passed to an LLM for generation.

13. Demonstration and Results

• Execution: Run python lang_extract_rag.py.
• Metadata Extraction: Lang Extract extracts entities from each document.
• Normalization: Missing versions are assigned default values.
• Filtering Impact:
  • Queries with smart filters (metadata) return fewer, more relevant documents.
  • Queries without filters return all documents.
• Example: "How do I authenticate with oath in version 2.0?" With filters, only the relevant document is retrieved. Without filters, four documents are retrieved.

14. Secondary Dense Embedding Search

• Necessity: For cases where metadata filtering reduces the search space but still returns multiple documents, a secondary dense embedding search or multi-vector representation is needed.
• Benefit: Metadata filters help reduce the amount of data processed.

15. Conclusion

• Summary: Lang Extract can be used for metadata generation and subsequent filtering in RAG systems.
• Availability: Code is available in the video description.
• Actionable Insight: Using proper metadata filters improves retrieval accuracy and reduces the amount of data processed by the LLM.