Turn ANY File into LLM Knowledge in SECONDS

Key Concepts

• Retrieval Augmented Generation (RAG): A method for enhancing large language models (LLMs) with external knowledge.
• Data Curation: The process of preparing documents for use in a RAG pipeline.
• Dockling: A free and open-source Python tool for extracting data from various file types for RAG.
• Hybrid Chunking: A chunking strategy that uses embedding models to define semantic similarity between text segments.
• Vector Database: A database that stores data as vectors, enabling efficient similarity searches.
• Embedding Model: A machine learning model that converts text into numerical vectors representing its semantic meaning.
• OCR (Optical Character Recognition): Technology that enables the extraction of text from images, such as scanned documents or PDFs.
• ASR (Automatic Speech Recognition): Technology that converts audio into text.

Dockling: A Tool for Data Curation in RAG Pipelines

The Problem: Limited Knowledge in LLMs and Complex Data Types

Large language models (LLMs) often have limited and general knowledge, making them unsuitable for tasks requiring specific or up-to-date information. Simply dumping documents into a chat interface is insufficient for effective knowledge retrieval. Retrieval Augmented Generation (RAG) addresses this by curating external knowledge for LLMs, enabling them to become experts on specific data. However, the data curation step can be challenging, especially when dealing with diverse file types like PDFs, Word documents, audio files, and video recordings. Extracting data seamlessly from these formats for a RAG pipeline is a significant hurdle.

Dockling: A Solution for Complex Data Extraction

Dockling is presented as a free and open-source Python tool designed to simplify data curation for RAG pipelines. It allows users to work with complex data types, including PDFs with tables and diagrams, Word documents, and audio/video files. Dockling aims to streamline the process of preparing data, regardless of its complexity, for use in RAG implementations.

Getting Started with Dockling

Dockling is a Python package that can be installed using pip. The tool's documentation and examples are available online. The presenter also provides a link to a complete AI agent template that utilizes Dockling in its RAG pipeline.

Dockling Features and File Type Support

Dockling supports various file types and offers features for:

• Simple Extraction: Extracting text and tables from PDF documents.
  • Example: Extracting text from a complex PDF with code examples, diagrams, and tables using a few lines of code.
• Multiple File Formats: Working with different file formats seamlessly.
  • Example: Processing PDFs, Word documents, and Markdown files without specifying the file extension.
• Audio File Transcription: Transcribing audio files using speech-to-text models.
  • Requires additional dependencies like FFmpeg and OpenAI Whisper.
  • Example: Transcribing a 30-second audio file in 10 seconds using Whisper Turbo.

Chunking Strategies with Dockling

Dockling assists with the chunking process, which involves splitting documents into smaller, manageable pieces for LLMs to retrieve. The presenter highlights hybrid chunking as a particularly effective strategy.

• Hybrid Chunking: Uses an embedding model to determine semantic similarity between text segments, ensuring that chunks contain coherent ideas.
  • The embedding model helps define where to split the document while preserving the core ideas.
  • Example: Processing a PDF and splitting it into 23 chunks, with varying token lengths based on semantic similarity.

Dockling's Role in a Complete RAG AI Agent

The presenter showcases a complete RAG AI agent template that integrates Dockling for data extraction and chunking. The agent uses:

• Postgres with PG Vector: As the vector database.
• Hybrid Chunking: To prepare the data for the vector database.
• Pydantic AI: To create the AI agent.

The agent can answer questions based on the knowledge base curated with Dockling, demonstrating the end-to-end RAG pipeline.

Additional Dockling Features and Resources

The presenter encourages viewers to explore the example section of Dockling's documentation for more use cases and customization options, including:

• Custom Conversion: Using different OCR backends for text extraction.
• Visual Grounding: Highlighting the specific parts of a document that the agent used to answer a question.

Dockling vs. Crawl for AI

The presenter positions Dockling as the go-to tool for document data extraction, while Crawl for AI is recommended for website data.

Conclusion

Dockling is presented as a critical tool for RAG implementations, simplifying data extraction and preparation from various file types. Its features, including hybrid chunking and support for audio transcription, make it a valuable asset for building AI agents and applications that require external knowledge integration. The presenter emphasizes the importance of data curation in RAG pipelines and positions Dockling as a key enabler for this process.