Structuring the Unstructured - Cedric Clyburn, Red Hat

Key Concepts

• Unstructured Data: Raw information in formats like PDFs, images, tables, and diagrams that lack a predefined data model, making them difficult for LLMs to process accurately.
• Docling: An open-source tool (part of the Linux Foundation) designed to parse, extract, and convert unstructured documents into structured formats like Markdown, JSON, and Pydantic objects.
• RAG (Retrieval-Augmented Generation): An AI framework that retrieves data from external sources to ground LLM responses.
• Chunkless RAG: A methodology where the document structure (e.g., a markdown outline) serves as the retrieval index, allowing agents to navigate sections without traditional vector database chunking.
• MCP (Model Context Protocol): A standard that allows AI agents (like Claude Code or Cursor) to interact with external tools (like Docling) to perform document processing tasks.
• OCR (Optical Character Recognition): Technology used to convert scanned images of text into machine-readable text.

---

1. The Challenge of Unstructured Data

Cedric Clyburn emphasizes that context is the most critical factor in building AI applications. A significant portion of enterprise data remains "trapped" in unstructured formats (PDFs, presentations, technical docs).

• The Problem: Naive parsing often leads to truncated text, merged columns, and lost data (e.g., tables and images), which causes LLMs to hallucinate or produce incorrect outputs.
• The Risk: Poor data processing can lead to the propagation of "nonsensical terms" in scientific literature, as AI models misinterpret scanned documents.
• Cost/Efficiency: Using frontier models for massive document parsing is expensive ($30+ per million tokens) and non-deterministic.

2. Docling: A Local, Open-Source Solution

Docling provides a middle ground by offering a fast, local, and cost-effective library for document transformation.

• Methodology: It uses a combination of OCR and vision models to maintain document layout consistency.
• Key Features:
  • Format Conversion: Converts PDFs to Markdown, JSON, or Pydantic data types.
  • Table Extraction: Specifically identifies and extracts tables into data frames, preserving their structure.
  • Image Annotation: Uses vision language models (VLMs) to describe and annotate images, making them searchable for RAG applications.
  • Scalability: Can be deployed as a REST API (docling-serve) in Kubernetes or as an MCP server for agentic workflows.

3. Step-by-Step Processes

A. Document Conversion & Extraction

1. Ingestion: Use DocumentConverter to load a source file (e.g., PDF).
2. Parsing: The tool identifies layout elements (headers, text, tables, images).
3. Export: Convert the parsed object into the desired format (Markdown/JSON).
4. Validation: Use Pydantic models to ensure the output structure is consistent and programmatically accessible.

B. Chunkless RAG Implementation

Instead of traditional vector-based chunking:

1. Outline Generation: Create a markdown outline of the document.
2. Agentic Search: An agent iterates through the document sections, checking for relevance to the user's query.
3. Retrieval: Once the relevant section is identified, the agent pulls the full text from that specific section to generate an answer.

4. Real-World Applications & Case Studies

• Hugging Face (Leandro): Used Docling to process Common Crawl PDFs. By utilizing CPU-based processing instead of naive VLM/OCR approaches, they achieved a 50x cost savings.
• Red Hat: The team uses Docling to process thousands of product documentation PDFs, ensuring that technical data remains accurate and accessible for internal AI agents.
• Enterprise Automation: Using the Docling MCP server, developers can integrate document processing directly into IDEs (like VS Code) to automate tasks like "summarize this PDF" or "extract action items from these meeting notes."

5. Notable Quotes

• "Unstructured data is becoming this new context layer for AI." — Attributed to Jensen Huang (Nvidia).
• "Data is the foundation for all AI systems. In order to leverage that data, we have to properly ingest different file formats with accuracy." — Cedric Clyburn.

6. Synthesis and Conclusion

The shift toward "agentic" AI requires a more sophisticated approach to data ingestion than simple text extraction. Docling addresses the "garbage in, garbage out" problem by providing a robust, open-source pipeline that preserves document structure. By moving away from expensive, proprietary cloud-based parsing toward local, structured extraction, organizations can build more reliable, cost-effective, and scalable RAG and agentic systems. The integration of Docling with the Model Context Protocol (MCP) further democratizes these capabilities, allowing AI agents to interact with complex documents as easily as they interact with code.