RAG with Distributed Cloud AppConnect, BIG-IP and NetApp

Key Concepts

• RAG (Retrieval-Augmented Generation): Enhancing AI inferencing by providing context from external data sources.
• LLM (Large Language Model): The AI model used for generating responses.
• Embedding LLM: LLM used to create vector embeddings of data for semantic search.
• Generative LLM: LLM used to generate the final answer based on retrieved context.
• F5 Distributed Cloud: A platform providing networking and security services across multiple cloud environments and on-premises locations.
• App Connect: F5 Distributed Cloud's load balancing and application delivery service.
• Network Connect: F5 Distributed Cloud's layer 3 IP reachability solution for connecting sites.
• S3 (Simple Storage Service): An object storage service protocol.
• NFS (Network File System): A distributed file system protocol.
• SMB (Server Message Block): A network file sharing protocol.
• Vector Database: A database optimized for storing and searching vector embeddings.
• Semantic Similarity: Measuring the similarity between the meaning of text or data.
• CE (Customer Edge): A component of F5 Distributed Cloud deployed at each site.
• WAF (Web Application Firewall): A security measure to protect web applications from attacks.
• API Protection: Security measures to protect APIs from abuse and attacks.
• Ollama: A tool for running LLMs locally.
• Hugging Face: A platform for sharing and using machine learning models.

1. Overview of F5 Distributed Cloud and RAG Integration

The video demonstrates how F5 Distributed Cloud enables flexible and context-enriched RAG (Retrieval-Augmented Generation) by connecting an LLM (Large Language Model) running in one location (Redmond, Washington) to remote data stores in various locations, including public clouds (Azure, AWS) and corporate sites (San Jose office with a NetApp appliance). The core value is providing context-aware AI inferencing by parsing internal data and using it to generate meaningful answers.

2. Connectivity Options: Layer 3 vs. Layer 4-7

• Layer 3 (Network Connect): Provides IP reachability between sites, allowing inside interfaces to communicate with each other.
• Layer 4-7 (App Connect): Uses load balancers to project protocols like S3 over HTTPS or TCP load balancers for NFS/SMB volumes.

The video focuses on App Connect, specifically setting up an HTTPS load balancer to project access to files and folders on a NetApp appliance in San Jose to the LLM in Redmond.

3. App Connect Configuration for S3 Access

The example details configuring App Connect to project data from San Jose to Redmond using the S3 protocol:

• Service Name: A fully qualified domain name (FQDN), either internet-compliant or a private DNS name.
• Protocol: HTTPS with a private certificate (or a publicly trusted certificate if exposed to the internet).
• Port: 443.
• Origin Pool: The NetApp appliance in San Jose, specified by its private IP address (10.xx) behind a secure mesh Site 2CE, communicating locally on port 443.
• Health Checks: Ensuring the availability of the appliance.
• Security: Options to add Web Application Firewall (WAF) and API protection (especially useful for the API-centric S3 protocol).
• Availability Scope: Restricted to the inside interface of the CE in the Redmond location, not exposed to the general internet.

4. Data Access and LLM Integration

Regardless of using Layer 3 or Layer 4-7 connectivity, remote data can be mounted locally in Redmond using commands like mount for NFS or similar commands for S3. In a self-hosted LLM environment, tools like Olama or Hugging Face can be used to load embedding LLMs (e.g., by Nomic) and generative LLMs.

The demonstration shows that App Connect makes folders in the San Jose office available through the S3 protocol. These folders can be used as RAG material by ingesting files as embeddings and storing them in a vector database.

5. RAG in Action: Context-Aware Question Answering

The video illustrates the power of context with a RAG solution. A vague question is posed to the LLM. Thanks to RAG, the system finds relevant data through semantic similarity in the vector database and provides it to the LLM, resulting in a meaningful answer. The system also provides attribution, showing the specific documents, pages, and chunks that contributed to the answer.

6. Key Benefits and Technologies

• RAG adds context to AI inferencing.
• F5 Distributed Cloud enables access to disparate data sources (SMB shares, NFS volumes, S3 buckets) across the enterprise.
• Worldwide high-bandwidth network for inter-site connections.
• Two main technologies:
  • Network Connect (Layer 3): IP reachability between sites.
  • App Connect (Layer 4-7): Load balancing and application delivery, projecting origin pool members to specific consumer sites.

7. Synthesis/Conclusion

F5 Distributed Cloud provides the infrastructure and tools necessary to implement context-enriched RAG solutions. By offering both Layer 3 and Layer 4-7 connectivity options, it enables organizations to securely and efficiently connect their LLMs to data sources distributed across various locations, enhancing the accuracy and relevance of AI-driven insights. The demonstration highlights the ease of configuring App Connect to project S3 data from a remote site to an LLM, showcasing the platform's ability to facilitate context-aware AI inferencing.