Deep Dive into Foundry IQ

Foundry IQ & Azure AI Search: Deep Dive into Knowledge Integration for AI Applications

Key Concepts:

• Foundry IQ: A knowledge layer built on Azure AI Search, enabling AI applications to access and utilize diverse data sources beyond their initial training corpus.
• Retrieval-Augmented Generation (RAG): The process of retrieving relevant information from external sources to augment the input prompt for a generative model, improving response quality.
• Azure AI Search: A cloud search service used to index and search various data sources, providing an API for accessing relevant information.
• Lexical Search: Keyword-based search matching terms directly.
• Semantic Search: Search based on the meaning of the query, using vector embeddings to find conceptually similar information.
• Vector Index: An index created using vector embeddings representing the semantic meaning of data chunks.
• Embedding Model: A model that converts text into high-dimensional vectors representing its semantic meaning.
• Reciprocal Rank Fusion (RRF): A technique for combining search results from different methods (lexical and semantic) to improve overall ranking.
• Agentic RAG/Multi-hop RAG: A more sophisticated RAG approach where the AI agent intelligently plans and executes multiple queries across different knowledge sources.
• Knowledge Source: A specific data source (e.g., blob storage, database, SharePoint site, Fabric OneLake) indexed for search.
• Knowledge Base: A collection of knowledge sources grouped together for a specific purpose.
• Extractive Data: Returning raw data chunks as search results.
• Answer Synthesis: Generating a complete answer based on retrieved information.
• Reasoning Effort (Minimal, Low, Medium): Controls the complexity of the AI agent's planning and query execution.

1. The Need for External Knowledge & RAG

Generative AI models are trained on a finite corpus of data with a specific cutoff date, excluding non-public information. To enable these models to utilize information outside of their training data, a mechanism for providing external data is required. This is achieved through Retrieval-Augmented Generation (RAG). The quality of the retrieved information directly impacts the quality of the generated response – “Garbage in, garbage out.” RAG involves retrieving relevant information and adding it to the prompt before sending it to the model.

2. Azure AI Search as the Foundation

Azure AI Search provides the infrastructure for retrieving this external information. It exposes an API and creates indexes to facilitate searching across various data sources. The core function is to receive a query, search for relevant data, and return it to the application for inclusion in the prompt.

3. Information vs. Knowledge: The Semantic Search Advantage

While Azure AI Search supports basic keyword-based (lexical) search, generative models benefit from semantic search. Natural language is nuanced, with words having multiple meanings and idioms requiring contextual understanding. Semantic search utilizes vector embeddings – high-dimensional representations of text meaning – to find conceptually similar information.

• Process: Data is chunked into blocks, and an embedding model creates vectors representing the semantic meaning of each chunk. When a query is received, it's also converted into a vector, and the system finds the closest matching vectors in the index.
• RRF & Re-ranking: Azure AI Search combines lexical and semantic search results using Reciprocal Rank Fusion (RRF) and then re-ranks the results based on semantic similarity, ensuring the most relevant information is returned.

4. Introducing Foundry IQ: From Information to Knowledge

Foundry IQ builds upon Azure AI Search, moving beyond simply retrieving information to providing a true knowledge layer. It enables “agentic RAG” or “multi-hop RAG,” allowing the AI to intelligently query multiple knowledge sources in a single request.

• Single-shot RAG vs. Agentic RAG: Traditional RAG (single-shot) retrieves information from a single source. Foundry IQ enables the AI to formulate a plan, query multiple sources, and synthesize information.
• Knowledge Sources: Foundry IQ introduces the concept of “knowledge sources,” which can include:
  • Azure AI Search indexes (traditional data sources)
  • Fabric OneLake data
  • Fabric IQ (Fabric’s enterprise ontology)
  • SharePoint sites
  • Microsoft Copilot (via MCP - Microsoft Capability Protocol)
  • Web (powered by Bing)

5. Foundry IQ Architecture & Configuration

• Knowledge Bases: Foundry IQ organizes knowledge sources into “knowledge bases” – collections of sources relevant to a specific domain or purpose. A knowledge base can contain up to 10 knowledge sources (depending on the Azure AI Search SKU).
• Indexing: Foundry IQ indexes data from various sources, creating vector indexes for semantic search. Remote knowledge sources (e.g., SharePoint via Work IQ) leverage existing semantic indexes.
• Microsoft Copilot Protocol (MCP): Foundry IQ can leverage MCP to access search tools exposed by Copilot, allowing the AI to utilize Copilot’s capabilities without needing to understand the underlying implementation.
• Configuration via Microsoft Foundry: Users configure knowledge bases and sources through the Microsoft Foundry interface, selecting data sources, defining descriptions, and setting retrieval instructions.

6. Reasoning Effort & Retrieval Strategies

Foundry IQ offers different levels of “reasoning effort” (Minimal, Low, Medium) that control the complexity of the AI’s query planning:

• Minimal: The AI simply searches all knowledge sources without planning or source selection.
• Low & Medium: The AI analyzes the query, considers knowledge source descriptions and retrieval instructions, and formulates a query plan, selectively querying relevant sources. Medium effort includes a self-reflection step to refine the results.

7. Extractive Data vs. Answer Synthesis

Foundry IQ offers two retrieval modes:

• Extractive Data: Returns raw data chunks, suitable for AI agents that perform their own reasoning.
• Answer Synthesis: Generates a complete answer based on the retrieved information, ideal for simpler chat applications. This mode is required when using the web as a knowledge source.

8. Data, Limits & SKUs

• Azure AI Search SKUs: Different Azure AI Search SKUs support varying numbers of knowledge bases and knowledge sources. (Free: 3 sources/bases, Basic: 15, S1/S2/S3: higher limits – check current documentation).
• Debugging & Transparency: The Foundry IQ interface provides detailed debugging information, showing the query plan, retrieval calls, and iterations used to generate the answer.

Conclusion:

Foundry IQ represents a significant advancement in knowledge integration for AI applications. By building upon Azure AI Search and introducing the concept of knowledge bases, agentic RAG, and configurable reasoning effort, it empowers developers to create more intelligent and context-aware AI solutions. The ability to seamlessly integrate diverse data sources, including structured data, unstructured content, and enterprise ontologies, unlocks the potential for AI to address complex business challenges and deliver valuable insights. The key takeaway is that Foundry IQ moves beyond simply providing information to delivering actionable knowledge that fuels AI-driven innovation.