Agentic Search for Context Engineering — Leonie Monigatti, Elastic

Key Concepts

• Agentic Search: An approach where an AI agent autonomously decides when, how, and what to search for to gather context, rather than relying on a fixed retrieval pipeline.
• Context Engineering: The process of curating and managing the information fed into an LLM’s context window to optimize response quality.
• Agentic RAG (Retrieval-Augmented Generation): A system where the agent uses tools to perform multi-hop retrieval, query rewriting, and selective information gathering.
• Tooling: Specialized functions (e.g., Semantic Search, ESQL, Shell/Bash) that allow agents to interact with external data sources.
• Progressive Disclosure: A technique for managing context window size by loading detailed skill instructions or documentation only when the agent specifically needs them.
• Low Floor/High Ceiling: A design philosophy where tools are easy to use for simple tasks (low floor) but powerful enough to handle complex, unexpected queries (high ceiling).

---

1. Evolution of Retrieval Pipelines

• Fixed RAG: The original model where user input is used verbatim as a vector search query. Limitations: It retrieves data even when unnecessary (confusing the LLM) and struggles with multi-hop queries that require iterative searching.
• Agentic RAG: Replaces the fixed pipeline with an agent that decides if a search is needed, which tool to use, and whether to refine the query.

2. Context Sources and Search Tools

The speaker identifies several key context sources, each requiring specific tools:

• Local Files: Accessed via search_files or shell tools.
• Databases: Accessed via semantic_search or query execution tools (e.g., ESQL).
• Web: Accessed via web search tools.
• Long-term Memory: Accessed via dedicated memory tools.
• Shell/Bash Tools: Highly versatile tools that allow agents to run CLI commands (ls, grep, curl) to navigate files or interact with database APIs. Warning: These require sandboxing to prevent unauthorized file deletion or system damage.

3. Building Robust Search Tools

The speaker highlights common failure modes and mitigation strategies:

• Failure Modes:
  1. Agent fails to call any tool (relies on parametric knowledge).
  2. Agent calls the wrong tool.
  3. Agent generates incorrect parameters for the tool.
• Optimization Strategies:
  • Tool Descriptions: Move beyond one-liners. Include "trigger conditions" (when to use/not use) and "relationships" (e.g., "call this skill before this tool").
  • Error Handling: Implement try-except blocks within tools to return error messages to the agent, allowing it to self-correct and rewrite queries.
  • System Prompts: Reinforce tool usage instructions in the system prompt if the agent consistently ignores tool descriptions.

4. Practical Methodologies

• General Purpose vs. Specialized: Start with general-purpose tools (like ESQL execution) to log agent behavior. Once patterns emerge, build specialized tools for frequent, narrow tasks to improve efficiency and reduce error rates.
• Agent Skills: Use markdown-based skill definitions that are injected into the system prompt. This provides the agent with "how-to" documentation for complex tasks (e.g., syntax rules for ESQL) without cluttering the context window permanently.
• Hybrid Approaches: Combining tools (e.g., using a database tool for retrieval and a shell tool for verification) often yields higher accuracy than relying on a single tool.

5. Notable Quotes

• "Context engineering is about 80% agentic search."
• "Doing good search is incredibly difficult and that's why we have many different techniques."
• "If you notice your agent is taking four or five tool calls per question, that's too many... it's an indicator that the tool is too difficult for it to use."

6. Synthesis and Conclusion

Effective agentic search requires a balanced stack of tools. Developers should avoid the "silver bullet" fallacy and instead curate a mix of specialized tools (for efficiency and low error rates) and general-purpose tools (for handling complex, unforeseen queries). Success is achieved by logging agent behavior, implementing robust error handling, and using progressive disclosure to keep the context window clean while providing the agent with the necessary documentation to perform complex operations.