Building a Smarter AI Agent with Neural RAG - Will Bryk, Exa.ai

Key Concepts

• PageRank: Google's original algorithm for ranking search results based on the web's graph structure.
• Transformers: Neural network architecture used in models like GPT-3, capable of understanding language subtleties.
• Embeddings: Numerical representations of documents or queries that capture meaning and context, not just keywords.
• Keyword Index: Traditional search engine approach of mapping words to documents containing them.
• Neural Search: Using embeddings and transformer models to understand the meaning of queries and documents.
• LLM (Large Language Model): AI models like GPT-3 and GPT-4 that can generate human-quality text and understand complex instructions.
• API (Application Programming Interface): A set of rules and specifications that software programs can follow to communicate with each other.

The Evolution of Search: From Keywords to AI

The speaker begins by contrasting the state of search in 1998, dominated by keyword-based search engines like Google using PageRank, with the emergence of powerful language models like GPT-3 in 2021. He notes the limitations of keyword search, which often fails to understand the nuances of user queries, and envisions a future where search engines leverage AI to deeply understand both queries and documents.

The Birth of Exa: A Search Engine for the AI Era

Driven by the limitations of existing search engines, the speaker founded Exa with the goal of building a search engine that combines the power of transformer models with web search. The core idea was to represent documents as embeddings, capturing their meaning and context, rather than relying on keywords. This allows for more nuanced and accurate search results.

Overcoming Challenges and Embracing the LLM Revolution

The speaker recounts the early days of Exa, including joining Y Combinator and experimenting with different transformer architectures. The launch of ChatGPT in November 2022 initially caused concern, as it seemed like LLMs might replace search engines altogether. However, the speaker quickly realized that LLMs have limitations, particularly in accessing and processing real-time information from the web.

LLMs Need Search: The Information Theory Argument

The speaker presents a compelling argument for why LLMs will always need search engines. He points out that the size of the web (in the exabyte range) far exceeds the capacity of LLMs to store all information in their weights (a few terabytes for GPT-4). Additionally, the web is constantly updating, making it impossible for LLMs to stay current without relying on external search.

The Mismatch Between Traditional Search and AI Needs

The speaker argues that traditional search engines, optimized for human users, are not well-suited for AI applications. Humans are lazy and prefer simple keyword queries, while AIs can handle complex queries and process vast amounts of information. This necessitates a new type of search engine designed specifically for AI.

Key Differences: How AI Search Differs from Human Search

The speaker highlights several key differences between how AI and humans use search:

• Precise and Controllable Information: AIs need search engines that return exactly what they ask for, not just what humans are likely to click on.
• Context-Rich Queries: AIs can leverage extensive context to formulate more specific and relevant queries.
• Comprehensive Knowledge: AIs can process and analyze large volumes of information, requiring search engines to return a much wider range of results.

The Query Space: Expanding the Possibilities of Search

The speaker visualizes the space of possible queries as a large circle, with traditional search engines only able to handle a small subset of simple keyword queries. He argues that AI has opened up new possibilities for semantic queries, complex queries, and queries that no one has even thought of yet.

Exa: One API to Access All Web Information

The speaker concludes by reiterating Exa's mission to provide a single API that can handle any type of query, empowering AI systems to access and process all the information they need from the web.

Code Demo: Building an AI Agent with Exa

The speaker provides a brief code demonstration showcasing how to use the Exa API to build an AI agent. The agent combines neural search and keyword search to retrieve information about engineers in San Francisco who like information retrieval. He also mentions the newly launched research endpoint, which performs deep research and returns structured output.

Conclusion

The speaker presents a compelling vision for the future of search, where AI-powered search engines play a crucial role in enabling AI systems to access and process information from the web. He argues that traditional search engines are not well-suited for this new era and that Exa is building a search engine specifically designed for the needs of AI. The key takeaway is that AI needs a different kind of search – one that is precise, controllable, context-aware, and comprehensive.