Build AI agents for e-commerce with ADK + Vector Search

Key Concepts

• RAG (Retrieval Augmented Generation): A system where an LLM retrieves information from a knowledge base (vector search) before generating a response.
• Vector Search: A search method that uses embeddings to find similar items based on semantic meaning.
• Embeddings: Numerical representations of data (text, images) that capture their semantic meaning.
• Multimodal Search: Searching across different data types (text, images) using multimodal embeddings.
• Hybrid Search: Combining semantic search (vector search) with keyword search.
• Task Type Embeddings: Embeddings trained to capture the relationship between queries and relevant documents, rather than just semantic similarity.
• ADK (Agent Development Kit): An open-source framework by Google for building AI agents.
• Agent as a Tool: A design pattern where one agent uses another agent as a tool to perform a specific task.
• Generative Recommendations: Using AI agents and vector search to generate personalized and trend-aware recommendations.

Vector Search and Embeddings

• Typical RAG System: User query -> LLM -> Vector Search (retrieval) -> LLM (generation) -> User response.
• Challenges with Usual RAG Systems:
  • Multimodal Search: Handling queries that involve both descriptive information (e.g., "cups with dancing figures") and specific identifiers (e.g., product number "123 ABC").
  • Recommendations: Providing meaningful suggestions that go beyond simple text similarity.
• Advanced Practices for Vector Search:
  • Multimodal Search:
    • Uses multimodal models to generate embeddings that are shared across different modalities (images and text).
    • Enables text-to-image and image-to-text searches.
    • Example: Searching for "cups with dancing figures" by analyzing images, without relying on text descriptions.
    • Vertex AI Embeddings API can be used to generate multimodal embeddings.
  • Hybrid Search:
    • Combines semantic search and keyword search in a single index.
    • Addresses the limitations of semantic search with product names or newly added products.
    • Example: Searching for "1234" (product number) using keyword search to find items with that keyword in the product description or title.
  • Task Type Embeddings:
    • Addresses the issue where simple similarity search fails due to different semantics between queries and documents.
    • Uses dual encoder or tutor models to learn the relationship between query and document domains.
    • Vertex AI Embeddings API provides pre-trained tutor models for task type embeddings.
    • Example: Using task type embeddings to find relevant birthday present recommendations, even if the text descriptions are not directly similar.

AI Agents and Vector Search

• Challenges for E-commerce Websites:
  • Smart Recommendations: Providing personalized and trend-aware recommendations.
  • Handling ambiguous or vague questions (e.g., "birthday present for 10 years old son").
• Solution: Combining AI Agents with Vector Search:
  • The AI agent takes the user query and triggers a Google search to research current trends.
  • The agent generates a set of queries for finding interesting items.
  • The search agent runs these queries against the vector search index.
  • This approach allows for a more intelligent and multifaceted search strategy.
• Showers Conscious Demo:
  • Demonstrates the combination of AI agents and vector search.
  • Users can ask ambiguous questions (e.g., "birthday present for 10 years old son").
  • The AI agent uses vector search and issues multiple queries to get results.
  • Deep research mode uses Google search to research what items people are buying for similar queries.
  • The agent generates a large number of queries (e.g., 100) and picks the best results.
  • The agent can also understand images and provide recommendations based on uploaded images.

Implementing the Agent with ADK

• ADK (Agent Development Kit):
  • An open-source framework developed by Google for building AI agents.
  • Supports Gemini and third-party models.
  • Supports real-time audio and image streaming.
• Deep Research Mode Implementation:
  • The UI agent uses Google search to learn about current trends for finding birthday presents.
  • The UI agent asks the search agent to generate 20 queries per item category.
  • The agent repeats this for five categories (total 100 queries).
  • All queries use multimodal embeddings, task type embeddings, and keyword embeddings.
  • The queries are sent to the vector search in parallel.
  • The search agent performs multimodal item curation, reviewing item images and descriptions.
  • Gemini curates items by analyzing images and user intent.
• Code Example (Notebook Sample):
  • Install ADK: pip install google-adk
  • Set environment variables (project ID, location, etc.).
  • Define a test function for the agent.
  • Define a basic shop agent with Gemini.
  • Define a function to call the vector search backend (HTTP request).
  • Wrap the vector search function with an ADK tool named find_shopping_items.
  • Extend the shop agent with the search capability by adding instructions and the find_shopping_items tool.
  • Define a research agent that uses Google search to generate queries.
  • Finalize the shop agent by combining the research agent and the vector search capability.
• Agent as a Tool Design Pattern:
  • The main agent (shop agent) controls the multi-agent system.
  • The sub-agent (research agent) is used as a tool to perform a specific task.
  • This design pattern allows for a more controlled and focused user experience.

Synthesis/Conclusion

The video demonstrates how to build a sophisticated e-commerce recommendation system by combining AI agents (using Google's ADK) with advanced vector search techniques. It highlights the limitations of simple similarity search and introduces multimodal search, hybrid search, and task type embeddings as solutions. The "Showers Conscious" demo showcases how an AI agent can leverage Google search and vector search to provide personalized and trend-aware recommendations. The video also provides a code walkthrough, illustrating how to implement these concepts using ADK and Vertex AI. The key takeaway is that by combining the power of AI agents with advanced vector search practices, it's possible to create more intelligent and helpful e-commerce experiences.