How to build an AI Agent Startup from India? $1 Trillion Opportunity - Decoded | Manav Garg | Ep. 03

Key Concepts:

• AI Agents: Autonomous software capable of executing instructions, understanding various inputs (voice, chat, image, video), and learning through feedback loops.
• LLMs (Large Language Models): Fundamental building blocks of AI agents, capable of absorbing natural language and predicting the next word or action.
• Agentic Workflow: A completely autonomous workflow where multiple AI agents interact with each other to achieve a specific goal.
• Context, Memory, Planning: Three fundamental primitives enabling agentic capabilities.
• Model Layer: The underlying AI model that powers the agent, with value derived from narrow use cases and reliability, not just the model itself.
• Evaluation Framework: A system for measuring the performance and reliability of AI agents, crucial for iterative improvement.
• Global Benchmark: A standard of excellence against which AI agents are measured, often requiring founders to step outside their comfort zone.
• 100x Engineer: The concept of AI agents augmenting software engineers to become significantly more productive.
• Agent Experience: Focus on how the agent interacts with the environment, gets stuck, and recovers.

1. Defining AI Agents and Their Evolution

• AI Agent Definition: An AI agent is software that takes instructions as input and autonomously executes them to produce an output. It can understand various input formats (voice, chat, image, video) and learn through feedback.
• Example: Alexa/Google Home is a V1 agent. It responds to simple queries like "What's the weather?" or "Tell me the time of the next train." More sophisticated tasks include recording TV shows.
• Evolution:
  • Pre-ChatGPT Era: Input and data had to be structured.
  • ChatGPT Era (V2): Input can be in any format (email, voice), and data need not be stored in a structured way.
  • V3 (Current): Agents autonomously talk to each other to achieve a goal. Example: Sales process with account executive, SDR, and sales engineer agents. The focus shifts from tactics to goals.

2. Technological Shifts Enabling AI Agents

• LLMs as the Foundation: LLMs predict the next word, building a general model of the world. Example: Predicting the killer in a detective novel requires understanding relationships and concepts.
• From Words to Actions: LLMs can now predict the next action, enabling automation. Example: A calendar agent can manage schedules, considering preferences and constraints.
• Communication Between Agents: The mode of communication is evolving towards natural language or vectorized mathematical numbers. Function calling is a V1 approach.
• Cloud's mCP Protocol: A new protocol for linking various tools and servers, allowing AI agents to understand different languages and APIs.

3. Fundamental Primitives of AI Agents

• Context: Understanding the broader situation and user intent.
• Memory: Remembering past interactions and feedback to refine future actions.
• Planning: Completing tasks by breaking them down into smaller steps, updating context, and adjusting the plan based on feedback.

4. Market Opportunity and Potential Impact

• Consumer Example: AI-powered search engines can become "goals engines," orchestrating various agents (Expedia, Kayak) to book holidays.
• Enterprise Example: In sales and marketing, AI agents can augment human workers, driving efficiencies and tapping into both software and people budgets.
• Salesforce Example: Salesforce's $25 billion revenue is minuscule compared to the $1 trillion sales and marketing budget globally.
• Software Engineering: AI agents can automate the software development lifecycle, from writing code to testing and debugging.
• Impact on Workforce: AI will make engineers more efficient, allowing them to tackle bigger problems and be more creative.

5. Emergent's Approach to AI-Powered Software Engineering

• Autonomous Coding Agents: Emergent is building agents that can automate the entire software development lifecycle, from writing code to setting up environments and running tests.
• Agent Experience: Focus on how the agent interacts with the environment, gets stuck, and recovers.
• 100x Engineer Vision: AI agents will augment software engineers to become significantly more productive.
• Human-in-the-Loop Approach: Emergent's agents work with human operators or guides, allowing for control and oversight.
• Customer Success: Early pilots show phenomenal results, such as compressing months of work into days.
• Tech Debt Focus: Initial focus on areas like library upgrades and integrations, which are repeatable and easy to experiment with.
• Logistics Integration Example: Reducing the time for new logistics partner integrations from 1-4 months to 2 days.

6. Building an AI Agent-First Company: Five Steps

1. Narrow Focus: Identify a specific use case where agents can be reliable.
2. Build a Good Evaluation System: Create a framework for measuring performance and reliability. Start with public evals like SWE-bench.
3. Experimentation: Accelerate the rate of experimentation within the company.
4. Get Closer to Customer: Understand their needs and build realistic evaluation sets.
5. Build a Team: Hire AI experts or learn through experimentation.

7. Key Challenges and Considerations

• Reliability: Making agents reliable is a significant challenge.
• Reasoning: Improving the reasoning capabilities of AI models is crucial.
• Speed: The speed of execution and learning is essential.
• Adaptability: Being adaptable to new models and technologies is critical.
• Global Benchmarks: Setting global benchmarks and competing against the best is necessary for success.
• Customer Obsession: Customer obsession should drive product direction, not competition.

8. Google's Approach to AI Adoption

• Centralized Model Building: Google centralizes model building in the DeepMind team.
• Service Provider-Customer Relationship: DeepMind provides model capabilities to other teams within Google.
• Evaluation Framework: Constant evaluation of new models to see if they can be used at the application layer.
• Checks and Balances: Internal customers hold the model team accountable for providing capabilities.

9. Differentiating Factors for Valley Companies

• Talent: Ambitious and adaptable talent with a growth mindset.
• Know-How: Insatiable appetite to learn and experiment.
• Quick Learning Feedback Loop: Obsession with quick learning feedback loops and hypothesis testing.
• Global Benchmarks: Aspiring to be world-class and competing against global benchmarks.
• Vulnerability: Being prepared to be vulnerable and learn from failures.

10. Synthesis/Conclusion

The AI agent market presents a massive opportunity across various industries. The key to success lies in focusing on narrow use cases, building reliable agents, and continuously experimenting and learning. Founders must embrace a global mindset, set high benchmarks, and prioritize customer obsession. The future of software engineering and many other fields will be shaped by AI agents, and those who embrace this technology will be best positioned to thrive.