How agents will unlock the $500B promise of AI - Donald Hruska, Retool

Key Concepts

• Agentic AI: AI systems that can reason, act, and self-verify, going beyond simple text completion.
• Vibe Coding: Rapid software development using AI tools like Cursor and Windsurf, where developers describe the desired outcome and the AI generates the code.
• React Framework (for Agents): A framework for building agents that instructs the agent to reason, act, reason, act until it determines that it's come up with a final answer.
• LLM (Large Language Model): The core AI model used in agents for decision-making and code generation.
• Tools (for Agents): Functions or external services that agents can access and use to perform actions.
• Managed Agent Platform: A platform that provides pre-built infrastructure, connectors, and observability for deploying and managing AI agents.
• Build vs. Buy Decision: The strategic choice between developing AI agents in-house or using a managed platform.
• Observability: The ability to monitor and understand the behavior of AI agents, including token usage, costs, and runtime information.

AI in Enterprise: From Toy Chatbots to Agentic AI

The speaker, Donald from Retool, discusses the shift from basic AI applications like chatbots to more sophisticated agentic AI in enterprises. Despite significant investment in AI infrastructure, many companies are still struggling to implement AI effectively. The key point is that enterprises can now build agents with guardrails that plug into real production systems.

• Growth in AI Spending: Anthropic's annualized revenue grew from $1 billion in December to $3 billion at the end of May, and OpenAI is projected to reach $12 billion in revenue by the end of 2025. This growth is largely driven by enterprise AI spending.
• Coding Transformation: Tools like Cursor and Windsurf are transforming coding workflows, with engineers using LLMs for code generation and review.
• Vibe Coding and its Power: Vibe coding allows developers to quickly create software by describing the desired outcome to AI tools, which then generate the code. Rick Rubin called vibe coding "the punk rock of software."

Building Agents: Easier Than Deployment

While building a basic AI agent can be relatively simple (around 100 lines of code in JavaScript or Python using the React framework), deploying it to production in an enterprise environment is challenging.

• Agent Architecture: An agent consists of an LLM wrapped in an execution loop that can read, decide, call tools, and self-verify.
• Agent Loop: The agent loop involves the LLM deciding when a tool needs to be invoked, calling the tool, passing the result back to the LLM, and determining when a final answer has been reached.
• Challenges in Production: Enterprises face challenges such as single sign-on, role-based access control, secure integration with external services, audit logs, compliance (e.g., SOC 2), and internationalization.
• Risks of Vibe-Coded Logic: The Information reported on the risks of using vibe-coded logic in production, citing real-world examples of vulnerabilities introduced by AI-generated code.
• Model Hallucinations and Security: Models can hallucinate, produce unpredictable results, and pose security risks if not carefully managed. Cost overruns are also a concern.
• Importance of Evals: Evals are an important safeguard in making your non-deterministic agent as deterministic as you can.

Four Approaches to Agent Implementation

Donald outlines four approaches to implementing AI agents, each with its own trade-offs:

1. Build from Scratch: Full control, purpose-built, but high lift. Requires AI/ML engineers and fine-tuning LLMs.
2. Use a Framework (e.g., LangGraph): Medium lift, flexible, but tied to the framework. Offers control over memory modes.
3. Agent Platform (e.g., Retool Agents): Low lift to production, opinionated defaults, but tied to the platform. Provides hosting, connectors, and observability.
4. Verticalized Agent: Dialed in for one use case, minimal flexibility beyond that.

Build vs. Buy: A Strategic Decision

The decision to build or buy an AI agent depends on the specific use case and the company's priorities.

• Core Product vs. Commodity Workflow: Build agents for core products or competitive advantages; buy for commodity workflows needed in days, not quarters.
• Risk Assessment: Consider the risks of engineers debugging business logic versus dealing with infrastructure issues like OAuth.
• Evaluation Criteria for Managed Platforms: Evaluate the breadth of connectors, built-in permissioning, compliance, audit trails, and observability.
• Cost Considerations: Factor in token costs, infrastructure costs, and engineering costs.

Observability: Understanding Agent Behavior

Observability is crucial for understanding token usage, estimated costs, and runtime information. Platforms should allow users to drill down into specific agent runs.

The Future of AI Agents: A Hybrid Approach

The speaker draws an analogy to how businesses approach software development today, suggesting a hybrid approach for AI agents.

• Hand-Built vs. Platform-Based: Companies will likely have a few hand-built agents for core use cases and a long tail of business use cases hosted on a platform.
• Examples: Stripe uses React for customer-facing software and Retool for internal tooling. Cursor might use an agent platform for customer support as they grow.

Real-World Impact and Cost Reduction

Retool customers like AWS, ClickUp, and dcript have seen significant benefits from automating business processes with AI.

• Cost Savings: ClickUp saved over $200,000 in vendor costs and reduced headcount.
• Time Savings: dcript estimated saving hundreds of hours of work weekly with the 50 apps they built.
• Work Automation: Retool customers have automated over 100 million hours of work to date.

The Democratization of Information and Potential

AI and agents are expected to enhance human capabilities and unlock limitless potential, similar to how the printing press democratized access to information.

• Inference Cost Reduction: Inference costs have dropped dramatically, with cost per token decreasing by 99.7% from 2022 to 2024.
• Increased Interest: Google searches for AI agents have increased 11x in the last 16 months.

Conclusion

The key takeaway is that the focus should be on helping engineers create the most leverage and choosing the right tool for the job, rather than seeking a single solution to automate everything.

Questions and Answers

• The speaker confirmed that the "build vs. buy" philosophy was developed internally at Retool.
• Retool aims to use its own platform as much as possible and will build custom solutions only when necessary.
• On-prem support for Retool Agents is coming soon, including support for air-gapped customers.