The New Stack Agent - Glenn Solomon of Notable Capital & Burkay Gur of Fal.ai

Key Concepts

• Generative AI
• Media Models (Image, Video, Audio)
• Inference Optimization
• Model Customization (LoRAs, Fine-tuning)
• AI Infrastructure
• Developer Platform
• Speed & Scale
• Cost of Creativity vs. Cost of Creation
• AI Adoption in Enterprises
• Computing Paradigm Shifts

Purpose of the Show

The New Stack Agents podcast/livestream aims to cover the rapidly evolving field of Generative AI. It seeks to provide insights into the latest trends, technologies, and applications by engaging in conversations with industry experts, founders, and investors. The goal is to build a community and foster a dialogue around the transformative potential of AI.

Introduction and Guests

The hosts, Alex Williams and Frederick, introduce the guests: Burkai Gur, co-founder and CEO of Fal AI, and Glenn Solomon, Managing Partner from Notable Capital. The discussion focuses on Fal AI's platform and the broader trends in the generative AI space.

Burkai Gur and the Genesis of Fal AI

• Early Days: Burkai started Fal AI in early 2021, driven by the belief that compute and infrastructure for machine learning would be the next big trend, similar to the rise of Snowflake and Data Dog. Initially, the focus was on providing a runtime for Python.
• Shift to Media Models: The release of models like DALL-E 2 and ChatGPT led to a shift towards generative media models, particularly image models. The key insight was the massive demand for GPU-based inference due to the larger size and consumer-facing nature of these models.
• Early Challenges: In the pre-ChatGPT era, Fal AI focused on building datasets for enterprise machine learning models. Post-ChatGPT, the focus shifted to interfacing with consumers and addressing the inefficiency of running image generation models.
• Optimization Focus: Early efforts were dedicated to optimizing models like Stable Diffusion 1.5, focusing on making them run faster. This involved combining custom kernel implementations and other optimizations.
• Evolution Beyond Optimization: Fal AI has evolved beyond being just an optimization company, establishing a broader position in the market by focusing on the end-to-end developer experience and the applications of these models.

Glenn Solomon's Perspective on AI Investment

• Hesitation and Observation: Notable Capital initially hesitated to invest heavily in GenAI companies, taking time to assess the market and identify enduring advantages.
• Attraction to Fal AI: Fal AI's value proposition resonated with both model providers and developers. The platform's focus on speed and cost-effectiveness in running models was seen as a key differentiator.
• Enterprise Adoption: Glenn highlights the growing enterprise adoption of GenAI, with companies moving beyond testing and implementing real-world applications.
• Impact on Software Development: GenAI is significantly impacting the software industry, changing how software is written and tested. This is leading to more software being created, with developers' roles evolving.
• Computing Paradigm Shifts: Glenn emphasizes that the best venture capital opportunities arise during significant computing paradigm shifts, and he believes GenAI represents the next massive shift.

Fal AI's Technical Architecture for Speed

• End-to-End Platform: Fal AI owns the entire developer experience, requiring them to optimize for speed at every level, from inference to media delivery.
• Region-Sensitive Gateway: For low-latency audio models, Fal AI built a region-sensitive gateway to route requests to the nearest GPU, addressing the lack of native AI infrastructure.
• Incremental Optimization: Fal AI's optimization engine is developed incrementally, focusing on new models and applying existing optimizations while adding new ones at the compiler or kernel level.
• Custom CDN: Fal AI uses a custom CDN built on top of existing infrastructure, offering multi-cloud and multi-region content delivery with features like access control and streaming.
• Dynamic Model Nature: Fal AI's architecture accounts for the dynamic nature of models, particularly the use of LoRAs for model customization, which requires handling changes in the neural network structure.

Use Cases and Model Customization

• Importance of LoRAs: LoRAs (Low-Rank Adaptation) are extremely important for media models, allowing for customization and steering models towards specific use cases.
• Retail and E-commerce: Fal AI is seeing significant demand from the retail and e-commerce sectors, where companies are using the platform to improve the creative process of product photography, variations, and content generation.
• API Integrations and Custom Engagements: Fal AI offers API integrations, a user-friendly UI for non-technical users, and custom engagements for post-training and model customization.
• Open and Closed Models: Fal AI supports both open-source and proprietary models, recognizing the importance of providing developers with access to the latest and greatest models.

Impact on Creative Industries

• Cost of Creation vs. Cost of Creativity: Generative AI reduces the cost of creation but not necessarily the cost of creativity. Taste and creative direction remain essential.
• Empowering Creators: Generative AI empowers individuals who previously lacked the means to create high-quality content, leading to a net positive impact on the creative landscape.
• Iteration Speed: Speed and ease of use are crucial for creative workflows, allowing for rapid iteration and experimentation.

Monitoring and Observability

• Importance of Uptime and SLAs: Monitoring and observability are critical for ensuring uptime and meeting SLAs, particularly for enterprise customers.
• New Infrastructure Challenges: The AI infrastructure stack presents unique challenges compared to the traditional web stack.
• Industry Standards: Fal AI aims to meet industry standards for uptime, acknowledging the challenges of achieving high reliability in the AI space.

Opportunities for Startups

• Focus on Personas: Glenn suggests focusing on three key personas: software developers (including AI developers), data professionals, and security professionals.
• AI Applications in Verticals: There are opportunities for AI applications in various verticals, including media, search, direct marketing, healthcare, and finance.
• Security: With the growth of AI, there is an increasing need for new security tools to address emerging risks.

Conclusion

The conversation highlights the transformative potential of generative AI and the importance of infrastructure and platforms like Fal AI in enabling developers and enterprises to leverage these technologies. The discussion emphasizes the need for speed, scalability, and customization in AI solutions, as well as the evolving roles of developers and creatives in the age of AI.