Deepseek, Stargate and AI's $600 Billion Question with Sequoia's David Cahn

Key Concepts

• AI Model Commoditization
• Scaling Laws in AI
• Data Center Investment and Revenue
• AI Search as a Killer App
• Developer Infrastructure
• Go-to-Market Strategy
• Authenticity in Venture Capital
• Religion and AI Consciousness

Deep Seek and AI Model Commoditization

• Deep Seek: A Chinese AI model trained by a hedge fund, notable for its smaller size and distilled nature (trained on outputs from existing models).
• Key Point: The model's low training cost signals a potential commoditization of AI Foundation models.
• Counterpoint: While significant, Deep Seek might be overhyped compared to the implications of Ilya Sutskever's (Ilya Sutskever is the guy who basically invented modern deep learning) talk at NeurIPS.
• Ilya Sutskever's Perspective: Sutskever suggests that pre-training and scaling laws are becoming less critical, raising questions about the source of future AI breakthroughs.
• Market Reaction: The stock market's negative reaction to the news is seen as a lagging response to trends evident six months prior.
• Positive View: Cheaper models benefit the application layer and AI builders by reducing GPU and inference costs.
• Satya Nadella's View: Microsoft's CEO has long predicted model distillation and lower costs, which he sees as beneficial for Microsoft's hosting services.

Stargate and Data Center Investment

• Stargate: A proposed $500 billion investment in data centers.
• Contrasting Directions: Stargate (large data centers, scaling laws) contrasts with Deep Seek (smaller, cheaper models).
• Microsoft's Revealed Preference: Microsoft has a right of first refusal for OpenAI's data center builds, indicating its continued investment in large-scale infrastructure.
• Shift in Funding: A shift from equity-funded data centers (Microsoft, Google, Amazon) to credit-funded or levered data centers is occurring.
• Ben Thompson's Analysis: Compares this shift to the late stages of the dot-com era.
• Potential Ramifications: Could be positive if data centers lead to AGI, but may counter the narrative of smaller, cheaper models.
• Seoa's Perspective: Seoa maintains a long-term view and is not overly reactive to day-to-day market movements.

AI's $600 Billion Question

• Core Question: Where is the revenue to justify massive AI investments?
• Napkin Math: A calculation to estimate the revenue needed to pay back data center investments.
• Assumptions:
  • $150 billion in Nvidia GPU revenue in 2024.
  • $1 spent on GPUs requires $1 spent on data center infrastructure.
  • Startups need a 50% gross margin.
• Calculation:
  • $150 billion (GPUs) * 2 = $300 billion (Data Centers)
  • $300 billion * 2 = $600 billion (Required Revenue)
• Key Point: $600 billion is the revenue needed to pay back one year of investment.
• Current Status: The revenue gap identified in the summer of 2024 remains largely unchanged. OpenAI is generating the most revenue, but big tech companies haven't fully unlocked AI revenue streams.
• Funding Source: Data center investments are currently funded by existing profits from the lucrative cloud business of major players like Amazon, Google, and Microsoft.
• Prisoner's Dilemma: Cloud companies are compelled to invest in AI to avoid falling behind competitors, even if immediate returns are uncertain.
• Competitive Race: Companies believe in AI's potential but are also driven by fear of being outpaced.

AI Search as a Killer App

• AI Search: Identified as a key area for revenue generation and productivity uplift.
• Perplexity Example: Used as an example of a powerful research tool that synthesizes information and provides direct answers.
• Shift from Navigational to Informational Search: AI search moves beyond simply finding links to providing synthesized answers.
• Cognitive Architecture: The idea that AI search engines can be tailored to different professions and patterns of thought.
• AI Search for Different Professions: Examples include Harvey (legal), OpenEvidence (doctors).
• Four Dimensions of Differentiation:
  1. Intent Extraction: Understanding the user's intent behind a query.
  2. Proprietary Data: Leveraging unique datasets for better insights.
  3. Formatting: Presenting information in a way that suits the user's needs.
  4. Cognitive Architecture: Mapping the AI search experience to the user's thought processes.

Developer Infrastructure and AI Investment Thesis

• Early Investment Thesis: Focus on making developers more productive.
• Examples: Superbase, Replit, Confluent, Databricks, Starburst.
• AI Thesis Origin: Inspired by Snowflake's ability to make charts for better business decisions.
• Aha Moment: The idea of automating decisions and processes with data, leading to a potentially trillion-dollar opportunity.
• Weights & Biases Influence: Anecdotes of companies like John Deere using AI to automate processes reinforced the AI thesis.

Investment Learnings

• Weights & Biases Lesson: Patience is key when you have a great team, product, and belief in the market.
• Hugging Face Lesson: Sometimes it's clear which companies will succeed, and you need to do everything to win the deal.
• Mistakes: Underestimating the difficulty of go-to-market.
• Key Takeaways:
  • Need a good sales team and a commercially oriented founder.
  • Must understand the buyer's perspective and offer a significantly better product.
• Founder Preferences: Technical founders with a passion for go-to-market.
• Founder Growth: The rate of growth of a company is constrained by the rate of growth of the founder.

Evolution as an Investor

• Early Focus: Identifying great founders and companies.
• Shift in Perspective: The importance of being a "True Believer" and going "native" with investments.
• Key Question: In 5 years, when others give up, will I still believe in this company and mission?
• Affirmative Investment Decisions: Making investments based on long-term belief, even if short-term prospects are uncertain.
• Runway Example: Investing in the team before they developed stable diffusion because of their deep passion for video.
• Form Energy Example: Investing in a radical energy storage solution because of the importance of the problem and the strength of the team.
• People Matter: The team is as important as the problem being solved.

Future Investment Areas

• Global Supply Chain: Rebuilding the supply chain in the West, potentially creating a "Shenzhen" equivalent.
• AI and Robotics: The intersection of AI and robotics is expected to be fruitful.

Authenticity and Humility in Venture Capital

• Authenticity: Being true to oneself and building genuine relationships with founders.
• One-on-One Relationships: Focusing on individual connections rather than a large public persona.
• Aggressiveness in Private: Being assertive in one-on-one conversations but avoiding public displays of aggression.
• Mantra: "Everybody knows everything," emphasizing the transparency of human interactions.

Looking Smart vs. Being Right

• Concept Origin: Inspired by the speaker's father, who emphasized the distinction between form and substance.
• Silicon Valley Example: Steve Jobs showing up barefoot to Sequoia, where Don Valentine looked past the presentation to the substance of the idea.
• Tension: The conflict between making decisions that look smart in the short term versus those that are substantively right in the long term.
• Importance of Authenticity: Working with people who present their true selves and are honest about their beliefs.

Religion and AI

• Religion as a Framework: Religion attempts to answer the question of what it means to be human.
• Common Core: Despite different cultural contexts, religions share a common core of confronting consciousness, truth, and essential questions.
• Gödel, Escher, Bach: The book explores the unique element of human consciousness as self-reflection and recursiveness.
• AI and Self-Reflection: The open question of whether AI can be self-reflective and conscious.
• Empathy: A fundamentally human trait resulting from self-reflection.
• Religious Question: The question of whether AI will be unconscious or conscious is fundamentally a religious one.
• Continuity of Analysis: Religion offers a long history of grappling with deep questions about human existence.

Synthesis/Conclusion

The conversation explores the rapidly evolving landscape of AI, from the commoditization of models to the massive investments in data centers. It emphasizes the importance of long-term thinking, authentic relationships, and a focus on solving meaningful problems. The discussion also delves into the philosophical implications of AI, particularly the question of consciousness and the role of religion in understanding the human condition. The key takeaway is that success in AI requires not only technical expertise but also a deep understanding of human values and a commitment to building a better future.