Scale AI CEO on Meta’s $14B deal, scaling Uber Eats to $80B, & what frontier labs are building next

Key Concepts AI Model Evolution (Knowing to Doing), Data Labeling, Training Data, Evals (Evaluation Data), Expert Data Labeling, Reinforcement Learning (RL) Environments, Generalizability of Data, Gross Margins, Independent Thinking, Not Losing as a Precursor to Winning, Curious Problem Solver, Working Across People (Collaboration), Good Leader, Uber Eats, Scour, Uber Direct, Midjourney V3.

---

Introduction to Jason Droege and Scale AI's Mission

The discussion introduces Jason Droege, the new CEO of Scale AI, in his first interview since taking over from Alex Wang, who now leads Meta's super intelligence team. The interview addresses the current perception that AI is not delivering on its enterprise promises, noting that robust automation for important processes typically takes six to twelve months. This is likened to major tech revolutions requiring foundational, often unseen, operational effort ("digging up every single road"). A core theme is the evolution of AI models from "knowing things" to "doing things," shifting the focus to "what can it do for me?" and how AI agents make decisions.

Jason Droege's background includes co-founding Scour with Travis Kalanick before Uber, and most notably, launching and leading Uber Eats, which grew into a multi-billion dollar business and was crucial for Uber's survival during the pandemic. Scale AI is highlighted as a pioneer in data labeling, training data, and creating evaluations (evals) for AI labs. The conversation delves into how AI models become smarter through human expert input, the changing market of data training, and the future role of humans in AI development.

---

Early Career Lessons: The Scour Experience

Droege shared a pivotal lesson from his time co-founding Scour, a peer-to-peer file-sharing app, with Travis Kalanick when they were 19-20 years old: "everything's negotiable" in business and startups. He recounted the chaotic financing process with initial investors Ron Burkle and Mike Ovitz, where deal terms constantly changed, and demands for equity escalated from 50% to 80% in a single day. This experience taught them that "there is no way to do things. There is just the way that you can negotiate your way through the world," a lesson that profoundly influenced both him and Kalanick at Uber.

Scour eventually faced legal challenges, being sued for a "quarter of a trillion dollars" by the RIAA and MPAA for facilitating free content sharing. The case settled for $1 million, revealing that established companies often "make up numbers" to drive competitors into bankruptcy rather than following a fixed playbook. This provided a "cold splash of water about how the real world really works."

---

Scale AI's Independence and the Meta Deal

Jason Droege clarified the nature of the recent Meta deal, stating that Scale AI remains a fully independent company. Meta invested "a little bit over $14 billion" to acquire "49% of the company non-voting stock," without taking a new board seat (Alex Wang continues to fill a board seat). There is no preferential access or relationship with Meta beyond a long-standing data partnership that might expand. All privacy and data security protocols remain in place. Only "about 15 people went over in the transaction" to Meta.

Scale AI currently employs "about 1,000 people" and operates two major businesses, each generating "hundreds of millions of revenue," effectively housing "two unicorns inside the company." Droege emphasized that Scale AI has "grown every month since the deal happened," countering some public narratives.

---

The Evolution of AI Data Labeling: From Generalists to Experts

Scale AI's history began in 2016 with Alex Wang's insight into the importance of data for models, initially focusing on autonomous vehicle labeling, then computer vision (including for the Department of Defense in 2020), and eventually generative AI. As AI models have improved, their data needs have evolved significantly.

• Past (18 months ago): Tasks were basic, such as comparing two short stories and editing them for preference.
• Present: Tasks are highly sophisticated, taking "hours of time" and requiring "PhDs and professionals." Examples include:
  • Building an entire website by one of the world's best web developers.
  • Explaining nuanced cancer topics to a model.
• Expert Network Statistics: "80% of the people in our expert network have a bachelor's degree or greater," with "15% having a PhD." PhDs on the network earn "significant amounts of money."
• Scale's Role: Scale AI has been at the forefront of this shift, adapting to and even driving the need for expert data labeling by identifying model deficiencies and offering expert-driven solutions.
• Finding Experts: This is challenging. Key methods include referrals (experts enjoy contributing their knowledge to AI), campus programs (engaging professors and students), and traditional platforms like LinkedIn. Providing a great experience is crucial for retention.

---

Reinforcement Learning, AI Agents, and Data Generalizability

The conversation highlighted the trend of AI models moving from "knowing things" to "doing things," which involves Reinforcement Learning (RL) Environments. These are "sandboxes for AI agents to play in to accomplish a goal so that they can learn how to accomplish that goal."

• Example: Salesforce Agent: An AI agent might navigate a Salesforce instance to perform a business process, requiring it to recognize data, understand configurations, and achieve high reliability. Crucially, the agent must know when to "pop it up to a human being for feedback" if accuracy is low.
• Challenge: The permutations of environments, goals, software systems, data types, and user complexities are "enormous."
• Scale's Research Focus: Determining "how generalizable is each individual task or each individual environment" to avoid collecting "45 trillion combinations." The goal is to provide "the most valuable data to model builders that accomplishes the goal of making agents as useful as possible for their end users."
• Specific Data Examples:
  • For website building, data can include code, annotated decisions ("I made this decision for this reason"), or explanations of why a website is broken for debugging tools.
  • For short stories, it involves comparing model-generated texts and suggesting improvements.
  • For Salesforce, it's teaching an agent how to "book a meeting with a prospect."
• Healthcare System Case Study (Scale's Solutions Business): Scale built a tool for a healthcare system to address doctor backlogs and improve diagnoses for rare cases. Doctors typically read 200-300 pages of documentation. The AI tool reads these documents, highlighting the top 5-10 critical points (e.g., non-obvious allergies conflicting with medication). This requires digitizing human judgment and deep subject matter expertise, as off-the-shelf models have limits. Enterprises are increasingly doing their own labeling to capture specific, nuanced judgment unique to their culture and objectives, which is becoming a bottleneck Scale helps unblock.
• AI's Strengths: AI excels at automating human processes that are 10-20% accurate, potentially boosting them to 50-80%. However, for processes already 98% accurate, achieving the remaining 2% with AI is still challenging.

---

The Future of Human-in-the-Loop AI and the "White-Collar Apocalypse"

Droege believes that the "history of data labeling is a history of new beginnings," with Scale constantly adapting to new data needs. He argues that the idea of AI no longer needing external human data implies that "no new human skill and no new human knowledge is important enough to put into these models," which he finds "pretty far out there." Humans will remain "in the loop" because these systems need to work for us.

Regarding the "white-collar apocalypse," Droege takes a "practical side," dismissing the idea of it happening in the next one to two years as "very far-fetched." He emphasizes human adaptability, noting that throughout technological history, people have always adapted to change.

---

Evals and the Reality of AI Adoption in Enterprises

Evaluations (evals) are crucial, especially for enterprise and government customers, as they "establish the benchmark for like what good looks like." For instance, in the healthcare example, a doctor would create evals to define what the AI should discover in a patient's report. AI systems are probabilistic, making the distinction between "good" and "correct" nuanced.

Droege highlighted a common misconception: people assume AI is trained on all human knowledge. In reality, "people are sitting around teaching AI things it doesn't know, filling gaps," and correcting its understanding. This "operational chiseling" is akin to "laying broadband means you need to dig up every single road in America." While AI models are "remarkable" (e.g., consistent punctuation), their improvement is a combination of "computational power, model improvement and data."

Looking ahead 2-3 years, Droege predicts AI technology will advance to a point where it "will push the change management and policy makers to say like, 'ooh, what do we do with this because it's getting pretty close.'"

Addressing the "AI not delivering on the promise" narrative in enterprises (e.g., MIT study on pilot failures), Droege views the reported 95% failure rate as "clickbait" and "hyperbolic." He attributes it to the "denominator effect" – it's easy to start AI projects. Achieving robust automation for "important process" takes "six to twelve months," involving legal, policy, regulatory approval, change management, and ensuring accuracy. When successful, the impact is profound, but "the time to get there is just longer than what people are selling." His summary: "Easy to learn, hard to master."

---

Product Lessons: Customer Obsession and Independent Thinking

Droege emphasized the importance of being close to customers, but not taking their words literally. Instead, he focuses on understanding their "underlying incentives" (financial, ego, career growth), believing that "show me the incentive and I'll show you the outcome."

• Uber Eats Restaurant Economics Case Study: When launching Uber Eats, Droege's team couldn't get restaurants to share unit economics. They independently researched by ordering food, getting supplier catalogs, and matching ingredient costs to build their own view of ingredient vs. labor costs. This revealed that restaurants have a "70 to 80% incremental gross margin product" if demand triples. This insight allowed them to confidently charge a 30% commission (which restaurants initially found too high, but the market cleared at ~25%), providing incremental demand.
• Urgency of the Buyer: Droege stressed that a product must address the customer's most urgent, top-of-mind problems, not just provide value. Otherwise, "you're just going to have a long road to a small town."
• Independent Thinking: This is crucial for finding "alpha in the market." Droege constantly questions why he, among "a million entrepreneurs," might have a unique insight and why he is the one to pursue it. He describes himself as a "contrarian personality type" who seeks "the thing that's true that people don't believe is true." He also advises against "falling in love with your ideas," instead being willing to discard them for the mission of serving the customer.

---

Business Building: Setting a High Bar and the Uber Eats Journey

Droege outlined two key factors for successful new businesses:

1. Founder as a Force of Nature: The most important factor is the founder's energy to pivot and persevere through years of hardship.
2. Business Model Knowledge: Understanding what constitutes a good business model (e.g., marketplaces, SaaS, recurring revenue, sticky, network effects) helps filter out bad ideas. These models are "more valuable at scale than big scale than low scale." Passion for the problem is also essential.

• Uber Eats Exploration: Droege kept a "very, very wide aperture on ideas."
  • Failed Idea: Convenience Van: They launched vans with 250 SKUs in DC, but it was a "flop" because they didn't understand retail (lacked key convenience items like cigarettes, beer, slurpies).
  • Grocery: Unit economics (pick-packing) "terrified" him.
  • Generalized Point-to-Point Delivery (Uber Direct): Also a "flop" as consumers didn't have the need in 2014.
  • Uber Eats Success: Food delivery "popping off on all signals," unit economics worked, and it offered a compelling solution to enable independent restaurants and local economies.
• Uber Eats Growth: Launched in December 2015 in Toronto, generating "$20,000 for the sales" in two hours. It grew from $0 to "$20 billion" in 4.5 years by the time Droege left, and is now "pushing $80 billion," with COVID accelerating its growth from $20 billion to $50 billion in about a year. Droege acknowledged that "luck is part of the game."
• McDonald's Deal: Droege initially rejected McDonald's approach, wanting to support local restaurants. However, his team convinced him, leading to an exclusive relationship that brought "an insane number of customers" and caused the business to "hockey stick again." He believes his initial resistance helped secure a better deal. The global onboarding of McDonald's was "mayhem" for the less than two-year-old Uber Eats business.

---

Gross Margins and the "Not Losing" Philosophy

Droege views gross margins as a crucial "filter" for business viability. High gross margins combined with healthy churn curves are a strong indicator of a healthy business, suggesting significant value addition and differentiation. He uses it as a "litmus test": if an idea can't achieve a 60% gross margin, it quickly reveals underlying problems like strong competition or lack of differentiation. He cited Costco and Walmart as examples of businesses with low gross margins that succeed by using price to achieve massive scale, absorb demand, and create high barriers to entry.

His life motto, "the end is never the end," underpins his philosophy of "not losing as a precursor to winning." In a tech culture that often promotes "just go for it," Droege advocates for entrepreneurs to "look at the risk profile" and make "asymmetrically positive decisions." Survival is paramount, as "most people just give up before they could get their timing right." He shared an example of self-funding a used golf club business after the dot-com bust, which was profitable but "painful the entire way" due to hubris and failing to anticipate margin compression. This taught him the value of "thinking upfront to save yourself a lot of pain downstream."

---

Hiring and Team Building

Droege has a nuanced view on hiring:

• Specific Roles: For certain roles (e.g., researchers, those with critical customer relationships), "you absolutely need the right experience" due to the fast-moving market. These constitute about 5% of roles.
• General Interview Criteria: For most roles, he focuses on three qualities:
  1. Curious problem solver: Can articulate their problem-solving approach verbally.
  2. Works across people: Humble and collaborative.
  3. Good leader.
• Adaptability: Given the changing world, adaptability is key, and direct experience is not always one-to-one relevant.
• Uber Eats Management Team: Droege built his Uber Eats management team as an "organism of strengths," minimizing conflicts. This team remained largely consistent from $0 to $20 billion, demonstrating that knowing each other's strengths and weaknesses and compensating for each other was more important than classic advice about scale experience. He believes in people's ability to learn and adapt. These core human qualities (problem-solving, leadership, collaboration) are fundamental to human success and will remain relevant in an AI-driven world.

---

AI Corner and Conclusion

Droege uses AI daily as a "tutor" for new concepts in the rapidly evolving AI space, especially when his team members don't have time for in-depth explanations. He uses voice mode on his commute to learn. He also uses AI to quickly identify "the most important thing" in internal documents, addressing the "broadcast problem" in organizations. He cited the founders of Perplexity, who have a rule to ask AI before asking a human.

His favorite books include The Selfish Gene, The Road Less Traveled, Good to Great, and Thinking, Fast and Slow. He recently enjoyed the Formula 1 movie. His favorite recent product discovery is Midjourney V3, which accurately rendered a scene from an old script he fed it, highlighting its potential for "emotionally life-changing" applications like animating family photos.

Droege reiterated his life motto, "the end is never the end," as a reminder to persevere through challenges. Despite his initial aversion, he orders McDonald's frequently on Uber Eats as a family treat.

He concluded by encouraging listeners to follow him on X (@JDroege) and to visit Scale.com/careers, noting that Scale AI has 250 open roles across its growing data, applications, and services businesses, including recent two $100 million government contracts. He emphasized that the public narrative often doesn't fully represent the "amazing work" and "ton of value" Scale AI's teams and customers are achieving.