Scaling and the Road to Human-Level AI | Anthropic Co-founder Jared Kaplan

Key Concepts

• Scaling Laws (Pre-training & Reinforcement Learning)
• Pre-training (Imitating human-written data)
• Reinforcement Learning (Optimizing for helpful, honest, and harmless behavior)
• Flexibility of AI (Ability to handle various modalities)
• Time Horizon for AI Tasks (Length of tasks AI can perform)
• Organizational Knowledge (AI understanding company context)
• Memory (AI retaining task-specific progress)
• Oversight (AI understanding fine-grained nuances)
• AI Integration (Incorporating AI into existing systems)
• Human-AI Collaboration (Humans managing and sanity-checking AI work)

From Physics to AI: A Personal Journey

Jared Kaplan, formerly a theoretical physicist, discusses his transition to AI. His initial interest in physics stemmed from a desire to understand the universe's fundamental principles, including determinism and free will. Frustration with the slow progress in physics, coupled with the growing excitement around AI, led him to switch fields. Skeptical at first, he was eventually convinced of AI's potential and joined Anthropic.

How Contemporary AI Models Work: Pre-training and Reinforcement Learning

Kaplan outlines the two primary phases in training modern AI models:

1. Pre-training: Models learn to predict the next word in a sequence based on vast amounts of human-written text and multimodal data. This phase teaches the model the underlying correlations in the data. Example: The model learns that the word "elephant" is unlikely to follow "speaker at a journal club."
2. Reinforcement Learning (RL): Models are fine-tuned using feedback to reinforce desired behaviors (helpful, honest, harmless) and discourage undesirable ones. This involves human or crowdworker input on which responses are better. Example: Early versions of Claude were trained by having people pick the best response in a conversation.

Scaling Laws: The Engine of AI Progress

Kaplan emphasizes the importance of scaling laws in both pre-training and RL:

• Pre-training Scaling Laws: As compute and dataset size increase, model performance improves predictably. This was discovered by asking simple questions about the importance of big data and model size. The precision of these trends, observed across many orders of magnitude, provided conviction that AI would continue to improve predictably.
• Reinforcement Learning Scaling Laws: Research on AlphaGo (and later Hex) showed that increasing compute in the RL phase also leads to predictable performance gains, measured by ELO scores (chess ratings). Andy Jones's work on Hex demonstrated these straight-line scaling trends.

The core driver of AI progress is not necessarily increased researcher intelligence, but rather the systematic application of scaling compute for both pre-training and RL.

Unlocking Capabilities: Flexibility and Time Horizon

Kaplan describes AI capabilities along two axes:

1. Flexibility: The ability of AI to handle various modalities and meet users where they are. AlphaGo, while super-intelligent, was limited to the Go board. Modern AI is becoming increasingly multimodal.
2. Time Horizon: The length of tasks that AI can perform. This is increasing steadily, doubling roughly every 7 months according to research by an organization called "meter." This suggests that AI will eventually be able to handle tasks that take days, weeks, months, or even years.

He envisions a future where AI models, or millions of them working together, can perform the work of entire human organizations or even the entire scientific community.

What's Left: Knowledge, Memory, and Oversight

To achieve human-level AI, Kaplan identifies several key ingredients:

1. Relevant Organizational Knowledge: AI models need to learn to work within organizations, understanding context like a long-term employee.
2. Memory: AI needs to retain progress on long-term tasks and use those memories effectively. This is being built into Claude 4.
3. Oversight: AI needs to understand fine-grained nuances and solve fuzzy tasks. This requires developing AI models that can generate more nuanced reward signals for reinforcement learning, enabling them to do things like tell good jokes or have good taste in research.

Other necessary ingredients include training AI to do more complex tasks and scaling compute across different domains (text, multimodal, robotics).

Preparing for the Future: Building, Integrating, and Adopting

Kaplan offers advice for preparing for the future of AI:

1. Build things that don't quite work yet: AI is improving rapidly, so products that are currently limited by AI capabilities may become viable with future model releases.
2. Leverage AI for AI integration: AI can help speed up the process of integrating AI into products, companies, and science.
3. Identify areas for rapid AI adoption: Software engineering has seen rapid AI integration, but the question is what other fields can grow as quickly.

Claude 4 and the Future of AI: Q&A with Diana

Kaplan discusses Claude 4, highlighting improvements in its ability to act as an agent (especially for coding), its supervision capabilities, and its memory. He emphasizes that scaling laws suggest a smooth curve towards human-level AI.

He is most excited about Claude's memory, which unlocks longer and longer horizon tasks. He believes that AI skeptics often focus on AI's mistakes, but a key difference between human and AI intelligence is that AI's judgment and generative capabilities are much closer. This means that humans can play a crucial role in managing and sanity-checking AI's work.

Kaplan notes that while many companies are selling AI as a co-pilot, some are now selling AI as a direct replacement for full workflows. He believes that the level of acceptable performance determines whether AI can be fully automated.

He references Dario Amodei's "Machines of Love and Grace" essay, envisioning a future of human-AI collaboration. He suggests that AI's breadth of knowledge, gained during pre-training, can be particularly useful in areas like biology, psychology, and history, where putting together a large number of pieces of information is key.

He identifies finance and law as potential greenfield areas for AI development, along with integrating AI into existing businesses.

The Physicist's Perspective: Precision and Macro Trends

Kaplan explains how his physics background has helped him in AI research. He emphasizes the importance of looking for the biggest picture and making macro trends as precise as possible. He would ask simple questions to brilliant AI researchers to make sure that the trends are as precise as possible.

He notes that studying approximations where neural networks are very big, similar to approximations used in physics, has been useful. He also emphasizes the importance of asking naive, dumb questions, as AI is a relatively new field with many unanswered basic questions.

He believes that interpretability in AI is more like biology or neuroscience than physics.

Scaling Laws: When Will They Break?

Kaplan states that he primarily uses scaling laws to diagnose whether AI training is broken. If scaling laws appear to be failing, his first inclination is to suspect that there is a problem with the AI training process, such as an incorrect neural network architecture or a bottleneck in training.

Compute and Precision: The Future of AI Efficiency

Kaplan acknowledges the high compute power required for AI and the need to improve efficiency. He notes that companies are working to make both AI training and inference more efficient. He expects to see significant gains in inference efficiency over time, including the use of lower precision formats like FP4.

He jokes that computers may eventually return to binary. He believes that AI is currently in a state of disequilibrium, with rapid improvements and unrealized potential. He questions whether AI will ever reach an equilibrium where it is inexpensive and not changing quickly.

He addresses the Jevons paradox, where increased efficiency can lead to increased consumption. He believes that as AI becomes more capable, it will be worth paying for frontier capabilities. He also questions whether all the value is at the frontier or whether there is significant value in cheaper, less capable systems.

Advice for Staying Relevant: Building, Integrating, and Understanding

Kaplan advises the audience to focus on understanding how AI models work, efficiently leveraging and integrating them, and building at the frontier.

Q&A: Exponential Growth and Task Creation

In response to audience questions, Kaplan discusses the potential for exponential growth in the time horizon of AI tasks. He suggests that the ability to self-correct and identify mistakes is crucial for longer horizon tasks.

He also discusses the process of creating tasks for reinforcement learning, noting that it involves a mix of AI-generated tasks and human-created tasks. He hopes that AI will be able to leverage AI more and more in the future, but humans will still be involved.

Synthesis/Conclusion

Jared Kaplan's talk provides a comprehensive overview of the current state of AI, emphasizing the importance of scaling laws, pre-training, and reinforcement learning. He highlights the key ingredients needed to achieve human-level AI, including knowledge, memory, and oversight. He offers practical advice for preparing for the future of AI, focusing on building, integrating, and adopting AI technologies. The Q&A session provides further insights into the challenges and opportunities in the field, including the potential for exponential growth in AI capabilities and the ongoing need for human involvement in task creation and oversight. The main takeaway is that AI is progressing rapidly and predictably, driven by scaling compute and data, and that there are significant opportunities for those who can understand, integrate, and leverage these technologies.